英文:
Generating/collecting URLs from "NEO Earth Close Approaches" database to link to Orbit Viewer
问题
我正在处理一个个人项目，利用NEO Earth Close Approaches数据库中的数据。我正在使用pandas清理/准备数据集，以在Tableau可视化中使用，但我非常好奇听听其他人如何解决一个特定的问题。
在浏览数据库时，表格包括“View CA”列，该列链接到一个轨道查看器，显示每次近地点接近。下载CSV或Excel文件不提供这些链接。我想在我的数据框中包含这些链接。
我已经找到了链接的格式，通过检查元素，例如，Object (2023 LD) 的View CA链接如下：
在链接的末尾，您可以看到...desig=2023%20LD&cajd=2460104.544743703&使用Object/designation（从2023 LD获取的desig=2023%20LD...）和cajd=...使用对象的近地点日期，采用朱利安日期格式，我已经创建了一个转换函数来生成。
首先，我提出的解决方案是否可行？其次，您会如何解决这个挑战？正如我所说，我最好奇的是是否有其他可能更简单的方法来做到这一点，因为我花了太多时间来解决我目前的解决方案。
谢谢！
到目前为止，我的解决方案是使用正则表达式函数和cajd转换函数基于可下载格式中可用的数据生成链接。
英文:
I'm working on a personal project utilizing data from the NEO Earth Close Approaches database. I'm cleaning/prepping the dataset with pandas to be used within a Tableau Viz, but I am very curious to hear how others would approach one particular issue.
When viewing the database in browser, the table includes the column "View CA", which links out to an Orbit Viewer showing each close approach. Downloading the CSV or Excel file does not provide these links. I would like to have these links in my dataframe.
I have figured out the format of the link through inspect element; for example, the View CA link for Object (2023 LD):
At the end of the link, you can see that ...desig=2023%20LD&cajd=2460104.544743703& uses the Object/designation (desig=2023%20LD... from 2023 LD) and cajd=... uses the object's close approach date in Julian Date format, which I have created a conversion function to generate.
Firstly, is my outlined solution a sane way to go about this? Secondly, how would you approach this challenge yourself? My main curiosity as I stated is whether there are other, possibly simpler ways to do this as I took way too much time to figure out my solution where it currently is.
Thanks!
So far my solution has been to use regex functions and the cajd conversion function to generate the links based on the data available from the downloadable format.
答案1
得分: 0
import urllib
import requests
import pandas as pd
def get_link(desig, jd, dist):
scale = 0.1
if dist < 0.02:
scale = 1.0
elif dist < 0.05:
scale = 0.5
elif dist < 0.1:
scale = 0.2
ov_par = ''
ov_par += '#load=&orientation=0,0,0,1'
# ov_par += '&viewfrom=above'
ov_par += '&lookat=Earth'
ov_par += '&interval=2'
ov_par += '&eclipticaxis=false'
ov_par += '&eclipticgrid=false'
ov_par += '&largeFont=false' # + str(isSafari)
ov_par += '&distance=29919.57414'
ov_par += '&pitch=0&roll=0&yaw=0'
ov_par += '&scale=' + format(scale, '.3f')
ov_par += '&rotateX=-30.20870289631195'
ov_par += '&rotateY=38.134339235185024'
ov_par += '&desig=DESIG'
ov_par += '&cajd=JD&'
ov_par = '/ca/ov/' + ov_par
ov_par = ov_par.replace('DESIG', urllib.parse.quote(desig))
ov_par = ov_par.replace('JD', str(jd))
return 'https://cneos.jpl.nasa.gov' + ov_par
api_url= 'https://ssd-api.jpl.nasa.gov/cad.api?diameter=1&dist-max=0.05&fullname=1&nea-comet=1&rating=1&www=1'
data = requests.get(api_url).json()
df = pd.DataFrame(data['data'], columns=data['fields'])
df['jd'] = pd.to_numeric(df['jd'])
df['dist'] = pd.to_numeric(df['dist'])
for des, jd, dist in zip(df['des'], df['jd'], df['dist']):
print(get_link(des.split('|')[-1], jd, dist))
英文:
Looking at the source code of the page, the URLs are constructed on the fly via JavaScript. The Js function that constructs the links looks like this (you can find it through the Web Developer Tools):
$scope.data.onRenderView = function (desig, jd, dist) {
// console.log('DEBUG: enter onRenderView()...'); // TESTING
//
// Set the viewer scale based on the approach distance:
var scale = 0.1;
if ( dist < 0.02 ) { scale = 1.0; }
else if ( dist < 0.05 ) { scale = 0.5; }
else if ( dist < 0.1 ) { scale = 0.2; }
//
// Set the orbit-viewer query-parameters string:
var ov_par = '';
ov_par += '#load=&orientation=0,0,0,1';
// ov_par += '&viewfrom=above';
ov_par += '&lookat=Earth';
ov_par += '&interval=2';
ov_par += '&eclipticaxis=false';
ov_par += '&eclipticgrid=false';
ov_par += '&largeFont=' + isSafari;
ov_par += '&distance=29919.57414';
ov_par += '&pitch=0&roll=0&yaw=0';
ov_par += '&scale=' + scale.toFixed(3);
ov_par += '&rotateX=-30.20870289631195';
ov_par += '&rotateY=38.134339235185024';
ov_par += '&desig=DESIG';
ov_par += '&cajd=JD&';
ov_par = '/ca/ov/' + encodeURI(ov_par);
ov_par = ov_par.replace('DESIG', desig);
ov_par = ov_par.replace('JD', jd);
$sce.trustAsResourceUrl(ov_par);
//
// Remove any prior viewer then add the new viewer to the DOM:
$('#orbit-viewer-iframe').remove();
var iframe = '<iframe id="orbit-viewer-iframe" style="width: 1000px; height: 600px;" src="';
iframe += ov_par;
iframe += '"></iframe>';
$('#orbit-viewer-container').append(iframe);
//
// Set a flag to show the new viewer:
$scope.data.orbitViewerShow = true;
};
So to get all links you need 3 parameters desig, jd, dist. The page gets them via Ajax call. So to construct the links you can use next example:
import urllib
import requests
import pandas as pd
def get_link(desig, jd, dist):
scale = 0.1
if dist < 0.02:
scale = 1.0
elif dist < 0.05:
scale = 0.5
elif dist < 0.1:
scale = 0.2
ov_par = ''
ov_par += '#load=&orientation=0,0,0,1'
# ov_par += '&viewfrom=above'
ov_par += '&lookat=Earth'
ov_par += '&interval=2'
ov_par += '&eclipticaxis=false'
ov_par += '&eclipticgrid=false'
ov_par += '&largeFont=false' #+ str(isSafari)
ov_par += '&distance=29919.57414'
ov_par += '&pitch=0&roll=0&yaw=0'
ov_par += '&scale=' + format(scale, '.3f')
ov_par += '&rotateX=-30.20870289631195'
ov_par += '&rotateY=38.134339235185024'
ov_par += '&desig=DESIG'
ov_par += '&cajd=JD&'
ov_par = '/ca/ov/' + ov_par
ov_par = ov_par.replace('DESIG', urllib.parse.quote(desig))
ov_par = ov_par.replace('JD', str(jd))
return 'https://cneos.jpl.nasa.gov' + ov_par
api_url= 'https://ssd-api.jpl.nasa.gov/cad.api?diameter=1&dist-max=0.05&fullname=1&nea-comet=1&rating=1&www=1'
data = requests.get(api_url).json()
df = pd.DataFrame(data['data'], columns=data['fields'])
df['jd'] = pd.to_numeric(df['jd'])
df['dist'] = pd.to_numeric(df['dist'])
for des, jd, dist in zip(df['des'], df['jd'], df['dist']):
print(get_link(des.split('|')[-1], jd, dist))
Prints:
https://cneos.jpl.nasa.gov/ca/ov/#load=&orientation=0,0,0,1&lookat=Earth&interval=2&eclipticaxis=false&eclipticgrid=false&largeFont=false&distance=29919.57414&pitch=0&roll=0&yaw=0&scale=1.000&rotateX=-30.20870289631195&rotateY=38.134339235185024&desig=2023%20LD&cajd=2460104.544743703&
https://cneos.jpl.nasa.gov/ca/ov/#load=&orientation=0,0,0,1&lookat=Earth&interval=2&eclipticaxis=false&eclipticgrid=false&largeFont=false&distance=29919.57414&pitch=0&roll=0&yaw=0&scale=0.500&rotateX=-30.20870289631195&rotateY=38.134339235185024&desig=2023%20JB3&cajd=2460105.360445169&
https://cneos.jpl.nasa.gov/ca/ov/#load=&orientation=0,0,0,1&lookat=Earth&interval=2&eclipticaxis=false&eclipticgrid=false&largeFont=false&distance=29919.57414&pitch=0&roll=0&yaw=0&scale=1.000&rotateX=-30.20870289631195&rotateY=38.134339235185024&desig=2023%20LA&cajd=2460107.524286449&
https://cneos.jpl.nasa.gov/ca/ov/#load=&orientation=0,0,0,1&lookat=Earth&interval=2&eclipticaxis=false&eclipticgrid=false&largeFont=false&distance=29919.57414&pitch=0&roll=0&yaw=0&scale=0.500&rotateX=-30.20870289631195&rotateY=38.134339235185024&desig=488453&cajd=2460107.536744452&
...
The df looks like this:
des orbit_id jd cd dist dist_min dist_max v_rel v_inf t_sigma_f h diameter diameter_sigma fullname rating
0 bK23L00D|2023 LD 2 2460104.544743703 2023-Jun-09 01:04 0.00635081714223801 0.0063401113338132 0.00636152260566893 9.10079205473851 9.05457441910954 < 00:01 26.371 None None (2023 LD) 0
1 bK23J03B|2023 JB3 13 2460105.360445169 2023-Jun-09 20:39 0.0362163376263994 0.0361880148384531 0.0362446602808189 6.94939728103983 6.93880250289472 < 00:01 24.174 None None (2023 JB3) 0
2 bK23L00A|2023 LA 3 2460107.524286449 2023-Jun-12 00:35 0.0044093539762013 0.00439127221463214 0.00442743413650688 10.4456580761954 10.3876472288082 < 00:01 25.208 None None (2023 LA) 1
3 a0488453|488453 93 2460107.536744452 2023-Jun-12 00:53 0.0211423059659372 0.0211421868604232 0.0211424250714549 21.4724651802065 21.4665951888416 < 00:01 19.28 None None 488453 (1994 XD) 2
4 bK22W04N|2022 WN4 21 2460108.897696208 2023-Jun-13 09:33 0.0276624541212163 0.0275584911929838 0.027766703533987 15.0756063294164 15.0692157693588 00:08 21.76 None None (2022 WN4) 1
...
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论