如何向URL添加日期以获取网站的历史数据

huangapple
go评论72阅读模式
英文:

How Do I add dates to a URL to retrieve Historical data from a website

问题

我有这个API https://api.weather.com/v2/pws/observations/current?apiKey=e1f10a1e78da46f5b10a1e78da96f525&stationId=KNYNEWYO1585&numericPrecision=decimal&format=json&units=e，我正在尝试从2023年04月30日检索数据。

然而，我遇到了以下错误

JSONDecodeError                           Traceback (most recent call last)
<ipython-input-10-873917b622c8> in <module>
     10 endpoint = f"https://api.weather.com/v2/pws/observations/historical.json?apiKey={apiKey}&stationId={station_id}&format=json&units=e&date={start_date}&endDate={end_date}"
     11 
---> 12 response = requests.get(endpoint).json()["observations"]
     13 weather_data = sorted(response, key=lambda k: k["epoch"])

是否有人可以帮助我正确设置端点的语法？我认为错误在于这里，因为API链接适用于当前时间，但不适用于我尝试检索数据的特定日期。非常感谢任何帮助。

from datetime import datetime
import requests
from tabulate import tabulate

apiKey = "e1f10a1e78da46f5b10a1e78da96f525"
station_id = "KNYNEWYO1585"
start_date = "2023-04-30"
end_date = "2023-04-30"

endpoint = f"https://api.weather.com/v2/pws/observations/historical.json?apiKey={apiKey}&stationId={station_id}&format=json&units=e&date={start_date}&endDate={end_date}"

response = requests.get(endpoint).json()["observations"]
weather_data = sorted(response, key=lambda k: k["epoch"])

header = [
    "Time", "Temperature", "Dew Point", "Humidity", "Wind",
    "Wind Speed", "Wind Gust", "Pressure", "Precip.", "Conditions",
]

table = []
for item in weather_data:
    row = [
        datetime.fromtimestamp(item["epoch"]).strftime('%I:%M %p'),
        item["temp"],
        f'{item["dewpt"]} °F',
        f'{item["humidity"]} %',
        item["winddir"],
        item["wspd"],
        f'{item["wgust"] if "wgust" in item else 0} mph',
        f'{item["pressure"]} in',
        f'{item["precip_total"] if "precip_total" in item else "0.0 in"}',
        item["wx_phrase"],
    ]
    table.append(row)

print(tabulate(table, headers=header, tablefmt="pretty"))
英文:

So I've this API https://api.weather.com/v2/pws/observations/current?apiKey=e1f10a1e78da46f5b10a1e78da96f525&stationId=KNYNEWYO1585&numericPrecision=decimal&format=json&units=e and I'm trying to retrieve data from 2023-04-30

However, I get the following error

JSONDecodeError                           Traceback (most recent call last)
<ipython-input-10-873917b622c8> in <module>
     10 endpoint = f"https://api.weather.com/v2/pws/observations/historical.json?apiKey={apiKey}&stationId={station_id}&format=json&units=e&date={start_date}&endDate={end_date}"
     11 
---> 12 response = requests.get(endpoint).json()["observations"]
     13 weather_data = sorted(response, key=lambda k: k["epoch"])

Can anyone help me with the proper syntax for the endpoint? I believe the mistake lies here because the API link works for the current time but not for the specific date I'm trying to retrieve data for. Any assistance would be greatly appreciated

from datetime import datetime
import requests
from tabulate import tabulate

apiKey = "e1f10a1e78da46f5b10a1e78da96f525"
station_id = "KNYNEWYO1585"
start_date = "2023-04-30"
end_date = "2023-04-30"

endpoint = f"https://api.weather.com/v2/pws/observations/historical.json?apiKey={apiKey}&stationId={station_id}&format=json&units=e&date={start_date}&endDate={end_date}"

response = requests.get(endpoint).json()["observations"]
weather_data = sorted(response, key=lambda k: k["epoch"])

header = [
    "Time", "Temperature", "Dew Point", "Humidity", "Wind",
    "Wind Speed", "Wind Gust", "Pressure", "Precip.", "Conditions",
]

table = []
for item in weather_data:
    row = [
        datetime.fromtimestamp(item["epoch"]).strftime('%I:%M %p'),
        item["temp"],
        f'{item["dewpt"]} °F',
        f'{item["humidity"]} %',
        item["winddir"],
        item["wspd"],
        f'{item["wgust"] if "wgust" in item else 0} mph',
        f'{item["pressure"]} in',
        f'{item["precip_total"] if "precip_total" in item else "0.0 in"}',
        item["wx_phrase"],
    ]
    table.append(row)

print(tabulate(table, headers=header, tablefmt="pretty"))

答案1

得分: 1

你正在查询的终端点是错误的。而且负载也是错误的。

要获取给定日期的所有表格读数，您需要这样做：

import datetime
from urllib.parse import urlencode

import pandas as pd
import requests
from tabulate import tabulate

payload = {
    "stationId": "KNYNEWYO1585",
    "format": "json",
    "units": "e",
    "date": "20230430",
    "numericPrecision": "decimal",
    "apiKey": "e1f10a1e78da46f5b10a1e78da96f525",
}
endpoint = "https://api.weather.com/v2/pws/history/all?"

headers = {
    "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) "
                  "AppleWebKit/537.36 (KHTML, like Gecko) "
                  "Chrome/113.0.0.0 Safari/537.36 Edg/113.0.1774.57",
}

data_points = (
    requests
    .get(endpoint + urlencode(payload), headers=headers)
    .json()["observations"]
)

table = []
for data_point in data_points:
    table.append(
        [
            datetime.datetime.fromtimestamp(data_point["epoch"]).strftime("%I:%M %p"),
            data_point["imperial"]["dewptAvg"],
            data_point["imperial"]["tempAvg"],
            data_point["humidityAvg"],
            data_point["winddirAvg"],
            data_point["imperial"]["windspeedAvg"],
            data_point["imperial"]["windgustAvg"],
            data_point["imperial"]["pressureMax"],
            data_point["imperial"]["precipRate"],
            data_point["imperial"]["precipTotal"],
            data_point["uvHigh"],
            data_point["solarRadiationHigh"],
        ]
    )

columns = [
    "Time",
    "Dew Point ℉",
    "Temp ℉",
    "Humidity %",
    "Wind",
    "Wind Speed mph",
    "Gust mph",
    "Pressure in",
    "Precip Rate in/hr",
    "Precip Total in",
    "UV",
    "Solar Radiation W/m^2",
]

df = pd.DataFrame(table, columns=columns)
print(tabulate(df, headers="keys", tablefmt="psql", showindex=False))

使用上述代码，您可以重建表格（根据您的喜好）。

+----------+----------------+-----------+--------------+--------+------------------+------------+---------------+---------------------+-------------------+------+-------------------------+
| Time     |   Dew Point ℉ |   Temp ℉ |   Humidity % |   Wind |   Wind Speed mph |   Gust mph |   Pressure in |   Precip Rate in/hr |   Precip Total in |   UV |   Solar Radiation W/m^2 |
|----------+----------------+-----------+--------------+--------+------------------+------------+---------------+---------------------+-------------------+------+-------------------------|
| 06:04 AM |           50.6 |      50.9 |           99 |    351 |              0.5 |        2.8 |         29.79 |                0    |              0    |    0 |                     0   |
| 06:09 AM |           50.7 |      51   |           99 |     48 |              1.1 |        2.8 |         29.79 |                0    |              0    |    0 |                     0   |
| 06:14 AM |           50.7 |      51   |           99 |    289 |              0.4 |        3.2 |         29.79 |                0    |              0    |    0 |                     0   |
| 06:15 AM |           50.6 |      50.9 |           99 |    257 |              2   |        2.7 |         29.78 |                0    |              0    |    0 |                     0   |
| 06:23 AM |           50.6 |      50.9 |           99 |     15 |              0   |        2.6 |         29.79 |                0    |              0    |    0 |                     0   |
| 06:34 AM |           50.6 |      50.9 |           99 |    252 |              0.7 |        2.6 |         29.78 |                0    |              0    |    0 |                     0   |
| 06:36 AM |           50.6 |      50.9 |           99 |     29 |              0   |        2.2 |         29.78 |                0    |              0    |    0 |                     0   |
| 06:48 AM |           50.6 |      50.9 |           99 |     21 |              1.3 |        2.8 |         29.79 |                0    |              0    |    0 |                     0   |
| 06:53 AM |           50.6 |      50.9 |           99 |    285 |              0.6 |        2.1 |         29.79 |                0    |              0    |    0 |                     0   |
| 06:59 AM |           50.6 |      50.9 |           99 |    297 |              1.9 |        4.8 |         29.79 |                0    |              0    |    0 |                     0   |
| 07:09 AM |           50.4 |      50.7 |           99 |     66 |              0   |        2.7 |         29.77 |                0    |              0    |    0 |                     0   |
| 07:15 AM |           50.4 |      50.7 |           99 |    352 |              2   |        5.1 |         29.77 |                0    |              0    |    0 |                     0   |
| 07:22 AM |           50.4 |      50.7 |           99 |     64 |              2.5 |        2.7 |         29.77 |                0    |              0    |    0 |                     0   |
| 07:28 AM |           50.4 |      50.7 |           99 |      4 |              1.6 |        4.3 |         29.76 |                0    |              0    |    

<details>
<summary>英文:</summary>

The endpoint you&#39;re querying is wrong. And so is the payload.

To get all the table readings for a given day, you need this:

```python
import datetime
from urllib.parse import urlencode

import pandas as pd
import requests
from tabulate import tabulate

payload = {
    &quot;stationId&quot;: &quot;KNYNEWYO1585&quot;,
    &quot;format&quot;: &quot;json&quot;,
    &quot;units&quot;: &quot;e&quot;,
    &quot;date&quot;: &quot;20230430&quot;,
    &quot;numericPrecision&quot;: &quot;decimal&quot;,
    &quot;apiKey&quot;: &quot;e1f10a1e78da46f5b10a1e78da96f525&quot;,

}
endpoint = &quot;https://api.weather.com/v2/pws/history/all?&quot;

headers = {
    &quot;User-Agent&quot;: &quot;Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) &quot;
                  &quot;AppleWebKit/537.36 (KHTML, like Gecko) &quot;
                  &quot;Chrome/113.0.0.0 Safari/537.36 Edg/113.0.1774.57&quot;,
}

data_points = (
    requests
    .get(endpoint + urlencode(payload), headers=headers)
    .json()[&quot;observations&quot;]
)

table = []
for data_point in data_points:
    table.append(
        [
            datetime.datetime.fromtimestamp(data_point[&quot;epoch&quot;]).strftime(&quot;%I:%M %p&quot;),
            data_point[&quot;imperial&quot;][&quot;dewptAvg&quot;],
            data_point[&quot;imperial&quot;][&quot;tempAvg&quot;],
            data_point[&quot;humidityAvg&quot;],
            data_point[&quot;winddirAvg&quot;],
            data_point[&quot;imperial&quot;][&quot;windspeedAvg&quot;],
            data_point[&quot;imperial&quot;][&quot;windgustAvg&quot;],
            data_point[&quot;imperial&quot;][&quot;pressureMax&quot;],
            data_point[&quot;imperial&quot;][&quot;precipRate&quot;],
            data_point[&quot;imperial&quot;][&quot;precipTotal&quot;],
            data_point[&quot;uvHigh&quot;],
            data_point[&quot;solarRadiationHigh&quot;],
        ]
    )

columns = [
    &quot;Time&quot;,
    &quot;Dew Point &#176;F&quot;,
    &quot;Temp &#176;F&quot;,
    &quot;Humidity %&quot;,
    &quot;Wind&quot;,
    &quot;Wind Speed mph&quot;,
    &quot;Gust mph&quot;,
    &quot;Pressure in&quot;,
    &quot;Precip Rate in/hr&quot;,
    &quot;Precip Total in&quot;,
    &quot;UV&quot;,
    &quot;Solar Radiation W/m^2&quot;,
]

df = pd.DataFrame(table, columns=columns)
print(tabulate(df, headers=&quot;keys&quot;, tablefmt=&quot;psql&quot;, showindex=False))

Using the above, you can reconstruct the table (to whatever liking you might have).

+----------+----------------+-----------+--------------+--------+------------------+------------+---------------+---------------------+-------------------+------+-------------------------+
| Time     |   Dew Point &#176;F |   Temp &#176;F |   Humidity % |   Wind |   Wind Speed mph |   Gust mph |   Pressure in |   Precip Rate in/hr |   Precip Total in |   UV |   Solar Radiation W/m^2 |
|----------+----------------+-----------+--------------+--------+------------------+------------+---------------+---------------------+-------------------+------+-------------------------|
| 06:04 AM |           50.6 |      50.9 |           99 |    351 |              0.5 |        2.8 |         29.79 |                0    |              0    |    0 |                     0   |
| 06:09 AM |           50.7 |      51   |           99 |     48 |              1.1 |        2.8 |         29.79 |                0    |              0    |    0 |                     0   |
| 06:14 AM |           50.7 |      51   |           99 |    289 |              0.4 |        3.2 |         29.79 |                0    |              0    |    0 |                     0   |
| 06:15 AM |           50.6 |      50.9 |           99 |    257 |              2   |        2.7 |         29.78 |                0    |              0    |    0 |                     0   |
| 06:23 AM |           50.6 |      50.9 |           99 |     15 |              0   |        2.6 |         29.79 |                0    |              0    |    0 |                     0   |
| 06:34 AM |           50.6 |      50.9 |           99 |    252 |              0.7 |        2.6 |         29.78 |                0    |              0    |    0 |                     0   |
| 06:36 AM |           50.6 |      50.9 |           99 |     29 |              0   |        2.2 |         29.78 |                0    |              0    |    0 |                     0   |
| 06:48 AM |           50.6 |      50.9 |           99 |     21 |              1.3 |        2.8 |         29.79 |                0    |              0    |    0 |                     0   |
| 06:53 AM |           50.6 |      50.9 |           99 |    285 |              0.6 |        2.1 |         29.79 |                0    |              0    |    0 |                     0   |
| 06:59 AM |           50.6 |      50.9 |           99 |    297 |              1.9 |        4.8 |         29.79 |                0    |              0    |    0 |                     0   |
| 07:09 AM |           50.4 |      50.7 |           99 |     66 |              0   |        2.7 |         29.77 |                0    |              0    |    0 |                     0   |
| 07:15 AM |           50.4 |      50.7 |           99 |    352 |              2   |        5.1 |         29.77 |                0    |              0    |    0 |                     0   |
| 07:22 AM |           50.4 |      50.7 |           99 |     64 |              2.5 |        2.7 |         29.77 |                0    |              0    |    0 |                     0   |
| 07:28 AM |           50.4 |      50.7 |           99 |      4 |              1.6 |        4.3 |         29.76 |                0    |              0    |    0 |                     0   |
| 07:34 AM |           50.5 |      50.8 |           99 |    261 |              0.5 |        3.1 |         29.77 |                0    |              0    |    0 |                     0   |
| 07:39 AM |           50.4 |      50.7 |           99 |     66 |              0   |        2.5 |         29.76 |                0    |              0    |    0 |                     0   |
| 07:44 AM |           50.5 |      50.8 |           99 |    107 |              0.5 |        3.5 |         29.76 |                0    |              0    |    0 |                     0   |
| 07:48 AM |           50.5 |      50.8 |           99 |      0 |              1   |        3.5 |         29.76 |                0    |              0    |    0 |                     0   |
| 07:54 AM |           50.4 |      50.7 |           99 |    300 |              0   |        3.4 |         29.76 |                0    |              0    |    0 |                     0   |
| 08:02 AM |           50.4 |      50.7 |           99 |    270 |              1.3 |        4   |         29.76 |                0    |              0    |    0 |                     0   |
| 08:09 AM |           50.2 |      50.5 |           99 |    274 |              1.9 |        5.8 |         29.75 |                0    |              0    |    0 |                     0   |
| 08:11 AM |           50.4 |      50.7 |           99 |     77 |              2.2 |        3.5 |         29.75 |                0    |              0    |    0 |                     0   |
AND A WHOLE LOT MORE ...

huangapple
  • 本文由 发表于 2023年6月8日 23:03:48
  • 转载请务必保留本文链接：https://go.coder-hub.com/76433218.html
  • python
  • web-scraping
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定