无法从Yahoo Finance的分析选项卡中获取数据。

huangapple
go评论129阅读模式
英文:

Can't fetch data from the analysis tab on Yahoo Finance

问题

  1. 我正在尝试从一个网站上抓取一些表格内容。这个[网站](https://finance.yahoo.com/quote/AAPL/analysis?p=AAPL)上的数据加载过程发生了巨大变化。以前，所需的数据可以在页面源代码的一些脚本标签中找到。我通过开发工具查看了端点，但没有在那里找到任何数据。不过，我不确定是否在那里漏掉了什么。我对位于“Revenue Estimate”下的表格感兴趣。以下是如何获取内容的示例代码。
  3. ```python
  4. import re
  5. import json
  6. import requests
  7. from pprint import pprint
  9. link = 'https://finance.yahoo.com/quote/AAPL/analysis?p=AAPL'
  11. headers = {
  12.     'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36',
  13. }
  14. with requests.Session() as s:
  15.     s.headers.update(headers)
  16.     res = s.get(link)
  17.     data = re.findall(r'root.App.main[^{]+([\s\S].*);', res.text)[0]
  18.     jsoncontent = json.loads(data)
  20.     try:
  21.         container = jsoncontent['context']['dispatcher']['stores']['QuoteSummaryStore']['earningsTrend']
  22.     except TypeError:
  23.         container = ""
  25.     pprint(container)

这是示例代码，用于从网站中获取数据。

  2. <details>
  3. <summary>英文:</summary>
  5. I&#39;m trying to scrape some tabular content from a website. The data loading process on this [website](https://finance.yahoo.com/quote/AAPL/analysis?p=AAPL) has changed dramatically. Previously, the necessary data could be found within some script tags in the page source. I looked into the endpoint through dev tools but could not find any data there. I&#39;m not sure if I missed anything in there, though. I&#39;m interested in the table located under `Revenue Estimate`. This is something how I could fetch the content.
  7.     import re
  8.     import json
  9.     import requests
  10.     from pprint import pprint
  11.     
  12.     link = &#39;https://finance.yahoo.com/quote/AAPL/analysis?p=AAPL&#39;
  13.     
  14.     headers = {
  15.         &#39;User-Agent&#39;: &#39;Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36&#39;,
  16.     }
  17.     with requests.Session() as s:
  18.         s.headers.update(headers)
  19.         res = s.get(link)
  20.         data = re.findall(r&#39;root.App.main[^{]+([\s\S].*);&#39;,res.text)[0]
  21.         jsoncontent = json.loads(data)
  22.     
  23.         # pprint(jsoncontent)
  24.     
  25.         try:
  26.             container = jsoncontent[&#39;context&#39;][&#39;dispatcher&#39;][&#39;stores&#39;][&#39;QuoteSummaryStore&#39;][&#39;earningsTrend&#39;]
  27.         except TypeError: container = &quot;&quot;
  28.     
  29.         pprint(container)
  33. </details>
  36. # 答案1
  37. **得分**: 1
  39. 请尝试使用以下代码：
  41. ```python
  42. import requests
  44. headers = {
  45.     'authority': 'finance.yahoo.com',
  46.     'accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9',
  47.     'accept-language': 'de,de-DE;q=0.9,en;q=0.8,en-GB;q=0.7,en-US;q=0.6,fr;q=0.5,de-CH;q=0.4,es;q=0.3',
  48.     'cache-control': 'no-cache',
  49.     'dnt': '1',
  50.     'pragma': 'no-cache',
  51.     'sec-ch-ua': '"Not_A Brand";v="99", "Microsoft Edge";v="109", "Chromium";v="109"',
  52.     'sec-ch-ua-mobile': '?0',
  53.     'sec-ch-ua-platform': '"Windows"',
  54.     'sec-fetch-dest': 'document',
  55.     'sec-fetch-mode': 'navigate',
  56.     'sec-fetch-site': 'same-origin',
  57.     'sec-fetch-user': '?1',
  58.     'upgrade-insecure-requests': '1',
  59.     'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36 Edg/109.0.1518.78',
  60. }
  62. params = {
  63.     'p': 'AAPL',
  64. }
  66. response = requests.get('https://finance.yahoo.com/quote/AAPL/analysis', params=params, headers=headers)

然后从 response.content 中解析所需的值。

英文:

Try using:

  1. import requests
  3. headers = {
  4.     &#39;authority&#39;: &#39;finance.yahoo.com&#39;,
  5.     &#39;accept&#39;: &#39;text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9&#39;,
  6.     &#39;accept-language&#39;: &#39;de,de-DE;q=0.9,en;q=0.8,en-GB;q=0.7,en-US;q=0.6,fr;q=0.5,de-CH;q=0.4,es;q=0.3&#39;,
  7.     &#39;cache-control&#39;: &#39;no-cache&#39;,
  8.     &#39;dnt&#39;: &#39;1&#39;,
  9.     &#39;pragma&#39;: &#39;no-cache&#39;,
  10.     &#39;sec-ch-ua&#39;: &#39;&quot;Not_A Brand&quot;;v=&quot;99&quot;, &quot;Microsoft Edge&quot;;v=&quot;109&quot;, &quot;Chromium&quot;;v=&quot;109&quot;&#39;,
  11.     &#39;sec-ch-ua-mobile&#39;: &#39;?0&#39;,
  12.     &#39;sec-ch-ua-platform&#39;: &#39;&quot;Windows&quot;&#39;,
  13.     &#39;sec-fetch-dest&#39;: &#39;document&#39;,
  14.     &#39;sec-fetch-mode&#39;: &#39;navigate&#39;,
  15.     &#39;sec-fetch-site&#39;: &#39;same-origin&#39;,
  16.     &#39;sec-fetch-user&#39;: &#39;?1&#39;,
  17.     &#39;upgrade-insecure-requests&#39;: &#39;1&#39;,
  18.     &#39;user-agent&#39;: &#39;Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36 Edg/109.0.1518.78&#39;,
  19. }
  21. params = {
  22.     &#39;p&#39;: &#39;AAPL&#39;,
  23. }
  25. response = requests.get(&#39;https://finance.yahoo.com/quote/AAPL/analysis&#39;, params=params, headers=headers)

and efter that parse the desired values from response.content.

答案2

得分: 1

你可以使用Pandas DataFrame来获取Revenue Estimate表格数据，如下所示：

  1. import requests
  2. import pandas as pd
  4. headers = {"user-agent": "Mozilla/5.0"}
  6. res = requests.get("https://finance.yahoo.com/quote/AAPL/analysis?p=AAPL&amp;guccounter=1", headers=headers).text
  7. # print(res)
  8. df = pd.read_html(res)[1]
  9. print(df)

输出：

  1.    Revenue Estimate Current Qtr. (Mar 2023) Next Qtr. (Jun 2023) Current Year (2023) Next Year (2024)
  2. 0  No. of Analysts                      24                   23                  39               36
  3. 1    Avg. Estimate                  93.19B               85.59B             392.39B          417.75B
  4. 2     Low Estimate                  91.81B               81.32B             378.62B          398.67B
  5. 3    High Estimate                  98.84B               90.12B             414.04B          438.76B
  6. 4   Year Ago Sales                  97.28B               82.96B             394.33B          392.39B
  7. 5  Sales Growth (year/est)                  -4.20%                3.20%              -0.50%            6.50%
英文:

You can use Pandas DataFrame to get the Revenue Estimate table data as follows:

  1. import requests
  2. import pandas as pd
  4. headers= {&quot;user-agent&quot;:&quot;Mozilla/5.0&quot;}
  6. res = requests.get(&quot;https://finance.yahoo.com/quote/AAPL/analysis?p=AAPL&amp;guccounter=1&quot;, headers=headers).text
  7. #print(res)
  8. df= pd.read_html(res)[1]
  9. print(df)

Output:

  1.       Revenue Estimate Current Qtr. (Mar 2023) Next Qtr. (Jun 2023) Current Year (2023) Next Year (2024)
  2. 0          No. of Analysts                      24                   23                  39               36
  3. 1            Avg. Estimate                  93.19B               85.59B             392.39B          417.75B
  4. 2             Low Estimate                  91.81B               81.32B             378.62B          398.67B
  5. 3            High Estimate                  98.84B               90.12B             414.04B          438.76B
  6. 4           Year Ago Sales                  97.28B               82.96B             394.33B          392.39B
  7. 5  Sales Growth (year/est)                  -4.20%                3.20%              -0.50%            6.50%

huangapple
  • 本文由 发表于 2023年2月8日 19:39:49
  • 转载请务必保留本文链接：https://go.coder-hub.com/75385266.html
  • python
  • python-3.x
  • python-requests
  • web-scraping
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定