如何使用BeautifulSoup抓取Yahoo Finance新闻标题?

huangapple go评论70阅读模式
英文:

How to scrape yahoo finance news headers with BeautifulSoup?

问题

I would like to scrape news from yahoo's finance, for a pair.

How does bs4's find() or find_all() work?

for this example:

如何使用BeautifulSoup抓取Yahoo Finance新闻标题?

with this link:

I'm trying to extract the data ... but no data is scraped. why? what's wrong?

I'm using this, but nothing is printed (except the tickers)

news_table_s = html.find_all("div", {"class": "Py(14px) Pos(r)"})
news_tables_s[ticker_s] = news_table_s
print("news_tables", news_tables_s)

I would like to extract the headers from a yahoo finance web page.

英文:

I would like to scrape news from yahoo's finance, for a pair.

How does bs4's find() or find_all() work?

for this example:

如何使用BeautifulSoup抓取Yahoo Finance新闻标题?

with this link:

I'm traying to extract the data ... but no data is scraped. why? what's wrong?

I'm using this, but nothing is printed (except the tickers)

html = BeautifulSoup(source_s, "html.parser")  # "html")
            
            news_table_s = html.find_all("div",{"class":"Py(14px) Pos(r)"})
            
            news_tables_s[ticker_s] = news_table_s
        print("news_tables", news_tables_s)

I would like to extract the headers from a yahoo finance web page.

答案1

得分: 0

你必须迭代你的 ResultSet 以获取其中的内容。

for e in html.find_all("div", {"class": "Py(14px) Pos(r)"}):
    print(e.h3.text)

建议 - 不要使用动态类来选择元素,使用更静态的ID或HTML结构,可以通过 css选择器 进行选择。

for e in html.select('div:has(>h3>a)'):
    print(e.h3.text)

示例

from bs4 import BeautifulSoup
import requests

url = 'https://finance.yahoo.com/quote/EURUSD%3DX?p=EURUSD%3DX'

html = BeautifulSoup(requests.get(url).text)

for e in html.select('div:has(>h3>a)'):
    print(e.h3.text)

输出

EUR/USD steadies, but bears sharpen claws as dollar feasts on Fed bets
EUR/USD Weekly Forecast – Euro Gives Up Early Gains for the Week
EUR/USD Forecast – Euro Plunges Yet Again on Friday
EUR/USD Forecast – Euro Gives Up Early Gains
EUR/USD Forecast – Euro Continues to Test the Same Indicator
Dollar gains as inflation remains sticky; sterling retreats
Siemens Issues Blockchain Based Euro-Denominated Bond on Polygon Blockchain
EUR/USD Forecast – Euro Rallies
FOREX-Dollar slips with inflation in focus; euro, sterling up on jobs data
FOREX-Jobs figures send euro, sterling higher; dollar slips before CPI
英文:

You have to iterate your ResultSet to get anything out.

for e in html.find_all("div",{"class":"Py(14px) Pos(r)"}):
    print(e.h3.text)

Recommendation - Do not use dynamic classes to select elements use more static ids or HTML structure, here selected via css selector

for e in html.select(' div:has(>h3>a)'):
        print(e.h3.text)

Example

from bs4 import BeautifulSoup
import requests

url='https://finance.yahoo.com/quote/EURUSD%3DX?p=EURUSD%3DX'

html = BeautifulSoup(requests.get(url).text)
            
for e in html.select(' div:has(>h3>a)'):
    print(e.h3.text)

Output

EUR/USD steadies, but bears sharpen claws as dollar feasts on Fed bets
EUR/USD Weekly Forecast – Euro Gives Up Early Gains for the Week
EUR/USD Forecast – Euro Plunges Yet Again on Friday
EUR/USD Forecast – Euro Gives Up Early Gains
EUR/USD Forecast – Euro Continues to Test the Same Indicator
Dollar gains as inflation remains sticky; sterling retreats
Siemens Issues Blockchain Based Euro-Denominated Bond on Polygon Blockchain
EUR/USD Forecast – Euro Rallies
FOREX-Dollar slips with inflation in focus; euro, sterling up on jobs data
FOREX-Jobs figures send euro, sterling higher; dollar slips before CPI

huangapple
  • 本文由 发表于 2023年2月19日 09:08:44
  • 转载请务必保留本文链接:https://go.coder-hub.com/75497393.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定