Python抓取返回None。

huangapple go评论70阅读模式
英文:

Python scraping returns none

问题

I'm trying to take a name from an HTML page with BeautifulSoup:

import urllib.request
from bs4 import BeautifulSoup

nightbot = 'https://nightbot.tv/t/tonyxzero/song_requests'
page = urllib.request.urlopen(nightbot)
soup = BeautifulSoup(page, 'html5lib')

list_item = soup.find('strong', attrs={'class': 'ng-binding'})
print(list_item)

But when I print print(list_item) I get a None as a reply. Is there a way to fix it?

英文:

I'm trying to take a name from a HTML page with BeautifulSoup:

import urllib.request
from bs4 import BeautifulSoup

nightbot = 'https://nightbot.tv/t/tonyxzero/song_requests'
page = urllib.request.urlopen(nightbot)
soup = BeautifulSoup(page, 'html5lib')

list_item = soup.find('strong', attrs={'class': 'ng-binding'})
print (list_item)

But when i print print(list_item) i get a none as reply. There is a way to fix it?

答案1

得分: 2

Webpage is rendered by javascript. So you have to use a package like selenium to get what you want.

You can try this:

CODE:

import urllib.request
from bs4 import BeautifulSoup
from selenium import webdriver

driver = webdriver.Firefox()
driver.get('https://nightbot.tv/t/tonyxzero/song_requests')

html = driver.page_source

soup = BeautifulSoup(html, 'html.parser')

list_item = soup.find('strong', attrs={'class': 'ng-binding'})
print (list_item)

RESULT:

<strong class="ng-binding" ng-bind="$state.current.title">Song Requests: TONYXZERO</strong>
英文:

Webpage is rendered by javascript. So you have to use a package like selenium to get what you want.

You can try this:

CODE:

import urllib.request
from bs4 import BeautifulSoup
from selenium import webdriver

driver = webdriver.Firefox()
driver.get(&#39;https://nightbot.tv/t/tonyxzero/song_requests&#39;)

html = driver.page_source

soup = BeautifulSoup(html, &#39;html.parser&#39;)

list_item = soup.find(&#39;strong&#39;, attrs={&#39;class&#39;: &#39;ng-binding&#39;})
print (list_item)

RESULT:

&lt;strong class=&quot;ng-binding&quot; ng-bind=&quot;$state.current.title&quot;&gt;Song Requests: TONYXZERO&lt;/strong&gt;

huangapple
  • 本文由 发表于 2020年1月6日 23:20:27
  • 转载请务必保留本文链接:https://go.coder-hub.com/59614625.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定