英文:
Python scraping returns none
问题
I'm trying to take a name from an HTML page with BeautifulSoup:
import urllib.request
from bs4 import BeautifulSoup
nightbot = 'https://nightbot.tv/t/tonyxzero/song_requests'
page = urllib.request.urlopen(nightbot)
soup = BeautifulSoup(page, 'html5lib')
list_item = soup.find('strong', attrs={'class': 'ng-binding'})
print(list_item)
But when I print print(list_item)
I get a None
as a reply. Is there a way to fix it?
英文:
I'm trying to take a name from a HTML page with BeautifulSoup:
import urllib.request
from bs4 import BeautifulSoup
nightbot = 'https://nightbot.tv/t/tonyxzero/song_requests'
page = urllib.request.urlopen(nightbot)
soup = BeautifulSoup(page, 'html5lib')
list_item = soup.find('strong', attrs={'class': 'ng-binding'})
print (list_item)
But when i print print(list_item)
i get a none
as reply. There is a way to fix it?
答案1
得分: 2
Webpage is rendered by javascript. So you have to use a package like selenium
to get what you want.
You can try this:
CODE:
import urllib.request
from bs4 import BeautifulSoup
from selenium import webdriver
driver = webdriver.Firefox()
driver.get('https://nightbot.tv/t/tonyxzero/song_requests')
html = driver.page_source
soup = BeautifulSoup(html, 'html.parser')
list_item = soup.find('strong', attrs={'class': 'ng-binding'})
print (list_item)
RESULT:
<strong class="ng-binding" ng-bind="$state.current.title">Song Requests: TONYXZERO</strong>
英文:
Webpage is rendered by javascript. So you have to use a package like selenium
to get what you want.
You can try this:
CODE:
import urllib.request
from bs4 import BeautifulSoup
from selenium import webdriver
driver = webdriver.Firefox()
driver.get('https://nightbot.tv/t/tonyxzero/song_requests')
html = driver.page_source
soup = BeautifulSoup(html, 'html.parser')
list_item = soup.find('strong', attrs={'class': 'ng-binding'})
print (list_item)
RESULT:
<strong class="ng-binding" ng-bind="$state.current.title">Song Requests: TONYXZERO</strong>
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论