英文:
Infinite scrolling of webpage using Selenium Python
问题
我想滚动直到所有元素都加载完毕。我的代码在前面7到8页运行良好，使用以下代码。
driver.get('https://www.bigbasket.com/ps/?q=rice')
last_height = driver.execute_script("return document.body.scrollHeight")
while True:
# 向下滚动到底部
driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
# 等待页面加载
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, '//div[@class="show-more"]/button')))
driver.find_element_by_xpath('//div[@class="show-more"]/button').click()
# 计算新的滚动高度并与上次滚动高度进行比较
new_height = driver.execute_script("return document.body.scrollHeight")
if new_height == last_height:
break
last_height = new_height
尽管不是所有元素都加载完毕，但最后的高度与新的高度相等。我需要滚动直到所有元素都加载完毕。
英文:
I want to scroll till all the element is loaded. My code works well for starting till 7 8 pages using this code.
driver.get('https://www.bigbasket.com/ps/?q=rice')
last_height = driver.execute_script("return document.body.scrollHeight")
while True:
# Scroll down to bottom
driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
# Wait to load page
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, '//div[@class="show-more"]/button')))
driver.find_element_by_xpath('//div[@class="show-more"]/button').click()
# Calculate new scroll height and compare with last scroll height
new_height = driver.execute_script("return document.body.scrollHeight")
if new_height == last_height:
break
last_height = new_height
The last height became equal to new height even though not all the element is loaded. I need to scroll till all element is loaded.
答案1
得分: 0
要[滚动](https://stackoverflow.com/a/54689339/7429447)到页面底部并不断点击<kbd>显示更多</kbd>按钮，您可以使用以下[_定位策略_](https://stackoverflow.com/questions/48054321/of-the-many-findelements-by-functions-in-selenium-when-would-you-use-one-over)：
- 代码块：
driver.get("https://www.bigbasket.com/ps/?q=rice")
while True:
try:
driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
print("浏览器滚动")
driver.execute_script("arguments[0].click();", WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, "//button[text()='显示更多']"))))
print("点击显示更多按钮")
time.sleep(3)
except WebDriverException:
print("不再滚动")
break
- 控制台输出：
浏览器滚动
点击显示更多按钮
浏览器滚动
点击显示更多按钮
浏览器滚动
点击显示更多按钮
...
...
请注意，上述内容中的链接将保持原样，不进行翻译。
英文:
To scroll down till the bottom of the page and keep on clicking the <kbd>Show More</kbd> button you can use the following locator strategy:
-
Code block:
driver.get("https://www.bigbasket.com/ps/?q=rice") while True: try: driver.execute_script("window.scrollTo(0, document.body.scrollHeight);") print("Browser scrolled") driver.execute_script("arguments[0].click();", WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, "//button[text()='Show More']")))) print("Show More button clicked") time.sleep(3) except WebDriverException: print("No more scrolling") break -
Console Output:
Browser scrolled Show More button clicked Browser scrolled Show More button clicked Browser scrolled Show More button clicked ... ...
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论