英文:
How to close clickable popup to continue scraping through Selenium in python
问题
我正在尝试使用`Selenium`在`python`中从网站上的表格中的可点击弹出窗口中提取信息到`pandas`数据框中,如果弹出窗口包含信息,似乎可以做到这一点。
```python
from selenium import webdriver
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.select import Select
import pandas as pd
import time
driver = webdriver.Chrome()
driver.get('https://mspotrace.org.my/Sccs_list')
time.sleep(20)
# 选择最大条目数
elem = driver.find_element_by_css_selector('select[name=dTable_length]')
select = Select(elem)
select.select_by_value('500')
time.sleep(15)
# 获取元素列表
elements = WebDriverWait(driver, 20).until(EC.presence_of_all_elements_located((By.XPATH, "//a[@title='View on Map']")))
# 循环遍历元素弹出窗口并将设施详情提取到DF
pos = 0
df = pd.DataFrame(columns=['facility_name', 'other_details'])
try:
for element in elements:
data = []
element.click()
time.sleep(3)
facility_name = driver.find_element_by_xpath('//h4[@class="modal-title"]').text
other_details = driver.find_element_by_xpath('//div[@class="modal-body"]').text
data.append(facility_name)
data.append(other_details)
df.loc[pos] = data
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button[aria-label='Close'] > span"))).click() # 关闭弹出窗口
time.sleep(10)
pos += 1
except:
print("没有地理位置信息")
pass
print(df)
然而,有些情况下会出现如下图所示的窗口,我需要点击'OK'以恢复抓取网页上的其他行,但我似乎无法找到要点击的元素来执行此操作。
[![enter image description here][1]][1]
<details>
<summary>英文:</summary>
I'm trying to scrape some information from clickable popups in a table on a website into a `pandas` dataframe using `Selenium` in `python` and it seems to be able to do this if the popups have information.
from selenium import webdriver
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.select import Select
import pandas as pd
import time
driver = webdriver.Chrome()
driver.get('https://mspotrace.org.my/Sccs_list')
time.sleep(20)
# Select maximum number of entries
elem = driver.find_element_by_css_selector('select[name=dTable_length]')
select = Select(elem)
select.select_by_value('500')
time.sleep(15)
# Get list of elements
elements = WebDriverWait(driver, 20).until(EC.presence_of_all_elements_located((By.XPATH, "//a[@title='View on Map']")))
# Loop through element popups and pull details of facilities into DF
pos = 0
df = pd.DataFrame(columns=['facility_name','other_details'])
try:
for element in elements:
data = []
element.click()
time.sleep(3)
facility_name = driver.find_element_by_xpath('//h4[@class="modal-title"]').text
other_details = driver.find_element_by_xpath('//div[@class="modal-body"]').text
data.append(facility_name)
data.append(other_details)
df.loc[pos] = data
WebDriverWait(driver,20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button[aria-label='Close'] > span"))).click() # close popup window
time.sleep(10)
pos+=1
except:
print("No geo location information")
pass
print(df)
However, there are cases when a window like below appears and I need to click 'OK' on this to resume scraping the other rows on the web page but I can't seem to be able to find the element to click on to do this.
[![enter image description here][1]][1]
[1]: https://i.stack.imgur.com/NdPqk.png
</details>
# 答案1
**得分**: 0
Selenium驱动程序提供了切换到警报上下文并与之交互的方法:
driver.switch_to().alert()
之后,根据警报类型,您可以执行所需的操作。要模拟单击“确定”:
driver.switch_to().alert().accept()
更多信息请参阅[此处](https://www.browserstack.com/guide/alerts-and-popups-in-selenium)。
<details>
<summary>英文:</summary>
Selenium driver provides methods to switch to alerts context and working with it:
driver.switch_to().alert()
After that, you can do whatever you want, depending on alert type. To simulate clicking on “OK”:
driver.switch_to().alert().accept()
More info [here](https://www.browserstack.com/guide/alerts-and-popups-in-selenium)
</details>
# 答案2
**得分**: 0
可以尝试使用 Python:
driver.switch_to.alert.accept()
但是,您的测试场景应该明确并且应该知道弹出窗口出现的位置。如果您不清楚,或者是“真正”随机的话,您可以检查一些在每个测试步骤中运行的钩子。
<details>
<summary>英文:</summary>
can you try for Pythonenter code here:
driver.switch_to.alert.accept()
But, your test scenario should be clear and should know where this pop up appears. If you don't know and "really" random, you can check some hooks that running for each test step
</details>
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论