如何通过Python中的Selenium关闭可点击的弹出窗口以继续进行网页数据抓取

huangapple go评论67阅读模式
英文:

How to close clickable popup to continue scraping through Selenium in python

问题

我正在尝试使用`Selenium``python`中从网站上的表格中的可点击弹出窗口中提取信息到`pandas`数据框中如果弹出窗口包含信息似乎可以做到这一点

```python
from selenium import webdriver
from selenium.webdriver.support.wait import WebDriverWait    
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.select import Select
import pandas as pd
import time

driver = webdriver.Chrome()
driver.get('https://mspotrace.org.my/Sccs_list')
time.sleep(20)

# 选择最大条目数
elem = driver.find_element_by_css_selector('select[name=dTable_length]')
select = Select(elem)
select.select_by_value('500')
time.sleep(15)

# 获取元素列表
elements = WebDriverWait(driver, 20).until(EC.presence_of_all_elements_located((By.XPATH, "//a[@title='View on Map']")))

# 循环遍历元素弹出窗口并将设施详情提取到DF
pos = 0
df = pd.DataFrame(columns=['facility_name', 'other_details'])

try:
    for element in elements:
        data = []
        element.click()
        time.sleep(3)
        facility_name = driver.find_element_by_xpath('//h4[@class="modal-title"]').text
        other_details = driver.find_element_by_xpath('//div[@class="modal-body"]').text
        data.append(facility_name)
        data.append(other_details)
        df.loc[pos] = data
        WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button[aria-label='Close'] > span"))).click()  # 关闭弹出窗口
        time.sleep(10)
        pos += 1
except:
    print("没有地理位置信息")
    pass

print(df)

然而,有些情况下会出现如下图所示的窗口,我需要点击'OK'以恢复抓取网页上的其他行,但我似乎无法找到要点击的元素来执行此操作。

[![enter image description here][1]][1]



<details>
<summary>英文:</summary>

I&#39;m trying to scrape some information from clickable popups in a table on a website into a `pandas` dataframe using `Selenium` in `python` and it seems to be able to do this if the popups have information. 

    from selenium import webdriver
    from selenium.webdriver.support.wait import WebDriverWait    
    from selenium.webdriver.common.by import By
    from selenium.webdriver.support import expected_conditions as EC
    from selenium.webdriver.support.select import Select
    import pandas as pd
    import time
    
    driver = webdriver.Chrome()
    driver.get(&#39;https://mspotrace.org.my/Sccs_list&#39;)
    time.sleep(20)
    
    # Select maximum number of entries
    elem = driver.find_element_by_css_selector(&#39;select[name=dTable_length]&#39;)
    select = Select(elem)
    select.select_by_value(&#39;500&#39;)
    time.sleep(15)
    
    # Get list of elements
    elements = WebDriverWait(driver, 20).until(EC.presence_of_all_elements_located((By.XPATH, &quot;//a[@title=&#39;View on Map&#39;]&quot;)))
    
    # Loop through element popups and pull details of facilities into DF
    pos = 0
    df = pd.DataFrame(columns=[&#39;facility_name&#39;,&#39;other_details&#39;])
    
    try:
        for element in elements:
            data = []
            element.click()
            time.sleep(3)
            facility_name = driver.find_element_by_xpath(&#39;//h4[@class=&quot;modal-title&quot;]&#39;).text
            other_details = driver.find_element_by_xpath(&#39;//div[@class=&quot;modal-body&quot;]&#39;).text
            data.append(facility_name)
            data.append(other_details)
            df.loc[pos] = data
            WebDriverWait(driver,20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, &quot;button[aria-label=&#39;Close&#39;] &gt; span&quot;))).click() # close popup window
            time.sleep(10)
            pos+=1
    except:
        print(&quot;No geo location information&quot;)
        pass

    print(df)

However, there are cases when a window like below appears and I need to click &#39;OK&#39; on this to resume scraping the other rows on the web page but I can&#39;t seem to be able to find the element to click on to do this.  

[![enter image description here][1]][1]


  [1]: https://i.stack.imgur.com/NdPqk.png

</details>


# 答案1
**得分**: 0

Selenium驱动程序提供了切换到警报上下文并与之交互的方法:

    driver.switch_to().alert()

之后,根据警报类型,您可以执行所需的操作。要模拟单击“确定”:

    driver.switch_to().alert().accept()

更多信息请参阅[此处](https://www.browserstack.com/guide/alerts-and-popups-in-selenium)。

<details>
<summary>英文:</summary>

Selenium driver provides methods to switch to alerts context and working with it:

    driver.switch_to().alert()

After that, you can do whatever you want, depending on alert type. To simulate clicking on “OK”: 

    driver.switch_to().alert().accept()

More info [here](https://www.browserstack.com/guide/alerts-and-popups-in-selenium)

</details>



# 答案2
**得分**: 0

可以尝试使用 Python:

    driver.switch_to.alert.accept()

但是,您的测试场景应该明确并且应该知道弹出窗口出现的位置。如果您不清楚,或者是“真正”随机的话,您可以检查一些在每个测试步骤中运行的钩子。

<details>
<summary>英文:</summary>

can you try for Pythonenter code here:

    driver.switch_to.alert.accept()

But, your test scenario should be clear and should know where this pop up appears. If you don&#39;t know and &quot;really&quot; random, you can check some hooks that running for each test step

</details>



huangapple
  • 本文由 发表于 2023年2月16日 09:01:48
  • 转载请务必保留本文链接:https://go.coder-hub.com/75466856.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定