使用Selenium Python进行网页抓取选择下拉选项。

huangapple go评论71阅读模式
英文:

Choose a drop down option using Selenium Python Webscraping

问题

我正在尝试使用Selenium和Python进行网页抓取,以根据我的当地公共交通时间表选择下拉路线选项。以下是网站链接:https://tmweb.pacebus.com/TMWebWatch/LiveDepartureTimes

我遇到的问题是这行代码:

find_route_takin = driver.find_element(By.CLASS_NAME,"active-result") 

产生了一个错误,错误消息如下:

Message: no such element: Unable to locate element: {"method":"css selector","selector":".active-result"}

以下是我尝试完成任务的代码:

import time
from bs4 import BeautifulSoup
from datetime import datetime
from selenium import webdriver
from selenium.webdriver.common.by import By
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.support.ui import Select

driver = webdriver.Chrome(ChromeDriverManager().install())
driver.maximize_window()
driver.get("https://tmweb.pacebus.com/TMWebWatch/LiveDepartureTimes")

driver.find_element(By.ID, "MainContent_routeList_chosen")

find_route_takin = driver.find_element(By.CLASS_NAME, "active-result")
select_route = Select(find_route_takin)
select_route.select_by_visible_text("565 - Grand Avenue")
time.sleep(20)

在网站的HTML中,第一个路线下拉菜单有一个包含许多li元素的ul元素,每个li元素代表一个具体的路线。我尝试查看了各种YouTube视频,但找不到解决我的问题的方法。我已经阅读了可能需要使用xpath而不是CLASS_NAME的建议,但是当我尝试右键单击并选择复制xpath时,仍然出现相同的错误消息。我希望运行程序时,会打开一个Chrome窗口并选择565号线 - Grand Avenue。我遇到的错误是:

Message: no such element: Unable to locate element: {"method":"css selector","selector":".active-result"}

英文:

I am trying to use selenium and python webscraping to choose a drop down option of routes based on my local public transit schedule. Here is the link to the site: https://tmweb.pacebus.com/TMWebWatch/LiveDepartureTimes

The issue I'm running into is that the line:

find_route_takin = driver.find_element(By.CLASS_NAME,"active-result") 

produces an error saying:

Message: no such element: Unable to locate element: {"method":"css selector","selector":".active-result"}

Here is the code for how I am trying to complete the task:

import time

from bs4 import BeautifulSoup
from datetime import datetime
from selenium import webdriver
from selenium.webdriver.common.by import By
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.support.ui import Select

driver = webdriver.Chrome(ChromeDriverManager().install())
driver.maximize_window()
driver.get("https://tmweb.pacebus.com/TMWebWatch/LiveDepartureTimes")

driver.find_element(By.ID, "MainContent_routeList_chosen")

find_route_takin = driver.find_element(By.CLASS_NAME,"active-result")
select_route = Select(find_route_takin)
select_route.select_by_visible_text("565 - Grand Avenue")
time.sleep(20)

Within the HTML of the site for the first drop-down for the routes, there is a ul object that holds many li objects that represent each individual route. I have tried looking through various YouTube videos, but I can't find a solution to my problem. I've read that perhaps I'll have to use xpath instead of CLASS_NAME, but when I tried copying the xpath by right-clicking and choosing copy xpath, I'm still prompted with the same error message. What I want is when I run the program, a chrome window is brought up and the 565 - Grand Avenue route is selected. The error I'm running into is

Message: no such element: Unable to locate element: {"method":"css selector","selector":".active-result"}

答案1

得分: 0

你可以尝试这种方式:

import time
from selenium.webdriver import Chrome, ChromeOptions
from selenium.webdriver.common.by import By
from selenium.webdriver.support.wait import WebDriverWait
import selenium.webdriver.support.expected_conditions as EC

options = ChromeOptions()
options.add_argument("--start-maximized")

driver = Chrome(options=options)

def choose_route(opt):

    driver.get("https://tmweb.pacebus.com/TMWebWatch/LiveDepartureTimes")
    WebDriverWait(driver, 10).until(EC.visibility_of_element_located((By.ID, 'MainContent_routeList_chosen'))).click()

    routes = driver.find_element(By.CSS_SELECTOR, 'ul.chosen-results')
    route_options = routes.find_elements(By.TAG_NAME, 'li')

    for option in route_options:
        if option.text == opt:
            option.click()

    time.sleep(10)

choose_route("565 - Grand Avenue")
# choose_route("213 - Green Bay Road")

需要注意的事项:

  1. 首先,我们等待Route下拉菜单在页面上可见/加载,以便点击它。
  2. 当我们点击后,下拉菜单展开,所有选项都加载到页面上。因此,我们找到所有的路线选项,这些选项都是位于带有类名chosen-resultsul标签下的li标签。
  3. 最后,根据文本选择选项并点击。

这就是全部,注意我们不能使用Select,因为下拉选项不在select标签下。

英文:

You may try this way:

import time
from selenium.webdriver import Chrome, ChromeOptions
from selenium.webdriver.common.by import By
from selenium.webdriver.support.wait import WebDriverWait
import selenium.webdriver.support.expected_conditions as EC

options = ChromeOptions()
options.add_argument("--start-maximized")

driver = Chrome(options=options)

def choose_route(opt):

    driver.get("https://tmweb.pacebus.com/TMWebWatch/LiveDepartureTimes")
    WebDriverWait(driver, 10).until(EC.visibility_of_element_located((By.ID, 'MainContent_routeList_chosen'))).click()

    routes = driver.find_element(By.CSS_SELECTOR, 'ul.chosen-results')
    route_options = routes.find_elements(By.TAG_NAME, 'li')

    for option in route_options:
        if option.text == opt:
            option.click()

    time.sleep(10)

choose_route("565 - Grand Avenue")
# choose_route("213 - Green Bay Road")

Things to note:

  1. first we wait for the Route drop-down to get visibly located/loaded on the page in order to click it.
  2. As we click, the drop-down gets open, and all the options get loaded on the page. So we find all the route options which are all the li tags under the ul tag with class name chosen-results
  3. And finally choose the option based on the text and click.

That's all, and notice that we can not use Select because the drop-down options are not under a select tag.

huangapple
  • 本文由 发表于 2023年5月25日 04:31:08
  • 转载请务必保留本文链接:https://go.coder-hub.com/76327210.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定