无法使用Selenium（Python）点击搜索按钮。

huangapple
go评论107阅读模式
英文:

Unable to click on the Search button using Selenium (Python)

问题

我正在尝试从以下网站中提取表格（使用Python + Selenium）：

https://www.ser-ag.com/en/resources/notifications-market-participants/management-transactions.html#/

在我成功输入表格中的日期参数后，我无法通过.click()点击“搜索”按钮。不知何故，Selenium找不到XPath。你能帮助我吗？

  1. # Auf Button suchen klicken
  2. # suchen = driver.find_element(By.XPATH, '//*[contains(@id="")]/div/div/div[1]/div/div[3]/div[1]/button[2]')
  3. # suchen.click()
  4. # time.sleep(2)
  6. button_xpath = "//*[@id='vwHpLbo9TZFKRdgg']/div/div/div[1]/div/div[3]/div[1]/button[2]"
  8. wait = WebDriverWait(driver, 10)  # 等待最多10秒
  9. button = wait.until(EC.presence_of_element_located((By.XPATH, button_xpath)))
  10. button.click()

请注意，由于文本中包含HTML和代码片段，我已将其保留在翻译中。如果您需要进一步的帮助，请告诉我。

英文:

I am trying to scrape the table of the following website (using Python + Selenium):

https://www.ser-ag.com/en/resources/notifications-market-participants/management-transactions.html#/

After I have successfully entered the date parameters in the table, I cannot click the "Search" button via .click(). Somehow Selenium does not find the Xpath. Can you help me?

  1. # Auf Button suchen klicken
  2. # suchen = driver.find_element(By.XPATH, '//*[contains(@id="")]/div/div/div[1]/div/div[3]/div[1]/button[2]')
  3. # suchen.click()
  4. # time.sleep(2)
  6. button_xpath = "//*[@id='vwHpLbo9TZFKRdgg']/div/div/div[1]/div/div[3]/div[1]/button[2]"
  8. wait = WebDriverWait(driver, 10)  # Wait up to 10 seconds
  9. button = wait.until(EC.presence_of_element_located((By.XPATH, button_xpath)))
  10. button.click()

答案1

得分: 0

尝试这个，可能的原因是按钮在驱动器屏幕之外，这将“滚动”到按钮并点击，（按钮代码与您的代码相同）

pd：请发布您的错误消息

  1. from selenium.webdriver.common.action_chains import ActionChains
  3. button = wait.until(EC.presence_of_element_located((By.XPATH, button_xpath)))
  4. ActionChains(self).move_to_element(button).click().perform()
英文:

Try this, probably the reason is button is outside the driver screen, this will "scroll" to button and click, (button code is same on your code)

pd: post your error message please

  1. from selenium.webdriver.common.action_chains import ActionChains
  3. button = wait.until(EC.presence_of_element_located((By.XPATH, button_xpath)))
  4. ActionChains(self).move_to_element(button).click().perform()

答案2

得分: 0

你首先必须点击<kbd>接受所有 cookies</kbd>按钮，然后尝试定位并点击<kbd>搜索</kbd>按钮。

看下面的工作代码：

  1. driver = webdriver.Chrome()
  2. driver.get('https://www.ser-ag.com/en/resources/notifications-market-participants/management-transactions.html#/')
  3. driver.maximize_window()
  4. button_xpath = "//button[text()='搜索']"
  5. wait = WebDriverWait(driver, 10)  # 等待最多10秒
  7. # 点击接受所有 cookies 按钮
  8. Accept_All_Btn = wait.until(EC.element_to_be_clickable((By.ID, "onetrust-accept-btn-handler")))
  9. driver.execute_script("arguments[0].click();", Accept_All_Btn)
  11. button = wait.until(EC.element_to_be_clickable((By.XPATH, button_xpath)))
  12. button.click()
英文:

You have to click on <kbd>Accept All cookies</kbd> button first, and then try to locate and click <kbd>Search</kbd> button.

See the below working code:

  1. driver = webdriver.Chrome()
  2. driver.get(&#39;https://www.ser-ag.com/en/resources/notifications-market-participants/management-transactions.html#/&#39;)
  3. driver.maximize_window()
  4. button_xpath = &quot;//button[text()=&#39;Search&#39;]&quot;
  5. wait = WebDriverWait(driver, 10)  # Wait up to 10 seconds
  7. # Click on Accept All cookies button
  8. Accept_All_Btn = wait.until(EC.element_to_be_clickable((By.ID, &quot;onetrust-accept-btn-handler&quot;)))
  9. driver.execute_script(&quot;arguments[0].click();&quot;, Accept_All_Btn)
  11. button = wait.until(EC.element_to_be_clickable((By.XPATH, button_xpath)))
  12. button.click()

答案3

得分: 0

以下是获取数据的一种方法（所有数据都包括在内）：

  1. import requests
  2. import pandas as pd
  4. headers= {
  5.     'User-Agent':'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36'
  6. }
  8. url = 'https://www.ser-ag.com/sheldon/management_transactions/v1/overview.json?pageSize=3000&pageNumber=0&sortAttribute=byDate&fromDate=20220710&toDate=20230710'
  9. r = requests.get(url, headers=headers)
  10. df = pd.json_normalize(r.json(), record_path=['itemList'])
  11. print(df)

终端中的结果：

  1.     buySellIndicator   correcteeId  correctorId               ISIN  notificationId  notificationSubmitter  notificationSubmitterId  obligorFunctionCode  obligorRelatedPartyInd securityDescription securityTypeCode  transactionAmountCHF transactionConditions transactionDate  transactionSize  transactionAmountPerSecurityCHF
  2. 0                  2  CH0019199550   T1N7700039    Alpine Select AG         ALPINE                     1                 L        7            981.0        20230707            90             10.9
  3. 1                  1  CH0019199550   T1N7700021    Alpine Select AG         ALPINE                     1                 L        7            2889.0        20230707           270             10.7
  4. 2                  1  CH0473243506   T1N7700013  ONE swiss bank SA          SBP                      1  Acquisition à la valeur nominale dans le cadre...        7           50000.0  Acquisition en raison de l'exercice de stock  20230707  50000  1.0
  5. 3                  1  CH0022427626   T1N7600015        LEM Holding SA                L        7          108495.0        20230706            50             2169.9
  6. 4                  1  CH0022427626   T1N7500017        LEM Holding SA                L        7          108660.0        20230705            50             2173.2
  7. ...
  8. 2050               1        T1M7B00046      CH0023405456   T1M7B00053  DUFRY AG  DUFRY  1  L        7  1980000.0  20220711  60000  33.0
  9. 2051               1        T1M7B00012      CH0001341608   T1M7B00061  Hypothekarbank Lenzburg AG  BKHYPOL  1        7  2639.0  20220711  1  2639.0
  10. 2052               1             CH0022427626  T1M7B00038  LEM Holding SA  LEM  2  L  7  172570.0  20220711  100  1725.7
  11. 2053               2             CH0005059438  T1M7B00020  nebag ag  NEBAG  1  7  32900.0  20220711  3500  9.4
  13. 2054 rows × 16 columns

数据从API端点的页面中提取：您需要爬取特定的端点。您可以通过检查页面上的脚本所发出的网络调用来找到它。

英文:

Here is one way of getting that data (all of it):

  1. import requests
  2. import pandas as pd
  4. headers= {
  5.     &#39;User-Agent&#39;:&#39;Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36&#39;
  6. }
  8. url = &#39;https://www.ser-ag.com/sheldon/management_transactions/v1/overview.json?pageSize=3000&amp;pageNumber=0&amp;sortAttribute=byDate&amp;fromDate=20220710&amp;toDate=20230710&#39;
  9. r = requests.get(url, headers=headers)
  10. df = pd.json_normalize(r.json(), record_path=[&#39;itemList&#39;])
  11. print(df)

Result in terminal:

  1.  	buySellIndicator 	correcteeId 	correctorId 	ISIN 	notificationId 	notificationSubmitter 	notificationSubmitterId 	obligorFunctionCode 	obligorRelatedPartyInd 	securityDescription 	securityTypeCode 	transactionAmountCHF 	transactionConditions 	transactionDate 	transactionSize 	transactionAmountPerSecurityCHF
  2. 0 	2 			CH0019199550 	T1N7700039 	Alpine Select AG 	ALPINE 	1 	L 		7 	981.0 		20230707 	90 	10.9
  3. 1 	1 			CH0019199550 	T1N7700021 	Alpine Select AG 	ALPINE 	1 	L 		7 	2889.0 		20230707 	270 	10.7
  4. 2 	1 			CH0473243506 	T1N7700013 	ONE swiss bank SA 	SBP 	1 		Acquisition a la valeur nominale dans le cadre... 	7 	50000.0 	Acquisition en raison de l exercice de stock\n... 	20230707 	50000 	1.0
  5. 3 	1 			CH0022427626 	T1N7600015 	LEM Holding SA 	LEM 	2 	L 		7 	108495.0 		20230706 	50 	2169.9
  6. 4 	1 			CH0022427626 	T1N7500017 	LEM Holding SA 	LEM 	2 	L 		7 	108660.0 		20230705 	50 	2173.2
  7. ... 	... 	... 	... 	... 	... 	... 	... 	... 	... 	... 	... 	... 	... 	... 	... 	...
  8. 2049 	1 			CH0019396990 	T1M7C00010 	Ypsomed Holding AG 	YPSOMED 	2 			7 	20100.0 		20220712 	150 	134.0
  9. 2050 	1 	T1M7B00046 		CH0023405456 	T1M7B00053 	DUFRY AG 	DUFRY 	1 	L 		7 	1980000.0 		20220711 	60000 	33.0
  10. 2051 	1 	T1M7B00012 		CH0001341608 	T1M7B00061 	Hypothekarbank Lenzburg AG 	BKHYPOL 	1 			7 	2639.0 		20220711 	1 	2639.0
  11. 2052 	1 			CH0022427626 	T1M7B00038 	LEM Holding SA 	LEM 	2 	L 		7 	172570.0 		20220711 	100 	1725.7
  12. 2053 	2 			CH0005059438 	T1M7B00020 	nebag ag 	NEBAG 	1 			7 	32900.0 		20220711 	3500 	9.4
  14. 2054 rows &#215; 16 columns

Data is being hydrated in page from an API endpoint: you need to scrape that particular endpoint. You can find it by inspecting the Network calls made by the scripts on that page.

答案4

得分: 0

以下是您要翻译的代码部分：

  1. import time
  2. from selenium import webdriver
  3. from selenium.webdriver import ChromeOptions
  4. from selenium.webdriver.common.by import By
  5. from selenium.webdriver.support import expected_conditions as EC
  6. from selenium.webdriver.support.wait import WebDriverWait
  8. options = ChromeOptions()
  10. options.add_argument("--start-maximized")
  11. options.add_experimental_option("excludeSwitches", ["enable-automation"])
  13. driver = webdriver.Chrome(options=options)
  14. wait = WebDriverWait(driver, 10)
  16. url = 'https://www.ser-ag.com/en/resources/notifications-market-participants/management-transactions.html#/'
  17. driver.get(url)
  19. # 等待并点击“接受所有Cookie”按钮
  20. wait.until(EC.presence_of_element_located((By.CSS_SELECTOR, 'button#onetrust-accept-btn-handler'))).click()
  21. # 等待并点击“搜索”按钮
  22. wait.until(EC.presence_of_all_elements_located((By.CSS_SELECTOR, 'button.Button')))[1].click()
  23. time.sleep(5)

注意事项：

  • 您可以使用简单的CSS选择器button.Button来定位<kbd>搜索</kbd>按钮。
  • 有两个使用此选择器获取的按钮。第一个是<kbd>清除筛选</kbd>按钮，第二个是<kbd>搜索</kbd>按钮。
英文:

Here's the way you can try:

  1. import time
  2. from selenium import webdriver
  3. from selenium.webdriver import ChromeOptions
  4. from selenium.webdriver.common.by import By
  5. from selenium.webdriver.support import expected_conditions as EC
  6. from selenium.webdriver.support.wait import WebDriverWait
  8. options = ChromeOptions()
  10. options.add_argument(&quot;--start-maximized&quot;)
  11. options.add_experimental_option(&quot;excludeSwitches&quot;, [&quot;enable-automation&quot;])
  13. driver = webdriver.Chrome(options=options)
  14. wait = WebDriverWait(driver, 10)
  16. url = &#39;https://www.ser-ag.com/en/resources/notifications-market-participants/management-transactions.html#/&#39;
  17. driver.get(url)
  19. # wait and click on the &quot;Accept all Cookies&quot; button
  20. wait.until(EC.presence_of_element_located((By.CSS_SELECTOR, &#39;button#onetrust-accept-btn-handler&#39;))).click()
  21. # wait and click on the &quot;Search&quot; button
  22. wait.until(EC.presence_of_all_elements_located((By.CSS_SELECTOR, &#39;button.Button&#39;)))[1].click()
  23. time.sleep(5)

Things to notice:

  • You can locate the <kbd>Search</kbd> button with a simple CSS Selector button.Button
  • there are two buttons that you get using the selector. The first is the <kbd>Clear Filter</kbd> button and 2nd one is the <kbd>Search</kbd> button.

huangapple
  • 本文由 发表于 2023年7月10日 16:54:41
  • 转载请务必保留本文链接：https://go.coder-hub.com/76652186.html
  • python
  • selenium-webdriver
  • web-scraping
  • xpath
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定