无法在Python的Selenium中找到嵌套的阴影DOM元素。

huangapple
go评论116阅读模式
英文:

Unable to find element in nested shodow root in Selenium in python

问题

I have read similar answers but it doesnt seem to be working for me . I am trying to scrape product names from a website

  1. driver = webdriver.Chrome(DRIVER_PATH,options=options)
  2. website = 'https://bodyandmind.com/california/longbeachdispensary/longbeachmenu/?category=flower#inventory'
  3. driver.get(website)
  4. time.sleep(3)
  5. confirmation = driver.find_element(By.CSS_SELECTOR,'button.age-gate-submit.age-gate-submit-yes').click()
  6. time.sleep(3)
  8. root1=driver.find_element(By.CSS_SELECTOR, "weave-ordering").shadow_root
  10. root2=root1.find_element(By.CSS_SELECTOR, "weave-app").shadow_root
  12. root3=root2.find_element(By.CSS_SELECTOR, "weave-inventory").shadow_root
  14. root4=root3.find_element(By.CSS_SELECTOR, "weave-product-list").shadow_root
  16. root5=root4.find_element(By.CSS_SELECTOR, "weave-category-section").shadow_root
  18. root6=root5.find_element(By.CSS_SELECTOR, "weave-product").shadow_root
  20. element=root6.find_element(By.CSS_SELECTOR, "[class='product-container']")
  22. print(element.text)

Running it is giving me the following error :

  1. Traceback (most recent call last):
  2.   File "/Users/japneeshsingh/Desktop/try.py", line 29, in <module>
  3.     root1=driver.find_element(By.CSS_SELECTOR, "weave-ordering").shadow_root
  4.   File "/Users/japneeshsingh/opt/anaconda3/lib/python3.8/site-packages/selenium/webdriver/remote/webelement.py", line 252, in shadow_root
  5.     return self._execute(Command.GET_SHADOW_ROOT)["value"]
  6.   File "/Users/japneeshsingh/opt/anaconda3/lib/python3.8/site-packages/selenium/webdriver/remote/webelement.py", line 403, in _execute
  7.     return self._parent.execute(command, params)
  8.   File "/Users/japneeshsingh/opt/anaconda3/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 440, in execute
  9.     self.error_handler.check_response(response)
  10.   File "/Users/japneeshsingh/opt/anaconda3/lib/python3.8/site-packages/selenium/webdriver/remote/errorhandler.py", line 245, in check_response
  11.     raise exception_class(message, screen, stacktrace)
  12. selenium.common.exceptions.NoSuchShadowRootException: Message: no such shadow root
英文:

I have read similar answers but it doesnt seem to be working for me . I am trying to scrape product names from a website

  1. driver = webdriver.Chrome(DRIVER_PATH,options=options)
  2. website = &#39;https://bodyandmind.com/california/longbeachdispensary/longbeachmenu/?category=flower#inventory&#39;
  3. driver.get(website)
  4. time.sleep(3)
  5. confirmation = driver.find_element(By.CSS_SELECTOR,&#39;button.age-gate-submit.age-gate-submit-yes&#39;).click()
  6. time.sleep(3)
  8. root1=driver.find_element(By.CSS_SELECTOR, &quot;weave-ordering&quot;).shadow_root
  10. root2=root1.find_element(By.CSS_SELECTOR, &quot;weave-app&quot;).shadow_root
  12. root3=root2.find_element(By.CSS_SELECTOR, &quot;weave-inventory&quot;).shadow_root
  14. root4=root3.find_element(By.CSS_SELECTOR, &quot;weave-product-list&quot;).shadow_root
  16. root5=root4.find_element(By.CSS_SELECTOR, &quot;weave-category-section&quot;).shadow_root
  18. root6=root5.find_element(By.CSS_SELECTOR, &quot;weave-product&quot;).shadow_root
  20. element=root6.find_element(By.CSS_SELECTOR, &quot;[class=&#39;product-container&#39;]&quot;)
  22. print(element.text)

Running it is giving me the following error :

  1. Traceback (most recent call last):
  2.   File &quot;/Users/japneeshsingh/Desktop/try.py&quot;, line 29, in &lt;module&gt;
  3.     root1=driver.find_element(By.CSS_SELECTOR, &quot;weave-ordering&quot;).shadow_root
  4.   File &quot;/Users/japneeshsingh/opt/anaconda3/lib/python3.8/site-packages/selenium/webdriver/remote/webelement.py&quot;, line 252, in shadow_root
  5.     return self._execute(Command.GET_SHADOW_ROOT)[&quot;value&quot;]
  6.   File &quot;/Users/japneeshsingh/opt/anaconda3/lib/python3.8/site-packages/selenium/webdriver/remote/webelement.py&quot;, line 403, in _execute
  7.     return self._parent.execute(command, params)
  8.   File &quot;/Users/japneeshsingh/opt/anaconda3/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py&quot;, line 440, in execute
  9.     self.error_handler.check_response(response)
  10.   File &quot;/Users/japneeshsingh/opt/anaconda3/lib/python3.8/site-packages/selenium/webdriver/remote/errorhandler.py&quot;, line 245, in check_response
  11.     raise exception_class(message, screen, stacktrace)
  12. selenium.common.exceptions.NoSuchShadowRootException: Message: no such shadow root

答案1

得分: 1

以下是翻译好的部分：

解决方案

要提取产品名称，您需要使用 querySelectorAll() 方法，并且您可以使用以下 定位策略：

  1. driver.get("https://bodyandmind.com/california/longbeachdispensary/longbeachmenu/?category=flower#inventory")
  2. WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button.age-gate-submit.age-gate-submit-yes"))).click()
  3. time.sleep(15)
  4. elements = driver.execute_script("return document.querySelector('weave-ordering').shadowRoot.querySelector('weave-app').shadowRoot.querySelector('weave-inventory').shadowRoot.querySelector('weave-product-list').shadowRoot.querySelector('weave-category-section').shadowRoot.querySelectorAll('weave-product')")
  5. for element in elements:
  6.     ele = element.shadow_root
  7.     e = ele.find_element(By.CSS_SELECTOR, "h2")
  8.     print(e.text)

控制台输出：

  1. 805 Glue
  2. 805 Glue
  3. 805 Glue
  4. 805 Glue Sugar Shake
  5. Ajo Blanco
  6. Alien Cookies
  7. Animal Face
  8. Artemis
  9. BK Satellite
  10. Banana Sherbet
  11. Banana Sherbet
  12. Biskante
  13. Black Diamond Smalls
  14. Black Diamond X
  15. Black Triangle OG
  16. Black Triangle OG
  17. Blue Cookiez
  18. Blue Dream
  19. Blue Dream
  20. Blue Dream
  21. Blue Dream Sugar Shake
  22. Blue Lotus
  23. Blueberry Mac
  24. Blueberry Skittles
  25. Blueberry Skittles
  26. Cadi...
  27. （更多内容，请参考原文）

参考资料

您可以在以下链接中找到一些相关的详细讨论：

英文:

The product names within multiple #shadow-root (open)

无法在Python的Selenium中找到嵌套的阴影DOM元素。

Solution

To extract the product names you need to use querySelectorAll() and you can use the following locator strategies:

  1. driver.get(&quot;https://bodyandmind.com/california/longbeachdispensary/longbeachmenu/?category=flower#inventory&quot;)
  2. WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, &quot;button.age-gate-submit.age-gate-submit-yes&quot;))).click()
  3. time.sleep(15)
  4. elements = driver.execute_script(&quot;return document.querySelector(&#39;weave-ordering&#39;).shadowRoot.querySelector(&#39;weave-app&#39;).shadowRoot.querySelector(&#39;weave-inventory&#39;).shadowRoot.querySelector(&#39;weave-product-list&#39;).shadowRoot.querySelector(&#39;weave-category-section&#39;).shadowRoot.querySelectorAll(&#39;weave-product&#39;)&quot;)
  5. for element in elements:
  6. 	ele = element.shadow_root
  7. 	e = ele.find_element(By.CSS_SELECTOR, &quot;h2&quot;)
  8. 	print(e.text)

Console Output:

  1. 805 Glue
  2. 805 Glue
  3. 805 Glue
  4. 805 Glue Sugar Shake
  5. Ajo Blanco
  6. Alien Cookies
  7. Animal Face
  8. Artemis
  9. BK Satellite
  10. Banana Sherbet
  11. Banana Sherbet
  12. Biskante
  13. Black Diamond Smalls
  14. Black Diamond X
  15. Black Triangle OG
  16. Black Triangle OG
  17. Blue Cookiez
  18. Blue Dream
  19. Blue Dream
  20. Blue Dream
  21. Blue Dream Sugar Shake
  22. Blue Lotus
  23. Blueberry Mac
  24. Blueberry Skittles
  25. Blueberry Skittles
  26. Cadillac Rainbowz
  27. Cali Berry
  28. Cereal Milk Sugar Shake
  29. Chauffeur
  30. Cheetah Piss
  31. Chem Dawg
  32. Chem Dawg
  33. Cherry Cheesecake
  34. Cherry Gas
  35. Cherry Gorilla
  36. Cherry Popperz
  37. Cherry Sababa
  38. Cherry Zest #4
  39. Cocolato
  40. Crop Duster
  41. DJ Short Blueberry
  42. Diesel Tov
  43. Double Diesel
  44. Drip
  45. Dutch Treat
  46. Electric Blue
  47. Fantasma
  48. French Laundry
  49. Fritter Glitter
  50. Fruit Bubblegum
  51. GMO Cookies Smalls
  52. GMO-Sherb
  53. Garlic Starship
  54. Gelato
  55. Gelato 41
  56. Gelato Quin
  57. Gelatti
  58. Ghost OG Smalls
  59. Goji Berry Runtz
  60. Grape Ambrosia
  61. Grape Cake
  62. Grape Octane
  63. Grapes Of Wrath
  64. Gravitron
  65. Grease Bucket #9
  66. Grease Monkey
  67. Gruntz
  68. Hella Jelly
  69. Highrise
  70. Holy Moly!
  71. Honey Lemon Hibiscus
  72. Infused Baby Jeeter Grapefruit Romulan
  73. Jelly Fish CBD
  74. Kiwi Sorbet
  75. Kosher Kush
  76. Krypto Chronic
  77. Kush Cake x Hybrid Kief Power Pack
  78. Kush Mints
  79. Kush Mints
  80. Kush Mints
  81. Kush Mints Sugar Shake
  82. L&#39;Chaim Lemon
  83. Lava Cake Smalls
  84. Lemon Cherry Gelato
  85. Lemon Granita
  86. Lemon z
  87. Lilac Diesel
  88. Lilac Diesel
  89. MAC 1
  90. MVP Cookies
  91. Mac 1
  92. Mac 1
  93. Mac1
  94. Mac1 x Hybrid Kief Power Pack
  95. Mafia Funeral
  96. Malibu Pure Kush
  97. Mango Haze
  98. Mango Haze
  99. Mimosa
  100. Modified Grapes
  101. Moonana Wreck
  102. Motor Breath Smalls
  103. Mule Fuel
  104. NF1
  105. ORO

References

You can find a couple of relevant detailed discussions in:

答案2

得分: 1

你的问题出在weave-product这行。如果你移除那行并调整你的find_element()调用，那么它就会工作（假设你在搜索之前等待页面完全加载）：

  1. root1=driver.find_element(By.CSS_SELECTOR, &quot;weave-ordering&quot;).shadow_root
  2. root2=root1.find_element(By.CSS_SELECTOR, &quot;weave-app&quot;).shadow_root
  3. root3=root2.find_element(By.CSS_SELECTOR, &quot;weave-inventory&quot;).shadow_root
  4. root4=root3.find_element(By.CSS_SELECTOR, &quot;weave-product-list&quot;).shadow_root
  5. root5=root4.find_element(By.CSS_SELECTOR, &quot;weave-category-section&quot;).shadow_root
  6. element=root5.find_element(By.CSS_SELECTOR, &quot;[class=&#39;product-container&#39;]&quot;)

要继续获取各个产品的名称，请添加以下内容：

  1. all_products = element.find_elements(By.CSS_SELECTOR, &quot;weave-product&quot;)
  2. for product in all_products:
  3.     print(product.text.split(&quot;\n&quot;)[0])

这将输出：

  1. Pacific Stone
  2. Pacific Stone
  3. Pacific Stone
  4. Maven Genetics
  5. Mazel Tov
  6. Fig Farms
  7. Cream of The Crop
  8. Alien Labs
  9. Claybourne Co.
  10. Claybourne Co.
  11. Alien Labs
  12. West Coast Trading Company
  13. Maven Genetics
  14. ...
英文:

Your issue is with the weave-product line. If you remove that line and shift your find_element() calls, then it works (assuming you wait for the page to fully load before searching):

  1. root1=driver.find_element(By.CSS_SELECTOR, &quot;weave-ordering&quot;).shadow_root
  2. root2=root1.find_element(By.CSS_SELECTOR, &quot;weave-app&quot;).shadow_root
  3. root3=root2.find_element(By.CSS_SELECTOR, &quot;weave-inventory&quot;).shadow_root
  4. root4=root3.find_element(By.CSS_SELECTOR, &quot;weave-product-list&quot;).shadow_root
  5. root5=root4.find_element(By.CSS_SELECTOR, &quot;weave-category-section&quot;).shadow_root
  6. element=root5.find_element(By.CSS_SELECTOR, &quot;[class=&#39;product-container&#39;]&quot;)

To continue and get individual product names, add the following:

  1. all_products = element.find_elements(By.CSS_SELECTOR, &quot;weave-product&quot;)
  2. for product in all_products:
  3.     print(product.text.split(&quot;\n&quot;)[0])

That outputs:

  1. Pacific Stone
  2. Pacific Stone
  3. Pacific Stone
  4. Maven Genetics
  5. Mazel Tov
  6. Fig Farms
  7. Cream of The Crop
  8. Alien Labs
  9. Claybourne Co.
  10. Claybourne Co.
  11. Alien Labs
  12. West Coast Trading Company
  13. Maven Genetics
  14. ...

huangapple
  • 本文由 发表于 2023年6月29日 06:32:50
  • 转载请务必保留本文链接：https://go.coder-hub.com/76577092.html
  • python
  • selenium-chromedriver
  • selenium-webdriver
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定