Unable to find element that looks like it's dynamically generated

I'm trying to find the email address text located in the below URL. You can clearly see the email address but I believe the text is generated dynamically through Javascript/React. When I copy the XPATH or CSS Selector and try to find the element like I would any other element I just get an error saying the element cannot be found.

I've tried time.sleep(30) to give the page time to fully load but that's not the issue.

I've tried:

driver.find_element(By.XPATH, '//*[@id="mount_0_0_D8"]/div/div[1]/div/div[5]/div/div/div[3]/div/div/div[1]/div[1]/div/div/div[4]/div[2]/div/div[1]/div[2]/div/div[1]/div/div/div/div/div[2]/div[2]/div/ul/div[2]/div[2]/div/div/span')

You can see from the snippet below that the email is visible but is between some ::before and ::after text I've not seen before.

https://www.facebook.com/homieestudio

Any ideas on how to consistently pull back the email address here? I'm using Chromedriver.

The clue is, the Email Address will always have the @ sign.

Solution

To extract the text Info@homieestudio.co.uk ideally you need to induce WebDriverWait for the visibility_of_element_located() and you can use either of the following locator strategies:

• Using XPATH and text attribute:

  driver.get('https://www.facebook.com/homieestudio')
  print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//span[contains(., '@')]"))).text)
  

• Using XPATH and get_attribute("textContent"):

  driver.get('https://www.facebook.com/homieestudio')
  print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//span[contains(., '@')]"))).get_attribute("textContent"))
  

• Console output:

  Info@homieestudio.co.uk
  

• Note : You have to add the following imports :

  from selenium.webdriver.support.ui import WebDriverWait
  from selenium.webdriver.common.by import By
  from selenium.webdriver.support import expected_conditions as EC
  

>You can find a relevant discussion in How to retrieve the text of a WebElement using Selenium - Python

You can find the email address using the below

wait = WebDriverWait(driver, 20)
element = wait.until(EC.visibility_of_element_located((By.XPATH, "//span[@dir='auto'][contains(text(), 'info@homieestudio.co.uk')]")))

OR

wait = WebDriverWait(driver, 20)
element = wait.until(EC.presence_of_element_located((By.XPATH, "//span[@dir='auto'][contains(text(), 'info@homieestudio.co.uk')]")))

IMPORT

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC