How to correctly get the links from a webpage using Selenium and BeautifulSoup?

Sorry if this isn't phrased correctly, but basically I'm trying to find the recipe links from https://www.foodnetwork.com/recipes and scrape the title, ingredients and instructions from each recipe page, then write it into an output file called "recipe_output.txt". Here is how I've attempted it:

Set up the webdriver:

options = webdriver.ChromeOptions()
options.add_argument("--headless")  # Run the browser in headless mode (without a GUI)
driver = webdriver.Chrome(options=options)

Define the URL of the recipe page:

listing_url = "https://www.foodnetwork.com/recipes"

Load the page with the webdriver:

driver.get(listing_url)

Get the page's content:

html_content = driver.execute_script("return document.documentElement.outerHTML")

Parse the HTML content with BeautifulSoup:

soup = BeautifulSoup(html_content, "html.parser")

Here is where the issue is, when I try to find the recipe links:

recipe_links = soup.find_all("a", class_="m-MediaBlock__a-HeadlineText")
print("Found {} recipe links".format(len(recipe_links)))

Every time I try to execute, I get the output "Found 0 recipe links" and I am unsure why.

Here are how the elements for the links to each recipe on the recipes page are formatted:

<h3 class="m-MediaBlock__a-Headline">
    <a href="//www.foodnetwork.com/recipes/food-network-kitchen/salad-stuffed-peppers-9970168">
      <span class="m-MediaBlock__a-HeadlineText">Salad-Stuffed Peppers</span>
      
    </a>
  </h3>

I thought I would find all recipes with the "a" tag and the "m-MediaBlock__a-HeadlineText" class in order to get the links, but that is not right.

I've tried other things like using the "a" tag with the "m-MediaBlock__a-Headline" class or the "span" tag with the "m-MediaBlock__a-HeadlineText" class, but none of that works either. Please, if anyone could help me figure out what I'm doing wrong I would very much appreciate it.

Here is the rest of the code for reference:

with open("recipe_output.txt", mode="w", encoding="utf-8") as file:
    for link in recipe_links:
        recipe_url = "https:" + link["href"]
        driver.get(recipe_url)
        recipe_html_content = driver.execute_script("return document.documentElement.outerHTML")
        recipe_soup = BeautifulSoup(recipe_html_content, "html.parser")
        
        title_element = recipe_soup.find("span", class_="o-AssetTitle__a-HeadlineText")
        if title_element is not None:
            recipe_title = title_element.text.strip()
            file.write("Title: " + recipe_title)
            
        ingredient_elements = recipe_soup.find_all("p", class_="o-Ingredients__a-Ingredient")
        ingredients = []
        for ingredient_element in ingredient_elements:
            ingredient_name_element = ingredient_element.find("span", class_="o-Ingredients__a-Ingredient--CheckboxLabel")
            if ingredient_name_element is not None:
                ingredient_name = ingredient_name_element.text.strip()
                
                # Clean the ingredient name
                ingredient_name = re.sub(r"\s*\xa0\s*", " ", ingredient_name)  # Remove "xa0" characters
                if ingredient_name != "Deselect All":
                    ingredients.append(ingredient_name)
                    
        file.write("Ingredients:\n")
        for ingredient in ingredients:
            file.write("- {}\n".format(ingredient))

        instructions_elements = recipe_soup.find_all("li", class_="o-Method__m-Step")
        instructions = [re.sub(r"\s*\xa0\s*", " ", instruction.text.strip()) for instruction in instructions_elements]
        file.write("Instructions:\n")
        for i, instruction in enumerate(instructions, start=1):
            file.write("{}. {}\n".format(i, instruction))
            
        file.write("----------\n")

# Quit the webdriver
driver.quit()

I should also add that nothing is being written into the output file.

That class is applied not to the "a" tag, but to a "span" tag. I changed one line in your code and got 64 links (which i think is what you're looking for):

recipe_links = soup.find_all("span", class_="m-MediaBlock__a-HeadlineText")