Extracting Recipes from HTML

Welcome back! In the previous lesson, you learned how to use AI to generate new recipes from a list of ingredients. Now, let’s take the next step: extracting recipes from real-world web pages.

Many cooking websites have great recipes, but the information is often buried in messy HTML code. Our goal is to build a script that can take a raw HTML file, use AI to extract a clean recipe, and then store that recipe in our database. This process is a key part of making our AI Cooking Helper smarter and more useful.

By the end of this lesson, you’ll understand how to automate recipe extraction from HTML using Python, prompt templates, and your existing database setup.

Before we dive in, let’s briefly remind ourselves how we previously generated recipes with AI.

In the last lesson, you learned how to:

• Use prompt templates to ask the AI for a recipe based on a list of ingredients.
• Send these prompts to the AI and receive a structured recipe in response.
• Parse the AI’s response and use it in your Flask app.

This time, instead of generating a recipe from scratch, we’ll use the AI to extract a recipe from a messy HTML page. The process is similar, but the input and prompts are a bit different.

Let’s look at the big picture before we break things down.

The script you’ll be working with is called extract_and_store_recipe.py. Its job is to:

1. Read a raw HTML file from your computer.
2. Use AI to extract a clean recipe from that HTML.
3. Parse the AI’s response into structured data (name, ingredients, steps).
4. Store the recipe in your database, making sure not to add duplicates.

Here’s a simple diagram of the flow:

This script brings together everything you’ve learned so far: prompt templates, LLM calls, and database operations.

The first step is to get the AI to read the HTML and return a clean recipe. We do this by sending it a carefully crafted prompt.

Let’s look at how the script prepares and sends this prompt.

We use two prompt templates:

• A system prompt that tells the AI what its job is.
• A user prompt that gives the AI the actual HTML to process.

Here’s how the script loads and fills in these templates:

from app.llm import generate_response

def extract_recipe_from_html(html_str):
    response = generate_response(
        system_prompt_name="recipe_extract_system",
        user_prompt_name="recipe_extract_user",
        variables={"html": html_str}
    )
    # ... (rest of the function)

• generate_response is a function that loads the prompt templates, fills in the {{html}} variable with your HTML, and sends the request to the AI.
• The AI is told to extract a recipe and return it in a specific format.

System prompt:

You are an expert at extracting structured information from unstructured HTML content. Your job is to read messy HTML from cooking websites and convert it into a clean recipe format.

Return your output in the following format:

Name: <Recipe Name>
Ingredients:
- item1
- item2
- ...
Steps:
1. Step one.
2. Step two.
3. ...

User prompt:

Extract a clean cooking recipe from the following raw HTML page:

{{html}}

When the script runs, it replaces {{html}} with the actual HTML content. The AI then returns a recipe in the requested format.

Once the AI returns its response, we need to turn that text into structured data and save it to the database.

The script uses a function called parse_recipe_string to break the AI’s response into parts:

def parse_recipe_string(response):
    name = None
    parsed_ingredients = []
    steps = []

    lines = response.strip().splitlines()
    section = None

    for line in lines:
        if line.lower().startswith("name:"):
            name = line.split(":", 1)[1].strip()
            section = None
        elif line.lower().startswith("ingredients:"):
            section = "ingredients"
        elif line.lower().startswith("steps:"):
            section = "steps"
        elif section == "ingredients" and line.strip().startswith("-"):
            parsed_ingredients.append(line.strip()[1:].strip())
        elif section == "steps" and line.strip()[:2].replace('.', '').isdigit():
            steps.append(line.strip())

    return {
        "name": name,
        "ingredients": parsed_ingredients,
        "steps": steps
    }

• The function reads each line of the AI’s response.
• It looks for the Name:, Ingredients:, and Steps: sections.
• It collects the recipe name, a list of ingredients, and a list of steps.

You might find the code familiar, as its the same logic we used in the generate_recipe function inside routes.py

Example output:

{
    "name": "Classic Pancakes",
    "ingredients": ["flour", "milk", "eggs", "sugar", "baking powder"],
    "steps": [
        "1. Mix all dry ingredients.",
        "2. Add milk and eggs, then stir.",
        "3. Cook on a hot griddle until golden."
    ]
}

Now, let’s see how the script saves the recipe:

def store_recipe_in_db(recipe_data):
    if recipe_data == {}:
        print("❌ Error generating recipe")
        return
    
    app = create_app()
    with app.app_context():
        # Avoid duplicate insert
        existing = Recipe.query.filter_by(name=recipe_data['name']).first()
        if existing:
            print(f"⚠️ Recipe '{recipe_data['name']}' already exists.")
            return

        # Create Recipe entry
        recipe = Recipe(name=recipe_data['name'], steps='\n'.join(recipe_data['steps']))

        # Handle ingredients
        for ing_name in recipe_data['ingredients']:
            ing_name = ing_name.strip().lower()
            ingredient = Ingredient.query.filter_by(name=ing_name).first()
            if not ingredient:
                ingredient = Ingredient(name=ing_name)
                db.session.add(ingredient)
            recipe.ingredients.append(ingredient)

        db.session.add(recipe)
        db.session.commit()
        print(f"✅ Recipe '{recipe.name}' stored in database.")

• The function first checks if the recipe data is empty.
• It opens a database session and checks if a recipe with the same name already exists.
• If not, it creates a new Recipe object and adds each ingredient, creating new ones if needed.
• Finally, it saves everything to the database.

You might also find the code familiar, as its the same logic we used in the add_manual_recipe.py script with avoidance of duplicate insert added.

Example output:

✅ Recipe 'Classic Pancakes' stored in database.

or, if the recipe already exists:

⚠️ Recipe 'Classic Pancakes' already exists.

In this lesson, you learned how to extract a recipe from a messy HTML page using AI and store it in your database. You saw how the script:

• Reads an HTML file,
• Uses prompt templates to guide the AI,
• Parses the AI’s response into structured data,
• And saves the recipe and its ingredients to your database.

Next, you’ll get hands-on practice running and modifying this script. You’ll see how it works with real HTML files and learn how to troubleshoot and improve the extraction process. Great job making it this far — let’s keep building your AI Cooking Helper!