Welcome back! In the previous lesson, you learned how to use AI to generate new recipes from a list of ingredients. Now, let’s take the next step: extracting recipes from real-world web pages.

Many cooking websites have great recipes, but the information is often buried in messy HTML code. Our goal is to build a script that can take a raw HTML file, use AI to extract a clean recipe, and then store that recipe in our database. This process is a key part of making our AI Cooking Helper smarter and more useful.

By the end of this lesson, you’ll understand how to automate recipe extraction from HTML using prompt templates, LLM calls, and your existing database setup.

Before we dive in, let’s briefly remind ourselves how we previously generated recipes with AI.

In the last lesson, you learned how to:

• Use prompt templates to ask the AI for a recipe based on a list of ingredients.
• Send these prompts to the AI and receive a structured recipe in response.
• Parse the AI’s response and use it in your application.

This time, instead of generating a recipe from scratch, we’ll use the AI to extract a recipe from a messy HTML page. The process is similar, but the input and prompts are a bit different.

Let’s look at the big picture before we break things down.

The script you’ll be working with is called extract_and_store_recipe.py. Its job is to:

1. Read a raw HTML file from your computer.
2. Use AI to extract a clean recipe from that HTML.
3. Parse the AI’s response into structured data (name, ingredients, steps).
4. Store the recipe in your database, making sure not to add duplicates.

Here’s a simple diagram of the flow:

This script brings together everything you’ve learned so far: prompt templates, LLM calls, and database operations.

The first step is to get the AI to read the HTML and return a clean recipe. We do this by sending it a carefully crafted prompt.

We use two prompt templates:

• A system prompt that tells the AI what its job is.
• A user prompt that gives the AI the actual HTML to process.

Here’s how the script loads and fills in these templates:

from app.llm import generate_response

def extract_recipe_from_html(html_str):
    response = generate_response(
        system_prompt_name="recipe_extract_system",
        user_prompt_name="recipe_extract_user",
        variables={"html": html_str}
    )
    # ... (rest of the function)

• generate_response is a function that loads the prompt templates, fills in the {{html}} variable with your HTML, and sends the request to the AI.
• The AI is told to extract a recipe and return it in a specific format.

System prompt:

You are an expert at extracting structured information from unstructured HTML content. Your job is to read messy HTML from cooking websites and convert it into a clean recipe format.

Return your output in the following format:

Name: <Recipe Name>
Ingredients:
- item1
- item2
- ...
Steps:
1. Step one.
2. Step two.
3. ...

User prompt:

Extract a clean cooking recipe from the following raw HTML page:

{{html}}

When the script runs, it replaces {{html}} with the actual HTML content. The AI then returns a recipe in the requested format.

Once the AI returns its response, we need to turn that text into structured data and save it to the database.

The script uses a function called parse_recipe_string to break the AI’s response into parts:

def parse_recipe_string(response):
    name = None
    parsed_ingredients = []
    steps = []

    lines = response.strip().splitlines()
    section = None

    for raw in lines:
        line = raw.strip()
        low = line.lower()
        if low.startswith("name:"):
            name = line.split(":", 1)[1].strip()
            section = None
        elif low.startswith("ingredients:"):
            section = "ingredients"
        elif low.startswith("steps:"):
            section = "steps"
        elif section == "ingredients" and line.startswith("-"):
            parsed_ingredients.append(line[1:].strip())
        elif section == "steps":
            prefix = line[:2].replace(".", "").replace(")", "")
            if prefix.isdigit() or line:
                steps.append(line)

    return {
        "name": name,
        "ingredients": parsed_ingredients,
        "steps": steps
    }

• The function reads each line of the AI’s response.
• It looks for the Name:, Ingredients:, and Steps: sections.
• It collects the recipe name, a list of ingredients, and a list of steps.

Example output:

{
    "name": "Classic Pancakes",
    "ingredients": ["flour", "milk", "eggs", "sugar", "baking powder"],
    "steps": [
        "1. Mix all dry ingredients.",
        "2. Add milk and eggs, then stir.",
        "3. Cook on a hot griddle until golden."
    ]
}

Now, let’s see how the script saves the recipe using direct session management and normalization.

from sqlalchemy.orm import Session
from app.db.session import SessionLocal
from app.db.models import Recipe, Ingredient

def _normalize(s):
    return (s or "").strip().lower()

def store_recipe_in_db(recipe_data):
    if not recipe_data or recipe_data == {}:
        print("❌ Error generating recipe")
        return

    session: Session = SessionLocal()
    try:
        name = (recipe_data.get("name") or "").strip()
        if not name:
            print("❌ Extracted recipe has no name.")
            return

        # Avoid duplicate insert (case-insensitive)
        existing = (
            session.query(Recipe)
            .filter(Recipe.name.ilike(name))
            .first()
        )
        if existing:
            print(f"⚠️ Recipe '{name}' already exists (id={existing.id}).")
            return

        # Create Recipe entry
        steps_list = [s.strip() for s in (recipe_data.get("steps") or []) if s.strip()]
        recipe = Recipe(name=name, steps="\n".join(steps_list))
        session.add(recipe)

        # Handle ingredients
        for ing in recipe_data.get("ingredients", []):
            ing_name = _normalize(ing)
            if not ing_name:
                continue
            ingredient = session.query(Ingredient).filter(Ingredient.name == ing_name).first()
            if not ingredient:
                ingredient = Ingredient(name=ing_name)
                session.add(ingredient)
                session.flush()  # get id for association immediately
            recipe.ingredients.append(ingredient)

        session.commit()
        print(f"✅ Recipe '{recipe.name}' stored in database (id={recipe.id}).")
    except Exception as e:
        session.rollback()
        print("❌ Error:", e)
        raise
    finally:
        session.close()

• The function first checks if the recipe data is empty.
• It creates a new database session using SessionLocal.
• It checks for duplicate recipes using a case-insensitive query (ilike).
• Ingredient names are normalized (stripped and lowercased) before checking or inserting.
• If the recipe does not exist, it creates a new Recipe object and adds each ingredient, creating new ones if needed.
• Finally, it commits the transaction and closes the session.

Example output:

✅ Recipe 'Classic Pancakes' stored in database (id=5).

or, if the recipe already exists:

⚠️ Recipe 'Classic Pancakes' already exists (id=5).

In this lesson, you learned how to extract a recipe from a messy HTML page using AI and store it in your database. You saw how the script:

• Reads an HTML file,
• Uses prompt templates to guide the AI,
• Parses the AI’s response into structured data,
• And saves the recipe and its ingredients to your database using direct session management and normalization.

Next, you’ll get hands-on practice running and modifying this script. You’ll see how it works with real HTML files and learn how to troubleshoot and improve the extraction process. Great job making it this far — let’s keep building your AI Cooking Helper!