Welcome back! In the previous lesson, you learned how to use AI to generate new recipes from a list of ingredients. Now, let’s take the next step: extracting recipes from real-world web pages.

Many cooking websites have great recipes, but the information is often buried in messy HTML code. Our goal is to build a script that can take a raw HTML file, use AI to extract a clean recipe, and then store that recipe in our database. This process is a key part of making our AI Cooking Helper smarter and more useful.

By the end of this lesson, you’ll understand how to automate recipe extraction from HTML using prompt templates, LLM calls, and your existing database setup.

Before we dive in, let’s briefly remind ourselves how we previously generated recipes with AI.

In the last lesson, you learned how to:

• Use prompt templates to ask the AI for a recipe based on a list of ingredients.
• Send these prompts to the AI and receive a structured recipe in response.
• Parse the AI’s response and use it in your application.

This time, instead of generating a recipe from scratch, we’ll use the AI to extract a recipe from a messy HTML page. The process is similar, but the input and prompts are a bit different.

Let’s look at the big picture before we break things down.

The script you’ll be working with is called extract_and_store_recipe.py. Its job is to:

1. Read a raw HTML file from your computer.
2. Use AI to extract a clean recipe from that HTML.
3. Parse the AI’s response into structured data (name, ingredients, steps).
4. Store the recipe in your database using the ORM, making sure not to add duplicates.

Here’s a simple diagram of the flow:

This script brings together everything you’ve learned so far: prompt templates, LLM calls, and database operations.

The first step is to get the AI to read the HTML and return a clean recipe. We do this by sending it a carefully crafted prompt.

We use two prompt templates:

• A system prompt (recipe_extract_system) that tells the AI it is an expert at structured information extraction.
• A user prompt (recipe_extract_user) that provides the raw HTML content.

Here’s how the script loads and fills in these templates using our LLM manager:

from recipes.llm import generate_response

def extract_recipe_from_html(html_str):
    response = generate_response(
        system_prompt_name="recipe_extract_system",
        user_prompt_name="recipe_extract_user",
        variables={"html": html_str},
    )

    if not response:
        raise ValueError("Failed to extract recipe from HTML.")

    return parse_recipe_string(response)

• generate_response is our centralized function that loads the templates and replaces the {{html}} variable with the actual website content.
• The AI is instructed to return the data in a specific text-based format so we can easily parse it later.

System prompt:

You are an expert at extracting structured information from unstructured HTML content. Your job is to read messy HTML from cooking websites and convert it into a clean recipe format.

Return your output in the following format:

Name: <Recipe Name>
Ingredients:
- item1
- item2
- ...
Steps:
1. Step one.
2. Step two.
3. ...

User prompt:

Extract a clean cooking recipe from the following raw HTML page:

{{html}}

Once the AI returns its response, we need to turn that text into structured data. The script uses a function called parse_recipe_string to break the text into parts.

The logic iterates through the lines of the response, looking for specific keywords to identify which section of the recipe it is currently reading.

def parse_recipe_string(response):
    try:
        name = None
        parsed_ingredients = []
        steps = []

        lines = response.strip().splitlines()
        section = None

        for raw in lines:
            line = raw.strip()
            low = line.lower()
            if low.startswith("name:"):
                name = line.split(":", 1)[1].strip()
                section = None
            elif low.startswith("ingredients:"):
                section = "ingredients"
            elif low.startswith("steps:"):
                section = "steps"
            elif section == "ingredients" and line.startswith("-"):
                parsed_ingredients.append(line[1:].strip())
            elif section == "steps":
                # Clean up prefixes like '1.' or '2)' to get the raw step text
                prefix = line[:2].replace(".", "").replace(")", "")
                if prefix.isdigit() or line:
                    steps.append(line)

        return {"name": name, "ingredients": parsed_ingredients, "steps": steps}
    except Exception:
        return {}

• It uses the prefix logic to handle different ways the AI might number the steps (like 1. or 1)).
• This results in a clean dictionary containing the recipe name, a list of ingredient strings, and a list of step strings.

After parsing, the final step is to save this data into our database. We use the Object-Relational Mapper (ORM) to handle the database tables and relationships.

To ensure the data is saved correctly, we use transaction.atomic(). This ensures that if something goes wrong (like a crash while saving ingredients), none of the partial data is saved, keeping our database clean.

from django.db import transaction
from recipes.models import Ingredient, Recipe

def _normalize(value):
    return (value or "").strip().lower()

def store_recipe_in_db(recipe_data):
    if not recipe_data:
        print("Error generating recipe")
        return

    name = (recipe_data.get("name") or "").strip()
    if not name:
        print("Extracted recipe has no name.")
        return

    # Check if a recipe with this name already exists (case-insensitive)
    existing = Recipe.objects.filter(name__iexact=name).first()
    if existing:
        print(f"Recipe '{name}' already exists (id={existing.id}).")
        return

    steps_list = [s.strip() for s in (recipe_data.get("steps") or []) if s.strip()]

    # Use a transaction to ensure everything is saved together
    with transaction.atomic():
        # Create the Recipe object
        recipe = Recipe.objects.create(name=name, steps="\n".join(steps_list))

        for ing in recipe_data.get("ingredients", []):
            ing_name = _normalize(ing)
            if not ing_name:
                continue
            
            # Find the ingredient or create it if it doesn't exist
            ingredient, _created = Ingredient.objects.get_or_create(name=ing_name)
            
            # Link the ingredient to the recipe
            recipe.ingredients.add(ingredient)

    print(f"Recipe '{recipe.name}' stored in database (id={recipe.id}).")

Here is how the database logic works:

1. Duplicate Prevention: We use Recipe.objects.filter(name__iexact=name) to check if the recipe is already in our database before trying to save it.
2. Atomic Transactions: The with transaction.atomic(): block wraps all the saving operations. If any part fails, the whole block is rolled back.
3. Automatic Creation: Ingredient.objects.get_or_create() is a powerful method that looks for an existing ingredient by name. If it finds one, it uses it; otherwise, it creates a new record automatically.
4. Normalization: We convert ingredient names to lowercase using _normalize so that Flour, flour, and FLOUR are all treated as the same ingredient.

In this lesson, you learned how to extract a recipe from a messy HTML page using AI and store it in your database. You saw how the script:

• Reads an HTML file and sends it to the AI using structured prompt templates.
• Uses generate_response from our LLM utility to manage the interaction.
• Parses the AI’s text response into a structured data format, cleaning up numbering and prefixes.
• Safely saves the data using atomic transactions and ORM methods like get_or_create to prevent duplicates and ensure data integrity.

Next, you’ll get hands-on practice running this script yourself. You’ll see how it handles real-world website content and learn how to verify that the recipes are correctly stored in your database!