Extracting Recipes From HTML

Introduction: Turning Web Pages into Usable Recipes

Welcome back! In the previous lesson, you learned how to use AI to generate new recipes from a list of ingredients. Now, let’s take the next step: extracting recipes from real-world web pages. Many cooking websites have great recipes, but the information is often buried in messy HTML code. Our goal is to build a script that can take a raw HTML file, use AI to extract a clean recipe, and then store that recipe in our database. This process is a key part of making our AI Cooking Helper smarter and more useful. By the end of this lesson, you’ll understand how to automate recipe extraction from HTML using prompt templates, LLM calls, and your existing database setup .

Quick Recall: Recipe Generation with AI

Before we dive in, let’s briefly remind ourselves how we previously generated recipes with AI . In the last lesson, you learned how to: Use prompt templates to ask the AI for a recipe based on a list of ingredients. Send these prompts to the AI and receive a structured recipe in response. Parse the AI ’s response and use it in your application. This time, instead of generating a recipe from scratch, we’ll use the AI to extract a recipe from a messy HTML page. The process is similar, but the input and prompts are a bit different.

How the Extraction Script Works

Let’s look at the big picture before we break things down. The script you’ll be working with is called extract_and_store_recipe.py . Its job is to: Read a raw HTML file from your computer. Use AI to extract a clean recipe from that HTML . Parse the AI ’s response into structured data (name, ingredients, steps). Store the recipe in your database using the ORM, making sure not to add duplicates. Here’s a simple diagram of the flow: This script brings together everything you’ve learned so far: prompt templates, LLM calls, and database operations .

Using Prompts to Extract Recipes from HTML

The first step is to get the AI to read the HTML and return a clean recipe. We do this by sending it a carefully crafted prompt.

Join the 1M+ learners on CodeSignal

Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal

Python

from recipes.llm import generate_response

def extract_recipe_from_html(html_str):
    response = generate_response(
        system_prompt_name="recipe_extract_system",
        user_prompt_name="recipe_extract_user",
        variables={"html": html_str},
    )

    if not response:
        raise ValueError("Failed to extract recipe from HTML.")

    return parse_recipe_string(response)

from recipes.llm import generate_response

def extract_recipe_from_html(html_str):
    response = generate_response(
        system_prompt_name="recipe_extract_system",
        user_prompt_name="recipe_extract_user",
        variables={"html": html_str},
    )

    if not response:
        raise ValueError("Failed to extract recipe from HTML.")

    return parse_recipe_string(response)

text

You are an expert at extracting structured information from unstructured HTML content. Your job is to read messy HTML from cooking websites and convert it into a clean recipe format.

Return your output in the following format:

Name: <Recipe Name>
Ingredients:
- item1
- item2
- ...
Steps:
1. Step one.
2. Step two.
3. ...

You are an expert at extracting structured information from unstructured HTML content. Your job is to read messy HTML from cooking websites and convert it into a clean recipe format.

Return your output in the following format:

Name: <Recipe Name>
Ingredients:
- item1
- item2
- ...
Steps:
1. Step one.
2. Step two.
3. ...

text

Extract a clean cooking recipe from the following raw HTML page:

{{html}}

Extract a clean cooking recipe from the following raw HTML page:

{{html}}

Python

def parse_recipe_string(response):
    try:
        name = None
        parsed_ingredients = []
        steps = []

        lines = response.strip().splitlines()
        section = None

        for raw in lines:
            line = raw.strip()
            low = line.lower()
            if low.startswith("name:"):
                name = line.split(":", 1)[1].strip()
                section = None
            elif low.startswith("ingredients:"):
                section = "ingredients"
            elif low.startswith("steps:"):
                section = "steps"
            elif section == "ingredients" and line.startswith("-"):
                parsed_ingredients.append(line[1:].strip())
            elif section == "steps":
                # Clean up prefixes like '1.' or '2)' to get the raw step text
                prefix = line[:2].replace(".", "").replace(")", "")
                if prefix.isdigit() or line:
                    steps.append(line)

        return {"name": name, "ingredients": parsed_ingredients, "steps": steps}
    except Exception:
        return {}

def parse_recipe_string(response):
    try:
        name = None
        parsed_ingredients = []
        steps = []

        lines = response.strip().splitlines()
        section = None

        for raw in lines:
            line = raw.strip()
            low = line.lower()
            if low.startswith("name:"):
                name = line.split(":", 1)[1].strip()
                section = None
            elif low.startswith("ingredients:"):
                section = "ingredients"
            elif low.startswith("steps:"):
                section = "steps"
            elif section == "ingredients" and line.startswith("-"):
                parsed_ingredients.append(line[1:].strip())
            elif section == "steps":
                # Clean up prefixes like '1.' or '2)' to get the raw step text
                prefix = line[:2].replace(".", "").replace(")", "")
                if prefix.isdigit() or line:
                    steps.append(line)

        return {"name": name, "ingredients": parsed_ingredients, "steps": steps}
    except Exception:
        return {}

Python

from django.db import transaction
from recipes.models import Ingredient, Recipe

def _normalize(value):
    return (value or "").strip().lower()

def store_recipe_in_db(recipe_data):
    if not recipe_data:
        print("Error generating recipe")
        return

    name = (recipe_data.get("name") or "").strip()
    if not name:
        print("Extracted recipe has no name.")
        return

    # Check if a recipe with this name already exists (case-insensitive)
    existing = Recipe.objects.filter(name__iexact=name).first()
    if existing:
        print(f"Recipe '{name}' already exists (id={existing.id}).")
        return

    steps_list = [s.strip() for s in (recipe_data.get("steps") or []) if s.strip()]

    # Use a transaction to ensure everything is saved together
    with transaction.atomic():
        # Create the Recipe object
        recipe = Recipe.objects.create(name=name, steps="\n".join(steps_list))

        for ing in recipe_data.get("ingredients", []):
            ing_name = _normalize(ing)
            if not ing_name:
                continue
            
            # Find the ingredient or create it if it doesn't exist
            ingredient, _created = Ingredient.objects.get_or_create(name=ing_name)
            
            # Link the ingredient to the recipe
            recipe.ingredients.add(ingredient)

    print(f"Recipe '{recipe.name}' stored in database (id={recipe.id}).")

from django.db import transaction
from recipes.models import Ingredient, Recipe

def _normalize(value):
    return (value or "").strip().lower()

def store_recipe_in_db(recipe_data):
    if not recipe_data:
        print("Error generating recipe")
        return

    name = (recipe_data.get("name") or "").strip()
    if not name:
        print("Extracted recipe has no name.")
        return

    # Check if a recipe with this name already exists (case-insensitive)
    existing = Recipe.objects.filter(name__iexact=name).first()
    if existing:
        print(f"Recipe '{name}' already exists (id={existing.id}).")
        return

    steps_list = [s.strip() for s in (recipe_data.get("steps") or []) if s.strip()]

    # Use a transaction to ensure everything is saved together
    with transaction.atomic():
        # Create the Recipe object
        recipe = Recipe.objects.create(name=name, steps="\n".join(steps_list))

        for ing in recipe_data.get("ingredients", []):
            ing_name = _normalize(ing)
            if not ing_name:
                continue
            
            # Find the ingredient or create it if it doesn't exist
            ingredient, _created = Ingredient.objects.get_or_create(name=ing_name)
            
            # Link the ingredient to the recipe
            recipe.ingredients.add(ingredient)

    print(f"Recipe '{recipe.name}' stored in database (id={recipe.id}).")

Python

from recipes.llm import generate_response

def extract_recipe_from_html(html_str):
    response = generate_response(
        system_prompt_name="recipe_extract_system",
        user_prompt_name="recipe_extract_user",
        variables={"html": html_str},
    )

    if not response:
        raise ValueError("Failed to extract recipe from HTML.")

    return parse_recipe_string(response)

text

You are an expert at extracting structured information from unstructured HTML content. Your job is to read messy HTML from cooking websites and convert it into a clean recipe format.

Return your output in the following format:

Name: <Recipe Name>
Ingredients:
- item1
- item2
- ...
Steps:
1. Step one.
2. Step two.
3. ...

text

Extract a clean cooking recipe from the following raw HTML page:

{{html}}

Python

def parse_recipe_string(response):
    try:
        name = None
        parsed_ingredients = []
        steps = []

        lines = response.strip().splitlines()
        section = None

        for raw in lines:
            line = raw.strip()
            low = line.lower()
            if low.startswith("name:"):
                name = line.split(":", 1)[1].strip()
                section = None
            elif low.startswith("ingredients:"):
                section = "ingredients"
            elif low.startswith("steps:"):
                section = "steps"
            elif section == "ingredients" and line.startswith("-"):
                parsed_ingredients.append(line[1:].strip())
            elif section == "steps":
                # Clean up prefixes like '1.' or '2)' to get the raw step text
                prefix = line[:2].replace(".", "").replace(")", "")
                if prefix.isdigit() or line:
                    steps.append(line)

        return {"name": name, "ingredients": parsed_ingredients, "steps": steps}
    except Exception:
        return {}

Python

from django.db import transaction
from recipes.models import Ingredient, Recipe

def _normalize(value):
    return (value or "").strip().lower()

def store_recipe_in_db(recipe_data):
    if not recipe_data:
        print("Error generating recipe")
        return

    name = (recipe_data.get("name") or "").strip()
    if not name:
        print("Extracted recipe has no name.")
        return

    # Check if a recipe with this name already exists (case-insensitive)
    existing = Recipe.objects.filter(name__iexact=name).first()
    if existing:
        print(f"Recipe '{name}' already exists (id={existing.id}).")
        return

    steps_list = [s.strip() for s in (recipe_data.get("steps") or []) if s.strip()]

    # Use a transaction to ensure everything is saved together
    with transaction.atomic():
        # Create the Recipe object
        recipe = Recipe.objects.create(name=name, steps="\n".join(steps_list))

        for ing in recipe_data.get("ingredients", []):
            ing_name = _normalize(ing)
            if not ing_name:
                continue
            
            # Find the ingredient or create it if it doesn't exist
            ingredient, _created = Ingredient.objects.get_or_create(name=ing_name)
            
            # Link the ingredient to the recipe
            recipe.ingredients.add(ingredient)

    print(f"Recipe '{recipe.name}' stored in database (id={recipe.id}).")

Introduction: Turning Web Pages into Usable Recipes

Quick Recall: Recipe Generation with AI

How the Extraction Script Works

Using Prompts to Extract Recipes from HTML

Join the 1M+ learners on CodeSignal

Loading the Prompt Templates

Example: What the AI Sees

Parsing the AI’s Response

Parsing Logic

Storing the Recipe in the Database

Using Transactions and ORM Methods

Summary and What’s Next