Extracting Recipes from HTML

Introduction: Turning Web Pages into Usable Recipes

Welcome back! In the previous lesson, you learned how to use AI to generate new recipes from a list of ingredients. Now, let’s take the next step: extracting recipes from real-world web pages. Many cooking websites have great recipes, but the information is often buried in messy HTML code. Our goal is to build a script that can take a raw HTML file, use AI to extract a clean recipe, and then store that recipe in our database. This process is a key part of making our AI Cooking Helper smarter and more useful. By the end of this lesson, you’ll understand how to automate recipe extraction from HTML using prompt templates, LLM calls, and your existing database setup.

Quick Recall: Recipe Generation with AI

Before we dive in, let’s briefly remind ourselves how we previously generated recipes with AI. In the last lesson, you learned how to: Use prompt templates to ask the AI for a recipe based on a list of ingredients. Send these prompts to the AI and receive a structured recipe in response. Parse the AI’s response and use it in your application. This time, instead of generating a recipe from scratch, we’ll use the AI to extract a recipe from a messy HTML page. The process is similar, but the input and prompts are a bit different.

How the Extraction Script Works

Let’s look at the big picture before we break things down. The script you’ll be working with is called extract_and_store_recipe.py . Its job is to: Read a raw HTML file from your computer. Use AI to extract a clean recipe from that HTML . Parse the AI’s response into structured data (name, ingredients, steps). Store the recipe in your database, making sure not to add duplicates. Here’s a simple diagram of the flow: This script brings together everything you’ve learned so far: prompt templates, LLM calls, and database operations.

Using Prompts to Extract Recipes from HTML

The first step is to get the AI to read the HTML and return a clean recipe. We do this by sending it a carefully crafted prompt.

Join the 1M+ learners on CodeSignal

Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal

Python

from app.llm import generate_response

def extract_recipe_from_html(html_str):
    response = generate_response(
        system_prompt_name="recipe_extract_system",
        user_prompt_name="recipe_extract_user",
        variables={"html": html_str}
    )
    # ... (rest of the function)

from app.llm import generate_response

def extract_recipe_from_html(html_str):
    response = generate_response(
        system_prompt_name="recipe_extract_system",
        user_prompt_name="recipe_extract_user",
        variables={"html": html_str}
    )
    # ... (rest of the function)

You are an expert at extracting structured information from unstructured HTML content. Your job is to read messy HTML from cooking websites and convert it into a clean recipe format.

Return your output in the following format:

Name: <Recipe Name>
Ingredients:
- item1
- item2
- ...
Steps:
1. Step one.
2. Step two.
3. ...

You are an expert at extracting structured information from unstructured HTML content. Your job is to read messy HTML from cooking websites and convert it into a clean recipe format.

Return your output in the following format:

Name: <Recipe Name>
Ingredients:
- item1
- item2
- ...
Steps:
1. Step one.
2. Step two.
3. ...

Extract a clean cooking recipe from the following raw HTML page:

{{html}}

Extract a clean cooking recipe from the following raw HTML page:

{{html}}

Python

def parse_recipe_string(response):
    name = None
    parsed_ingredients = []
    steps = []

    lines = response.strip().splitlines()
    section = None

    for raw in lines:
        line = raw.strip()
        low = line.lower()
        if low.startswith("name:"):
            name = line.split(":", 1)[1].strip()
            section = None
        elif low.startswith("ingredients:"):
            section = "ingredients"
        elif low.startswith("steps:"):
            section = "steps"
        elif section == "ingredients" and line.startswith("-"):
            parsed_ingredients.append(line[1:].strip())
        elif section == "steps":
            prefix = line[:2].replace(".", "").replace(")", "")
            if prefix.isdigit() or line:
                steps.append(line)

    return {
        "name": name,
        "ingredients": parsed_ingredients,
        "steps": steps
    }

def parse_recipe_string(response):
    name = None
    parsed_ingredients = []
    steps = []

    lines = response.strip().splitlines()
    section = None

    for raw in lines:
        line = raw.strip()
        low = line.lower()
        if low.startswith("name:"):
            name = line.split(":", 1)[1].strip()
            section = None
        elif low.startswith("ingredients:"):
            section = "ingredients"
        elif low.startswith("steps:"):
            section = "steps"
        elif section == "ingredients" and line.startswith("-"):
            parsed_ingredients.append(line[1:].strip())
        elif section == "steps":
            prefix = line[:2].replace(".", "").replace(")", "")
            if prefix.isdigit() or line:
                steps.append(line)

    return {
        "name": name,
        "ingredients": parsed_ingredients,
        "steps": steps
    }

Python

{
    "name": "Classic Pancakes",
    "ingredients": ["flour", "milk", "eggs", "sugar", "baking powder"],
    "steps": [
        "1. Mix all dry ingredients.",
        "2. Add milk and eggs, then stir.",
        "3. Cook on a hot griddle until golden."
    ]
}

{
    "name": "Classic Pancakes",
    "ingredients": ["flour", "milk", "eggs", "sugar", "baking powder"],
    "steps": [
        "1. Mix all dry ingredients.",
        "2. Add milk and eggs, then stir.",
        "3. Cook on a hot griddle until golden."
    ]
}

Python

from sqlalchemy.orm import Session
from app.db.session import SessionLocal
from app.db.models import Recipe, Ingredient

def _normalize(s):
    return (s or "").strip().lower()

def store_recipe_in_db(recipe_data):
    if not recipe_data or recipe_data == {}:
        print("❌ Error generating recipe")
        return

    session: Session = SessionLocal()
    try:
        name = (recipe_data.get("name") or "").strip()
        if not name:
            print("❌ Extracted recipe has no name.")
            return

        # Avoid duplicate insert (case-insensitive)
        existing = (
            session.query(Recipe)
            .filter(Recipe.name.ilike(name))
            .first()
        )
        if existing:
            print(f"⚠️ Recipe '{name}' already exists (id={existing.id}).")
            return

        # Create Recipe entry
        steps_list = [s.strip() for s in (recipe_data.get("steps") or []) if s.strip()]
        recipe = Recipe(name=name, steps="\n".join(steps_list))
        session.add(recipe)

        # Handle ingredients
        for ing in recipe_data.get("ingredients", []):
            ing_name = _normalize(ing)
            if not ing_name:
                continue
            ingredient = session.query(Ingredient).filter(Ingredient.name == ing_name).first()
            if not ingredient:
                ingredient = Ingredient(name=ing_name)
                session.add(ingredient)
                session.flush()  # get id for association immediately
            recipe.ingredients.append(ingredient)

        session.commit()
        print(f"✅ Recipe '{recipe.name}' stored in database (id={recipe.id}).")
    except Exception as e:
        session.rollback()
        print("❌ Error:", e)
        raise
    finally:
        session.close()

from sqlalchemy.orm import Session
from app.db.session import SessionLocal
from app.db.models import Recipe, Ingredient

def _normalize(s):
    return (s or "").strip().lower()

def store_recipe_in_db(recipe_data):
    if not recipe_data or recipe_data == {}:
        print("❌ Error generating recipe")
        return

    session: Session = SessionLocal()
    try:
        name = (recipe_data.get("name") or "").strip()
        if not name:
            print("❌ Extracted recipe has no name.")
            return

        # Avoid duplicate insert (case-insensitive)
        existing = (
            session.query(Recipe)
            .filter(Recipe.name.ilike(name))
            .first()
        )
        if existing:
            print(f"⚠️ Recipe '{name}' already exists (id={existing.id}).")
            return

        # Create Recipe entry
        steps_list = [s.strip() for s in (recipe_data.get("steps") or []) if s.strip()]
        recipe = Recipe(name=name, steps="\n".join(steps_list))
        session.add(recipe)

        # Handle ingredients
        for ing in recipe_data.get("ingredients", []):
            ing_name = _normalize(ing)
            if not ing_name:
                continue
            ingredient = session.query(Ingredient).filter(Ingredient.name == ing_name).first()
            if not ingredient:
                ingredient = Ingredient(name=ing_name)
                session.add(ingredient)
                session.flush()  # get id for association immediately
            recipe.ingredients.append(ingredient)

        session.commit()
        print(f"✅ Recipe '{recipe.name}' stored in database (id={recipe.id}).")
    except Exception as e:
        session.rollback()
        print("❌ Error:", e)
        raise
    finally:
        session.close()

✅ Recipe 'Classic Pancakes' stored in database (id=5).

✅ Recipe 'Classic Pancakes' stored in database (id=5).

⚠️ Recipe 'Classic Pancakes' already exists (id=5).

⚠️ Recipe 'Classic Pancakes' already exists (id=5).

Python

from app.llm import generate_response

def extract_recipe_from_html(html_str):
    response = generate_response(
        system_prompt_name="recipe_extract_system",
        user_prompt_name="recipe_extract_user",
        variables={"html": html_str}
    )
    # ... (rest of the function)

You are an expert at extracting structured information from unstructured HTML content. Your job is to read messy HTML from cooking websites and convert it into a clean recipe format.

Return your output in the following format:

Name: <Recipe Name>
Ingredients:
- item1
- item2
- ...
Steps:
1. Step one.
2. Step two.
3. ...

Extract a clean cooking recipe from the following raw HTML page:

{{html}}

Python

def parse_recipe_string(response):
    name = None
    parsed_ingredients = []
    steps = []

    lines = response.strip().splitlines()
    section = None

    for raw in lines:
        line = raw.strip()
        low = line.lower()
        if low.startswith("name:"):
            name = line.split(":", 1)[1].strip()
            section = None
        elif low.startswith("ingredients:"):
            section = "ingredients"
        elif low.startswith("steps:"):
            section = "steps"
        elif section == "ingredients" and line.startswith("-"):
            parsed_ingredients.append(line[1:].strip())
        elif section == "steps":
            prefix = line[:2].replace(".", "").replace(")", "")
            if prefix.isdigit() or line:
                steps.append(line)

    return {
        "name": name,
        "ingredients": parsed_ingredients,
        "steps": steps
    }

Python

{
    "name": "Classic Pancakes",
    "ingredients": ["flour", "milk", "eggs", "sugar", "baking powder"],
    "steps": [
        "1. Mix all dry ingredients.",
        "2. Add milk and eggs, then stir.",
        "3. Cook on a hot griddle until golden."
    ]
}

Python

from sqlalchemy.orm import Session
from app.db.session import SessionLocal
from app.db.models import Recipe, Ingredient

def _normalize(s):
    return (s or "").strip().lower()

def store_recipe_in_db(recipe_data):
    if not recipe_data or recipe_data == {}:
        print("❌ Error generating recipe")
        return

    session: Session = SessionLocal()
    try:
        name = (recipe_data.get("name") or "").strip()
        if not name:
            print("❌ Extracted recipe has no name.")
            return

        # Avoid duplicate insert (case-insensitive)
        existing = (
            session.query(Recipe)
            .filter(Recipe.name.ilike(name))
            .first()
        )
        if existing:
            print(f"⚠️ Recipe '{name}' already exists (id={existing.id}).")
            return

        # Create Recipe entry
        steps_list = [s.strip() for s in (recipe_data.get("steps") or []) if s.strip()]
        recipe = Recipe(name=name, steps="\n".join(steps_list))
        session.add(recipe)

        # Handle ingredients
        for ing in recipe_data.get("ingredients", []):
            ing_name = _normalize(ing)
            if not ing_name:
                continue
            ingredient = session.query(Ingredient).filter(Ingredient.name == ing_name).first()
            if not ingredient:
                ingredient = Ingredient(name=ing_name)
                session.add(ingredient)
                session.flush()  # get id for association immediately
            recipe.ingredients.append(ingredient)

        session.commit()
        print(f"✅ Recipe '{recipe.name}' stored in database (id={recipe.id}).")
    except Exception as e:
        session.rollback()
        print("❌ Error:", e)
        raise
    finally:
        session.close()

✅ Recipe 'Classic Pancakes' stored in database (id=5).

⚠️ Recipe 'Classic Pancakes' already exists (id=5).

Introduction: Turning Web Pages into Usable Recipes

Quick Recall: Recipe Generation with AI

How the Extraction Script Works

Using Prompts to Extract Recipes from HTML

Join the 1M+ learners on CodeSignal

Loading the Prompt Templates

Example: What the AI Sees

Parsing and Saving the Recipe

Parsing the AI’s Response

Storing the Recipe in the Database

Summary And What’s Next