Speeding Up Workflows with Parallelization

Introduction & Context

Welcome back! You've mastered sequential workflows with prompt chaining and conditional workflows with intelligent routing. Now it's time to unlock dramatic performance improvements by learning parallel processing — executing multiple independent OpenAI API calls simultaneously instead of waiting for each one to complete. In this lesson, you'll discover how to transform workflows that take minutes into operations that complete in seconds. You'll learn the difference between synchronous and asynchronous programming, master the asyncio library, and build a system that asks multiple questions to GPT-5 at the same time.

What "Synchronous" Really Means

Before we dive into the technical details, let's clarify terms that often confuse beginners: synchronous and asynchronous . Outside of programming, "synchronous" means "happening at the same time" — think synchronized swimming or clocks ticking together. In programming, though, "synchronous" means the opposite : operations are coordinated in sequence, not in parallel. When we say "synchronous API calls," we mean calls that happen one after another, waiting for each to complete before starting the next. "Asynchronous" API calls, on the other hand, can be launched together and run concurrently — they don't wait for each other to finish. This might seem backwards at first, but once you understand this distinction, the terms "synchronous" (sequential) and "asynchronous" (parallel) will make much more sense throughout this lesson.

The Parallelization Workflow Pattern

Let's understand the high-level pattern we'll be implementing. This workflow has two distinct phases that work together to provide both speed and comprehensive results: Phase 1: Parallel Research Gathering Launch multiple independent OpenAI API calls simultaneously Each call researches a different aspect of your topic (attractions, transportation, culture) All questions run concurrently, completing in roughly the time of the slowest individual request Results are collected and preserved in their original order Phase 2: Sequential Result Synthesis Combine all parallel research into a single comprehensive dataset Send the aggregated information to GPT-5 with instructions for synthesis Generate a unified, actionable final result (like a complete travel guide) This sequential step ensures all information is properly integrated This two-phase approach maximizes both efficiency and quality: you get the speed benefits of parallel processing for data gathering while maintaining coherent analysis through sequential aggregation. It's particularly powerful for research tasks, analysis workflows, and any scenario where you need to quickly gather diverse information and synthesize it into actionable insights.

Join the 1M+ learners on CodeSignal

Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal

Understanding Sync vs Async OpenAI Clients

When working with the OpenAI Responses API, you can choose between two client types: one for synchronous (step-by-step) operations and one for asynchronous (parallel) operations. The difference between them determines whether your program waits for each GPT-5 response before moving on or whether it can send multiple requests at once.

With the standard OpenAI client, each API call is synchronous — your code waits for a response before continuing. This is simple but can be slow if you have many independent tasks.

Python

from openai import OpenAI

# Initialize the standard OpenAI client
client = OpenAI()

# Synchronous approach - each call waits for the previous one to finish
response1 = client.responses.create(...)  # Wait for this to complete
response2 = client.responses.create(...)  # Then wait for this to complete

from openai import OpenAI

# Initialize the standard OpenAI client
client = OpenAI()

# Synchronous approach - each call waits for the previous one to finish
response1 = client.responses.create(...)  # Wait for this to complete
response2 = client.responses.create(...)  # Then wait for this to complete

In contrast, the AsyncOpenAI client supports asynchronous operations. This means you can start several GPT-5 API calls at the same time, and your program will continue running while waiting for responses. This is ideal for running many independent tasks in parallel.

Python

import asyncio
from openai import AsyncOpenAI

# Initialize the async OpenAI client
client = AsyncOpenAI()

# Asynchronous approach - launch multiple requests concurrently (inside an async function)
async def run_parallel():
    tasks = [
        client.responses.create(...),  # Request A
        client.responses.create(...),  # Request B
    ]
    # Run both at the same time; results are returned in the order of the tasks list
    response1, response2 = await asyncio.gather(*tasks)
    
# Run the async function
asyncio.run(run_parallel())

import asyncio
from openai import AsyncOpenAI

# Initialize the async OpenAI client
client = AsyncOpenAI()

# Asynchronous approach - launch multiple requests concurrently (inside an async function)
async def run_parallel():
    tasks = [
        client.responses.create(...),  # Request A
        client.responses.create(...),  # Request B
    ]
    # Run both at the same time; results are returned in the order of the tasks list
    response1, response2 = await asyncio.gather(*tasks)
    
# Run the async function
asyncio.run(run_parallel())

In summary:

Use the synchronous client for simple, sequential workflows where each step depends on the previous one.
Use the asynchronous client when you want to launch multiple independent GPT-5 API calls at once, dramatically improving performance for batch or parallel tasks.

Choosing the right client type is the key to optimizing your OpenAI workflows for both simplicity and speed.

Aggregating Results for Final Analysis

With all our parallel research complete, let's build the aggregation phase that synthesizes everything into a comprehensive result. This sequential step ensures all information is properly integrated:

Python

async def create_travel_plan(results):
    """
    Aggregate all research results into a comprehensive travel plan
    """
    # Combine all Q&A pairs into a formatted string
    combined_research = ""
    for question, answer in results:
        combined_research += f"{question}\n{answer}\n\n"
    
    # Create aggregator prompt
    aggregator_prompt = f"Create a brief Paris travel guide based on this research:\n\n{combined_research}\n\nMake it concise and practical."
    
    # Send to GPT-5 for final synthesis
    response = await client.responses.create(
        model="gpt-5",
        instructions="You are a travel planner. Create brief, useful guides.",
        input=[{"role": "user", "content": aggregator_prompt}],
        reasoning={"effort": "minimal"},
        store=False
    )
    
    return response.output_text

async def create_travel_plan(results):
    """
    Aggregate all research results into a comprehensive travel plan
    """
    # Combine all Q&A pairs into a formatted string
    combined_research = ""
    for question, answer in results:
        combined_research += f"{question}\n{answer}\n\n"
    
    # Create aggregator prompt
    aggregator_prompt = f"Create a brief Paris travel guide based on this research:\n\n{combined_research}\n\nMake it concise and practical."
    
    # Send to GPT-5 for final synthesis
    response = await client.responses.create(
        model="gpt-5",
        instructions="You are a travel planner. Create brief, useful guides.",
        input=[{"role": "user", "content": aggregator_prompt}],
        reasoning={"effort": "minimal"},
        store=False
    )
    
    return response.output_text

The aggregation step continues to use minimal reasoning effort since we're simply synthesizing already-researched information into a concise guide.

Running the Complete Parallel Workflow

Let's bring it all together into a complete workflow that demonstrates the full power of parallel processing followed by intelligent aggregation:

Python

async def main():
    """
    Execute parallel research and create comprehensive travel plan
    """
    print(f"Starting {len(questions)} research questions in parallel...")
    
    # Create tasks for all questions
    tasks = [ask_question(question) for question in questions]
    
    # Execute all tasks in parallel
    results = await asyncio.gather(*tasks)
    
    print("\nResearch completed!")
    
    # Aggregate results into final travel plan
    travel_plan = await create_travel_plan(results)
    
    print("\nParis Travel Guide:")
    print(travel_plan)

# Run the parallel workflow
if __name__ == "__main__":
    asyncio.run(main())

async def main():
    """
    Execute parallel research and create comprehensive travel plan
    """
    print(f"Starting {len(questions)} research questions in parallel...")
    
    # Create tasks for all questions
    tasks = [ask_question(question) for question in questions]
    
    # Execute all tasks in parallel
    results = await asyncio.gather(*tasks)
    
    print("\nResearch completed!")
    
    # Aggregate results into final travel plan
    travel_plan = await create_travel_plan(results)
    
    print("\nParis Travel Guide:")
    print(travel_plan)

# Run the parallel workflow
if __name__ == "__main__":
    asyncio.run(main())

When you run this workflow, you'll see the power of parallel execution unfold in three distinct stages:

Instant Launch: All three "🔄 Asking" messages appear immediately as the API calls fire off simultaneously
Concurrent Completion: The "✅ Answered" messages arrive as GPT-5 finishes each response — often in a different order than they were asked, proving your requests are truly running in parallel
Intelligent Synthesis: All this concurrent research gets woven together into a comprehensive travel guide that combines the speed benefits of parallel processing with thoughtful analysis

This visual progression clearly demonstrates how your requests execute concurrently rather than waiting for each other, transforming what could be a slow sequential process into a fast, efficient workflow that delivers both speed and quality.

text

Starting 3 research questions in parallel...
🔄 Asking: What are the top 3 must-see attractions in Paris?
🔄 Asking: What is the most efficient way to get around Paris as a tourist?
🔄 Asking: What are important cultural etiquette tips for visitors to France?
✅ Answered: What are the top 3 must-see attractions in Paris?
✅ Answered: What are important cultural etiquette tips for visitors to France?
✅ Answered: What is the most efficient way to get around Paris as a tourist?

Research completed!

Paris Travel Guide:
Paris in brief: a practical guide

Must-see (top 3)
- Eiffel Tower: Iconic views. Book summit tickets in advance; best at sunset. Top photo spots: Trocadéro, Champ de Mars. Metro: Bir-Hakeim/Trocadéro.
- Louvre Museum: Reserve timed entry; closed Tuesdays. Quicker entrances: Carrousel or Porte des Lions. Metro: Palais Royal–Musée du Louvre.
- Sainte-Chapelle: Go on a sunny day for the stained glass. Consider a combo ticket with the Conciergerie. Metro: Cité.

Getting around
- Metro/RER + walking is fastest; most sights are in zones 1–2.
- Tickets: Tap a contactless bank card for single rides or get a Navigo Easy (load t+). In town Mon–Sun and riding a lot? Navigo Découverte weekly pass is best value (bring a small ID photo). Check fares in the Bonjour RATP app.
- Buses: Slower but scenic; same tickets.
- Bikes: Vélib' is great for 10–20 min trips; use bike lanes and avoid rush-hour boulevards if new to city cycling.
- Taxis/ride-hailing (Taxi Parisien, Uber/Bolt): Best late at night or with luggage; traffic can be heavy at peak times.
- Airports: CDG—RER B to central Paris. Orly—Metro line 14 goes directly; OrlyBus is a good backup.
- Night service: Last metro ~00:45 (later Fri/Sat). Noctilien night buses run overnight.
- Practical tips: Use Citymapper or Bonjour RATP for live routing. Always validate; tap out on RER. Watch for pickpockets on lines 1, 4, and RER B. Many metro stations have stairs — buses are more accessible.

Etiquette essentials
- Greet on entry/exit: "Bonjour/Bonsoir, Monsieur/Madame"; "Au revoir/Bonne journée."
- Start in French; use vous with strangers. Key words: s'il vous plaît, merci, pardon, excusez‑moi.
- Keep voices low; queue properly; escalators: stand right, walk left.
- Dress smart‑casual; cover shoulders in churches.
- Dining: Hands visible, napkin on lap. Wait for "Bon appétit." Bread on the table — tear pieces. Tap water is free: ask for "une carafe d'eau." Service included — round up small change; 5–10% only for excellent service. Separate checks are less common — ask "séparés, s'il vous plaît?" Espresso after meals; milky coffees mostly at breakfast.
- Phones on silent in restaurants and on transit. Ask before photographing people.

Handy phrases
- Bonjour / Bonsoir
- Excusez‑moi
- S'il vous plaît
- Merci beaucoup
- Parlez‑vous anglais ?
- Je ne parle pas bien français.
- Où sont les toilettes, s'il vous plaît ?

Starting 3 research questions in parallel...
🔄 Asking: What are the top 3 must-see attractions in Paris?
🔄 Asking: What is the most efficient way to get around Paris as a tourist?
🔄 Asking: What are important cultural etiquette tips for visitors to France?
✅ Answered: What are the top 3 must-see attractions in Paris?
✅ Answered: What are important cultural etiquette tips for visitors to France?
✅ Answered: What is the most efficient way to get around Paris as a tourist?

Research completed!

Paris Travel Guide:
Paris in brief: a practical guide

Must-see (top 3)
- Eiffel Tower: Iconic views. Book summit tickets in advance; best at sunset. Top photo spots: Trocadéro, Champ de Mars. Metro: Bir-Hakeim/Trocadéro.
- Louvre Museum: Reserve timed entry; closed Tuesdays. Quicker entrances: Carrousel or Porte des Lions. Metro: Palais Royal–Musée du Louvre.
- Sainte-Chapelle: Go on a sunny day for the stained glass. Consider a combo ticket with the Conciergerie. Metro: Cité.

Getting around
- Metro/RER + walking is fastest; most sights are in zones 1–2.
- Tickets: Tap a contactless bank card for single rides or get a Navigo Easy (load t+). In town Mon–Sun and riding a lot? Navigo Découverte weekly pass is best value (bring a small ID photo). Check fares in the Bonjour RATP app.
- Buses: Slower but scenic; same tickets.
- Bikes: Vélib' is great for 10–20 min trips; use bike lanes and avoid rush-hour boulevards if new to city cycling.
- Taxis/ride-hailing (Taxi Parisien, Uber/Bolt): Best late at night or with luggage; traffic can be heavy at peak times.
- Airports: CDG—RER B to central Paris. Orly—Metro line 14 goes directly; OrlyBus is a good backup.
- Night service: Last metro ~00:45 (later Fri/Sat). Noctilien night buses run overnight.
- Practical tips: Use Citymapper or Bonjour RATP for live routing. Always validate; tap out on RER. Watch for pickpockets on lines 1, 4, and RER B. Many metro stations have stairs — buses are more accessible.

Etiquette essentials
- Greet on entry/exit: "Bonjour/Bonsoir, Monsieur/Madame"; "Au revoir/Bonne journée."
- Start in French; use vous with strangers. Key words: s'il vous plaît, merci, pardon, excusez‑moi.
- Keep voices low; queue properly; escalators: stand right, walk left.
- Dress smart‑casual; cover shoulders in churches.
- Dining: Hands visible, napkin on lap. Wait for "Bon appétit." Bread on the table — tear pieces. Tap water is free: ask for "une carafe d'eau." Service included — round up small change; 5–10% only for excellent service. Separate checks are less common — ask "séparés, s'il vous plaît?" Espresso after meals; milky coffees mostly at breakfast.
- Phones on silent in restaurants and on transit. Ask before photographing people.

Handy phrases
- Bonjour / Bonsoir
- Excusez‑moi
- S'il vous plaît
- Merci beaucoup
- Parlez‑vous anglais ?
- Je ne parle pas bien français.
- Où sont les toilettes, s'il vous plaît ?

Introduction & Context

What "Synchronous" Really Means

The Parallelization Workflow Pattern

Join the 1M+ learners on CodeSignal

Understanding Sync vs Async OpenAI Clients

AsyncIO Fundamentals for GPT-5 Workflows

Running Async Code with asyncio.run()

Concurrent Execution with asyncio.gather()

Creating Async Functions for GPT-5 Calls

Preparing the List of Questions

Building Parallel Task Collections

Aggregating Results for Final Analysis

Running the Complete Parallel Workflow

Performance Benefits and Use Cases

Summary & Practice Preparation