Welcome back! You've mastered sequential workflows with prompt chaining and conditional workflows with intelligent routing. Now it's time to unlock dramatic performance improvements by learning parallel processing — executing multiple independent OpenAI API calls simultaneously instead of waiting for each one to complete.
In this lesson, you'll discover how to transform workflows that take minutes into operations that complete in seconds. You'll learn the difference between synchronous and asynchronous programming, master the asyncio library, and build a system that asks multiple questions to GPT at the same time.
What "Synchronous" Really Means
Before we dive into the technical details, let's clarify terms that often confuse beginners: synchronous and asynchronous.
Outside of programming, "synchronous" means "happening at the same time" — think synchronized swimming or clocks ticking together. In programming, though, "synchronous" means the opposite: operations are coordinated in sequence, not in parallel.
When we say "synchronous API calls," we mean calls that happen one after another, waiting for each to complete before starting the next. "Asynchronous" API calls, on the other hand, can be launched together and run concurrently — they don't wait for each other to finish.
This might seem backwards at first, but once you understand this distinction, the terms "synchronous" (sequential) and "asynchronous" (parallel) will make much more sense throughout this lesson.
The Parallelization Workflow Pattern
Let's understand the high-level pattern we'll be implementing. This workflow has two distinct phases that work together to provide both speed and comprehensive results:
Phase 1: Parallel Research Gathering
Launch multiple independent OpenAI API calls simultaneously
Each call researches a different aspect of your topic (attractions, transportation, culture)
All questions run concurrently, completing in roughly the time of the slowest individual request
Results are collected and preserved in their original order
Phase 2: Sequential Result Synthesis
Combine all parallel research into a single comprehensive dataset
Send the aggregated information to GPT with instructions for synthesis
Generate a unified, actionable final result (like a complete travel guide)
This sequential step ensures all information is properly integrated
This two-phase approach maximizes both efficiency and quality: you get the speed benefits of parallel processing for data gathering while maintaining coherent analysis through sequential aggregation. It's particularly powerful for research tasks, analysis workflows, and any scenario where you need to quickly gather diverse information and synthesize it into actionable insights.
Join the 1M+ learners on CodeSignal
Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal
Understanding Sync vs Async OpenAI Clients
When working with the OpenAI Responses API, you can choose between two client types: one for synchronous (step-by-step) operations and one for asynchronous (parallel) operations. The difference between them determines whether your program waits for each GPT response before moving on or whether it can send multiple requests at once.
With the standard OpenAI client, each API call is synchronous — your code waits for a response before continuing. This is simple but can be slow if you have many independent tasks.
from openai import OpenAI# Initialize the standard OpenAI clientclient = OpenAI()# Synchronous approach - each call waits for the previous one to finishresponse1 = client.responses.create(...) # Wait for this to completeresponse2 = client.responses.create(...) # Then wait for this to complete
In contrast, the AsyncOpenAI client supports asynchronous operations. This means you can start several GPT API calls at the same time, and your program will continue running while waiting for responses. This is ideal for running many independent tasks in parallel.
import asynciofrom openai import AsyncOpenAI# Initialize the async OpenAI clientclient = AsyncOpenAI()# Asynchronous approach - launch multiple requests concurrently (inside an async function)async def run_parallel(): tasks = [ client.responses.create(...), # Request A client.responses.create(...), # Request B ] # Run both at the same time; results are returned in the order of the tasks list response1, response2 = await asyncio.gather(*tasks)# Run the async functionasyncio.run(run_parallel())
In summary:
Use the synchronous client for simple, sequential workflows where each step depends on the previous one.
Use the asynchronous client when you want to launch multiple independent GPT API calls at once, dramatically improving performance for batch or parallel tasks.
Choosing the right client type is the key to optimizing your OpenAI workflows for both simplicity and speed.
AsyncIO Fundamentals for GPT Workflows
The asyncio library provides an event loop that manages multiple operations simultaneously, switching between them efficiently rather than blocking on any single operation. The async keyword transforms a regular function into a coroutine that can be paused and resumed, while await pauses execution until an asynchronous operation completes.
import asynciofrom openai import AsyncOpenAI# Initialize the async OpenAI clientclient = AsyncOpenAI()async def your_async_function(question): # The 'await' allows other tasks to run while waiting for the API response response = await client.responses.create( model="gpt-6-luna", instructions="You are a travel expert. Give brief, helpful answers.", input=[{"role": "user", "content": question}], reasoning={"effort": "none"}, store=False ) return response.output_text
This approach is particularly effective for I/O-bound operations like API calls, where much of the time is spent waiting for network responses. We set the reasoning effort to "none" to further optimize response times.
Running Async Code with asyncio.run()
To execute async functions, you need an event loop. asyncio.run() creates an event loop, runs your async function, and cleans up afterward. This is the standard entry point for async programs:
async def main(): # Call your async function and await the result result = await your_async_function("What are the top 3 must-see attractions in Paris?")# Run the async main function - this creates and manages the event loopif __name__ == "__main__": asyncio.run(main())
This pattern of wrapping your async code in a main() function and calling it with asyncio.run() is the standard approach for async programs. The asyncio.run() function handles all the event loop management automatically, making it the simplest way to execute async code.
Concurrent Execution with asyncio.gather()
The real power of async programming comes from running multiple operations concurrently. asyncio.gather() starts multiple coroutines simultaneously and waits for all of them to complete, returning results in the original order:
async def main(): # Create coroutine objects (these don't run yet) tasks = [ your_async_function("What are the top attractions in Paris?"), your_async_function("How do I get around Paris?"), your_async_function("What are French cultural norms?") ] # Execute all tasks concurrently and wait for all to complete # Note: Results are returned in the same order as the original tasks results = await asyncio.gather(*tasks)if __name__ == "__main__": asyncio.run(main())
The key insight: while one API call waits for GPT's response, the event loop can initiate or continue processing other API calls. This transforms sequential waiting time into concurrent execution time.
Creating Async Functions for GPT Calls
Now that you understand the fundamentals, let's build the foundation of our parallel workflow by creating an async function specifically designed for OpenAI API calls. This function will handle individual questions while being optimized for concurrent execution.
from openai import AsyncOpenAI# Initialize the async OpenAI clientclient = AsyncOpenAI()async def ask_question(question): """ Async function to ask GPT a single question """ print(f"🔄 Asking: {question}") # Send async request to GPT response = await client.responses.create( model="gpt-6-luna", instructions="You are a travel expert. Give brief, helpful answers.", input=[{"role": "user", "content": question}], reasoning={"effort": "none"}, store=False ) # Extract the answer answer = response.output_text print(f"✅ Answered: {question}") return question, answer
The print statements help visualize when each question starts and completes, while returning a tuple of (question, answer) makes it easy to match responses back to their original questions when processing parallel results. The instructions parameter ensures consistent, focused responses from GPT.
Preparing the List of Questions
With our async function ready, let's define the independent research questions that will form the parallel component of our workflow. Parallel processing shines when you have independent problems that don't rely on each other's answers:
# List of independent questions for Paris trip planningquestions = [ "What are the top 3 must-see attractions in Paris?", "What is the most efficient way to get around Paris as a tourist?", "What are important cultural etiquette tips for visitors to France?"]
These questions cover different aspects of travel planning (attractions, transportation, culture) and are completely independent of each other, making them perfect candidates for parallel execution.
Building Parallel Task Collections
Now let's put asyncio.gather() to work by creating multiple tasks that execute simultaneously. This is where the parallel magic happens:
async def main(): """ Execute parallel research and create comprehensive travel plan """ print(f"Starting {len(questions)} research questions in parallel...") # Create tasks for all questions tasks = [ask_question(question) for question in questions] # Execute all tasks in parallel results = await asyncio.gather(*tasks) print("\nResearch completed!")
The list comprehension creates coroutine objects representing work to be done, while asyncio.gather(*tasks) starts all coroutines simultaneously and returns results in the original order regardless of completion sequence. Each result is a tuple containing the question and its corresponding answer.
Aggregating Results for Final Analysis
With all our parallel research complete, let's build the aggregation phase that synthesizes everything into a comprehensive result. This sequential step ensures all information is properly integrated:
async def create_travel_plan(results): """ Aggregate all research results into a comprehensive travel plan """ # Combine all Q&A pairs into a formatted string combined_research = "" for question, answer in results: combined_research += f"{question}\n{answer}\n\n" # Create aggregator prompt aggregator_prompt = f"Create a brief Paris travel guide based on this research:\n\n{combined_research}\n\nMake it concise and practical." # Send to GPT for final synthesis response = await client.responses.create( model="gpt-6-luna", instructions="You are a travel planner. Create brief, useful guides.", input=[{"role": "user", "content": aggregator_prompt}], reasoning={"effort": "none"}, store=False ) return response.output_text
The aggregation step continues to use no reasoning effort since we're simply synthesizing already-researched information into a concise guide.
Running the Complete Parallel Workflow
Let's bring it all together into a complete workflow that demonstrates the full power of parallel processing followed by intelligent aggregation:
async def main(): """ Execute parallel research and create comprehensive travel plan """ print(f"Starting {len(questions)} research questions in parallel...") # Create tasks for all questions tasks = [ask_question(question) for question in questions] # Execute all tasks in parallel results = await asyncio.gather(*tasks) print("\nResearch completed!") # Aggregate results into final travel plan travel_plan = await create_travel_plan(results) print("\nParis Travel Guide:") print(travel_plan)# Run the parallel workflowif __name__ == "__main__": asyncio.run(main())
When you run this workflow, you'll see the power of parallel execution unfold in three distinct stages:
Instant Launch: All three "🔄 Asking" messages appear immediately as the API calls fire off simultaneously
Concurrent Completion: The "✅ Answered" messages arrive as GPT finishes each response — often in a different order than they were asked, proving your requests are truly running in parallel
Intelligent Synthesis: All this concurrent research gets woven together into a comprehensive travel guide that combines the speed benefits of parallel processing with thoughtful analysis
This visual progression clearly demonstrates how your requests execute concurrently rather than waiting for each other, transforming what could be a slow sequential process into a fast, efficient workflow that delivers both speed and quality.
This two-stage approach provides significant performance benefits while maintaining result quality. The parallel research phase completes in roughly the time of the slowest individual question, while the aggregation phase ensures all information is properly synthesized into a usable travel plan.
This pattern works well for any scenario where you need to:
Research multiple independent topics quickly
Aggregate diverse information into a unified result
Balance speed with comprehensive analysis
The performance benefits are most significant when you have many independent research topics or when individual API calls have high latency.
Summary & Practice Preparation
You've mastered parallel processing patterns that transform slow sequential workflows into lightning-fast concurrent operations. The combination of parallel research gathering and sequential result synthesis provides both speed and quality, making it ideal for complex analysis tasks like travel planning, market research, or technical evaluations.
In the upcoming exercises, you'll apply these patterns to real-world scenarios and learn to handle the nuances of concurrent GPT workflows. Remember: use parallel processing for independent research tasks, then aggregate results sequentially for comprehensive final analysis.
from openai import OpenAI# Initialize the standard OpenAI clientclient = OpenAI()# Synchronous approach - each call waits for the previous one to finishresponse1 = client.responses.create(...) # Wait for this to completeresponse2 = client.responses.create(...) # Then wait for this to complete
Python
import asynciofrom openai import AsyncOpenAI# Initialize the async OpenAI clientclient = AsyncOpenAI()# Asynchronous approach - launch multiple requests concurrently (inside an async function)async def run_parallel(): tasks = [ client.responses.create(...), # Request A client.responses.create(...), # Request B ] # Run both at the same time; results are returned in the order of the tasks list response1, response2 = await asyncio.gather(*tasks)# Run the async functionasyncio.run(run_parallel())
Python
import asynciofrom openai import AsyncOpenAI# Initialize the async OpenAI clientclient = AsyncOpenAI()async def your_async_function(question): # The 'await' allows other tasks to run while waiting for the API response response = await client.responses.create( model="gpt-6-luna", instructions="You are a travel expert. Give brief, helpful answers.", input=[{"role": "user", "content": question}], reasoning={"effort": "none"}, store=False ) return response.output_text
Python
async def main(): # Call your async function and await the result result = await your_async_function("What are the top 3 must-see attractions in Paris?")# Run the async main function - this creates and manages the event loopif __name__ == "__main__": asyncio.run(main())
Python
async def main(): # Create coroutine objects (these don't run yet) tasks = [ your_async_function("What are the top attractions in Paris?"), your_async_function("How do I get around Paris?"), your_async_function("What are French cultural norms?") ] # Execute all tasks concurrently and wait for all to complete # Note: Results are returned in the same order as the original tasks results = await asyncio.gather(*tasks)if __name__ == "__main__": asyncio.run(main())
Python
from openai import AsyncOpenAI# Initialize the async OpenAI clientclient = AsyncOpenAI()async def ask_question(question): """ Async function to ask GPT a single question """ print(f"🔄 Asking: {question}") # Send async request to GPT response = await client.responses.create( model="gpt-6-luna", instructions="You are a travel expert. Give brief, helpful answers.", input=[{"role": "user", "content": question}], reasoning={"effort": "none"}, store=False ) # Extract the answer answer = response.output_text print(f"✅ Answered: {question}") return question, answer
Python
# List of independent questions for Paris trip planningquestions = [ "What are the top 3 must-see attractions in Paris?", "What is the most efficient way to get around Paris as a tourist?", "What are important cultural etiquette tips for visitors to France?"]
Python
async def main(): """ Execute parallel research and create comprehensive travel plan """ print(f"Starting {len(questions)} research questions in parallel...") # Create tasks for all questions tasks = [ask_question(question) for question in questions] # Execute all tasks in parallel results = await asyncio.gather(*tasks) print("\nResearch completed!")
Python
async def create_travel_plan(results): """ Aggregate all research results into a comprehensive travel plan """ # Combine all Q&A pairs into a formatted string combined_research = "" for question, answer in results: combined_research += f"{question}\n{answer}\n\n" # Create aggregator prompt aggregator_prompt = f"Create a brief Paris travel guide based on this research:\n\n{combined_research}\n\nMake it concise and practical." # Send to GPT for final synthesis response = await client.responses.create( model="gpt-6-luna", instructions="You are a travel planner. Create brief, useful guides.", input=[{"role": "user", "content": aggregator_prompt}], reasoning={"effort": "none"}, store=False ) return response.output_text
Python
async def main(): """ Execute parallel research and create comprehensive travel plan """ print(f"Starting {len(questions)} research questions in parallel...") # Create tasks for all questions tasks = [ask_question(question) for question in questions] # Execute all tasks in parallel results = await asyncio.gather(*tasks) print("\nResearch completed!") # Aggregate results into final travel plan travel_plan = await create_travel_plan(results) print("\nParis Travel Guide:") print(travel_plan)# Run the parallel workflowif __name__ == "__main__": asyncio.run(main())