Implementing Parallelization Patterns

Welcome back! You've mastered sequential workflows with prompt chaining and conditional workflows with intelligent routing. Now it's time to unlock dramatic performance improvements by learning parallel processing — executing multiple independent OpenAI API calls simultaneously instead of waiting for each one to complete.

In this lesson, you'll discover how to transform workflows that take minutes into operations that complete in seconds. You'll learn the difference between sequential and parallel execution, master Ruby's built-in Thread class, and build a system that asks multiple questions to GPT-5 at the same time.

Before we dive into the technical details, let's clarify terms that often confuse beginners: sequential and parallel.

In everyday language, "sequential" means things happen one after another — like waiting in a queue at a bakery, where each customer is served completely before the next one steps forward. In programming, sequential execution means your code runs one operation at a time: it starts a task, waits for it to finish, then starts the next one.

Parallel execution, on the other hand, means multiple tasks run at the same time — like a bakery where several customers are being served simultaneously by different staff members. In Ruby terms, this means launching multiple threads that each handle their own OpenAI API call independently, without waiting for the others.

For OpenAI API workflows:

• Sequential requests: Call GPT-5 for question 1, wait for the response, then call GPT-5 for question 2, wait, then question 3 — total time is the sum of all individual wait times.
• Threaded parallel requests: Launch all three GPT-5 calls at once using threads — total time is roughly equal to the slowest individual response, not the sum of all of them.

This distinction is the key insight of this lesson: when your tasks are independent of each other, running them in parallel with threads can dramatically cut your total execution time.

In Ruby, the standard OpenAI::Client handles requests one at a time when called sequentially — your code waits for GPT-5 to respond before moving on to the next call. This is simple and easy to reason about, but slow when you have multiple independent questions to ask.

Sequential approach — each request waits for the previous one to finish:

require "openai"

client = OpenAI::Client.new

# Sequential: question 2 cannot start until question 1 is done
response1 = client.responses.create(model: "gpt-5", input: [...])
response2 = client.responses.create(model: "gpt-5", input: [...])
response3 = client.responses.create(model: "gpt-5", input: [...])

If each request takes 3 seconds, the total runtime is 9 seconds.

Threaded parallel approach — all requests fire at the same time:

require "openai"

client = OpenAI::Client.new

# Parallel: all three requests start immediately
threads = [
  Thread.new { client.responses.create(model: "gpt-5", input: [...]) },
  Thread.new { client.responses.create(model: "gpt-5", input: [...]) },
  Thread.new { client.responses.create(model: "gpt-5", input: [...]) }
]

# Wait for all threads to finish and collect results in original order
results = threads.map(&:value)

If each request takes 3 seconds, the total runtime is roughly 3 seconds — because all three are running concurrently. The same OpenAI::Client is used in both cases; it's Ruby threads that enable the parallelism, not a different client type.

In summary:

• Use sequential calls for simple workflows where each step depends on the previous one.
• Use threads when you have multiple independent GPT-5 calls that can run simultaneously, dramatically improving performance for batch or parallel tasks.

Ruby's built-in Thread class lets you run code concurrently by creating lightweight execution contexts — each thread runs independently and can make its own API calls without blocking the others. This is particularly effective for I/O-bound operations like OpenAI API calls, where most of the time is spent waiting for a network response rather than doing computation.

Here are the three building blocks you'll use throughout this lesson:

Thread.new — starts a new thread immediately, running the block you pass to it:

thread = Thread.new do
  # This code runs concurrently with the rest of your program
  client.responses.create(model: "gpt-5", input: [...])
end

thread.value — waits for a thread to finish and returns the value from the last expression in its block. If the thread hasn't finished yet, value blocks until it does:

result = thread.value  # Waits for the thread to complete, then returns its result

threads.map(&:value) — the idiomatic Ruby way to wait for all threads in an array to finish and collect their results in the original order:

threads = [thread1, thread2, thread3]
results = threads.map(&:value)  # Returns [result1, result2, result3]

This is particularly powerful for OpenAI API calls. While one thread waits for GPT-5's response, the operating system can schedule other threads to do their own waiting — all API calls progress simultaneously. Crucially, results are always returned in the original order of the threads array, regardless of which thread finishes first.

Now let's build the foundation of our parallel workflow: a regular Ruby method that handles one GPT-5 question at a time. Because each thread gets its own call to this method, no special syntax is needed — it's just a plain Ruby method:

MODEL = "gpt-5"

def ask_question(client, question)
  puts "🔄 Asking: #{question}"

  response = client.responses.create(
    model: MODEL,
    input: [
      text_message("developer", "You are a travel expert. Give brief, helpful answers."),
      text_message("user", question)
    ],
    reasoning: { effort: "minimal" },
    store: false
  )

  answer = response.output_text
  puts "✅ Answered: #{question}"
  [question, answer]
end

A few things to notice:

• The method accepts client as a parameter so that the same client instance can be safely shared across multiple threads.
• The text_message helper (introduced in earlier lessons) builds the role-based typed content blocks the Responses API expects — a "developer" system prompt sets the persona and a "user" message carries the question.
• The puts statements help visualize when each question starts and finishes — especially useful for confirming that requests are truly running in parallel.
• The method returns [question, answer] as a two-element array, making it easy to match each answer back to its original question when processing the collected results.
• Keeping reasoning: { effort: "minimal" } minimizes individual response times, which amplifies the overall performance gain of running requests in parallel.

With our helper method ask_question ready, let's define the independent research questions that will power the parallel phase of our workflow. Parallel processing is most valuable when you have independent problems that don't rely on each other's answers — the perfect scenario for multi-topic travel research:

questions = [
  "What are the top 3 must-see attractions in Paris?",
  "What is the most efficient way to get around Paris as a tourist?",
  "What are important cultural etiquette tips for visitors to France?"
]

These three questions cover distinct aspects of travel planning (attractions, transportation, and culture) — and are completely independent of each other — none of the answers requires knowing another answer first. This independence is exactly what makes them ideal candidates for parallel execution with threads.

Now let's put threads to work. The pattern is straightforward: use questions.map to create a thread for each question, start each one with Thread.new, then collect all results with threads.map(&:value):

puts "Starting #{questions.length} research questions in parallel..."

threads = questions.map do |question|
  Thread.new do
    ask_question(client, question)
  end
end

results = threads.map(&:value)

Here's what happens step by step:

1. questions.map { Thread.new { ... } } — iterates over each question and immediately starts a new thread for it. By the time the map call finishes, all three threads are already running concurrently.
2. Thread.new { ask_question(client, question) } — each thread independently calls ask_question, which fires its own OpenAI API request. All three API calls are in-flight at the same time.
3. threads.map(&:value) — iterates over the threads array, calling .value on each one. This waits for each thread to finish and collects its return value. Results arrive in the same order as the original questions array, regardless of which API call finishes first.

The final results is an array of [question, answer] pairs, one for each question, ready to be passed to the aggregation step.

With all parallel research complete, the second phase synthesizes everything into a comprehensive result. This aggregation step is sequential by design — it needs all the research before it can produce a unified guide:

def create_travel_plan(client, results)
  combined_research = results.map { |question, answer| "#{question}\n#{answer}\n" }.join("\n")

  aggregator_prompt = <<~PROMPT
    Create a brief Paris travel guide based on this research:

    #{combined_research}

    Make it concise and practical.
  PROMPT

  response = client.responses.create(
    model: MODEL,
    input: [
      text_message("developer", "You are a travel planner. Create brief, useful guides."),
      text_message("user", aggregator_prompt)
    ],
    reasoning: { effort: "minimal" },
    store: false
  )

  response.output_text
end

Key points about this method:

• results.map { |question, answer| ... }.join("\n") — Ruby's array destructuring unpacks each [question, answer] pair cleanly, and join assembles them into a single formatted string for the prompt.
• The heredoc (<<~PROMPT) keeps the multi-line aggregator_prompt readable without messy string concatenation.
• text_message("developer", ...) sets the system-level persona for the aggregator — a travel planner rather than a travel expert — using the same role-based typed content format as previous lessons.
• This sequential API call ensures GPT-5 sees the complete picture before synthesizing a coherent, actionable travel guide.

Let's bring the full workflow together. The structure mirrors the two-phase pattern: parallel threads for research gathering, then a sequential call for synthesis:

# ---- Parallel execution using Ruby threads ----
puts "Starting #{questions.length} research questions in parallel..."

threads = questions.map do |question|
  Thread.new do
    ask_question(client, question)
  end
end

results = threads.map(&:value)

puts "\nResearch completed!"

travel_plan = create_travel_plan(client, results)
puts "\nParis Travel Guide:"
puts travel_plan

When you run this script, you'll see the two-phase workflow play out clearly:

1. Instant launch: All three "🔄 Asking" messages appear almost simultaneously, confirming every thread started right away.
2. Concurrent completion: The "✅ Answered" messages arrive as GPT-5 finishes each response — potentially in a different order than they were asked, proving the requests ran truly in parallel.
3. Sequential synthesis: Once threads.map(&:value) has collected all results, create_travel_plan sends a single aggregation request that weaves all the research into a cohesive guide.

Starting 3 research questions in parallel...
🔄 Asking: What are the top 3 must-see attractions in Paris?
🔄 Asking: What is the most efficient way to get around Paris as a tourist?
🔄 Asking: What are important cultural etiquette tips for visitors to France?
✅ Answered: What are the top 3 must-see attractions in Paris?
✅ Answered: What is the most efficient way to get around Paris as a tourist?
✅ Answered: What are important cultural etiquette tips for visitors to France?

Research completed!

Paris Travel Guide:
Paris in a snapshot: a 2–3 day guide
...

The total runtime is roughly the time of the slowest individual question — not the sum of all three — because they all run concurrently inside their own threads.

This two-phase approach delivers significant performance improvements while maintaining result quality. The parallel research phase completes in roughly the time of the slowest individual question, while the sequential aggregation phase ensures all information is properly synthesized into a usable travel plan.

This pattern works well for any scenario where you need to:

• Research multiple independent topics quickly
• Aggregate diverse information into a unified result
• Balance speed with comprehensive analysis

The performance benefits are most significant when you have many independent research topics or when individual API calls have high latency.

You've mastered parallel processing with Ruby threads — a pattern that transforms slow sequential API workflows into fast concurrent operations. By spinning up one thread per question with Thread.new and collecting all results with threads.map(&:value), you get the research speed of parallelism without any complex machinery.

The key takeaway is the two-phase structure:

• Parallel phase: Use threads to fire off independent GPT-5 calls simultaneously, reducing total wait time to roughly the duration of the slowest single request.
• Sequential synthesis phase: Once all threads complete, aggregate the results in a single GPT-5 call that weaves everything into a coherent, actionable output.

In the upcoming exercises, you'll apply these patterns to real-world scenarios and learn to handle the nuances of concurrent GPT-5 workflows. Remember: reach for Ruby threads whenever you have independent research tasks, then aggregate sequentially for a high-quality final result.