Parallel Agent Orchestration

Welcome back! In the previous lessons, you built a concurrent Ruby agent system that can:

1. Run multiple conversations at the same time.
2. Execute multiple tool calls concurrently inside a single agent turn.

Now we will combine those ideas into a higher-level architecture: parallel agent orchestration.

In this pattern, one orchestrator agent receives a complex user request, breaks it into independent subtasks, delegates those subtasks to specialized agents, and then synthesizes the results. The delegated agents are wrapped as normal OpenAI function tools, so the orchestrator can call them just like any other tool.

The key principle for this unit is:

Agent tools should return final, JSON-serializable values. The Agent#run method owns concurrency by running same-turn tool calls in Ruby threads.

That means a wrapped agent tool should call the delegated agent, wait for its final response, and return that response. It should not return raw concurrency primitives like Thread objects.

Agent orchestration is the pattern of using one coordinator agent to manage work across one or more specialized agents.

Think of it like a project manager. If a user asks for a report comparing the technology and manufacturing industries, the project manager does not need to research everything personally. Instead, it can split the work:

• Ask a researcher agent to investigate technology.
• Ask a researcher agent to investigate manufacturing.
• Compare the two returned summaries.
• Produce a final synthesized report.

Each delegated task is independent, so the model may emit multiple delegated tool calls in one response turn. If that happens, the Agent#run method executes those tool calls concurrently using Ruby threads.

This is the same tool-level parallelism you learned in Unit 2, but now the tools are more powerful: calling one tool can trigger a complete agent run.

First, we create a specialized researcher agent. It has one job: use a search tool to gather information, then summarize what it found.

def mock_search(query:)
  puts "🔎 Searching for: #{query}"
  sleep(1)
  "Result for #{query}: Data point XYZ"
end

The sleep(1) simulates slow network I/O. This makes concurrency easier to observe when multiple searches happen close together.

Next, we define the search tool schema:

search_schema = {
  "type" => "function",
  "name" => "search",
  "description" => "Search for information on a topic.",
  "parameters" => {
    "type" => "object",
    "properties" => {
      "query" => {
        "type" => "string",
        "description" => "The search query"
      }
    },
    "required" => ["query"],
    "additionalProperties" => false
  }
}

Then we create the researcher:

researcher = Agent.new(
  name: "researcher",
  system_prompt: (
    "You are a research assistant. Use the search tool to gather facts, then summarize your findings. " \
    "When several independent searches are useful, call the relevant search tools in the same turn when possible."
  ),
  tools: { "search" => method(:mock_search) },
  tool_schemas: [search_schema],
  max_turns: 15
)

The researcher does not need to know anything about orchestration. It only needs to be good at its focused role: searching and summarizing.

To let another agent call the researcher, we wrap it as a function tool using create_agent_tool.

def create_agent_tool(agent, description)
  tool_function = lambda do |message:|
    puts "🦾 Delegating to #{agent.name}..."

    _history, response = agent.run([text_message("user", message)])
    response
  end

  tool_schema = {
    "type" => "function",
    "name" => "#{agent.name}_tool",
    "description" => description,
    "parameters" => {
      "type" => "object",
      "properties" => {
        "message" => {
          "type" => "string",
          "description" => "The message to send to the agent"
        }
      },
      "required" => ["message"],
      "additionalProperties" => false
    }
  }

  [tool_function, tool_schema]
end

This helper returns two things:

1. tool_function — the Ruby callable that runs the delegated agent.
2. tool_schema — the OpenAI function tool schema the orchestrator can see.

The tool function is intentionally synchronous:

_history, response = agent.run([text_message("user", message)])
response

It blocks until the delegated agent finishes, then returns the final response text.

That may sound like it prevents concurrency, but it does not. The outer Agent#run method executes multiple tool calls in separate Ruby threads. Each tool should block until its own final value is ready, while the framework handles running multiple tools at once.

A tool should return a final result that can be serialized into the function_call_output message:

{
  type: "function_call_output",
  call_id: call_id,
  output: JSON.generate(result: result)
}

Good tool return values include:

• Strings
• Numbers
• Booleans
• Arrays
• Hashes
• Other JSON-serializable data

A tool should not return raw concurrency primitives such as:

• Thread
• Queue
• Mutex
• Sockets
• File handles

For this course, the tool contract is simple:

A tool may perform slow work internally, but it should return the final value, not the object used to perform the work.

This keeps responsibilities clear:

• The tool does the work and returns the final result.
• The agent framework runs multiple tools concurrently when the model emits multiple tool calls in the same turn.

Now we can wrap the researcher and give that wrapper to a manager agent:

research_fn, research_schema = create_agent_tool(
  researcher,
  "Delegate a focused research task to the researcher agent. Returns summarized findings."
)

The returned schema has a generated name based on the agent:

"name" => "researcher_tool"

Now we create the manager:

manager = Agent.new(
  name: "manager",
  system_prompt: (
    "You are a project manager. For multi-faceted reports, break the work into independent research tasks. " \
    "When possible, emit multiple researcher_tool calls in the same turn for independent facets, then synthesize " \
    "the returned research into one cohesive answer."
  ),
  tools: { research_schema["name"] => research_fn },
  tool_schemas: [research_schema],
  max_turns: 15
)

The manager has exactly one tool: researcher_tool.

From the model's perspective, this looks like a normal function. From Ruby's perspective, that function starts a full researcher.run(...) call and returns the researcher's final answer.

Parallel delegation depends on the model emitting multiple tool calls in the same turn.

If the model emits one tool call, Agent#run executes one tool call.

If the model emits multiple tool calls in the same response turn, Agent#run executes them concurrently:

tool_threads = tool_calls.map do |function_call|
  Thread.new do
    call_tool(function_call)
  end
end

function_outputs.concat(tool_threads.map(&:value))

This means the orchestrator prompt can encourage same-turn parallel delegation:

"When possible, emit multiple researcher_tool calls in the same turn for independent facets..."

But the runtime behavior is:

If multiple researcher_tool calls appear in one turn, they run concurrently.

This distinction is important because the model decides which tool calls to emit. The framework controls how same-turn tool calls are executed.

We send the manager a multi-faceted question:

prompt = "Compare the economic outlook of the technology industry vs. the manufacturing industry for 2024."

_history, final_report = manager.run([text_message("user", prompt)])

The initial message uses the same structured text_message helper used throughout the course:

def text_message(role, text)
  {
    role: role,
    content: [
      { type: "input_text", text: text }
    ]
  }
end

The manager may break the request into independent research tasks. If it emits both delegated calls in the same turn, each call to researcher_tool runs in its own Ruby thread.

When the manager emits multiple delegated calls in the same turn, you may see output like this:

🔧 [manager] Tool called: researcher_tool({"message"=>"Research the 2024 economic outlook for the global technology industry..."})
🦾 Delegating to researcher...
🔧 [manager] Tool called: researcher_tool({"message"=>"Research the 2024 economic outlook for the global manufacturing industry..."})
🦾 Delegating to researcher...

The two delegation messages appear close together because both researcher_tool calls are being executed by the outer agent's tool threads.

Each delegated researcher may then call its own tools:

🔧 [researcher] Tool called: search({"query"=>"2024 global technology industry outlook AI cloud semiconductors"})
🔎 Searching for: 2024 global technology industry outlook AI cloud semiconductors
🔧 [researcher] Tool called: search({"query"=>"2024 global manufacturing industry outlook PMI reshoring industrial production"})
🔎 Searching for: 2024 global manufacturing industry outlook PMI reshoring industrial production

This demonstrates nested concurrency:

• The manager can run multiple delegated agent tools concurrently.
• Each delegated researcher can run multiple search tools concurrently if the model emits same-turn search calls.

The output order may vary between runs because Ruby thread scheduling and model tool-call choices can vary.

A tempting mistake is to make the agent tool itself return a Thread:

tool_function = lambda do |message:|
  Thread.new do
    _history, response = agent.run([text_message("user", message)])
    response
  end
end

This starts background work, but it breaks the tool contract.

The outer Agent#run already runs tool calls in threads. If the tool returns a raw Thread, the outer tool runner treats that Thread object as the tool result. It does not automatically know that the thread's future value is the real result.

So instead of sending the delegated agent's response back to the model, the agent may serialize the thread object itself:

#<Thread:0x0000000108ad4c88 ...>

The correct pattern is to keep the tool synchronous:

tool_function = lambda do |message:|
  _history, response = agent.run([text_message("user", message)])
  response
end

Let Agent#run handle parallel execution across multiple tool calls. Do not leak raw Thread objects into tool results.

You have learned how to build a parallel agent orchestration system in Ruby. The core pattern is:

1. Create specialized agents for focused work.
2. Wrap those agents as synchronous tools using create_agent_tool.
3. Give those tools to an orchestrator agent.
4. Encourage the model to emit multiple tool calls in the same turn for independent subtasks.
5. Let Agent#run execute same-turn tool calls concurrently using Ruby threads.
6. Keep tool return values final and JSON-serializable.

This architecture gives you modularity and concurrency at the same time. Specialized agents stay focused, the orchestrator handles coordination, and the framework handles parallel execution.