Integrating Human Input Back to Agents

Introduction: Completing the Human-in-the-Loop Workflow

In the previous lesson, you completed Factor 6 by implementing pause and resume endpoints that give clients lifecycle control over agent workflows . Now, you'll implement Factor 7 — Contact humans with tool calls . As you learned when studying the 12-Factor Agents methodology, this factor treats human escalation as a first-class tool: when the agent lacks information, it doesn't guess or fail — it calls a structured ask_human tool, pauses itself, and waits for a response. The human's input is recorded in the context just like any other tool output, making the interaction auditable and reproducible . By the end of this lesson, you'll have a complete human-in-the-loop system where agents and users collaborate seamlessly to solve problems together.

Defining the ask_human Tool Schema

To allow the agent to request information from users, you need to create a new tool schema. The ask_human tool requires only one parameter: the question or prompt that the agent wants to present to the user. Create a new file at src/core/tools/schemas/ask_human.json with the following content: JSON{ "type": "function", "name": "ask_human", "description": "Ask the user for clarification or additional information.", "strict": true, "parameters": { "type": "object", "properties": { "question": { "type": "string", "description": "The question or prompt to ask the user" } }, "required": ["question"], "additionalProperties": false } } This schema follows the same structure as your other tools, like sum_numbers or final_answer. The name field identifies the function as ask_human, while the description tells the language model when it should use this tool. The parameters section defines a single required field called question that holds the text the agent wants to show to the user. By keeping the schema simple with just one string parameter, you make it easy for the language model to formulate clear questions without worrying about complex argument structures.

Loading the ask_human Schema in the Agent

Now you need to load this schema and register it alongside your existing tools in the Agent constructor. Open src/core/agent.ts and look at how the toolSchemas property is built inside the constructor: TypeScriptconstructor(options: AgentOptions = {}) { this.model = options.model ?? "gpt-5"; this.reasoningEffort = options.reasoningEffort ?? "low"; this.maxSteps = options.maxSteps ?? 10; const baseDir = dirname(fileURLToPath(import.meta.url)); this.systemPrompt = readFileSync(join(baseDir, "prompts", "base_system.md"), "utf8") + (options.extraInstructions ?? ""); this.toolSchemas = [ ...JSON.parse(readFileSync(join(baseDir, "tools", "schemas", "math.json"), "utf8")), JSON.parse(readFileSync(join(baseDir, "tools", "schemas", "final_answer.json"), "utf8")), // Include the ask_human schema so the model can escalate to a human (Factor 7) JSON.parse(readFileSync(join(baseDir, "tools", "schemas", "ask_human.json"), "utf8")) ] as Tool[]; } readFileSync reads the file contents synchronously at startup, and JSON.parse turns the raw JSON string into an object. The new ask_human schema is appended to this.toolSchemas after final_answer, making the tool visible to the language model during every agent step. The as Tool[] cast ensures TypeScript treats the parsed objects as valid OpenAI tool definitions.

Join the 1M+ learners on CodeSignal

Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal

Handling ask_human in the Agent's Step Logic

When the agent calls the ask_human tool, you need special tool dispatch logic that differs from regular tools. Regular tools execute immediately and return results, while final_answer terminates the workflow. The ask_human tool should pause the workflow and wait for external input before continuing. Inside the nextStep method, a new if branch handles this case:

TypeScript

if (functionCall.name === "ask_human") {
  // Human escalation: drop this call, pause the loop, and wait for input (Factor 7).
  // The ask_human function_call stays in context so the API can match the answer later.
  state.pending_tool_calls = state.pending_tool_calls.filter(
    (call) => call.call_id !== functionCall.call_id
  );
  state.status = "waiting_human_input";
  return state;
}

if (functionCall.name === "ask_human") {
  // Human escalation: drop this call, pause the loop, and wait for input (Factor 7).
  // The ask_human function_call stays in context so the API can match the answer later.
  state.pending_tool_calls = state.pending_tool_calls.filter(
    (call) => call.call_id !== functionCall.call_id
  );
  state.status = "waiting_human_input";
  return state;
}

When the agent detects an ask_human call, the handler performs three steps:

Removes the call from state.pending_tool_calls using .filter(...) to prevent reprocessing it later.
Sets state.status to waiting_human_input, which signals to the rest of your system that the agent is paused and waiting for user interaction.
Returns the state immediately without continuing the loop.

Unlike final_answer, which ends the workflow permanently, ask_human represents a temporary pause that will resume after the user provides input.

There is a critical detail to notice here: by the time the code reaches this if block, a state.context.push(...) call earlier in the nextStep loop has already appended the function_call entry — including its call_id and name — to the context. This means the ask_human call is preserved in the context even though it is removed from pending_tool_calls. This is essential because later, when the human responds, you will search through state.context to find this exact entry and extract its call_id, linking the human's answer back to the original question.

Constructing the Function Call Output and Securing the Lifecycle

With the call_id identified, you can construct the function_call_output that the language model expects and append it to the agent's context. TypeScript's ContextItem union type enforces the correct shape:

TypeScript

const humanResponse: ContextItem = {
  type: "function_call_output",
  call_id: callId,
  output: JSON.stringify({ answer })
};

const workingState: State = {
  ...current,
  status: "running",
  context: [...current.context, humanResponse]
};

store.save(workingState);
void runAgentInBackground(id, workingState);

response.json(workingState);

const humanResponse: ContextItem = {
  type: "function_call_output",
  call_id: callId,
  output: JSON.stringify({ answer })
};

const workingState: State = {
  ...current,
  status: "running",
  context: [...current.context, humanResponse]
};

store.save(workingState);
void runAgentInBackground(id, workingState);

response.json(workingState);

The humanResponse object links the output to the original ask_human call using the extracted call_id, and it wraps the human's text in a JSON string so the model receives a structured payload. Rather than mutating the existing state object, a new workingState is created using the spread operator with status set back to "running" and the updated context array. store.save(workingState) persists the change before the background task starts, ensuring the new context and status are durable even if the process restarts. void runAgentInBackground(id, workingState) then launches the agent loop in the background, and the endpoint returns the new working state immediately to the client.

Lifecycle guard on /agent/resume: Because the agent is now waiting for specific input linked to a call_id, a blind call to /agent/resume would allow the agent to continue without the human's answer, leaving the ask_human tool call permanently unmatched in the context. The /agent/resume handler already guards against this:

TypeScript

// A waiting agent must receive human input, not a blind resume (Factor 7)
if (current.status === "waiting_human_input") {
  throw new HttpError(400, "Agent is waiting for human input");
}

// A waiting agent must receive human input, not a blind resume (Factor 7)
if (current.status === "waiting_human_input") {
  throw new HttpError(400, "Agent is waiting for human input");
}

This means clients must always use /agent/provide_input to unblock a waiting agent, and /agent/resume will return a 400 error if called in this state. This is a validation guard, not an atomic lifecycle transition; if multiple workers or clients can race, pair it with the same conditional update or locking approach discussed in the pause/resume lesson.

Testing the Launch and Waiting Phase

To verify that the human-in-the-loop workflow functions correctly, create a test script that launches an agent with an incomplete prompt and polls until it enters the waiting_human_input state. The script uses top-level await and the State type imported from the core models:

TypeScript

import type { ContextItem, State } from "./src/core/models/state.js";

const baseUrl = "http://localhost:8000";

// 1. Launch a new agent that will likely ask for a missing value (radius/diameter)
const launchResponse = await fetch(`${baseUrl}/agent/launch`, {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({ input_prompt: "Can you figure out the area of a circle for me? Use pi = 3.14." })
});
const state = (await launchResponse.json()) as State;
console.log(`Launched agent with ID: ${state.id}`);

// 2. Poll until the agent waits for human input (or finishes on its own)
let currentState = state;
while (true) {
  const response = await fetch(`${baseUrl}/agent/state/${encodeURIComponent(state.id)}`);
  currentState = (await response.json()) as State;
  console.log(`Status: ${currentState.status}, Steps: ${currentState.steps}`);

  if (currentState.status === "waiting_human_input") {
    console.log(`Question: ${getAskHumanQuestion(currentState.context)}`);
    break;
  }

  if (["complete", "max_steps_reached", "failed"].includes(currentState.status)) {
    console.log(`Completed: ${currentState.final_answer}`);
    break;
  }

  await new Promise((resolve) => setTimeout(resolve, 1000));
}

function getAskHumanQuestion(context: ContextItem[]): string | null {
  for (const item of [...context].reverse()) {
    if ("type" in item && item.type === "function_call" && item.name === "ask_human") {
      const args = JSON.parse(item.arguments) as { question?: string };
      return args.question ?? null;
    }
  }
  return null;
}

Running this script produces output showing the agent launching and running until it needs human input:

Launched agent with ID: 6d60068d-eeee-4705-9dd7-b8381f095b31
Status: running, Steps: 0
Status: running, Steps: 0
Status: running, Steps: 0
Status: running, Steps: 0
Status: waiting_human_input, Steps: 2
Question: What is the circle's radius (or diameter)? Please provide one, and specify the units if you have them.

The output shows that the agent launched successfully and ran for two steps before calling ask_human to request the missing circle measurement. The question it formulated is clear and specific, asking for either radius/diameter. The getAskHumanQuestion helper mirrors the same "type" in item narrowing pattern used in getAskHumanCallId on the server, parsing the arguments string as JSON to extract the question field.

Introduction: Completing the Human-in-the-Loop Workflow

Defining the ask_human Tool Schema

Loading the ask_human Schema in the Agent

Join the 1M+ learners on CodeSignal

Handling ask_human in the Agent's Step Logic

Creating the provide_input Endpoint

Implementing the provide_input Route Handler

Extracting the call_id from the Context

Using the call_id to Validate the Context

Constructing the Function Call Output and Securing the Lifecycle

Testing the Launch and Waiting Phase

Testing the Lifecycle Guard

Providing Input and Verifying Completion

Summary