Introduction: Completing the Human-in-the-Loop Workflow
In the previous lesson, you completed Factor 6 by implementing pause and resume endpoints that give clients lifecycle control over agent workflows. Now, you'll implement Factor 7 — Contact humans with tool calls. As you learned when studying the 12-Factor Agents methodology, this factor treats human escalation as a first-class tool: when the agent lacks information, it doesn't guess or fail — it calls a structured ask_human tool, pauses itself, and waits for a response. The human's input is recorded in the context just like any other tool output, making the interaction auditable and reproducible. By the end of this lesson, you'll have a complete human-in-the-loop system where agents and users collaborate seamlessly to solve problems together.
Defining the ask_human Tool Schema
To allow the agent to request information from users, you need to create a new tool schema. The ask_human tool requires only one parameter: the question or prompt that the agent wants to present to the user. Create a new file at src/core/tools/schemas/ask_human.json with the following content:
JSON
{ "type": "function", "name": "ask_human", "description": "Ask the user for clarification or additional information.", "strict": true, "parameters": { "type": "object", "properties": { "question": { "type": "string", "description": "The question or prompt to ask the user" } }, "required": ["question"], "additionalProperties": false }}
This schema follows the same structure as your other tools, like sum_numbers or final_answer. The name field identifies the function as ask_human, while the description tells the language model when it should use this tool. The parameters section defines a single required field called question that holds the text the agent wants to show to the user. By keeping the schema simple with just one string parameter, you make it easy for the language model to formulate clear questions without worrying about complex argument structures.
Loading the ask_human Schema in the Agent
Now you need to load this schema and register it alongside your existing tools in the Agent constructor. Open src/core/agent.ts and look at how the toolSchemas property is built inside the constructor:
TypeScript
constructor(options: AgentOptions = {}) { this.model = options.model ?? "gpt-6-luna"; this.reasoningEffort = options.reasoningEffort ?? "low"; this.maxSteps = options.maxSteps ?? 10; const baseDir = dirname(fileURLToPath(import.meta.url)); this.systemPrompt = readFileSync(join(baseDir, "prompts", "base_system.md"), "utf8") + (options.extraInstructions ?? ""); this.toolSchemas = [ ...JSON.parse(readFileSync(join(baseDir, "tools", "schemas", "math.json"), "utf8")), JSON.parse(readFileSync(join(baseDir, "tools", "schemas", "final_answer.json"), "utf8")), // Include the ask_human schema so the model can escalate to a human (Factor 7) JSON.parse(readFileSync(join(baseDir, "tools", "schemas", "ask_human.json"), "utf8")) ] as Tool[];}
readFileSync reads the file contents synchronously at startup, and JSON.parse turns the raw JSON string into an object. The new ask_human schema is appended to this.toolSchemas after final_answer, making the tool visible to the language model during every agent step. The as Tool[] cast ensures TypeScript treats the parsed objects as valid OpenAI tool definitions.
Join the 1M+ learners on CodeSignal
Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal
Handling ask_human in the Agent's Step Logic
When the agent calls the ask_human tool, you need special tool dispatch logic that differs from regular tools. Regular tools execute immediately and return results, while final_answer terminates the workflow. The ask_human tool should pause the workflow and wait for external input before continuing. Inside the nextStep method, a new if branch handles this case:
if (functionCall.name === "ask_human") { // Human escalation: drop this call, pause the loop, and wait for input (Factor 7). // The ask_human function_call stays in context so the API can match the answer later. state.pending_tool_calls = state.pending_tool_calls.filter( (call) => call.call_id !== functionCall.call_id ); state.status = "waiting_human_input"; return state;}
When the agent detects an ask_human call, the handler performs three steps:
Removes the call from state.pending_tool_calls using .filter(...) to prevent reprocessing it later.
Sets state.status to waiting_human_input, which signals to the rest of your system that the agent is paused and waiting for user interaction.
Returns the state immediately without continuing the loop.
Unlike final_answer, which ends the workflow permanently, ask_human represents a temporary pause that will resume after the user provides input.
There is a critical detail to notice here: by the time the code reaches this if block, a state.context.push(...) call earlier in the nextStep loop has already appended the function_call entry — including its call_id and name — to the context. This means the ask_human call is preserved in the context even though it is removed from pending_tool_calls. This is essential because later, when the human responds, you will search through state.context to find this exact entry and extract its call_id, linking the human's answer back to the original question.
Creating the provide_input Endpoint
To accept human responses and resume agent execution, you need a new API endpoint. Rather than defining a separate request model class, Express lets you validate the incoming fields inline using the requireStringField helper already present in src/server/app.ts. The two fields you need are extracted at the top of the route handler:
const id = requireStringField(request.body, "id");const answer = requireStringField(request.body, "answer");
id identifies which agent state should receive the input, and answer carries the human's response to the agent's question. requireStringField throws an HttpError(400, ...) automatically if either field is missing or empty, so no additional validation boilerplate is needed before you begin working with the database.
Implementing the provide_input Route Handler
Now, implement the route handler that validates the request and prepares the state for resumption in src/server/app.ts:
app.post( "/agent/provide_input", asyncHandler(async (request, response) => { const id = requireStringField(request.body, "id"); const answer = requireStringField(request.body, "answer"); const current = store.get(id); if (!current) { throw new HttpError(404, "State not found"); } if (current.status !== "waiting_human_input") { throw new HttpError(400, `State is not waiting for human input. Current status: ${current.status}`); } // ... continued below }));
The handler uses store.get(id) to look up the state record, returning a 404HttpError if it does not exist. Before accepting the input, it checks that the state's current status is waiting_human_input. This prevents clients from accidentally sending answers to agents that are not actually waiting for them, which would corrupt the context with an unmatched tool output.
Extracting the call_id from the Context
Before you can add the human's answer to the agent's context, you need to determine which specific ask_human call the answer corresponds to. The language model requires this information because it tracks function calls and their outputs using unique call_id values. A module-level helper function at the bottom of src/server/app.ts searches backward through the context:
function getAskHumanCallId(state: State): string | null { for (const item of [...state.context].reverse()) { if ("type" in item && item.type === "function_call" && item.name === "ask_human") { return item.call_id; } } return null;}
The function creates a reversed copy of the context array and iterates through it. Because ContextItem is a discriminated union that includes both user messages (which only have role) and tool entries (which have type), the "type" in item check narrows the type before accessing item.type and item.name. When it finds a function_call entry whose name is ask_human, it returns the call_id immediately. If no matching call is found, the function returns null. Searching from the end of the context is both correct and efficient, since the most recent ask_human call is the one that triggered the current pause.
Using the call_id to Validate the Context
Back inside the provide_input route handler, you call this helper to retrieve the call_id needed for constructing the response:
const callId = getAskHumanCallId(current);if (!callId) { throw new HttpError(400, "Could not find ask_human call in state context");}
After extracting the call_id, the code validates that it exists. If getAskHumanCallId returns null, the endpoint throws a 400HttpError, indicating that the context is in an unexpected state — for example, if someone calls /agent/provide_input on a state that became waiting_human_input through some other mechanism without a corresponding ask_human function call in the context.
Constructing the Function Call Output and Securing the Lifecycle
With the call_id identified, you can construct the function_call_output that the language model expects and append it to the agent's context. TypeScript's ContextItem union type enforces the correct shape:
The humanResponse object links the output to the original ask_human call using the extracted call_id, and it wraps the human's text in a JSON string so the model receives a structured payload. Rather than mutating the existing state object, a new workingState is created using the spread operator with status set back to "running" and the updated context array. store.save(workingState) persists the change before the background task starts, ensuring the new context and status are durable even if the process restarts. void runAgentInBackground(id, workingState) then launches the agent loop in the background, and the endpoint returns the new working state immediately to the client.
Lifecycle guard on /agent/resume: Because the agent is now waiting for specific input linked to a call_id, a blind call to /agent/resume would allow the agent to continue without the human's answer, leaving the ask_human tool call permanently unmatched in the context. The /agent/resume handler already guards against this:
// A waiting agent must receive human input, not a blind resume (Factor 7)if (current.status === "waiting_human_input") { throw new HttpError(400, "Agent is waiting for human input");}
This means clients must always use /agent/provide_input to unblock a waiting agent, and /agent/resume will return a 400 error if called in this state.
This is a validation guard, not an atomic lifecycle transition; if multiple workers or clients can race, pair it with the same conditional update or locking approach discussed in the pause/resume lesson.
Testing the Launch and Waiting Phase
Testing the Lifecycle Guard
Before providing the answer, verify that the lifecycle guard on /agent/resume works. Because the agent is waiting for specific input linked to a call_id, a blind resume should be rejected. Add this check to your test script inside the if (currentState.status === "waiting_human_input") block:
if (currentState.status === "waiting_human_input") { // 3. Verify the lifecycle guard on /agent/resume console.log("Attempting blind resume (should fail with 400)..."); const resumeResponse = await fetch(`${baseUrl}/agent/resume`, { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ id: state.id }) }); console.log(`Resume response code: ${resumeResponse.status}`); if (resumeResponse.status === 400) { console.log("Correct! The API rejected a blind resume while waiting for input."); } // ... continued below}
Running this part of the script confirms the server correctly enforces the human-in-the-loop requirement:
Attempting blind resume (should fail with 400)...Resume response code: 400Correct! The API rejected a blind resume while waiting for input.
Providing Input and Verifying Completion
Summary
You've implemented Factor 7 — Contact humans with tool calls by building a complete human-in-the-loop system. The ask_human tool schema in ask_human.json, the provide_input endpoint in src/server/app.ts, and the getAskHumanCallId helper work together so that agents can request structured information from users, record the exchange in context like any other tool call, and resume seamlessly. The "type" in item narrowing pattern ensures type-safe context inspection across both the server helper and the test script. Combined with the launch, pause, and resume endpoints from earlier lessons (Factor 6), your API now supports rich, collaborative workflows between agents and humans.
if (functionCall.name === "ask_human") { // Human escalation: drop this call, pause the loop, and wait for input (Factor 7). // The ask_human function_call stays in context so the API can match the answer later. state.pending_tool_calls = state.pending_tool_calls.filter( (call) => call.call_id !== functionCall.call_id ); state.status = "waiting_human_input"; return state;}
TypeScript
const id = requireStringField(request.body, "id");const answer = requireStringField(request.body, "answer");
TypeScript
app.post( "/agent/provide_input", asyncHandler(async (request, response) => { const id = requireStringField(request.body, "id"); const answer = requireStringField(request.body, "answer"); const current = store.get(id); if (!current) { throw new HttpError(404, "State not found"); } if (current.status !== "waiting_human_input") { throw new HttpError(400, `State is not waiting for human input. Current status: ${current.status}`); } // ... continued below }));
TypeScript
function getAskHumanCallId(state: State): string | null { for (const item of [...state.context].reverse()) { if ("type" in item && item.type === "function_call" && item.name === "ask_human") { return item.call_id; } } return null;}
TypeScript
const callId = getAskHumanCallId(current);if (!callId) { throw new HttpError(400, "Could not find ask_human call in state context");}
// A waiting agent must receive human input, not a blind resume (Factor 7)if (current.status === "waiting_human_input") { throw new HttpError(400, "Agent is waiting for human input");}
To verify that the human-in-the-loop workflow functions correctly, create a test script that launches an agent with an incomplete prompt and polls until it enters the waiting_human_input state. The script uses top-level await and the State type imported from the core models:
TypeScript
import type { ContextItem, State } from "./src/core/models/state.js";const baseUrl = "http://localhost:8000";// 1. Launch a new agent that will likely ask for a missing value (radius/diameter)const launchResponse = await fetch(`${baseUrl}/agent/launch`, { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ input_prompt: "Can you figure out the area of a circle for me? Use pi = 3.14." })});const state = (await launchResponse.json()) as State;console.log(`Launched agent with ID: ${state.id}`);// 2. Poll until the agent waits for human input (or finishes on its own)let currentState = state;while (true) { const response = await fetch(`${baseUrl}/agent/state/${encodeURIComponent(state.id)}`); currentState = (await response.json()) as State; console.log(`Status: ${currentState.status}, Steps: ${currentState.steps}`); if (currentState.status === "waiting_human_input") { console.log(`Question: ${getAskHumanQuestion(currentState.context)}`); break; } if (["complete", "max_steps_reached", "failed"].includes(currentState.status)) { console.log(`Completed: ${currentState.final_answer}`); break; } await new Promise((resolve) => setTimeout(resolve, 1000));}function getAskHumanQuestion(context: ContextItem[]): string | null { for (const item of [...context].reverse()) { if ("type" in item && item.type === "function_call" && item.name === "ask_human") { const args = JSON.parse(item.arguments) as { question?: string }; return args.question ?? null; } } return null;}
Running this script produces output showing the agent launching and running until it needs human input:
text
Launched agent with ID: 6d60068d-eeee-4705-9dd7-b8381f095b31Status: running, Steps: 0Status: running, Steps: 0Status: running, Steps: 0Status: running, Steps: 0Status: waiting_human_input, Steps: 2Question: What is the circle's radius (or diameter)? Please provide one, and specify the units if you have them.
The output shows that the agent launched successfully and ran for two steps before calling ask_human to request the missing circle measurement. The question it formulated is clear and specific, asking for either radius/diameter. The getAskHumanQuestion helper mirrors the same "type" in item narrowing pattern used in getAskHumanCallId on the server, parsing the arguments string as JSON to extract the question field.
TypeScript
if (currentState.status === "waiting_human_input") { // 3. Verify the lifecycle guard on /agent/resume console.log("Attempting blind resume (should fail with 400)..."); const resumeResponse = await fetch(`${baseUrl}/agent/resume`, { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ id: state.id }) }); console.log(`Resume response code: ${resumeResponse.status}`); if (resumeResponse.status === 400) { console.log("Correct! The API rejected a blind resume while waiting for input."); } // ... continued below}
text
Attempting blind resume (should fail with 400)...Resume response code: 400Correct! The API rejected a blind resume while waiting for input.
Now, complete the test script by calling the /agent/provide_input endpoint with the missing information. This will unblock the agent, allowing it to resume its loop and reach a final answer:
TypeScript
// 4. Provide human input via /agent/provide_input await fetch(`${baseUrl}/agent/provide_input`, { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ id: state.id, answer: "The radius is 10 inches." }) }); console.log("Providing human input via /agent/provide_input..."); // 5. Poll for completion while (true) { const response = await fetch(`${baseUrl}/agent/state/${encodeURIComponent(state.id)}`); currentState = (await response.json()) as State; console.log(`Status: ${currentState.status}, Steps: ${currentState.steps}`); if (["complete", "max_steps_reached", "failed"].includes(currentState.status)) { console.log(`\nFinal answer: ${currentState.final_answer}`); break; } await new Promise((resolve) => setTimeout(resolve, 1000)); }
Running the full sequence produces output demonstrating the complete human-in-the-loop cycle:
text
Providing human input via /agent/provide_input...Status: running, Steps: 2Status: running, Steps: 2Status: running, Steps: 3Status: running, Steps: 4Status: complete, Steps: 6Final answer: The area is 314 square inches.
The agent resumes execution and reaches completion with the correct calculated area. If you inspect the final state's context, you will see the ask_human function call followed by the function_call_output containing the human's answer — both sharing the same call_id — providing a complete audit trail of the collaboration.
