Building Autonomous Gemini Agents

Introduction & Overview

Throughout this course, you've learned the fundamentals of integrating tools with Gemini : defining tool schemas, understanding Gemini's tool use responses, and executing single tool requests. However, the approach you've used so far is limited to handling only one tool call per conversation turn. While this is sufficient for simple tasks, many real-world problems require multiple sequential steps, and often the number and nature of these steps can't be determined in advance. In this lesson, we'll work together to transform Gemini from a single-turn tool user into an autonomous agent capable of iterative problem-solving. We'll build an agent class that can call tools, analyze results, decide what to do next, and continue this process until complex multi-step tasks are completed. This marks a shift from reactive tool usage to proactive, intelligent problem-solving — mirroring how humans approach complex challenges.

The Action-Feedback Loop Concept

Before we start coding, let's understand how autonomous agents operate through action-feedback loops . In this pattern, each tool execution provides information that influences the next decision. This iterative process mirrors human problem-solving: we take an action, observe the result, decide what to do next, and repeat until we reach our goal. The action-feedback loop consists of four key phases that repeat until task completion: Decision Phase: Gemini analyzes the current situation and determines the next action, which may include calling one or more tools. Action Phase: Our agent executes the requested tool(s) based on Gemini's instructions. Feedback Phase: The results from the tool execution(s) are captured and added to the conversation history. Evaluation Phase: Gemini reviews the new information, decides whether the task is complete or if additional steps are needed, and the loop continues. This loop structure enables complex problem-solving because each iteration builds upon previous results. For example, when solving a quadratic equation, Gemini might first calculate the discriminant, then use that result to determine if real solutions exist, then calculate the square root of the discriminant, and finally compute the two solutions. The key insight is that Gemini doesn't need to plan all steps in advance — it can adapt its approach based on intermediate results, just like a human mathematician working through a problem. Now, let's start building our agent class to make this iterative process possible.

Join the 1M+ learners on CodeSignal

Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal

Building Our Agent Class Foundation

Let's begin by creating the foundation of our autonomous agent using Gemini. We'll use the Google Generative AI Python SDK and configure it to use the "models/gemini-flash-latest" model. We'll also set up tool integration using Gemini's tool-calling API.

Here's the initial class definition and constructor:

Python

from google import genai
from google.genai import types

class Agent:
    BASE_SYSTEM_PROMPT = (
        "You are an autonomous agent that can take multiple tool-calling steps when helpful. "
        "The user only sees your response when you stop using tools, not your tool usage or reasoning steps. "
        "When you provide your answer without calling tools, make it complete and standalone.\n"
        "Additional instructions:\n"
    )

    def __init__(
        self,
        name,
        system_prompt="You are a helpful assistant.",
        model="models/gemini-flash-latest",
        tools=None,
        tool_schemas=None,
        max_turns=10,
        api_key=None
    ):
        client_kwargs = {}
        if api_key is not None:
            client_kwargs["api_key"] = api_key
        self.client = genai.Client(**client_kwargs) if client_kwargs else genai.Client()
        self.name = name
        self.model = model
        self.system_prompt = self.BASE_SYSTEM_PROMPT + system_prompt
        self.max_turns = max_turns

        # Avoid shared mutable defaults and protect against external mutation
        self.tools = {} if tools is None else dict(tools)                # name -> Python function
        self.tool_schemas = [] if tool_schemas is None else list(tool_schemas)  # list of tool schemas

from google import genai
from google.genai import types

class Agent:
    BASE_SYSTEM_PROMPT = (
        "You are an autonomous agent that can take multiple tool-calling steps when helpful. "
        "The user only sees your response when you stop using tools, not your tool usage or reasoning steps. "
        "When you provide your answer without calling tools, make it complete and standalone.\n"
        "Additional instructions:\n"
    )

    def __init__(
        self,
        name,
        system_prompt="You are a helpful assistant.",
        model="models/gemini-flash-latest",
        tools=None,
        tool_schemas=None,
        max_turns=10,
        api_key=None
    ):
        client_kwargs = {}
        if api_key is not None:
            client_kwargs["api_key"] = api_key
        self.client = genai.Client(**client_kwargs) if client_kwargs else genai.Client()
        self.name = name
        self.model = model
        self.system_prompt = self.BASE_SYSTEM_PROMPT + system_prompt
        self.max_turns = max_turns

        # Avoid shared mutable defaults and protect against external mutation
        self.tools = {} if tools is None else dict(tools)                # name -> Python function
        self.tool_schemas = [] if tool_schemas is None else list(tool_schemas)  # list of tool schemas

Key design decisions:

BASE_SYSTEM_PROMPT: Instructs Gemini to use tools as needed and only show the final answer to the user.
Constructor parameters:
- name: Identifier for the agent.
- system_prompt: Customizable for different domains.
- model: Specifies the Gemini model.
- tools and tool_schemas: Passed as dictionaries/lists and copied to avoid shared state.
- max_turns: Prevents infinite loops.
- api_key: Allows explicit configuration of the API key, though the SDK will automatically look for the GOOGLE_API_KEY environment variable if not provided.

This structure prepares us to implement the core functionality for autonomous, multi-step tool use with Gemini.

Complete Run Method

Here's the complete run method for our Gemini agent:

Python

def run(self, input_messages):
    messages = [dict(m) for m in input_messages]
    turn = 0

    while turn < self.max_turns:
        turn += 1

        response = self.gemini_generate(**self._build_request_args(messages))

        candidate = response.candidates[0]
        content = candidate.content

        messages.append({
            "role": "model",
            "parts": content.parts
        })

        # Detect tool calls
        tool_calls = [
            part for part in content.parts
            if getattr(part, "function_call", None)
        ]

        if tool_calls:
            tool_results = []  # list of function_response parts
            for tool_call in tool_calls:
                tool_result = self.call_tool(tool_call)
                tool_results.append(tool_result)

            messages.append({
                "role": "user",
                "parts": tool_results
            })
            continue
        else:
            response_text = self._extract_text(content.parts)
            return messages, response_text

    raise Exception("Max turns reached")

def run(self, input_messages):
    messages = [dict(m) for m in input_messages]
    turn = 0

    while turn < self.max_turns:
        turn += 1

        response = self.gemini_generate(**self._build_request_args(messages))

        candidate = response.candidates[0]
        content = candidate.content

        messages.append({
            "role": "model",
            "parts": content.parts
        })

        # Detect tool calls
        tool_calls = [
            part for part in content.parts
            if getattr(part, "function_call", None)
        ]

        if tool_calls:
            tool_results = []  # list of function_response parts
            for tool_call in tool_calls:
                tool_result = self.call_tool(tool_call)
                tool_results.append(tool_result)

            messages.append({
                "role": "user",
                "parts": tool_results
            })
            continue
        else:
            response_text = self._extract_text(content.parts)
            return messages, response_text

    raise Exception("Max turns reached")

Testing Our Autonomous Agent

Let's test our Gemini-based agent on a complex quadratic equation. We'll use math tools and schemas as before. Ensure you have your GOOGLE_API_KEY set as an environment variable so the SDK can access it securely.

Python

import json
import os
from agent import Agent
from functions import sum_numbers, multiply_numbers, subtract_numbers, divide_numbers, power, square_root

# Load the schemas from JSON file
with open('schemas.json', 'r') as f:
    tool_schemas = json.load(f)

tools = {
    "sum_numbers": sum_numbers,
    "multiply_numbers": multiply_numbers,
    "subtract_numbers": subtract_numbers,
    "divide_numbers": divide_numbers,
    "power": power,
    "square_root": square_root
}

# Create the Gemini agent 
# The SDK automatically uses the GOOGLE_API_KEY environment variable
agent = Agent(
    name="math_assistant",
    system_prompt="You are a helpful math assistant.",
    tools=tools,
    tool_schemas=tool_schemas,
    max_turns=15
)

# Initialize conversation with user message
messages = [{
    "role": "user",
    "parts": [{"text": "Solve this equation: 2x² - 7x + 3 = 0"}]
}]

# Run the agent
messages, result = agent.run(messages)

# Display the response
print("\nFinal response:")
print(result)

import json
import os
from agent import Agent
from functions import sum_numbers, multiply_numbers, subtract_numbers, divide_numbers, power, square_root

# Load the schemas from JSON file
with open('schemas.json', 'r') as f:
    tool_schemas = json.load(f)

tools = {
    "sum_numbers": sum_numbers,
    "multiply_numbers": multiply_numbers,
    "subtract_numbers": subtract_numbers,
    "divide_numbers": divide_numbers,
    "power": power,
    "square_root": square_root
}

# Create the Gemini agent 
# The SDK automatically uses the GOOGLE_API_KEY environment variable
agent = Agent(
    name="math_assistant",
    system_prompt="You are a helpful math assistant.",
    tools=tools,
    tool_schemas=tool_schemas,
    max_turns=15
)

# Initialize conversation with user message
messages = [{
    "role": "user",
    "parts": [{"text": "Solve this equation: 2x² - 7x + 3 = 0"}]
}]

# Run the agent
messages, result = agent.run(messages)

# Display the response
print("\nFinal response:")
print(result)

Sample output:

text

🔧 Tool called: power({'base': -7, 'exponent': 2})
🔧 Tool called: multiply_numbers({'a': 4, 'b': 2})
🔧 Tool called: multiply_numbers({'a': 8, 'b': 3})
🔧 Tool called: subtract_numbers({'a': 49, 'b': 24})
🔧 Tool called: square_root({'number': 25})
🔧 Tool called: multiply_numbers({'a': 2, 'b': 2})
🔧 Tool called: sum_numbers({'a': 7, 'b': 5})
🔧 Tool called: divide_numbers({'a': 12, 'b': 4})
🔧 Tool called: subtract_numbers({'a': 7, 'b': 5})
🔧 Tool called: divide_numbers({'a': 2, 'b': 4})

Final response:
The solutions to the equation 2x² - 7x + 3 = 0 are:

**x = 3** and **x = 0.5** (or x = 1/2)

You can verify these solutions by substituting back into the original equation:
- For x = 3: 2(3)² - 7(3) + 3 = 18 - 21 + 3 = 0 ✓
- For x = 0.5: 2(0.5)² - 7(0.5) + 3 = 0.5 - 3.5 + 3 = 0 ✓

🔧 Tool called: power({'base': -7, 'exponent': 2})
🔧 Tool called: multiply_numbers({'a': 4, 'b': 2})
🔧 Tool called: multiply_numbers({'a': 8, 'b': 3})
🔧 Tool called: subtract_numbers({'a': 49, 'b': 24})
🔧 Tool called: square_root({'number': 25})
🔧 Tool called: multiply_numbers({'a': 2, 'b': 2})
🔧 Tool called: sum_numbers({'a': 7, 'b': 5})
🔧 Tool called: divide_numbers({'a': 12, 'b': 4})
🔧 Tool called: subtract_numbers({'a': 7, 'b': 5})
🔧 Tool called: divide_numbers({'a': 2, 'b': 4})

Final response:
The solutions to the equation 2x² - 7x + 3 = 0 are:

**x = 3** and **x = 0.5** (or x = 1/2)

You can verify these solutions by substituting back into the original equation:
- For x = 3: 2(3)² - 7(3) + 3 = 18 - 21 + 3 = 0 ✓
- For x = 0.5: 2(0.5)² - 7(0.5) + 3 = 0.5 - 3.5 + 3 = 0 ✓

Our Gemini agent systematically applies the quadratic formula, making multiple tool calls and building upon previous results, just like a human mathematician.

Introduction & Overview

The Action-Feedback Loop Concept

Join the 1M+ learners on CodeSignal

Building Our Agent Class Foundation

Adding Helper Methods for State Management

Implementing Tool Execution

Building the Core Loop - Part 1: Understanding Stateless Design

Building the Core Loop - Part 2: Setting Up the Iteration

Building the Core Loop - Part 3: Handling Tool Calls

Building the Core Loop - Part 4: Managing Flow Control

Complete Run Method

Testing Our Autonomous Agent

Summary & Practice Preparation