Configuring Bedrock Responses

Introduction

Welcome back to your second lesson in Basics of GenAI Foundation Models with Amazon Bedrock! Having successfully established your first connection with Bedrock and witnessed the power of foundation models in action, you're now ready to take your AI interactions to the next level. In this lesson, we'll explore the sophisticated configuration options that allow you to fine-tune how AI models behave and respond to your requests. As you may recall from our previous lesson, we sent a basic message to Claude and received a comprehensive response about AWS Bedrock itself. While that interaction was impressive, we were essentially using the model's default settings. Today, we'll discover how to take control of the AI's behavior through inference parameters and system prompts, transforming you from a passive consumer of AI responses into an active director of AI behavior. These configuration tools are what separate basic AI usage from professional-grade applications that deliver consistent, reliable results.

Understanding Model Configuration Parameters

Before we dive into the code, let's build intuition around the key parameters that control how foundation models generate responses. Think of these parameters as dials on a sophisticated audio mixing board: each one controls a different aspect of the output, and adjusting them changes the character and quality of what you receive. The two most critical parameters are temperature and top-p, which work together to control the creativity and consistency of responses. temperature acts like a creativity dial: lower values produce more focused, predictable responses that stay close to the most likely answers, while higher values introduce more randomness and creative variation. top-p, on the other hand, controls vocabulary diversity by limiting which words the model considers at each step, effectively determining how adventurous the model gets with its word choices. A third important parameter is maxTokens, which simply limits how long the response can be. This isn't just about saving costs; it's about controlling the scope and depth of responses to match your specific needs, whether you want a brief summary or a detailed explanation.

Configuring System Prompts

Beyond controlling the randomness and length of responses, we can fundamentally shape how the AI behaves through system prompts . These are special instructions that define the AI's role, personality, and approach to answering questions. Think of a system prompt as hiring instructions for a new employee: it tells the AI what job it's being asked to do and how it should approach that work. Let's see how we define a system prompt that creates a specialized AWS technical assistant: Python # Define the system prompt that sets the AI's role and behavior system_prompt = "You are an AWS Technical Assistant. Provide clear, accurate information about AWS services." This system_prompt transforms our general-purpose AI model into a specialized technical consultant. The AI will now approach every question through the lens of an AWS expert, focusing on providing accurate, technical information rather than general knowledge. System prompts are incredibly powerful because they persist throughout the entire conversation, influencing every response the AI generates.

Join the 1M+ learners on CodeSignal

Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal

Now let's configure the inference parameters that will control the AI's response characteristics. These parameters are grouped together in an inferenceConfig dictionary that we'll pass to the Converse API:

Python

# Call Bedrock model with configurable inference parameters
response = client.converse(
    modelId=MODEL_ID,
    messages=messages,
    system=[{"text": system_prompt}],
    inferenceConfig={
        "maxTokens": 256,    # Limit response to 256 tokens
        "temperature": 0.2,  # Low temperature for more focused responses
        "topP": 0.9,         # Top-p sampling for response diversity
    },
)

# Call Bedrock model with configurable inference parameters
response = client.converse(
    modelId=MODEL_ID,
    messages=messages,
    system=[{"text": system_prompt}],
    inferenceConfig={
        "maxTokens": 256,    # Limit response to 256 tokens
        "temperature": 0.2,  # Low temperature for more focused responses
        "topP": 0.9,         # Top-p sampling for response diversity
    },
)

Here, we're setting conservative values that prioritize accuracy and consistency over creativity. The temperature of 0.2 keeps responses focused and deterministic, perfect for technical information where accuracy matters more than creativity. The top-p of 0.9 allows some vocabulary flexibility while maintaining precision. The maxTokens limit of 256 ensures we get concise, focused answers rather than lengthy explanations. Notice how the system prompt is passed as a list containing a dictionary with a "text" key — this structure allows Bedrock to support multiple system instructions if needed in the future.

For this lesson, we'll ask the AI to explain the very parameters we're configuring, creating an educational loop where the AI teaches us about its own configuration. Let's start by setting up our foundation components:

Python

import boto3

# Create a Bedrock Runtime client
client = boto3.client("bedrock-runtime")

# Define the model ID
MODEL_ID = "us.anthropic.claude-sonnet-5"

import boto3

# Create a Bedrock Runtime client
client = boto3.client("bedrock-runtime")

# Define the model ID
MODEL_ID = "us.anthropic.claude-sonnet-5"

This setup should look familiar from our previous lesson, but now we're preparing to use these foundational elements with much more sophisticated configuration options. Next, we'll craft our user message that leverages the system prompt by asking for technical guidance in a specific domain:

Python

# Create the user message asking about model configuration parameters
messages = [
    {
        "role": "user",
        "content": [
            {
                "text": (
                    "Explain temperature vs top-p briefly and suggest safe defaults for AWS technical Q&A."
                )
            }
        ],
    },
]

# Create the user message asking about model configuration parameters
messages = [
    {
        "role": "user",
        "content": [
            {
                "text": (
                    "Explain temperature vs top-p briefly and suggest safe defaults for AWS technical Q&A."
                )
            }
        ],
    },
]

The AI will approach this question as an AWS technical assistant, providing practical recommendations rather than just theoretical explanations.

Processing and Understanding the Response

With all our configuration pieces in place, let's handle the API call and process the response. The response processing remains similar to our previous lesson, but now we're working with an AI that has been specifically configured for our use case:

Python

try:
    # Extract and combine text content from the Bedrock response
    output_message = response.get("output", {}).get("message", {})
    parts = output_message.get("content", [])
    text_chunks = [part.get("text", "") for part in parts if isinstance(part, dict)]
    full_response = "".join(text_chunks).strip()

    # Print the AI's response
    print(full_response)
    
except Exception as e:
    print(f"Error: {e}")

try:
    # Extract and combine text content from the Bedrock response
    output_message = response.get("output", {}).get("message", {})
    parts = output_message.get("content", [])
    text_chunks = [part.get("text", "") for part in parts if isinstance(part, dict)]
    full_response = "".join(text_chunks).strip()

    # Print the AI's response
    print(full_response)
    
except Exception as e:
    print(f"Error: {e}")

The error handling remains crucial because we're now making more complex API calls with additional parameters that could potentially cause issues. Our response processing safely extracts the text content from the structured response format that Bedrock returns. When we run our configured model with the technical system prompt and conservative parameters, we receive a highly focused, practical response that demonstrates the power of our configuration choices.

When we run our complete code with the AWS technical assistant system prompt and conservative parameters, here's the actual response we receive:

text

## Temperature vs Top-p

**Temperature (0.0-1.0):**
- Controls randomness/creativity in responses
- Lower = more focused, deterministic answers
- Higher = more creative, varied responses

**Top-p (0.0-1.0):**
- Controls diversity by limiting token selection to top probability mass
- Lower = more conservative word choices
- Higher = considers more possible words

## Safe Defaults for AWS Technical Q&A

**Recommended settings:**
- **Temperature: 0.1-0.3** - Ensures consistent, accurate technical information
- **Top-p: 0.8-0.9** - Allows some flexibility while maintaining precision

**Why these ranges:**
- AWS technical content requires accuracy and consistency
- Low temperature prevents hallucination of service features or incorrect configurations
- Moderate top-p maintains natural language flow while staying factual
- These settings balance helpfulness with reliability for technical documentation and troubleshooting

For critical production guidance, consider **temperature: 0.1, top-p: 0.8** for maximum consistency.

## Temperature vs Top-p

**Temperature (0.0-1.0):**
- Controls randomness/creativity in responses
- Lower = more focused, deterministic answers
- Higher = more creative, varied responses

**Top-p (0.0-1.0):**
- Controls diversity by limiting token selection to top probability mass
- Lower = more conservative word choices
- Higher = considers more possible words

## Safe Defaults for AWS Technical Q&A

**Recommended settings:**
- **Temperature: 0.1-0.3** - Ensures consistent, accurate technical information
- **Top-p: 0.8-0.9** - Allows some flexibility while maintaining precision

**Why these ranges:**
- AWS technical content requires accuracy and consistency
- Low temperature prevents hallucination of service features or incorrect configurations
- Moderate top-p maintains natural language flow while staying factual
- These settings balance helpfulness with reliability for technical documentation and troubleshooting

For critical production guidance, consider **temperature: 0.1, top-p: 0.8** for maximum consistency.

Notice how our configuration choices directly influenced this response: the system prompt made the AI focus on AWS-specific technical guidance, the low temperature produced a structured, consistent format, and the token limit kept the response concise yet comprehensive. This demonstrates the power of thoughtful parameter configuration in creating reliable, professional AI interactions.

Excellent work! You've successfully learned how to configure Amazon Bedrock models using inference parameters and system prompts to create specialized, reliable AI assistants. These configuration skills transform you from a basic Bedrock user into someone who can craft AI interactions for specific professional needs, giving you the power to control the creativity, consistency, and expertise level of AI responses.

The combination of technical system prompts and conservative parameter settings creates AI assistants that provide reliable, consistent guidance — exactly what you need for production applications and technical documentation. In our upcoming practice exercises, you'll experiment with different parameter combinations and system prompts, gaining hands-on experience with the full range of configuration options that make Bedrock such a powerful platform for professional AI applications!

Introduction

Understanding Model Configuration Parameters

Configuring System Prompts

Join the 1M+ learners on CodeSignal

Managing Inference Parameters

Building the Complete Configuration Request

Processing and Understanding the Response

Conclusion and Next Steps