Exploring Completion Parameters

Welcome! Today, we’ll explore a key part of working with language models: completion parameters. If you want to control how creative, factual, short, or long an AI’s response is, this lesson is for you. Our goal is to understand what completion parameters are, why they matter, and how to use them to shape AI model responses. By the end, you’ll know how to experiment with these parameters in your own code.

When you use the OpenAI API, you send a prompt and get a response. But you can also control how the model responds using completion parameters. Completion parameters are settings that influence the model’s output. Think of them as dials to adjust the model’s behavior. The most common are:

• temperature: Controls randomness and creativity.
• max_tokens: Limits response length.
• top_p: Another way to control randomness.
• n: Number of completions to generate.

We’ll focus on temperature and max_tokens — the two most important for shaping responses, whether you want concise answers or creative stories.

The temperature parameter controls how creative the model’s output is. It ranges from 0 to 2, with 0.0 being very predictable and 1.0 being more creative.

• Low temperature (e.g., 0.2): Output is focused and predictable.
• High temperature (e.g., 1.0): Output is more diverse and creative.

A low temperature is best for reliable, factual, or consistent answers, such as summaries or Q&A. A high temperature is ideal for creative, varied, or brainstorming responses, like stories or idea generation. Adjusting this parameter lets you control the model’s creativity to fit your needs.

If you set temperature to an extreme value, such as 2.0, the model’s responses can become highly random, sometimes to the point of being nonsensical or off-topic. This is rarely useful in practice, but it can be interesting for experimentation or generating unexpected ideas. On the other hand, setting temperature to 0.0 makes the model as deterministic as possible, always returning the most likely next word.

Here’s a simple example:

from openai import OpenAI

client = OpenAI()
prompt = "Write a short story about a robot learning to paint."

# Low temperature: predictable
response_low = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": prompt}],
    max_tokens=60,
    temperature=0.2
)
print("Low temperature:\n", response_low.choices[0].message.content.strip())

# High temperature: creative
response_high = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": prompt}],
    max_tokens=60,
    temperature=1.0
)
print("\nHigh temperature:\n", response_high.choices[0].message.content.strip())

Run this code and notice: a low temperature gives a straightforward story, while a high temperature adds creative twists.

The max_tokens parameter sets the maximum length of the model’s response. A token is about a word or a few characters. Use max_tokens to keep responses short or allow longer outputs. For example, to get a brief answer, set max_tokens=20. For more detail, use max_tokens=100 or higher.

If you set max_tokens to 0, the model will not generate any output at all. This is generally not useful, but it’s good to know that setting this parameter too low can prevent the model from responding. Conversely, setting max_tokens very high allows for long, detailed responses, but may use more resources and take longer to generate.

from openai import OpenAI

client = OpenAI()
prompt = "Explain how a bicycle works."

# Short response
response_short = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": prompt}],
    max_tokens=20,
    temperature=0.5
)
print("Short response:\n", response_short.choices[0].message.content.strip())

# Longer response
response_long = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": prompt}],
    max_tokens=100,
    temperature=0.5
)
print("\nLonger response:\n", response_long.choices[0].message.content.strip())

The first response is concise; the second is more detailed. This is useful for fitting responses into a specific space or controlling verbosity.

Let’s combine these ideas with a code example that lets you try different temperature values and see the results.

from openai import OpenAI

def main():
    client = OpenAI()
    prompt = "Write a short story about a robot learning to paint."
    print("Prompt:", prompt)

    for temp in [0.2, 0.7, 1.0]:
        print(f"\n--- Temperature: {temp} ---")
        response = client.chat.completions.create(
            model="gpt-4o",
            messages=[{"role": "user", "content": prompt}],
            max_tokens=60,
            temperature=temp
        )
        if response and hasattr(response, "choices") and len(response.choices) > 0:
            print(response.choices[0].message.content.strip())
        else:
            print("No completion returned.")

if __name__ == "__main__":
    main()

Output example:

Prompt: Write a short story about a robot learning to paint.

--- Temperature: 0.2 ---
Artie the robot wanted to paint. He learned about colors and brushes, practicing every day. Soon, Artie created beautiful paintings, amazing everyone with his new talent.

--- Temperature: 0.7 ---
In a small town, a robot named Hue discovered painting. At first, Hue’s lines were shaky, but with practice, he painted vibrant sunsets and joyful scenes. People came from afar to see Hue’s art, inspired by his journey.

--- Temperature: 1.0 ---
Pixel, a curious robot, found a paintbrush and splashed colors everywhere—on walls, on canvases, even on himself! Each painting was wild and new, and soon, Pixel’s art became the talk of the city, inspiring robots and humans alike.

This code loops through three temperature values and prints the model’s response for each. Try running it and see how the story changes. A higher temperature means more creative stories. You can also change max_tokens to see how it affects response length. Experimenting like this helps you master completion parameters.

You learned what completion parameters are and why they matter. We focused on temperature (controls creativity) and max_tokens (controls length). You saw how to use them in code and how they affect the model’s output. Adjusting these lets you tailor AI responses for your needs, from concise answers to creative stories.

Now it’s your turn! Next, you’ll get hands-on practice with completion parameters. You’ll adjust temperature and max_tokens to see their effects. This will help you confidently control AI behavior in your own projects. Let’s begin!