Welcome back! In the previous lesson, you learned how to measure and interpret token usage in large language models (LLMs). That knowledge is important for understanding how efficiently a model processes information. In this lesson, we will focus on a different aspect of LLM behavior: the temperature parameter.
The temperature
setting is a key way to control how creative or predictable a model’s responses are. When you send a prompt to an LLM, the model can generate many possible outputs. The temperature
parameter lets you adjust how much randomness the model uses when picking its next word or phrase. A low temperature
makes the model more focused and deterministic, while a higher temperature
encourages more variety and creativity in the responses.
Understanding temperature
is important for benchmarking because it helps you see how the same model can behave differently depending on this setting. By the end of this lesson, you will know how to run simple experiments to observe these differences and interpret what they mean for your use case.
The temperature
parameter is a number, usually between 0 and 2, that you can set when generating text with an LLM. When the temperature
is set to 0
, the model will always pick the most likely next word, making its responses very predictable and consistent. As you increase the temperature
, the model becomes more willing to take risks and choose less likely words, which can make its responses more creative or surprising.
If you set the temperature
above 1
, the model becomes even more random and unpredictable. While the model is still valid and will generate output, the responses may start to lose coherence or relevance, as the model is more likely to select unusual or unexpected words. This can be useful for brainstorming or creative writing, but may not be suitable if you need reliable or factual answers.
On the other hand, setting the temperature
to a negative value is not valid. Most LLM APIs will return an error or ignore the setting if you try to use a negative temperature
. Always use a value of 0
or higher to ensure the model behaves as expected.
Let’s look at a practical example to see how temperature
affects model outputs. In this example, you will use the OpenAI
Python client to send the same prompt to the model three times, each with a different temperature
setting. The prompt asks the model to describe a completely fictional animal found in a magical forest.
Here is the code you will use:
In this code, you first define your prompt
and a list of three temperature
values: 0.0
, 0.7
, and 1.2
. For each temperature
, you send the prompt
to the model and print out the response. The only thing that changes between each run is the temperature
value.
When you run this code, you might see output like the following (your results may vary):
Notice how the response at temperature
0.0
is very straightforward and safe, while the response at 0.7
adds more creative details. At 1.2
, the model invents a completely new animal with imaginative features. This shows how increasing the temperature
leads to more diverse and creative outputs.
In this lesson, you learned what the temperature
parameter is and how it affects the behavior of large language models. You saw that low temperature
values make the model’s responses more predictable, while higher temperature
values encourage creativity and variety. You also worked through a code example that compared model outputs at different temperature
settings, helping you see these effects in action.
Next, you will get a chance to practice running your own temperature
experiments. Try different prompts and temperature
values to see how the model’s responses change. This hands-on practice will help you build intuition for when to use different temperature
settings in your own projects.
