Exploring Model Parameters
Exploring Model Parameters
Welcome back! In the previous lesson, you learned how to send a simple message to OpenAI's language model and receive a response using Go. Now, we will take a step further by exploring model parameters that allow you to customize the AI's responses. These parameters are crucial for tailoring the chatbot's behavior to meet specific needs. In this lesson, we will focus on four key parameters: MaxCompletionTokens, verbosity, ResponseFormat, and streaming. Understanding these parameters will enable you to control the length, level of detail, structure, and delivery of the AI's responses, enhancing your chatbot's functionality.
In this course we use gpt-6-luna, a reasoning model: it thinks briefly before it answers. Reasoning models manage their own sampling, so older controls such as Temperature, TopP, PresencePenalty, and FrequencyPenalty are not accepted by this model (they remain available on classic chat models). The parameters in this lesson are the ones that work with the model we use.
Controlling Response Length with Max Completion Tokens
The MaxCompletionTokens parameter sets a hard limit on the number of tokens the AI can generate in a response. A "token" can be a whole word or just part of a word. For example, "chatbot" might be one token, while "hello" could be split into two tokens: "hel" and "lo." It's important to note that token counts vary across different models, words, and languages — so the same text might have a different token count depending on these factors.
For a reasoning model, this limit covers both the text you see and the model's internal reasoning. If the limit is too small, the model can spend all of its tokens thinking and return an empty answer, with a FinishReason of "length". For this reason, treat MaxCompletionTokens as a generous safety cap rather than a way to make answers shorter.
Here's an example where we set MaxCompletionTokens to 500:
By setting MaxCompletionTokens to 500, you limit the AI's response to a maximum of 500 tokens. The model is not explicitly aware of the limit while generating text, so a response can be cut off if it reaches it. The main use of this parameter is to manage usage and control the cost of requests. You can check why a response ended by reading resp.Choices[0].FinishReason: "stop" means the model finished naturally, while "length" means it hit the limit.
Controlling Detail with Verbosity
The verbosity parameter guides how detailed the AI's answer is. It accepts three values: "low", "medium", and "high". A "low" value asks for a short, concise answer, while a "high" value asks for a more thorough and detailed one. Unlike MaxCompletionTokens, verbosity doesn't cut the response off; it influences how much the model chooses to say, and it has no effect on how correct the answer is.
The openai-go version used in this course has no dedicated field for verbosity yet. You can still send it by passing option.WithJSONSet("verbosity", "low") as an extra argument to the New call. WithJSONSet adds any field you name to the request exactly as you write it, so check the name and value carefully: a misspelled field name is rejected by the API.
Here's an example where we set verbosity to "low":
With a verbosity of "low", the AI is asked for a brief answer. Change it to "high" and run the code again to compare the two answers. Choosing the right level helps you match the answer to your use case, such as short replies for a quick customer service chat and detailed ones for a tutoring chatbot.
Requesting Structured Output with Response Format
By default, the AI answers in free-form text. The ResponseFormat parameter lets you ask for a specific output format instead. When you set it to a JSON object, the AI replies with valid JSON, which is much easier for your code to process. In Go, you do this with openai.ChatCompletionNewParamsResponseFormatUnion, setting its OfJSONObject field to &shared.ResponseFormatJSONObjectParam{} from the shared package. When you use this setting, the API requires the word "JSON" to appear in your prompt.
Consider the following code where we request a JSON response:
Because the prompt asks for the keys summary and colors, the response is a JSON object with those keys, which your code could read with encoding/json.
Example Response (the exact wording will differ):
Streaming Responses as They Are Generated
Normally, the API waits until the whole answer is ready and sends it back at once. With streaming, the API instead sends the answer in small chunks as they are produced, so your program can start showing text right away. This makes a chatbot feel much more responsive, especially for longer answers. With the openai-go client, you stream by calling NewStreaming instead of New.
Here is how to use it:
When you call NewStreaming, the client returns a stream instead of a single response. We loop through it with stream.Next() and stream.Current() and print the text of each chunk, found in chunk.Choices[0].Delta.Content, as soon as it arrives. After the loop, stream.Err() tells you whether anything went wrong.
Example: Implementing Model Parameters in Code
Let's bring it all together with a complete code example that incorporates the parameters we've discussed:
In this example, we use all four parameters to customize the AI's response. By adjusting these parameters, you can fine-tune the chatbot's behavior to meet your specific requirements. When you run this code, the JSON response appears piece by piece as it is generated, with a level of detail set by verbosity.
Summary and Preparation for Practice
In this lesson, we explored how to use model parameters to customize AI responses. You learned about the MaxCompletionTokens, verbosity, and ResponseFormat parameters, and about streaming with NewStreaming, and saw how they can be applied in code. These tools allow you to control the length, detail, structure, and delivery of the AI's responses, enhancing your chatbot's functionality. You also learned that the classic sampling controls (Temperature, TopP, and the penalties) belong to older chat models, while reasoning models like gpt-6-luna manage their own sampling.
As you move on to the practice exercises, I encourage you to experiment with different parameter settings to see their effects firsthand. This hands-on practice will reinforce what you've learned and prepare you for the next unit, where we'll delve deeper into managing conversation history and message types. Keep up the great work, and enjoy the journey of creating your chatbot with OpenAI!
