Exploring Model Parameters in PHP

Welcome back! In the previous lesson, you learned how to send a simple message to OpenAI's language model and receive a response. Now, we will take a step further by exploring model parameters that allow you to customize the AI's responses. These parameters are crucial for tailoring the chatbot's behavior to meet specific needs. In this lesson, we will focus on four key parameters: max_completion_tokens, verbosity, response_format, and streaming. Understanding these parameters will enable you to control the length, level of detail, structure, and delivery of the AI's responses, enhancing your chatbot's functionality.

In this course we use gpt-6-luna, a reasoning model: it thinks briefly before it answers. Reasoning models manage their own sampling, so older controls such as temperature, top_p, presence_penalty, and frequency_penalty are not accepted by this model (they remain available on classic chat models). The parameters in this lesson are the ones that work with the model we use.

The max_completion_tokens parameter sets a hard limit on the number of tokens the AI can generate in a response. A "token" can be a whole word or just part of a word. For example, "chatbot" might be one token, while "hello" could be split into two tokens: "hel" and "lo." It's important to note that token counts vary across different models, words, and languages — so the same text might have a different token count depending on these factors.

When you set max_completion_tokens, you specify the maximum number of tokens the AI can produce. For a reasoning model, this limit covers both the text you see and the model's internal reasoning. If the limit is too small, the model can spend all of its tokens thinking and return an empty answer, with a finish_reason of "length". For this reason, treat max_completion_tokens as a generous safety cap rather than a way to make answers shorter.

Here's an example where we set max_completion_tokens to 500:

$response = $client->chat()->create([
    'model' => 'gpt-6-luna',
    'messages' => [['role' => 'user', 'content' => $prompt]],
    'max_completion_tokens' => 500
]);

By setting max_completion_tokens to 500, you impose a hard limit on the number of tokens the AI can generate. The model isn't consciously summarizing or adjusting its content to fit within this limit; it simply stops when the limit is reached. Primarily, this parameter is valuable for managing usage rates and controlling the cost of API requests. You can check why a response ended by reading $response->choices[0]->finishReason: "stop" means the model finished naturally, while "length" means it hit the limit.

The verbosity parameter guides how detailed the AI's answer is. It accepts three values: 'low', 'medium', and 'high'. A 'low' value asks for a short, concise answer, while a 'high' value asks for a more thorough and detailed one. Unlike max_completion_tokens, verbosity doesn't cut the response off; it influences how much the model chooses to say, and it has no effect on how correct the answer is.

For example, consider the following code snippet where we set verbosity to 'low':

$response = $client->chat()->create([
    'model' => 'gpt-6-luna',
    'messages' => [['role' => 'user', 'content' => $prompt]],
    'max_completion_tokens' => 500,
    'verbosity' => 'low'
]);

With a verbosity of 'low', the AI gives a brief answer. Change it to 'high' and run the code again to see a more detailed explanation of the same question. Choosing the right level helps you match the answer to your use case, such as short replies for a quick customer service chat and detailed ones for a tutoring chatbot.

By default, the AI answers in free-form text. The response_format parameter lets you ask for a specific output format instead. When you set it to ['type' => 'json_object'], the AI replies with valid JSON, which is much easier for your code to process. When you use this setting, the API requires the word "JSON" to appear in your prompt.

Consider the following code where we request a JSON response:

$prompt = 'Explain how a rainbow forms. Reply as JSON with the keys "summary" and "colors".';

$response = $client->chat()->create([
    'model' => 'gpt-6-luna',
    'messages' => [['role' => 'user', 'content' => $prompt]],
    'max_completion_tokens' => 500,
    'response_format' => ['type' => 'json_object']
]);

Because the prompt asks for the keys summary and colors, the response is a JSON object with those keys, which your code could read with json_decode.

Example Response (the exact wording will differ):

{
  "summary": "A rainbow forms when sunlight is refracted, reflected, and dispersed by raindrops.",
  "colors": ["red", "orange", "yellow", "green", "blue", "indigo", "violet"]
}

Normally, the API waits until the whole answer is ready and sends it back at once. With streaming, the API instead sends the answer in small chunks as they are produced, so your program can start showing text right away. This makes a chatbot feel much more responsive, especially for longer answers. With the PHP client, you stream by calling createStreamed() instead of create().

Here is how to use it:

$stream = $client->chat()->createStreamed([
    'model' => 'gpt-6-luna',
    'messages' => [['role' => 'user', 'content' => $prompt]],
    'max_completion_tokens' => 500
]);

foreach ($stream as $response) {
    echo $response->choices[0]->delta->content ?? '';
}
echo PHP_EOL;

When you call createStreamed(), the client returns a stream instead of a single response. We loop through it with foreach and print the text of each chunk, found in $response->choices[0]->delta->content, as soon as it arrives.

Let's bring it all together with a complete code example that incorporates the parameters we've discussed:

<?php
require 'vendor/autoload.php';

// Fetch the api key
$apiKey = $_ENV['OPENAI_API_KEY'] ?? getenv('OPENAI_API_KEY');
$baseUrl = $_ENV['OPENAI_BASE_URL'] ?? getenv('OPENAI_BASE_URL');
$baseUrl = rtrim($baseUrl, '/');
if (!str_ends_with($baseUrl, '/v1')) {
    $baseUrl .= '/v1';
}

// Initialize the OpenAI client
$client = \OpenAI::factory()
    ->withApiKey($apiKey)
    ->withBaseUri($baseUrl)
    ->make();

// Define a simple user message to test the API
$prompt = "Explain how a rainbow forms. Reply as JSON with the keys \"summary\" and \"colors\".";

// Get a streamed response with specific parameters
$stream = $client->chat()->createStreamed([
    'model' => 'gpt-6-luna',
    'messages' => [['role' => 'user', 'content' => $prompt]],
    'max_completion_tokens' => 500,   // Caps the response length
    'verbosity' => 'high',            // Controls how detailed the response is
    'response_format' => ['type' => 'json_object']  // Requests a JSON response
]);

// Print each chunk of the response as it arrives
foreach ($stream as $response) {
    echo $response->choices[0]->delta->content ?? '';
}
echo PHP_EOL;
?>

In this example, we use all four parameters to customize the AI's response. By adjusting these parameters, you can fine-tune the chatbot's behavior to meet your specific requirements. When you run this code, the JSON response appears piece by piece as it is generated, with a level of detail set by verbosity.

In this lesson, we explored how to use model parameters to customize AI responses. You learned about the max_completion_tokens, verbosity, and response_format parameters, and about streaming with createStreamed(), and saw how they can be applied in code. These tools allow you to control the length, detail, structure, and delivery of the AI's responses, enhancing your chatbot's functionality. You also learned that the classic sampling controls (temperature, top_p, and the penalties) belong to older chat models, while reasoning models like gpt-6-luna manage their own sampling.

As you move on to the practice exercises, I encourage you to experiment with different parameter settings to see their effects firsthand. This hands-on practice will reinforce what you've learned and prepare you for the next unit, where we'll delve deeper into managing conversation history and message types. Keep up the great work, and enjoy the journey of creating your chatbot with OpenAI!