Welcome back! In the previous lesson, you learned how to send a simple message to OpenAI's language model and receive a response. Now, we will take a step further by exploring model parameters that allow you to customize the AI's responses. These parameters are crucial for tailoring the chatbot's behavior to meet specific needs. In this lesson, we will focus on four key parameters: max_tokens, temperature, presence_penalty, and frequency_penalty. Understanding these parameters will enable you to control the creativity, length, and content of the AI's responses, enhancing your chatbot's functionality.

The max_tokens parameter sets a hard limit on the number of tokens the AI can generate in its response. A "token" can be a whole word or just part of a word. For example, "chatbot" might be one token, while "hello" could be split into two tokens: "hel" and "lo." It's important to note that token counts vary across different models, words, and languages — so the same text might have a different token count depending on these factors.

When you set max_tokens, you specify the maximum number of tokens the AI can produce. This is a strict limit, meaning the model will stop generating text once it reaches this count, even if it results in an incomplete sentence or word.

Here's an example where we set max_tokens to 100:

import 'dotenv/config';
import OpenAI from 'openai';

const openai = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY,
});

async function getResponse(prompt) {
  const response = await openai.chat.completions.create({
    model: "gpt-4",
    messages: [{ role: "user", content: prompt }],
    max_tokens: 100
  });
  console.log("Assistant:", response.choices[0].message.content.trim());
}

getResponse("Describe a sunset over the ocean");

By setting max_tokens to 100, you impose a hard limit on the number of tokens the AI can generate in its response. This may result in responses being abruptly cut off if the model hasn't completed its intended thought. Importantly, the max_tokens parameter doesn't make the model inherently more concise or brief — it simply restricts response length. The model isn't consciously summarizing or adjusting content to fit within this limit; rather, it continues generating text until reaching the token limit. Primarily, this parameter is valuable for managing usage rates and controlling the cost of API requests.

Example Response:

Assistant: The sun dips below the horizon, casting a warm glow across the sky. The ocean reflects hues of orange, pink, and purple, creating a breathtaking view. Waves gently lap against the shore, adding a soothing soundtrack to the visual spectacle. As the sun disappears, the sky darkens, and stars begin to twinkle, marking the end of a beautiful day.

The temperature parameter is a fascinating aspect of AI interaction. It controls the randomness or creativity of the AI's responses. A lower temperature value, such as 0.2, makes the AI's output more deterministic and focused, often resulting in more predictable responses. Conversely, a higher temperature value, like 0.8, encourages the AI to generate more diverse and creative responses, which can be useful for tasks requiring imaginative outputs.

For example, consider the following code snippet where we set the temperature to 0.7:

async function getResponseWithTemperature(prompt) {
  const response = await openai.chat.completions.create({
    model: "gpt-4",
    messages: [{ role: "user", content: prompt }],
    temperature: 0.7
  });
  console.log("Assistant:", response.choices[0].message.content.trim());
}

getResponseWithTemperature("Describe a sunset over the ocean");

With a temperature of 0.7, the AI is likely to provide a response that balances creativity and coherence. Experimenting with different temperature values will help you find the right balance for your specific use case.

Example Response:

Assistant: As the sun sets over the ocean, it paints the sky with a palette of vibrant colors. The horizon glows with shades of orange and pink, while the water shimmers with golden reflections. Seagulls glide gracefully across the sky, silhouetted against the fading light. The gentle sound of waves lapping against the shore creates a serene atmosphere, inviting you to pause and appreciate the beauty of nature's daily masterpiece.

The presence_penalty parameter is a powerful tool for encouraging the AI to introduce new topics in its responses. It works by penalizing the AI for using words that have already appeared in the conversation, thus promoting diversity in the dialogue. A low presence_penalty value, such as 0.0, means the AI is less discouraged from repeating words, leading to more focused responses. In contrast, a high presence_penalty value, like 1.0, strongly encourages the AI to explore new topics, resulting in more varied and diverse responses.

Consider the following code where we set the presence_penalty to 0.6:

async function getResponseWithPresencePenalty(prompt) {
  const response = await openai.chat.completions.create({
    model: "gpt-4",
    messages: [{ role: "user", content: prompt }],
    presence_penalty: 0.6
  });
  console.log("Assistant:", response.choices[0].message.content.trim());
}

getResponseWithPresencePenalty("Describe a sunset over the ocean");

With a presence_penalty of 0.6, the AI is more likely to explore new topics and provide varied responses. This can be particularly useful in scenarios where you want to keep the conversation fresh and engaging.

Example Response:

Assistant: The sun's descent over the ocean is a mesmerizing sight. As it sinks lower, the sky transforms into a canvas of fiery oranges and deep purples. The ocean mirrors this spectacle, its surface glistening with the sun's final rays. In the distance, a sailboat drifts lazily, adding a touch of adventure to the tranquil scene. The air is filled with the scent of salt and the gentle rustle of palm trees swaying in the evening breeze.

The frequency_penalty parameter helps reduce repetition in the AI's responses by penalizing the AI for using the same words or phrases multiple times within its response. This encourages more varied and interesting outputs. A low frequency_penalty value, such as 0.0, allows for more repetition, which can be useful for maintaining focus on a specific topic. Conversely, a high frequency_penalty value, like 1.0, discourages repetition, promoting a more dynamic and varied dialogue.

While presence_penalty encourages the AI to introduce new topics by penalizing the use of words that have already appeared in the conversation, frequency_penalty focuses on reducing repetition by penalizing the AI for using the same words or phrases multiple times within its response. This distinction allows you to manage both the diversity of topics and the variety of language in the AI's outputs.

In the following example, we set the frequency_penalty to 0.3:

async function getResponseWithFrequencyPenalty(prompt) {
  const response = await openai.chat.completions.create({
    model: "gpt-4",
    messages: [{ role: "user", content: prompt }],
    frequency_penalty: 0.3
  });
  console.log("Assistant:", response.choices[0].message.content.trim());
}

getResponseWithFrequencyPenalty("Describe a sunset over the ocean");

By applying a frequency_penalty of 0.3, you can minimize repetitive content in the AI's responses, making the conversation more dynamic and engaging. This parameter is particularly useful when you want to maintain a lively and varied dialogue.

Example Response:

Assistant: The sunset over the ocean is a breathtaking display of nature's beauty. As the sun dips below the horizon, the sky is painted with vibrant hues of orange, pink, and purple. The ocean reflects these colors, creating a stunning mirror effect. The gentle sound of waves adds to the peaceful ambiance, making it a perfect moment to unwind and reflect on the day's events.

Let's bring it all together with a complete code example that incorporates all the parameters we've discussed:

import 'dotenv/config';
import OpenAI from 'openai';

const openai = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY,
});

async function getCustomResponse(prompt) {
  const response = await openai.chat.completions.create({
    model: "gpt-4",
    messages: [{ role: "user", content: prompt }],
    temperature: 0.7,  // Controls response creativity
    max_tokens: 100,   // Limits response length
    presence_penalty: 0.6,  // Encourages new topics
    frequency_penalty: 0.3  // Reduces repetition
  });
  console.log("Assistant:", response.choices[0].message.content.trim());
}

getCustomResponse("Describe a sunset over the ocean");

In this example, we use all four parameters to customize the AI's response. By adjusting these parameters, you can fine-tune the chatbot's behavior to meet your specific requirements. When you run this code, you should see a response that reflects the balance of creativity, length, and content diversity that you've set.

Example Response:

Assistant: As the sun sets over the ocean, it casts a warm glow across the horizon. The sky is a tapestry of colors, with shades of orange, pink, and purple blending seamlessly. The ocean mirrors this beauty, its surface shimmering with the sun's final rays. A gentle breeze carries the scent of salt and the sound of waves, creating a serene atmosphere. It's a moment of tranquility, inviting reflection and appreciation for the natural world.

In this lesson, we explored how to use model parameters to customize AI responses. You learned about the temperature, max_tokens, presence_penalty, and frequency_penalty parameters and saw how they can be applied in code. These tools allow you to control the creativity, length, and content of the AI's responses, enhancing your chatbot's functionality.

As you move on to the practice exercises, I encourage you to experiment with different parameter settings to see their effects firsthand. This hands-on practice will reinforce what you've learned and prepare you for the next unit, where we'll delve deeper into managing conversation history and message types. Keep up the great work, and enjoy the journey of creating your chatbot with OpenAI!