Exploring Model Parameters in JavaScript

Welcome back! In the previous lesson, you learned how to send a simple message to OpenAI's language model and receive a response. Now, we will take a step further by exploring model parameters that allow you to customize the AI's responses. These parameters are crucial for tailoring the chatbot's behavior to meet specific needs. In this lesson, we will focus on four key parameters: max_completion_tokens, verbosity, response_format, and stream. Understanding these parameters will enable you to control the length, level of detail, structure, and delivery of the AI's responses, enhancing your chatbot's functionality.

In this course we use gpt-6-luna, a reasoning model: it thinks briefly before it answers. Reasoning models manage their own sampling, so older controls such as temperature, top_p, presence_penalty, and frequency_penalty are not accepted by this model (they remain available on classic chat models). The parameters in this lesson are the ones that work with the model we use.

The max_completion_tokens parameter sets a hard limit on the number of tokens the AI can generate in a response. A "token" can be a whole word or just part of a word. For example, "chatbot" might be one token, while "hello" could be split into two tokens: "hel" and "lo." It's important to note that token counts vary across different models, words, and languages — so the same text might have a different token count depending on these factors.

When you set max_completion_tokens, you specify the maximum number of tokens the AI can produce. For a reasoning model, this limit covers both the text you see and the model's internal reasoning. If the limit is too small, the model can spend all of its tokens thinking and return an empty answer, with a finish_reason of "length". For this reason, treat max_completion_tokens as a generous safety cap rather than a way to make answers shorter.

Here's an example where we set max_completion_tokens to 500:

import 'dotenv/config';
import OpenAI from 'openai';

const openai = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY,
});

async function getResponse(prompt) {
  const response = await openai.chat.completions.create({
    model: "gpt-6-luna",
    messages: [{ role: "user", content: prompt }],
    max_completion_tokens: 500
  });
  console.log("Assistant:", response.choices[0].message.content.trim());
}

getResponse("Explain how a rainbow forms");

By setting max_completion_tokens to 500, you impose a hard limit on the number of tokens the AI can generate. The model isn't consciously summarizing or adjusting its content to fit within this limit; it simply stops when the limit is reached. Primarily, this parameter is valuable for managing usage rates and controlling the cost of API requests. You can check why a response ended by reading response.choices[0].finish_reason: "stop" means the model finished naturally, while "length" means it hit the limit.