Customizing Model Parameters in LangChain

Welcome to the next step in your LangChain journey! In the previous lesson, you mastered the basics of sending messages to an AI model. Now, it's time to unlock the full potential of your AI interactions by customizing model parameters. This crucial skill will empower you to tailor AI responses to meet your specific needs, making your AI more responsive and aligned with your goals. Get ready to dive deeper and enhance your AI experience by learning how to adjust these parameters effectively.

The model parameter lets you select which AI "brain" will power your conversations:

# Select a specific OpenAI model
chat = ChatOpenAI(
    model="gpt-5.6-luna",     # The fast, low-cost tier
    reasoning_effort="none"   # Answer directly, without a reasoning pass
)

This parameter determines the underlying AI model that processes your messages. Different models offer varying capabilities:

• gpt-5.6-luna: The fastest and lowest-cost option, ideal for straightforward chat
• gpt-5.6-terra: A good balance of capability and cost
• gpt-5.6-sol: The most advanced reasoning capabilities, but the highest cost

Think of this like choosing between different experts for different tasks—some are more specialized, some more general, and they come with different "hiring costs".

Notice the second parameter, reasoning_effort. Models in this family can spend extra time reasoning before they answer, and each model has its own default. Because that default is not always what you want, it is worth stating your choice explicitly: "none" asks the model to answer directly, which keeps responses fast and cheap. Model choice and reasoning effort are two separate dials—a low-cost model can still be asked to think harder, and a flagship model can still answer directly.

Each example below adds one more parameter on top of this model and reasoning setting, so you can see what each one does on its own.

Once you've selected your model, you'll want to control how creative it gets. This is where temperature comes in:

# Set the creativity level of the AI
chat = ChatOpenAI(
    model="gpt-5.6-luna",
    reasoning_effort="none",
    temperature=0.7           # Balanced creativity setting
)

Temperature values typically range from 0 to 2:

• Low temperature (0-0.3): More deterministic, focused, and predictable responses
• Medium temperature (0.4-0.7): Balanced creativity and coherence
• High temperature (0.8-2.0): More random, creative, and diverse outputs

For factual questions or coding help, turn the dial down. For creative writing or brainstorming, crank it up. It's like adjusting between "strictly follow the recipe" and "improvise with the ingredients".

While temperature controls how your AI thinks, max_tokens controls how much it says:

# Limit the length of the AI's response
chat = ChatOpenAI(
    model="gpt-5.6-luna",
    reasoning_effort="none",
    max_tokens=150            # Caps the response at approximately 100-120 words
)

This parameter sets a hard ceiling on how many tokens the model can generate in its response:

• Lower values (50-100): Results in brief responses that may be cut off before completing the thought
• Medium values (150-500): Provides enough space for most explanations, but complex answers might still be truncated
• Higher values (1000+): Allows for comprehensive responses with less risk of mid-sentence cutoffs

It's important to understand that the model has no awareness of this limit while generating its response. If a response would naturally exceed your max_tokens setting, it will simply be cut off mid-sentence or mid-thought. The model doesn't try to wrap up its answer as it approaches the limit. Without setting this limit, models might generate very lengthy responses, potentially increasing your costs and overwhelming your users with too much information.

Temperature isn't the only way to control randomness. top_p offers a complementary approach:

# Adjust how the AI selects its next words
chat = ChatOpenAI(
    model="gpt-5.6-luna",
    reasoning_effort="none",
    top_p=0.9                 # Consider only the most likely 90% of possible next words
)

This parameter determines how the model selects the next token in its response:

• Value of 1.0: Consider all possible next words
• Value of 0.9: Only consider the most likely words that add up to 90% probability
• Lower values (0.5-0.7): More focused, less surprising responses

While temperature adjusts "how random" the selection is, top_p filters "which options are even considered". They work well together—like controlling both the size of a menu and how randomly you select from it. Many developers find that adjusting temperature is sufficient, but top_p gives you another dimension of control.

So far every example has set reasoning_effort="none", which asks the model to answer straight away. But that dial has more than one position, and turning it up is what you do when a question actually requires working through steps:

# Let the model think before it answers
chat = ChatOpenAI(
    model="gpt-5.6-luna",
    reasoning_effort="high"   # Spend time reasoning before responding
)

The effort level controls how much internal work the model does before it starts writing its answer:

• "none": Answer immediately. Fastest and cheapest.
• "low": A brief pass before answering.
• "medium": A balanced amount of deliberation.
• "high": Work the problem through carefully before responding.

Higher effort costs you time and money on every call, and you pay it whether the question needed the extra thought or not. It's the difference between answering off the top of your head and working something out on paper first—one is quicker, the other is more reliable when the question is genuinely hard.

Knowing the levels matters less than knowing when to reach for them. The question to ask is whether getting the answer requires steps:

• Use "none" for lookups, rewording, formatting, simple classification, and ordinary conversation. Most chat traffic is this, and the extra effort would buy you nothing.
• Use "low" or "medium" when there is a little work to do—comparing options, following a short chain of conditions, tidying up messy input.
• Use "high" for multi-step problems: math with several stages, debugging, planning, or anything where a plausible-sounding wrong answer would cost you.

# A hard question deserves a higher effort level
chat = ChatOpenAI(
    model="gpt-5.6-luna",
    reasoning_effort="high",
    max_tokens=300            # Give the answer room to show its work
)

A useful habit is to start at "none" and raise the level only when you can point to answers that came back wrong. Reaching for "high" everywhere is the most common way to make an application slow and expensive without making it noticeably better.

A note on other parameters: you may encounter frequency_penalty and presence_penalty in older OpenAI examples. They discourage repeated words and encourage new topics respectively, and they are not supported by the gpt-5.6 family—another reminder that the parameters available to you depend on the model you chose.

Now that you understand each parameter individually, you can combine them to create your perfect AI recipe:

from langchain_openai import ChatOpenAI

# Initialize ChatOpenAI with a complete set of custom parameters
chat = ChatOpenAI(
    model="gpt-5.6-luna",     # Choose the fast, low-cost model
    reasoning_effort="none",  # Answer directly, without a reasoning pass
    temperature=0.7,          # Balanced creativity setting
    max_tokens=50,            # Keep responses concise
    top_p=0.9                 # Consider 90% of probability mass
)

# Send a message to the AI model
response = chat.invoke("Hello, can you tell me a joke?")

# Print the AI's response
print("AI Response:")
print(response.content)

Each parameter adjustment contributes to the overall behavior of your AI, allowing you to fine-tune it for specific use cases. Like a chef combining ingredients, you'll develop your own preferred "recipes" for different situations.

The best way to understand these parameters is to experiment with them. Try adjusting one parameter at a time to observe its effect on the AI's responses:

1. Set temperature to 0 vs. 1.5 for the same prompt
2. Compare max_tokens of 50 vs. 500
3. Ask the same multi-step question with reasoning_effort set to "none" and then "high"

Through experimentation, you'll develop an intuitive feel for how to configure the model for different use cases—just like a chef learns to adjust recipes by tasting as they go.

In this lesson, we explored how to customize model parameters to tailor AI responses to your specific needs. You learned about key parameters such as model selection, reasoning effort, temperature, max tokens, and top p. As you move on to the practice exercises, experiment with different parameter values to reinforce your understanding and achieve the desired AI behavior. This skill will be invaluable as you continue your journey into conversational AI with LangChain.