Creating the LLM Manager

Welcome to the lesson on creating the LLM Manager, a crucial component of the DeepResearcher project. In previous lessons, you learned about the design of DeepResearcher, the prompts module and how to make basic LLM calls. Now, we will focus on the LLM Manager, which facilitates interactions with language models like OpenAI's GPT. This manager is responsible for rendering prompts, sending them to the language model, and handling the responses. By the end of this lesson, you will understand how to set up and use the LLM Manager effectively.

To interact with OpenAI's language models, we need to set up an OpenAI client. This client requires an API key and a base URL, which are typically stored in environment variables for security reasons. Let's start by initializing the client.

import os
from openai import OpenAI
from ..prompts.prompt_manager import render_prompt_from_file

# Initialize OpenAI client (API key read from environment variable)
client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"), base_url=os.getenv("OPENAI_BASE_URL"))

In this code snippet:

• We import the os module to access environment variables.
• We import the OpenAI class from the openai package.
• We initialize the client by reading the API key and base URL from environment variables using os.getenv(). This approach keeps sensitive information secure and separate from your code.

The generate_response function is central to the LLM Manager. It renders system and user prompts, sends them to the language model, and returns the response. Let's break it down step-by-step.

First, we need to render the system and user prompts using the render_prompt_from_file function, which was covered in a previous lesson.

system_prompt = render_prompt_from_file("path/to/system_prompt.txt", variables)
user_prompt = render_prompt_from_file("path/to/user_prompt.txt", variables)

• system_prompt and user_prompt are generated by calling render_prompt_from_file with the respective prompt names and variables. This function replaces placeholders in the prompt templates with actual values.

Next, we send the rendered prompts to the language model using the client.

completion = client.chat.completions.create(
    model=model,
    messages=[
        {"role": "system", "content": system_prompt},
        {"role": "user", "content": user_prompt},
    ],
    reasoning_effort=reasoning_effort
)

• We use the client.chat.completions.create method to send the prompts.
• The model parameter specifies which language model to use, such as "gpt-6-luna."
• The messages parameter contains the system and user prompts.
• The reasoning_effort parameter controls the depth of reasoning in the response. A higher reasoning effort results in deeper reasoning.

Finally, we extract and return the response from the language model.

return completion.choices[0].message.content.strip()

• We access the first choice in the completion object and retrieve the message content.
• The strip() method removes any leading or trailing whitespace from the response.

The generate_boolean function interprets LLM responses as boolean values. It builds on the generate_response function.

First, we call generate_response to get the LLM's response.

response = generate_response("path/to/other/system_prompt.txt", "path/to/other/user_prompt.txt", variables, model, reasoning_effort)

• This line calls generate_response with a system prompt, an user prompt, the variables to enrich the prompts, the model and the reasoning_effort, to obtain a response from the LLM.

Next, we interpret the response as a boolean value using truthy keywords.

truthy_keywords = ["yes", "true", "correct", "affirmative", "certainly", "absolutely"]
return any(keyword in response.lower() for keyword in truthy_keywords)

• We define a list of truthy_keywords that represent affirmative responses.
• We use a generator expression to check if any of these keywords are present in the response (converted to lowercase).
• The function returns True if any keyword is found; otherwise, False.

Error handling is crucial when interacting with APIs. The LLM Manager includes error handling and logging to manage unexpected issues.

First, logging must be configured at the beginning of the script.

import logging

# Configure logging
logging.basicConfig(level=logging.INFO, format='[%(levelname)s] %(message)s')

• We import the logging module and configure it to display messages at the INFO level or higher.
• The format specifies how log messages are displayed, including the log level and message.

To handle API Errors, we will use a try-except block.

try:
    # Code to generate response
except APIError as e:
    logging.error(f"LLM API Error: {e}")
    return None
except Exception as e:
    logging.error(f"Unexpected error: {e}")
    return None

• We catch APIError to handle specific errors from the OpenAI API.
• We log the error message using logging.error.
• We also catch any other exceptions to handle unexpected errors gracefully.

In this lesson, you learned how to create the LLM Manager, a key component of the DeepResearcher tool. We covered setting up the OpenAI client, understanding the generate_response and generate_boolean functions, and implementing error handling and logging. These skills are essential for managing interactions with language models effectively.

As you move on to the practice exercises, you'll have the opportunity to apply what you've learned. Experiment with different prompt inputs and model parameters to see how they affect the responses. Congratulations on reaching this point in the course, and keep up the great work as you continue to build your DeepResearcher tool!