Building the LLM Manager

Welcome to the lesson on creating the LLM Manager in TypeScript, a crucial component of the AI Cooking Helper project. In previous lessons, you learned about the prompts module and how to make basic LLM calls. Now, we will focus on the LLM Manager, which facilitates interactions with language models like OpenAI's GPT. This manager is responsible for rendering prompts, sending them to the language model, and handling the responses. By the end of this lesson, you will understand how to set up and use the LLM Manager effectively in a TypeScript project.

In previous units, we learned how to set up the OpenAI Client to make requests:

import OpenAI from "openai";
import { config } from "../config";

// Initialize OpenAI client using API key from config
const client = new OpenAI({ apiKey: config.openaiApiKey });

Remember, in the Codesignal environment the variables needed like the API key are already configured for you, do not worry!

The generateResponse function is central to the LLM Manager. It renders system and user prompts, sends them to the language model, and returns the response. Let's break it down step by step.

First, we need to render the system and user prompts using the renderPromptFromFile function, which was covered in a previous lesson.

const system = await renderPromptFromFile(systemPromptName, variables);
const user = await renderPromptFromFile(userPromptName, variables);

• system and user are generated by calling renderPromptFromFile with the respective prompt names and variables. This function replaces placeholders in the prompt templates with actual values.

Next, we send the rendered prompts to the language model using the client.

const completion = await client.chat.completions.create({
  model,
  messages: [
    { role: "system", content: system },
    { role: "user", content: user },
  ],
  temperature,
});

We use the client.chat.completions.create method to send the prompts, like we saw in past units:

• The model parameter specifies which language model to use, such as gpt-4o.
• The messages parameter contains the system and user prompts.
• The temperature parameter controls the randomness of the response.

Finally, we extract and return the response from the language model.

return completion.choices[0]?.message?.content?.trim() ?? null;

• If there is no response, we return null.