Welcome to the second unit of our course on building a RAG-powered chatbot! In the previous lesson, we built a type-safe document processor that forms the retrieval component of our RAG system. Today, we'll focus on the conversational aspect by creating a chat engine that can maintain conversation history, all implemented in TypeScript.

While our document processor is excellent at finding relevant information, a complete RAG system needs a way to interact with users in a natural, conversational manner. This is where our chat engine comes in. The chat engine is responsible for managing the conversation flow, formatting prompts with relevant context, and maintaining a history of the interaction.

The chat engine we'll build today will:

1. Manage interactions with the language model
2. Optionally maintain a history of the conversation for display or logging
3. Format prompts with relevant context from our document processor
4. Provide methods to reset the conversation history when needed

By the end of this lesson, you'll have a fully functional, type-safe chat engine that can be integrated with the document processor we built previously to create a complete RAG system.

Let's begin by setting up the basic structure of our ChatEngine class in TypeScript. We'll use type annotations and access modifiers to ensure type safety and encapsulation.

import { ChatOpenAI } from "@langchain/openai";
import { HumanMessage, AIMessage } from "@langchain/core/messages";
import { ChatPromptTemplate, SystemMessagePromptTemplate, HumanMessagePromptTemplate } from "@langchain/core/prompts";

class ChatEngine {
    private chatModel: ChatOpenAI;
    private systemMessage: string;
    public conversationHistory: (HumanMessage | AIMessage)[];
    private prompt: ChatPromptTemplate;

    constructor() {
        // Initialize the chat model
        this.chatModel = new ChatOpenAI();

        // Define the system message that sets the behavior of the assistant
        this.systemMessage = (
            "You are a helpful assistant that ONLY answers questions based on the " +
            "provided context. If no relevant context is provided, do NOT answer the " +
            "question and politely inform the user that you don't have the necessary " +
            "information to answer their question accurately."
        );

        // Define the prompt template with explicit system and human messages
        this.prompt = ChatPromptTemplate.fromMessages([
            SystemMessagePromptTemplate.fromTemplate(this.systemMessage),
            HumanMessagePromptTemplate.fromTemplate(
                "Context:\n{context}\n\nQuestion: {question}"
            )
        ]);

        // Optionally, keep conversation history for display/logging only
        this.conversationHistory = [];
    }
}

export default ChatEngine;

Key points in this initialization:

1. Chat Model: We initialize this.chatModel using new ChatOpenAI() to create an instance of the OpenAI chat model for generating responses.
2. System Message: We define strict instructions that guide the AI's behavior, telling it to answer questions only based on the provided context and to politely decline if no relevant context is available.
3. Prompt Template: We use ChatPromptTemplate.fromMessages() to explicitly define both the system and human message templates. The system message sets the assistant's behavior, and the human message template includes placeholders for context and question.
4. Conversation History: We initialize an empty array to optionally keep track of the conversation for display or logging purposes. This history is not sent to the model in this implementation.

This structure ensures our chat engine can properly communicate with the language model while optionally maintaining a record of the conversation.

Let's take a closer look at how we define the prompt template in our ChatEngine class. This part is crucial for controlling how information is passed to the language model.

this.prompt = ChatPromptTemplate.fromMessages([
    SystemMessagePromptTemplate.fromTemplate(this.systemMessage),
    HumanMessagePromptTemplate.fromTemplate(
        "Context:\n{context}\n\nQuestion: {question}"
    )
]);

Here's what each component does and what it returns:

• SystemMessagePromptTemplate.fromTemplate
  This method takes a string template (our system instructions) and returns a SystemMessagePromptTemplate object. This object knows how to generate a system message for the chat model by filling in any placeholders if needed.

• HumanMessagePromptTemplate.fromTemplate
  This method takes a string template for the user's message (with placeholders for context and question) and returns a HumanMessagePromptTemplate object. This object can generate a human message for the chat model by filling in those placeholders.

• ChatPromptTemplate.fromMessages
  This method takes a list of message prompt templates (like the two above) and returns a ChatPromptTemplate object. This object can generate a full list of formatted messages (system and human) ready to be sent to the chat model, using the values you provide for the placeholders.

Key difference:

• The .fromTemplate methods create individual message templates (for either system or human messages).
• The .fromMessages method combines multiple message templates into a single prompt template that can generate the full message sequence for the chat model.

By using these together, we can clearly separate the instructions (system message) from the user input (human message), and then combine them into the exact format the language model expects.

Now that we have our basic class structure, let's implement the core functionality: sending messages and receiving responses. In TypeScript, we'll use type annotations for method parameters and return types to ensure type safety.

async sendMessage(userMessage: string, context: string = ""): Promise<string> {
    // Format the messages using the prompt template (includes system message)
    const messages = await this.prompt.formatMessages({
        context: context,
        question: userMessage
    });

    // Get the response from the model
    const response = await this.chatModel.invoke(messages);

    // Optionally, track the conversation for display/logging
    this.conversationHistory.push(new HumanMessage({ content: userMessage }));
    this.conversationHistory.push(new AIMessage({ content: response.content }));

    // Return the AI's response
    return response.content as string;
}

The sendMessage method takes two parameters: userMessage (the question from the user) and context (optional relevant information from our document processor).

1. Format Messages: We use our prompt template to fill in placeholders with the provided context and question. The system message is always included.
2. Get Response: We invoke the chat model with our formatted messages using await this.chatModel.invoke(messages).
3. Update History: We append the user's message and the AI's response to the conversation history for display or logging.
4. Return Result: We return the content of the response to be displayed to the user.

Note: In this implementation, conversation history is not sent to the model. Each response is based only on the current context and question, which is typical for RAG systems.

An important aspect of any chat system is the ability to manage the conversation state. Let's implement a method to reset the conversation history, using TypeScript's type annotations for the return type.

resetConversation(): string {
    // Reset the conversation history
    this.conversationHistory = [];
    return "Conversation history has been reset.";
}

The resetConversation method clears the conversation history. This is useful for display or logging purposes and allows users to start fresh when needed.

Let's see how our chat engine behaves when we don't provide any context. This is important because, in a RAG system, the assistant should not "hallucinate" answers — it should only respond based on the information it has.

Here's how you can test this scenario in TypeScript:

// app/main.ts
import ChatEngine from './ChatEngine';

// Initialize the chat engine
const chatEngine = new ChatEngine();

// Send a message without context (should politely decline)
const query1: string = "What is the capital of France?";
const response1: string = await chatEngine.sendMessage(query1);
console.log(`Question: ${query1}`);
console.log(`Answer: ${response1}`);

// Print conversation history
console.log("\nConversation history:");
console.log(chatEngine.conversationHistory);

In this example, we create an instance of ChatEngine and send a question without providing any context. According to our system message instructions, the assistant should politely inform the user that it doesn't have enough information to answer.

You should see output similar to:

Question: What is the capital of France?
Answer: I'm sorry, but based on the provided context, I don't have the necessary information to answer your question accurately. If you could provide additional context, I'd be happy to help.

Conversation history:
[
  HumanMessage { content: 'What is the capital of France?' },
  AIMessage { content: "I'm sorry, but based on the provided context, I don't have the necessary information to answer your question accurately. If you could provide additional context, I'd be happy to help." }
]

Notice that the assistant does not attempt to answer the question directly. Instead, it follows the instructions from the system message and responds appropriately. The conversation history contains strictly typed HumanMessage and AIMessage objects.

Now, let's test the chat engine with some relevant context. This simulates the scenario in which our document processor has retrieved useful information, and we want the assistant to answer using only that context.

// Send a message with context (should answer using only the context)
const context: string = `Paris is the capital and most populous city of France. 
The Eiffel Tower, the Louvre Museum, and Notre-Dame Cathedral are among its most famous landmarks.`;
const query2: string = "Tell me about the landmarks mentioned.";
const response2: string = await chatEngine.sendMessage(query2, context);
console.log(`\nQuestion with context: ${query2}`);
console.log(`Answer: ${response2}`);

// Print updated conversation history
console.log("\nUpdated conversation history:");
console.log(chatEngine.conversationHistory);

In this test, we provide a context string that mentions several Paris landmarks. The assistant should now generate an answer based only on this context.

The output will look something like:

Question with context: Tell me about the landmarks mentioned.
Answer: The Eiffel Tower, the Louvre Museum, and Notre-Dame Cathedral are among the most famous landmarks in Paris.

Updated conversation history:
[
  HumanMessage { content: 'What is the capital of France?' },
  AIMessage { content: "I'm sorry, but based on the provided context, I don't have the necessary information to answer your question accurately. If you could provide additional context, I'd be happy to help." },
  HumanMessage { content: 'Tell me about the landmarks mentioned.' },
  AIMessage { content: 'The Eiffel Tower, the Louvre Museum, and Notre-Dame Cathedral are among the most famous landmarks in Paris.' }
]

Here, the assistant uses only the information from the provided context to answer the question. This demonstrates how the chat engine, when combined with a document processor, can provide accurate, context-aware responses.

Finally, let's see how to reset the conversation history. This is useful if you want to clear the previous exchanges.

// Reset the conversation history (for display/logging only)
chatEngine.resetConversation();
console.log("\nConversation history has been reset.");

// Print conversation history after reset
console.log("\nConversation history after reset:");
console.log(chatEngine.conversationHistory);

After calling resetConversation(), the conversation history should be empty:

Conversation history has been reset.

Conversation history after reset:
[]

This confirms that the conversation history is cleared and ready for a new interaction.

In this lesson, we've built a type-safe chat engine for our RAG chatbot using explicit system and human message templates in TypeScript. We've learned how to:

1. Create a ChatEngine class that manages conversations with a language model, using TypeScript's type system and access modifiers
2. Define system messages to guide the AI's behavior
3. Format prompts with context and questions using templates
4. Optionally maintain conversation history for display or logging, with strict typing
5. Implement methods to send messages and reset conversation history, with clear type annotations
6. Test our chat engine with various scenarios

Our chat engine complements the document processor we built in the previous lesson. While the document processor handles the retrieval of relevant information, the chat engine manages the conversation and presents this information to the user in a natural way. In the next unit, we'll integrate the document processor and chat engine to create a complete RAG system. This integration will allow our chatbot to automatically retrieve relevant context from documents based on user queries, creating a seamless experience in which users can ask questions about their documents and receive informed, contextual responses.

Get ready to practice what you've learned and take your RAG chatbot to the next level!