Building a Chat Engine with Conversation History

Welcome to the second unit of our course on building a RAG-powered chatbot! In the previous lesson, we built a document processor that forms the retrieval component of our RAG system. Today, we'll focus on the conversational aspect by creating a chat engine that can maintain conversation history.

While our document processor is excellent at finding relevant information, a complete RAG system needs a way to interact with users in a natural, conversational manner. This is where our chat engine comes in. The chat engine is responsible for managing the conversation flow, formatting prompts with relevant context, and maintaining a history of the interaction.

The chat engine we'll build today will:

1. Manage interactions with the language model
2. Optionally maintain a history of the conversation for display or logging
3. Format prompts with relevant context from our document processor
4. Provide methods to reset the conversation history when needed

By the end of this lesson, you'll have a fully functional chat engine that can be integrated with the document processor we built previously to create a complete RAG system.

Let's begin by setting up the basic structure of our ChatEngine class. This class will encapsulate all the functionality needed for managing conversations with the language model.

import { ChatOpenAI } from "@langchain/openai";
import { HumanMessage, AIMessage } from "@langchain/core/messages";
import { ChatPromptTemplate, SystemMessagePromptTemplate, HumanMessagePromptTemplate } from "@langchain/core/prompts";

class ChatEngine {
    constructor() {
        // Initialize the chat model
        this.chatModel = new ChatOpenAI({
  model: "gpt-5.6-luna",
  reasoningEffort: "none",
});
        
        // Define the system message that sets the behavior of the assistant
        this.systemMessage = (
            "You are a helpful assistant that ONLY answers questions based on the " +
            "provided context. If no relevant context is provided, do NOT answer the " +
            "question and politely inform the user that you don't have the necessary " +
            "information to answer their question accurately."
        );

        // Define the prompt template with explicit system and human messages
        this.prompt = ChatPromptTemplate.fromMessages([
            SystemMessagePromptTemplate.fromTemplate(this.systemMessage),
            HumanMessagePromptTemplate.fromTemplate(
                "Context:\n{context}\n\nQuestion: {question}"
            )
        ]);

        // Optionally, keep conversation history for display/logging only
        this.conversationHistory = [];
    }
}

export default ChatEngine;

Key points in this initialization:

1. Chat Model: We initialize this.chatModel using new ChatOpenAI({ model: "gpt-5.6-luna", reasoningEffort: "none", }) to create an instance of the OpenAI chat model for generating responses.
2. System Message: We define strict instructions that guide the AI's behavior, telling it to answer questions only based on the provided context and to politely decline if no relevant context is available.
3. Prompt Template: We use ChatPromptTemplate.fromMessages() to explicitly define both the system and human message templates. The system message sets the assistant's behavior, and the human message template includes placeholders for context and question.
4. Conversation History: We initialize an empty array to optionally keep track of the conversation for display or logging purposes. This history is not sent to the model in this implementation.

This structure ensures our chat engine can properly communicate with the language model while optionally maintaining a record of the conversation.