Building a Chat Engine

Building a Chat Engine with Conversation History

Welcome to the second lesson of our course on building a retrieval-augmented generation (RAG) chatbot with Go! In the previous lesson, we built a document processor that forms the retrieval component of our RAG system. Today, we'll focus on the conversational aspect by creating a chat engine that can maintain conversation history and interact with language models.

While our document processor is excellent at finding relevant information, a complete RAG system needs a way to interact with users in a natural, conversational manner. This is where our chat engine comes in. The chat engine is responsible for managing the conversation flow, formatting prompts with relevant context, and maintaining a history of the interaction.

Understanding the Chat Engine

The chat engine we'll build today will:

  1. Manage interactions with the language model using LangChain Go
  2. Maintain a history of the conversation for display or logging
  3. Format prompts with relevant context from our document processor
  4. Provide methods to reset the conversation history when needed

By the end of this lesson, you'll have a fully functional chat engine that can be integrated with the document processor we built previously to create a complete RAG system.

Creating the ChatEngine Struct Structure

Let's begin by setting up the basic structure of our ChatEngine using a Go struct. This struct will encapsulate all the functionality needed for managing conversations with the language model.

package chatengine

import (
    "context"
    "fmt"

    "github.com/tmc/langchaingo/llms"
    "github.com/tmc/langchaingo/llms/openai"
)

type Message struct {
    Role    string
    Content string
}

type ChatEngine struct {
    LLM                 llms.Model
    SystemPrompt        string
    ConversationHistory []Message
}

func NewChatEngine(ctx context.Context) (*ChatEngine, error) {
    // Initialize OpenAI LLM for chat
    llm, err := openai.New(
        openai.WithModel("gpt-3.5-turbo"),
    )
    if err != nil {
        return nil, fmt.Errorf("failed to initialize OpenAI client: %w", err)
    }

    return &ChatEngine{
        LLM: llm,
        SystemPrompt: "You are a helpful assistant that ONLY answers questions based on the provided context. If no relevant context is provided, do NOT answer the question and politely inform the user that you don't have the necessary information to answer their question accurately.",
        ConversationHistory: []Message{},
    }, nil
}

Key points in this initialization:

  1. LLM Model: We initialize an OpenAI LLM client configured for chat with the gpt-3.5-turbo model, which is specifically designed for conversational interactions.

  2. System Prompt: We define strict instructions that guide the AI's behavior, telling it to answer questions only based on provided context and to politely decline if no relevant context is available.

  3. Conversation History: We initialize an empty slice to keep track of the conversation for display or logging purposes. This history is maintained locally but not necessarily sent to the model in typical RAG implementations.

  4. Message Struct: We use a Message struct to represent each message in the conversation, with a role (system, user, or assistant) and content.

This structure ensures our chat engine can properly communicate with the language model while maintaining a record of the conversation.

Building the Message Handling System

Now that we have our basic struct structure, let's implement the core functionality: sending messages and receiving responses. We'll create a SendMessage method that formats the prompt with context and interacts with the language model.

import (
    "bytes"
    "text/template"
)

func (ce *ChatEngine) SendMessage(ctx context.Context, userMessage string, context string) (string, error) {
    // Create a prompt template that includes system instructions, context, and question
    const promptTemplate = `{{.SystemPrompt}}

{{if .Context}}Context:
{{.Context}}
{{end}}
Question: {{.Question}}`

    // Define template data structure
    type TemplateData struct {
        SystemPrompt string
        Context      string
        Question     string
    }

    // Parse and execute the template
    tmpl, err := template.New("prompt").Parse(promptTemplate)
    if err != nil {
        return "", fmt.Errorf("failed to parse template: %w", err)
    }

    data := TemplateData{
        SystemPrompt: ce.SystemPrompt,
        Context:      context,
        Question:     userMessage,
    }

    var formattedPrompt bytes.Buffer
    err = tmpl.Execute(&formattedPrompt, data)
    if err != nil {
        return "", fmt.Errorf("failed to execute template: %w", err)
    }

    // Get response from the language model
    response, err := llms.GenerateFromSinglePrompt(ctx, ce.LLM, formattedPrompt.String())
    if err != nil {
        return "", fmt.Errorf("failed to get response from chat model: %w", err)
    }

    // Add the interaction to conversation history
    ce.ConversationHistory = append(ce.ConversationHistory, Message{
        Role:    "user",
        Content: userMessage,
    })
    ce.ConversationHistory = append(ce.ConversationHistory, Message{
        Role:    "assistant",
        Content: response,
    })

    return response, nil
}

The SendMessage method takes three parameters: a context.Context for managing the request lifecycle, userMessage (the question from the user), and context (optional relevant information from our document processor).

Here's what happens in this method:

  1. Template Creation: We use Go's text/template package to create a prompt template that combines the system prompt, context, and question. This is similar to how we formatted prompts in the previous lesson on asking questions with retrieved context.

  2. Template Execution: We populate the template with the system prompt, context (if provided), and the user's question.

  3. Model Interaction: We use llms.GenerateFromSinglePrompt to send our formatted prompt to the OpenAI model and receive a response.

  4. History Management: We append both the user's message and the AI's response to our conversation history for display or logging.

  5. Error Handling: We properly handle errors at each step and wrap them with descriptive messages.

Note: In this implementation, conversation history is maintained locally for display purposes but is not sent to the model. Each response is based only on the current context and question, which is typical for RAG systems to ensure responses are grounded in the provided context.

Implementing Conversation Management

An important aspect of any chat system is the ability to manage the conversation state. Let's implement methods to access and reset the conversation history:

func (ce *ChatEngine) GetConversationHistory() []Message {
    return ce.ConversationHistory
}

func (ce *ChatEngine) ResetConversation() {
    ce.ConversationHistory = []Message{}
}

The GetConversationHistory method returns the current conversation history, which can be useful for displaying the chat to users or for logging purposes.

The ResetConversation method clears the conversation history. This is useful when users want to start a new conversation or when testing different scenarios.

Testing Our Chat Engine Without Context

Let's see how our chat engine behaves when we don't provide any context. This is important because, in a RAG system, the assistant should not "hallucinate" answers — it should respond only based on the information it has.

Here's how you can test this scenario:

package main

import (
    "context"
    "fmt"
    "log"

    "codesignal/chatengine"
)

func main() {
    ctx := context.Background()

    // Initialize the chat engine
    engine, err := chatengine.NewChatEngine(ctx)
    if err != nil {
        log.Fatalf("failed to initialize chat engine: %v", err)
    }

    // Send a message without context (should politely decline)
    query := "What is the capital of France?"
    response, err := engine.SendMessage(ctx, query, "")
    if err != nil {
        log.Fatalf("error sending message: %v", err)
    }

    fmt.Printf("Question: %s\n", query)
    fmt.Printf("Answer: %s\n", response)

    // Print conversation history
    fmt.Println("\nConversation history:")
    history := engine.GetConversationHistory()
    for i, msg := range history {
        fmt.Printf("%d. [%s]: %s\n", i+1, msg.Role, msg.Content)
    }
}

When you run this code, you should see output similar to:

Question: What is the capital of France?
Answer: I'm sorry, but I don't have any context provided to answer your question. Without relevant information about France's capital in the context, I cannot provide an accurate answer. If you could provide additional context, I'd be happy to help.

Conversation history:
1. [user]: What is the capital of France?
2. [assistant]: I'm sorry, but I don't have any context provided to answer your question. Without relevant information about France's capital in the context, I cannot provide an accurate answer. If you could provide additional context, I'd be happy to help.

The assistant correctly refuses to answer because no context was provided, demonstrating that our system prompt is working as intended.

Testing With Context

Now, let's test the chat engine with some relevant context. This simulates the scenario where our document processor has retrieved useful information, and we want the assistant to answer using only that context.

package main

import (
    "context"
    "fmt"
    "log"

    "codesignal/chatengine"
)

func main() {
    ctx := context.Background()

    // Initialize the chat engine
    engine, err := chatengine.NewChatEngine(ctx)
    if err != nil {
        log.Fatalf("failed to initialize chat engine: %v", err)
    }

    // First, send a message without context
    query1 := "What is the capital of France?"
    response1, err := engine.SendMessage(ctx, query1, "")
    if err != nil {
        log.Fatalf("error sending message: %v", err)
    }

    fmt.Printf("Question: %s\n", query1)
    fmt.Printf("Answer: %s\n\n", response1)

    // Now send a message WITH context
    context := `Paris is the capital and most populous city of France. 
The Eiffel Tower, the Louvre Museum, and Notre-Dame Cathedral are among its most famous landmarks.`
    query2 := "Tell me about the landmarks mentioned in the context."

    response2, err := engine.SendMessage(ctx, query2, context)
    if err != nil {
        log.Fatalf("error sending message: %v", err)
    }

    fmt.Printf("Question: %s\n", query2)
    fmt.Printf("Answer: %s\n", response2)

    // Print updated conversation history
    fmt.Println("\nFull conversation history:")
    history := engine.GetConversationHistory()
    for i, msg := range history {
        fmt.Printf("%d. [%s]: %s\n", i+1, msg.Role, msg.Content)
    }
}

The output will look something like:

Question: What is the capital of France?
Answer: I'm sorry, but I don't have any context provided to answer your question. Without relevant information about France's capital in the context, I cannot provide an accurate answer. If you could provide additional context, I'd be happy to help.

Question: Tell me about the landmarks mentioned in the context.
Answer: Based on the provided context, Paris has three famous landmarks mentioned:

1. The Eiffel Tower - one of the most iconic symbols of Paris and France
2. The Louvre Museum - a world-renowned art museum
3. Notre-Dame Cathedral - a historic cathedral and architectural masterpiece

These landmarks are among the most famous attractions in Paris, which is the capital and most populous city of France.

Full conversation history:
1. [user]: What is the capital of France?
2. [assistant]: I'm sorry, but I don't have any context provided to answer your question. Without relevant information about France's capital in the context, I cannot provide an accurate answer. If you could provide additional context, I'd be happy to help.
3. [user]: Tell me about the landmarks mentioned in the context.
4. [assistant]: Based on the provided context, Paris has three famous landmarks mentioned:

1. The Eiffel Tower - one of the most iconic symbols of Paris and France
2. The Louvre Museum - a world-renowned art museum
3. Notre-Dame Cathedral - a historic cathedral and architectural masterpiece

These landmarks are among the most famous attractions in Paris, which is the capital and most populous city of France.

Here, the assistant properly refuses to answer the first question without context, but when provided with relevant information, it gives a detailed answer based solely on that context. This demonstrates how the chat engine, when combined with a document processor, can provide accurate, context-aware responses.

Resetting the Conversation

Finally, let's see how to reset the conversation history. This is useful if you want to clear the previous exchanges and start fresh.

package main

import (
    "context"
    "fmt"
    "log"

    "codesignal/chatengine"
)

func main() {
    ctx := context.Background()

    // Initialize the chat engine
    engine, err := chatengine.NewChatEngine(ctx)
    if err != nil {
        log.Fatalf("failed to initialize chat engine: %v", err)
    }

    // Simulate some conversation
    engine.SendMessage(ctx, "What is the capital of France?", "")
    engine.SendMessage(ctx, "Tell me about landmarks.", "Paris has the Eiffel Tower and Louvre Museum.")

    fmt.Printf("Conversation history before reset: %d messages\n", len(engine.GetConversationHistory()))

    // Reset the conversation history
    engine.ResetConversation()
    fmt.Println("Conversation history has been reset.")

    // Print conversation history after reset
    fmt.Printf("Conversation history after reset: %d messages\n", len(engine.GetConversationHistory()))
    
    history := engine.GetConversationHistory()
    for i, msg := range history {
        fmt.Printf("%d. [%s]: %s\n", i+1, msg.Role, msg.Content)
    }
}

After calling ResetConversation(), the conversation history should be empty:

Conversation history before reset: 4 messages
Conversation history has been reset.
Conversation history after reset: 0 messages

This confirms that the conversation history is cleared and ready for a new interaction.

Summary and Practice Preview

In this lesson, we've built a chat engine for our RAG chatbot using LangChain Go and proper integration with OpenAI's chat models. We've learned how to:

  1. Create a ChatEngine struct that manages conversations with a language model
  2. Initialize an OpenAI LLM client configured for chat interactions
  3. Define system prompts to guide the AI's behavior
  4. Format prompts with context and questions using Go's text/template package
  5. Use llms.GenerateFromSinglePrompt to interact with the language model
  6. Maintain conversation history for display or logging purposes
  7. Implement methods to access and reset conversation history
  8. Test our chat engine with various scenarios

Our chat engine complements the document processor we built in the previous lesson. While the document processor handles the retrieval of relevant information, the chat engine manages the conversation and presents this information to the user in a natural way. In the next unit, we'll integrate the document processor and chat engine to create a complete RAG system. This integration will allow our chatbot to automatically retrieve relevant context from documents based on user queries, creating a seamless experience where users can ask questions about their documents and receive informed, contextual responses.

Get ready to practice what you've learned and take your RAG chatbot to the next level!

Previous LessonNext Lesson: Integrating the RAG Chatbot
Sign up

Join the 1M+ learners on CodeSignal

Be a part of our community of 1M+ users who develop and demonstrate their skills on CodeSignal