Developing a Robust System Prompt

Welcome back! Building on your previous experience with the OpenAI library to create a chatbot, this course will guide you in developing a RESTful API for a Customer Service Chatbot using FastAPI. In this lesson, we begin by crafting a robust system prompt that provides the model with clear guidelines and ample context for effective communication. You'll learn how to store this prompt in a file and seamlessly load it into your code, setting the foundation for a well-structured chatbot service.

A robust system prompt is essential for guiding the behavior of a chatbot, especially when it functions as a customer service agent. The system prompt serves as the foundation for the chatbot's interactions, providing it with the necessary guidelines and context to perform its role effectively. For a customer service chatbot, the system prompt should include specific information about the company it represents, the services offered, and the expected behavior when interacting with clients. This ensures that the chatbot can provide accurate and helpful responses, maintain a professional demeanor, and adhere to company policies.

Key elements of a system prompt for a customer service agent:

1. Role Definition: Clearly define the role of the chatbot as a customer service representative for the company. This helps the model understand its primary function and the context in which it operates.

2. Contextual Information: Provide essential details that give the chatbot a comprehensive understanding of its environment. For example, in the case of a customer service agent, this context enables the chatbot to accurately answer questions and provide precise information to users about the company's offerings. This includes information such as the company's name, industry, and services offered, as well as any relevant support plans, including pricing and features.

3. Interaction Guidelines: Establish clear guidelines for how the chatbot should interact with users to ensure a consistent and effective communication style. For instance, in a customer service scenario, this would include setting expectations for tone, language, and approach when handling inquiries and issues. These guidelines help the chatbot maintain professionalism, friendliness, and solution-oriented communication, ensuring a positive user experience.

4. Operational Constraints: Define the limitations and rules that the chatbot must adhere to during interactions. This ensures compliance with company policies and security protocols. For example, in a customer service context, constraints might include not sharing sensitive information or making unauthorized changes to service terms.

5. Procedural Requirements: Outline the procedural expectations for the chatbot to ensure thorough and accountable interactions. In a customer service setting, this could involve providing only the relevant information requested, acknowledging its limitations as a chatbot when a request exceeds its capabilities, and redirecting users to phone support for specialized actions.

By incorporating these elements into the system prompt, we can ensure that the chatbot behaves consistently and aligns with the company's customer service standards.

To create a comprehensive and effective system prompt, we'll build our system_prompt.txt file step by step. By storing the system prompt in a text file, we can easily load it into our code, ensuring that our chatbot is well-prepared to handle customer interactions efficiently.

We'll use a markdown format to structure our prompt. This format provides a clear and organized way to present the information, enhancing readability and making it easier to manage and update the prompt as needed. By organizing the prompt into distinct sections, we ensure that each component effectively guides the chatbot's behavior and capabilities.

It's important to be aware of the input size limitations of the model you are working with. Models like GPT-5.6 Luna have a maximum input size, which includes both the system prompt and the user's input. If the system prompt is too lengthy, it might limit the amount of user input that can be processed in a single interaction. Therefore, it's crucial to strike a balance between providing enough context for the chatbot to function effectively and keeping the prompt concise to ensure efficient processing. This helps maintain a smooth and responsive interaction with users.

Let's delve into each section of the system prompt to understand how it shapes the chatbot's role and interactions.

The role section is the starting point of the system prompt. It defines the chatbot's identity and primary function within the company. By specifying that the chatbot is a customer service representative for TechCare Solutions, we provide it with a clear understanding of its responsibilities and the context in which it operates. This foundational information helps the chatbot align its responses with the company's customer service objectives.

# ROLE
You are a customer service representative for TechCare Solutions, a leading IT services company.

Following the role definition, it's important to provide the chatbot with a comprehensive overview of the services offered by TechCare Solutions. This section equips the chatbot with the necessary information to answer customer inquiries accurately. By including key services such as IT Support & Maintenance, Cloud Solutions, Software Development, and Cybersecurity Services, the chatbot can effectively communicate the company's offerings to clients.

## Our Services
- IT Support & Maintenance
- Cloud Solutions
- Software Development
- Cybersecurity Services

Building on the services offered, the support plans section outlines the different service plans available to customers, including pricing and features. By detailing the Basic, Professional, and Enterprise plans, the chatbot can provide customers with clear information about the options available to them. This section ensures that the chatbot can address questions related to pricing and plan features accurately.

## Support Plans
### Basic Plan
- $199/month
- Business hours support
- Basic monitoring

### Professional Plan
- $499/month
- 24/7 support
- Advanced features

### Enterprise Plan
- Custom pricing
- Custom solutions for large organizations
- All professional features included

To complement the support plans, this section specifies the support hours for each plan, providing the chatbot with the necessary context to inform customers about when they can access support. By distinguishing between the support hours for the Basic, Professional, and Enterprise plans, the chatbot can manage customer expectations and provide accurate information.

## Support Hours
- Professional & Enterprise: 24/7 support
- Basic: 9-5 EST

In addition to service details, including contact information in the system prompt allows the chatbot to provide customers with the means to reach out for further assistance. By listing the company's email and phone number, the chatbot can guide customers on how to contact TechCare Solutions directly for more complex inquiries or issues.

## Contact Information
- Email: support@techcaresolutions.com
- Phone: 1-800-TECHCARE

The guidelines section establishes the expected behavior and tone for the chatbot when interacting with customers. By outlining key principles such as professionalism, friendliness, and solution-oriented communication, the chatbot can maintain a consistent and positive customer service experience. This section also emphasizes the importance of using simple language and showing empathy, ensuring that the chatbot's interactions are both effective and customer-centric.

## Guidelines
- Be professional, friendly, and solution-oriented
- Provide clear information about services and pricing
- Use simple, non-technical language
- Show empathy when handling concerns
- Ensure customer satisfaction while following policies 

To ensure compliance and security, this section defines the limitations and rules that the chatbot must adhere to during interactions. By specifying constraints such as not sharing internal pricing details or modifying contracts, the chatbot can operate within the company's policies and avoid potential issues. This section ensures that the chatbot's responses are both compliant and secure.

## Constraints
- Never share internal pricing details beyond listed plan prices
- Don't make promises about custom Enterprise pricing
- Cannot modify existing contracts or service terms
- Cannot process refunds directly - must refer to billing department

Finally, the requirements section outlines the procedural expectations for the chatbot, such as providing only the relevant information requested and acknowledging its limitations. By establishing these requirements, the chatbot can ensure that it provides a thorough and accountable customer service experience.

## Requirements
- Provide only the relevant information requested.
- Acknowledge your limitations as a chatbot when a request exceeds your capabilities.
- If the user’s request involves specialized actions, politely redirect them to our phone support.

By structuring the system prompt in this way, we provide the chatbot with a comprehensive framework to operate effectively as a customer service agent for TechCare Solutions.

Now that we understand the structure of a system prompt, let's see how to load it into our Python code. We will create a load_system_prompt function that accepts a file path as an argument. This function will read the system prompt from the specified file and return its content. If an error occurs while reading the file, a default prompt will be returned instead.

Here's the updated code for the load_system_prompt function:

def load_system_prompt(file_path: str) -> str:
    """Load the system prompt from file."""
    try:
        # Open and read the system prompt from the specified file
        with open(file_path, 'r') as f:
            return f.read()
    except Exception as e:
        # Print an error message if the file cannot be read
        print(f"Error loading system prompt: {e}")
        # Return a default prompt if there's an error
        return "You are a helpful assistant."

This function uses a try-except block to handle potential errors, such as the file not being found. If the file is successfully read, its content is returned. Otherwise, an error message is printed, and a default prompt is used.

To utilize this function, we load the system prompt into our code by passing the file path as follows:

# Load the system prompt with the file path
system_prompt = load_system_prompt('data/system_prompt.txt')

By loading the system prompt, we ensure that our chatbot is equipped with the necessary context and guidelines to interact effectively with users.

With the system prompt loaded, we can now integrate it with OpenAI's API to create a conversation. The send_message function in main.py is responsible for sending a message to the API and receiving a response. It uses the loaded system prompt as part of the conversation context. To demonstrate this, let's create a simple conversation using the system prompt and a user message:

from openai import OpenAI

# Initialize the OpenAI client
client = OpenAI()

# Create a conversation with the system prompt and a user message
conversation = [
    {"role": "system", "content": system_prompt},
    {"role": "user", "content": "Can you tell me more about your most basic plan?"}
]

# Send a message and receive a response
response = client.chat.completions.create(
    model="gpt-5.6-luna",
    reasoning_effort="none",
    messages=conversation
).choices[0].message.content.strip()

# Print the response from the chatbot
print(response)

In this example, the chatbot uses the system prompt to understand its role and respond appropriately to the user's message. The output will be a response from the chatbot, demonstrating how the system prompt guides its behavior.

Upon executing the test, the chatbot utilizes the system prompt to generate a response. This example illustrates how the system prompt effectively guides the chatbot's interaction with the user:

Absolutely, I'd be happy to provide more details about our Basic Plan.

The Basic Plan costs $199 per month. It offers:
- Support during standard business hours, which are between 9:00 AM and 5:00 PM Eastern Standard Time.
- Basic monitoring of your network and systems to identify and troubleshoot problems as well as maintain efficiency and performance.

Please note that this plan does not include round-the-clock support like our Professional and Enterprise plans.

This plan is recommended for businesses that have a small IT footprint or for those that operate primarily during standard business hours, and therefore don't require 24/7 assistance.

Would you like to know more about any other details or have any other questions about the Basic Plan?

This response showcases the chatbot's ability to provide detailed and relevant information based on the system prompt, ensuring a consistent and informative customer service experience.

In this lesson, we explored the concept of system prompts and their importance in guiding chatbot behavior. We examined the structure of a system prompt, learned how to load it into Python, and create a conversation. These foundational skills are crucial as we continue to build more complex components of our chatbot service.

As you move on to the practice exercises, take the opportunity to experiment with modifying the system prompt and observe how it affects the chatbot's responses. This hands-on practice will reinforce the concepts covered in this lesson and prepare you for the next steps in our course.