Automatic Instruction Optimization with DSPy

Welcome to our lesson on Automatic Instruction Optimization with DSPy! In our previous lesson, we explored how Few-Shot Learning optimizers can enhance your DSPy programs by automatically selecting and generating examples to include in your prompts. Today, we'll take your optimization skills to the next level by focusing on a different approach: optimizing the actual instructions in your prompts.

While Few-Shot Learning optimizers focus on providing examples to guide the language model, Instruction Optimization optimizers focus on improving the natural language instructions themselves. Instead of asking, "What examples should I show the model?" these optimizers ask, "How should I phrase my request to the model?"

This distinction is important because the way you phrase your instructions can significantly impact the model's performance, even with the same underlying task. A well-crafted instruction can guide the model to produce better outputs without needing additional examples, or it can work alongside examples to further enhance performance.

DSPy offers two powerful instruction optimizers:

1. COPRO (Contrastive Prompt Optimization): Generates and refines new instructions for each step in your program, optimizing them through a process called coordinate ascent.

2. MIPROv2 (Minimum Instruction Prompt Optimization v2): Generates instructions that are aware of both your data and any demonstrations included in the prompt, using Bayesian Optimization to efficiently search the space of possible instructions.

These optimizers can be particularly valuable when you want to keep your prompts concise (reducing token usage) or when you're working with models that respond better to clear instructions than to examples. They can also be combined with few-shot learning techniques to get the best of both worlds.

Let's dive into each of these optimizers, understand how they work, and learn how to implement them in your DSPy programs.

COPRO, or Contrastive Prompt Optimization, is a powerful technique for automatically improving the instructions in your DSPy programs. The core idea behind COPRO is to generate multiple alternative instructions for each module in your program, evaluate them using your metric, and iteratively refine them to find the best-performing instructions.

The "contrastive" aspect of COPRO comes from how it learns from both successful and unsuccessful instruction variants. By comparing instructions that lead to correct outputs with those that don't, COPRO can identify patterns that make instructions effective for your specific task.

COPRO uses a process called coordinate ascent (a form of hill-climbing) to optimize instructions. Here's how it works:

1. For each module in your program, COPRO generates multiple alternative instructions.
2. It evaluates each alternative using your metric and training data.
3. It selects the best-performing instruction for each module.
4. It repeats this process for multiple iterations, each time generating new alternatives based on the current best instructions.

This iterative refinement allows COPRO to progressively improve the instructions, exploring the space of possible instructions in a structured way.

Let's look at the key parameters you'll need to configure when using COPRO:

• prompt_model: The language model used to generate new instruction candidates. This can be different from the model used in your program.
• metric: A function that evaluates the performance of your program with different instructions.
• breadth: The number of new instruction candidates to generate in each iteration (default is 16).
• depth: The number of iterations to run the optimization process (default is 2).
• init_temperature: The temperature used when generating new instruction candidates (default is 1.0). Higher values lead to more diverse candidates.
• verbose: Whether to print detailed information during the optimization process (default is False).

COPRO is particularly effective when you have a clear metric for evaluating your program's performance and when you want to optimize instructions without relying on examples. It's also useful when you want to explore a wide range of possible instructions to find creative alternatives you might not have considered.

In the next section, we'll see how to implement COPRO in your DSPy programs with a practical example.

Now that we understand the concept behind COPRO, let's implement it in a DSPy program. The implementation involves setting up the optimizer with appropriate parameters, compiling your program with the optimizer, and then using the optimized program.

Here's a complete example of how to implement COPRO:

from dspy.teleprompt import COPRO

# Define evaluation parameters
eval_kwargs = dict(num_threads=16, display_progress=True, display_table=0)

# Create the COPRO optimizer
copro_teleprompter = COPRO(
    prompt_model=model_to_generate_prompts,  # Model used to generate new instructions
    metric=your_defined_metric,              # Your evaluation metric
    breadth=num_new_prompts_generated,       # Number of new prompts to generate per iteration
    depth=times_to_generate_prompts,         # Number of iterations
    init_temperature=prompt_generation_temperature,  # Temperature for generation
    verbose=False                            # Whether to print detailed information
)

# Compile your program with the optimizer
compiled_program_optimized_signature = copro_teleprompter.compile(
    your_dspy_program,
    trainset=trainset,
    eval_kwargs=eval_kwargs
)

Let's break down this implementation:

First, we define eval_kwargs, which controls how the evaluation is performed during optimization. The num_threads parameter allows for parallel evaluation, which can significantly speed up the optimization process. The display_progress parameter shows a progress bar during evaluation, and display_table controls whether to show a detailed table of results (0 means no table).

Next, we create the COPRO optimizer with several key parameters:

• prompt_model: This is the language model that will generate new instruction candidates. It can be the same as the model used in your program, but you might want to use a more powerful model here to generate better candidates.
• metric: This is your evaluation function that measures how well your program performs with different instructions.
• breadth: This controls how many new instruction candidates to generate in each iteration. A higher value explores more alternatives but requires more computation.
• depth: This controls how many iterations of optimization to perform. More iterations can lead to better results but also require more computation.
• init_temperature: This controls the diversity of generated candidates. A higher value leads to more diverse (but potentially less focused) candidates.
• verbose: Set this to True if you want detailed output during the optimization process.

Finally, we compile our program with the optimizer, providing our training set and evaluation parameters. The result is an optimized version of our program with improved instructions.

When COPRO runs, it will generate multiple alternative instructions for each module in your program, evaluate them using your metric, and iteratively refine them to find the best-performing instructions. The output might look something like this:

Generating new prompts for module: YourModule
Evaluating 16 candidates...
[========================================] 100% 16/16
Best candidate improved performance from 0.75 to 0.82
Iteration 1/2 complete

Generating new prompts for module: YourModule
Evaluating 16 candidates...
[========================================] 100% 16/16
Best candidate improved performance from 0.82 to 0.85
Iteration 2/2 complete

Optimization complete. Final performance: 0.85

The optimized program can then be used just like your original program, but with the improved instructions. You can inspect the optimized instructions by looking at the signature attribute of each module in the compiled program.

COPRO is particularly effective when you have a clear metric for evaluating your program's performance and when you want to optimize instructions without relying on examples. It's also useful when you want to explore a wide range of possible instructions to find creative alternatives you might not have considered.

MIPROv2 (Minimum Instruction Prompt Optimization v2) is another powerful instruction optimizer in DSPy, but it takes a different approach compared to COPRO. While COPRO focuses solely on optimizing instructions, MIPROv2 can optimize both instructions and few-shot examples, making it a more comprehensive solution.

The core idea behind MIPROv2 is to generate instructions that are aware of both your data and any demonstrations included in the prompt. This data-aware and demonstration-aware approach allows MIPROv2 to create instructions that work well with the specific examples in your prompt.

MIPROv2 uses Bayesian Optimization to efficiently search the space of possible instructions. This approach is more sophisticated than the coordinate ascent used by COPRO and can often find better instructions with fewer evaluations. The Bayesian Optimization process builds a probabilistic model of the performance landscape and uses it to select promising candidates for evaluation.

Here are the key parameters you'll need to configure when using MIPROv2:

• metric: A function that evaluates the performance of your program with different instructions.
• auto: A string specifying the optimization intensity, which can be "light," "medium," or "heavy." This controls the number of optimization trials and other internal parameters.

When compiling a program with MIPROv2, you can also specify:

• max_bootstrapped_demos: The maximum number of examples to generate using your program (similar to BootstrapFewShot).
• max_labeled_demos: The maximum number of examples to use from your training set.
• requires_permission_to_run: Whether to ask for permission before running the optimization (useful for expensive runs).

The key difference between MIPROv2 and COPRO is that MIPROv2 is designed to work well with few-shot examples. It generates instructions that complement the examples in your prompt, creating a more cohesive and effective overall prompt. However, MIPROv2 can also be used in a zero-shot configuration (with no examples) if you prefer to rely solely on instructions.

MIPROv2 is particularly effective when you have a reasonable amount of training data (e.g., 200+ examples) and are willing to use more inference calls for a longer optimization run. It's also a good choice when you want to optimize both instructions and examples in a unified way.

In the next section, we'll see how to implement MIPROv2 in your DSPy programs with practical examples for both few-shot and zero-shot configurations.

Now that we understand the concept behind MIPROv2, let's implement it in a DSPy program. We'll look at two configurations: one with few-shot examples and one without (zero-shot).

Here's how to implement MIPROv2 with few-shot examples:

from dspy.teleprompt import MIPROv2

# Create the MIPROv2 optimizer
teleprompter = MIPROv2(
    metric=gsm8k_metric,
    auto="light",  # Can choose between light, medium, and heavy optimization runs
)

# Compile your program with the optimizer
optimized_program = teleprompter.compile(
    program.deepcopy(),
    trainset=trainset,
    max_bootstrapped_demos=3,  # Generate up to 3 examples using your program
    max_labeled_demos=4,       # Use up to 4 examples from your training set
    requires_permission_to_run=False,
)

In this example, we create a MIPROv2 optimizer with a specific metric and set the optimization intensity to "light." The auto parameter can be set to "light," "medium," or "heavy," with heavier settings performing more optimization trials but requiring more computation. For early experimentation, "light" is usually sufficient to observe improvement trends without long runtimes. Once you identify promising configurations or are preparing a production deployment, switching to "medium" or "heavy" can yield further gains.

When compiling the program, we specify both max_bootstrapped_demos and max_labeled_demos to include few-shot examples in the optimized prompts. The max_bootstrapped_demos parameter controls how many examples to generate using your program (similar to BootstrapFewShot), while max_labeled_demos controls how many examples to use directly from your training set.

The requires_permission_to_run parameter is set to False, meaning the optimization will run without asking for confirmation. You might want to set this to True for expensive optimization runs to avoid accidentally starting a long-running process.

Now, let's look at how to implement MIPROv2 in a zero-shot configuration (without examples):

from dspy.teleprompt import MIPROv2

# Create the MIPROv2 optimizer
teleprompter = MIPROv2(
    metric=gsm8k_metric,
    auto="light",  # Can choose between light, medium, and heavy optimization runs
)

# Compile your program with the optimizer in zero-shot mode
optimized_program = teleprompter.compile(
    program.deepcopy(),
    trainset=trainset,
    max_bootstrapped_demos=0,  # No generated examples
    max_labeled_demos=0,       # No examples from training set
    requires_permission_to_run=False,
)

The key difference in this zero-shot configuration is that both max_bootstrapped_demos and max_labeled_demos are set to 0, meaning no examples will be included in the optimized prompts. This is useful when you want to rely solely on instructions or when you're working with models that respond better to clear instructions than to examples.

When MIPROv2 runs, it will use Bayesian Optimization to search for effective instructions, evaluating each candidate on your training set using your metric. The output might look something like this:

Starting MIPROv2 optimization with auto="light"
Running 20 optimization trials...
Trial 1/20: Performance = 0.72
Trial 2/20: Performance = 0.68
Trial 3/20: Performance = 0.75
...
Trial 20/20: Performance = 0.83
Optimization complete. Best performance: 0.83

The optimized program can then be used just like your original program, but with the improved instructions (and possibly examples). You can inspect the optimized instructions by looking at the signature attribute of each module in the compiled program.

MIPROv2 is particularly effective when you have a reasonable amount of training data and are willing to use more inference calls for a longer optimization run. It's also a good choice when you want to optimize both instructions and examples in a unified way.

In this lesson, we've explored two powerful instruction optimizers in DSPy: COPRO and MIPROv2. Both optimizers aim to improve the natural language instructions in your prompts, but they take different approaches and have different strengths.

COPRO uses coordinate ascent to iteratively refine instructions, generating multiple alternatives and selecting the best performers. It focuses solely on optimizing instructions and is particularly effective when you want to explore a wide range of possible instructions without relying on examples.

MIPROv2 uses Bayesian Optimization to search for effective instructions, taking into account both your data and any demonstrations included in the prompt. It can optimize both instructions and few-shot examples, making it a more comprehensive solution. MIPROv2 is particularly effective when you have a reasonable amount of training data and want to optimize both instructions and examples in a unified way.

When deciding which optimizer to use, consider the following guidelines:

• If you want to focus solely on optimizing instructions without examples, COPRO is a good choice.
• If you want to optimize both instructions and examples, or if you have a larger training set, MIPROv2 is often more effective.
• If you prefer zero-shot prompts (without examples), you can use either optimizer, but MIPROv2 configured for zero-shot optimization often performs well.
• If you're willing to use more inference calls for a longer optimization run, MIPROv2 with auto="medium" or auto="heavy" can yield better results.

In the practice exercises that follow, you'll get hands-on experience with these optimizers. You'll implement both COPRO and MIPROv2, observe how they affect your program's performance, and develop an intuition for when to use each approach.

In the next lesson, we'll explore the final category of optimizers in DSPy: Automatic Finetuning. These optimizers go beyond prompt engineering to actually update the weights of the language model itself, offering even more powerful optimization capabilities.

Remember, optimization is an iterative process. After applying these instruction optimizers, you might want to try different configurations, combine them with few-shot learning techniques, or revisit your program design to further improve performance. The tools and techniques you've learned in this lesson provide a solid foundation for this iterative improvement process.