Finishing Optimization: Saving and Loading DSPy Programs

Welcome to our final lesson on optimizing with DSPy! Throughout this course, we've explored various optimization techniques, from Few-Shot Learning to Instruction Optimization, and most recently, Automatic Finetuning. You've learned how to enhance your DSPy programs by tuning prompts, instructions, and even model weights. But what happens after you've spent hours or days optimizing your program? How do you preserve that work so you don't have to repeat the optimization process every time you want to use your program?

This is where saving and loading optimized programs become crucial. Persistence is a fundamental aspect of any mature machine learning workflow, and DSPy provides straightforward mechanisms to save your optimized programs and load them later. This capability is especially important after you've run computationally expensive optimization processes like BootstrapFewShotWithRandomSearch or MIPROv2, which might take hours to complete.

Saving your optimized programs serves several important purposes:

1. It preserves your work across sessions, allowing you to close your environment and return to it later without losing progress.
2. It enables sharing optimized programs with teammates or deploying them to production environments.
3. It creates checkpoints in your optimization journey, letting you compare different optimization approaches.
4. It saves computational resources by eliminating the need to re-run optimization processes.

The ability to save and load programs fits naturally into the DSPy optimization workflow we've been building throughout this course. After defining your program, collecting data, implementing a metric, and running an optimizer, saving the result becomes the final step that makes your optimization work persistent and reusable.

Remember that different optimizers modify your program in different ways. Few-Shot Learning optimizers add examples to your prompts, Instruction Optimization techniques refine the instructions, and Finetuning updates model weights. Regardless of which optimization approach you've used, DSPy's saving and loading functionality preserves all these modifications, ensuring your optimized program works exactly the same when loaded as it did when saved.

In this lesson, we'll explore how to save your optimized programs to disk, examine the format of the saved files, and learn how to load these programs back into memory. By the end, you'll have the complete toolkit for optimizing DSPy programs and making those optimizations persistent and reusable.

After you've successfully optimized your DSPy program using any of the techniques we've covered in previous lessons, saving it to disk is straightforward. DSPy provides a simple .save() method that can be called on any optimized program. Let's look at how to use this method:

# Assuming you have an optimized program from any optimizer
optimized_program.save("my_optimized_program.json")

This single line of code saves your entire optimized program to a file named my_optimized_program.json in your current working directory. The .save() method takes a path parameter that specifies where to save the file. You can provide a simple filename as shown above, or a full path to save it in a specific directory:

# Save to a specific directory
optimized_program.save("/path/to/saved_models/my_optimized_program.json")

When you run this code, you'll see output similar to this:

Saved program to my_optimized_program.json

The saved file is in plain-text JSON format, which means you can open and inspect it with any text editor. This transparency is valuable for understanding what the optimizer generated and for debugging purposes. Let's take a moment to understand what gets saved in this file.

The JSON file contains all the parameters and steps in your source program, including:

1. The program's class name and structure
2. All optimized prompts with their instructions and examples
3. Any configuration parameters specific to your program
4. The signatures of each module in your program

For example, if you've used BootstrapFewShot to optimize your program, the saved file will include all the bootstrapped examples that were added to your prompts. If you've used COPRO or MIPROv2, it will include the optimized instructions. This comprehensive preservation ensures that when you load the program later, it will behave exactly as it did when you saved it.

Let's look at some best practices for saving your optimized programs:

First, establish a consistent naming convention. Include information about the optimizer used, the date, and perhaps a performance metric in the filename:

# Good naming convention
filename = f"qa_program_miprov2_acc{accuracy:.2f}_{date.today().strftime('%Y%m%d')}.json"
optimized_program.save(filename)

This produces filenames like qa_program_miprov2_acc0.87_20230615.json, making it easy to identify and compare different optimized versions of your program.

Second, organize your saved programs in a structured directory. Consider creating separate folders for different program types, optimizers, or projects:

import os

# Create directory if it doesn't exist
save_dir = "saved_programs/qa_system/miprov2"
os.makedirs(save_dir, exist_ok=True)

# Save program in the directory
optimized_program.save(os.path.join(save_dir, "optimized_qa.json"))

Third, consider saving metadata alongside your program. While the program itself contains the optimized prompts and configuration, you might want to save additional information like the dataset used, the optimization time, or notes about the optimization process:

import json

metadata = {
    "optimizer": "MIPROv2",
    "dataset_size": len(trainset),
    "optimization_time_minutes": 120,
    "notes": "Used auto='light' setting with 3 bootstrapped demos"
}

with open(os.path.join(save_dir, "optimized_qa_metadata.json"), "w") as f:
    json.dump(metadata, f, indent=2)

By following these practices, you'll create a well-organized library of optimized programs that you can easily navigate, compare, and reuse. In the next section, we'll explore how to load these saved programs back into memory.

Once you've saved your optimized DSPy program, you'll often need to load it back into memory for further use, evaluation, or deployment. DSPy makes this process just as straightforward as saving, with a simple .load() method. However, there are a few important details to understand to ensure your loaded program works correctly.

To load a previously saved program, you first need to initialize an instance of the same program class that was used to create the optimized program. Then, you call the .load() method on this instance, providing the path to the saved file:

# First, initialize an instance of your program class
loaded_program = YOUR_PROGRAM_CLASS()

# Then, load the saved program
loaded_program.load(path="my_optimized_program.json")

Let's break down what's happening here. The first line creates a new, unoptimized instance of your program class. This is necessary because DSPy needs to know the structure of your program before loading the optimized version. The second line loads the optimized parameters from the saved file into this instance, effectively transforming it into the optimized program.

When you run this code, you'll see output similar to this:

Loaded program from my_optimized_program.json

After loading, your program will behave exactly as it did when you saved it, with all the optimized prompts, instructions, and examples intact. You can immediately start using it for inference:

# Use the loaded program for inference
result = loaded_program(input_example)
print(result)

It's important to verify that your loaded program works correctly. One approach is to run it on a few examples from your validation set and check that the results match what you expected:

# Verify the loaded program works correctly
for example in validation_examples[:3]:
    result = loaded_program(example)
    print(f"Input: {example.question}")
    print(f"Output: {result.answer}")
    print("---")

If you encounter issues when loading a program, there are a few common troubleshooting steps to try:

First, ensure you're initializing the correct program class. The class must have the same structure as the one used to create the saved program. If your program class has changed since you saved the optimized version, you might need to revert to the original class definition.

Second, check that the path to the saved file is correct. If you're working in a different directory or environment than when you saved the program, you might need to provide the full path:

# Using full path to avoid directory issues
loaded_program.load(path="/absolute/path/to/my_optimized_program.json")

Third, if you're loading a program that was optimized with a finetuned model, make sure the model is available and properly initialized:

import dspy

# Load the finetuned model
LM = dspy.HFModel(checkpoint="path_to_finetuned_model", model="model_name")

# Initialize and load the program
loaded_program = YOUR_PROGRAM_CLASS()
loaded_program.load(path="finetuned_program.json")

# Set the language model for all predictors
for p in loaded_program.predictors():
    p.lm = LM
    p.activated = False

This ensures that the loaded program uses the correct finetuned model for inference.

Finally, if you're working with different versions of DSPy, there might be compatibility issues between saved programs. It's generally best to use the same version of DSPy for saving and loading. If you must use a different version, consider re-optimizing your program with the current version.

By understanding these details and following these practices, you can reliably load your optimized programs and integrate them into your workflows. In the next section, we'll explore practical applications and next steps for your DSPy optimization journey.

As you continue your DSPy optimization journey, consider asking yourself these questions to guide your next steps:

1. Did you define your task well? A clear task definition is the foundation of effective optimization.
2. Do you need more data? Additional examples can help your optimizers find better patterns.
3. Should you update your metric? Different metrics might better capture what you're trying to optimize for.
4. Could a more sophisticated optimizer help? You might start with simple Few-Shot Learning and progress to more complex approaches like MIPROv2.
5. Would adding complexity to your DSPy program improve results? Sometimes the program structure itself needs refinement.
6. Have you considered using multiple optimizers in sequence? As we saw earlier, combining optimization approaches can be powerful.

Congratulations on completing this course on optimizing with DSPy! You now have a comprehensive toolkit for creating, optimizing, and persisting DSPy programs. You've learned how to use Few-Shot Learning to incorporate examples into your prompts, Instruction Optimization to refine your instructions, Automatic Finetuning to update model weights, and finally, how to save and load your optimized programs for reuse and deployment.

Remember that optimization is an iterative process. Each cycle of programming, optimization, evaluation, and refinement brings you closer to your performance goals. By making your optimized programs persistent through saving and loading, you create a foundation for continuous improvement and practical application of your DSPy systems. Good luck using DSPy!