Modules: The DSPy Building Blocks

Welcome to our fourth lesson in the DSPy Programming course! In our previous lesson, we explored how to create custom signatures in DSPy, learning how to define the expected input and output behavior of our language model tasks using both string-based and class-based approaches.

Now that you understand how to create signatures, we're ready to take the next step: working with DSPy modules. If signatures are the contracts that specify what goes in and what comes out, modules are the implementations that fulfill those contracts. They're the actual building blocks you'll use to construct AI systems in DSPy.

Think of it this way: a signature says, "I need a function that takes a question and returns an answer," while a module says, "Here's how I'm going to transform that question into an answer." Modules encapsulate specific prompting techniques and reasoning strategies, allowing you to leverage sophisticated approaches without having to reinvent them.

DSPy modules are inspired by neural network modules in PyTorch, but instead of operating on tensors, they operate on natural language inputs and outputs. Just as you can compose PyTorch modules to build complex neural networks, you can compose DSPy modules to build complex language model applications.

Each module in DSPy is designed to implement a specific prompting technique, such as chain-of-thought reasoning or tool use. By combining these modules, you can create sophisticated AI systems that leverage the strengths of different approaches.

In this lesson, we'll explore the core DSPy modules, learn how to use and configure them, and see how they can be composed to build more complex systems. By the end, you'll have a solid understanding of how to use these building blocks to create powerful language model applications.

Working with DSPy modules follows a consistent three-step pattern that you'll use regardless of which specific module you're working with. This consistency makes DSPy code easy to read and maintain, even as your applications grow in complexity.

The three steps are:

1. Declare the module with a signature: This tells DSPy what inputs and outputs to expect.
2. Call the module with input arguments: This provides the actual data to process.
3. Access the outputs from the returned prediction: This retrieves the results.

Let's see this pattern in action with a simple example using the dspy.Predict module, which is the most basic module in DSPy:

# Step 1: Declare with a signature
classify = dspy.Predict('sentence -> sentiment: bool')

# Step 2: Call with input argument
sentence = "it's a charming and often affecting journey."
response = classify(sentence=sentence)

# Step 3: Access the output
print(response.sentiment)
# Output: True

In this example, we first declare a Predict module with a signature that takes a sentence and returns a boolean sentiment. Then we call the module with a specific sentence, and finally, we access the sentiment from the returned prediction.

This same pattern applies to all DSPy modules, regardless of their complexity. For instance, if we were using a more sophisticated module like ChainOfThought, the pattern would be the same:

# Step 1: Declare with a signature
math = dspy.ChainOfThought("question -> answer: float")

# Step 2: Call with input argument
response = math(question="Two dice are tossed. What is the probability that the sum equals two?")

# Step 3: Access the outputs
print(f"Reasoning: {response.reasoning}")
print(f"Answer: {response.answer}")

The only difference is that ChainOfThought adds an additional output field (reasoning) to capture the step-by-step thinking process.

You can also pass configuration options when declaring a module. For example, you might want to generate multiple completions:

# Step 1: Declare with a signature and configuration
classify = dspy.ChainOfThought('question -> answer', n=5)

# Step 2: Call with input argument
question = "What's something great about the ColBERT retrieval model?"
response = classify(question=question)

# Step 3: Access the outputs
for i, answer in enumerate(response.completions.answer):
    print(f"Completion {i+1}: {answer}")

This would generate five different answers to the question, which you can access through the completions attribute.

The consistent three-step pattern makes DSPy code predictable and easy to understand, even as you work with more complex modules and configurations. As we explore different modules in the next section, keep this pattern in mind—it will help you quickly grasp how to use each new module.

DSPy provides several built-in modules, each implementing a different prompting technique. These modules are designed to handle a wide range of tasks, from simple prediction to complex reasoning and tool use. Let's explore the core module types and see how they can be used.

The Predict module is the simplest and most fundamental module in DSPy. It directly implements a signature without adding any additional fields or behavior. This makes it perfect for straightforward tasks where you just need a direct mapping from inputs to outputs.

Here's an example using a class-based signature for sentiment classification:

from typing import Literal

class Classify(dspy.Signature):
    """Classify sentiment of a given sentence."""

    sentence: str = dspy.InputField()
    sentiment: Literal['positive', 'negative', 'neutral'] = dspy.OutputField()
    confidence: float = dspy.OutputField()

classify = dspy.Predict(Classify)
result = classify(sentence="This book was super fun to read, though not the last chapter.")
print(f"Sentiment: {result.sentiment}")
print(f"Confidence: {result.confidence}")
# Possible output:
#   Sentiment: positive
#   Confidence: 0.75

In this example, we define a Classify signature with fields for the input sentence, the output sentiment (restricted to three possible values), and a confidence score. The Predict module simply implements this signature, asking the language model to generate values for the output fields based on the input.

The ChainOfThought module implements the chain-of-thought prompting technique, which encourages the language model to think step by step before providing an answer. This often leads to more accurate results, especially for complex reasoning tasks.

When you use ChainOfThought, it automatically adds a reasoning field to your signature, capturing the model's step-by-step thinking process:

math = dspy.ChainOfThought("question -> answer: float")
result = math(question="Two dice are tossed. What is the probability that the sum equals two?")

print(f"Reasoning: {result.reasoning}")
print(f"Answer: {result.answer}")
# Possible output:
#   Reasoning: When two dice are tossed, each die has 6 faces, resulting in a total of 6 x 6 = 36 possible outcomes. The sum of the numbers on the two dice equals two only when both dice show a 1. This is just one specific outcome: (1, 1). Therefore, there is only 1 favorable outcome. The probability of the sum being two is the number of favorable outcomes divided by the total number of possible outcomes, which is 1/36.
#   Answer: 0.0277776

The reasoning field captures the model's thought process, showing how it arrived at the answer. This is valuable not only for improving accuracy but also for making the model's decisions more transparent and interpretable.

The ProgramOfThought module takes reasoning a step further by encouraging the language model to generate and execute code to solve problems. This is particularly useful for mathematical or algorithmic tasks where code can provide a more precise solution.

example = dspy.ProgramOfThought("question -> answer")

question = "I have 10 candies, I want to distribute them between my 5 friends, how many candies I have to give to each friend?"
result = example(question=question)

print(f"Question: {question}")
print(f"Final Predicted Answer (after ProgramOfThought process): {result.answer}")
# Output:
#   Question: I have 10 candies, I want to distribute them between my 5 friends, how many candies I have to give to each friend?
#   Final Predicted Answer (after ProgramOfThought process): Each friend will receive 2 candies.

Behind the scenes, ProgramOfThought might generate code like 10 / 5 and execute it to get the answer. This approach combines the natural language understanding of large language models with the precision of programmatic computation.

The ReAct module implements the Reasoning and Acting framework, which allows the language model to use tools to gather information or perform actions as part of its reasoning process. This is particularly useful for tasks that require external knowledge or computation.

def evaluate_math(expression: str) -> float:
    return dspy.PythonInterpreter({}).execute(expression)

def search_wikipedia(query: str) -> str:
    results = dspy.ColBERTv2(url='http://20.102.90.50:2017/wiki17_abstracts')(query, k=3)
    return [x['text'] for x in results]

react = dspy.ReAct("question -> answer: float", tools=[evaluate_math, search_wikipedia])

pred = react(question="What is 9362158 divided by the year of birth of David Gregory of Kinnairdy castle?")
print(pred.answer)
# Possible output: 5761.328

In this example, we provide the ReAct module with two tools: a function to evaluate mathematical expressions and a function to search Wikipedia. The module can then use these tools as part of its reasoning process, deciding when and how to use them to answer the question.

The ReAct module is particularly powerful for building agent-like systems that can interact with their environment to gather information and perform actions.

The MultiChainComparison module generates multiple reasoning paths and then compares them to select the best answer. This can lead to more robust results, especially for questions where different reasoning approaches might lead to different answers.

class MultipleChoice(dspy.Signature):
    """Answer multiple choice questions by comparing options."""

    question = dspy.InputField()
    options = dspy.InputField()
    answer = dspy.OutputField(desc="The correct answer (A, B, C, or D)")
    reasoning = dspy.OutputField(desc="Explanation for the answer")

predictor = dspy.Predict(MultipleChoice)

mc_solver = dspy.MultiChainComparison(MultipleChoice, M=3)

question = "What color is the sky?"
options = {"A": "Red", "B": "Blue", "C": "Black", "D": "Yellow"}

completions = [predictor(question=question, options=options) for _ in range(3)]

result = mc_solver(completions=completions, question=question, options=options)

print(f"Question: {question}")
print(f"Options: {options}")
print(f"Selected Answer: {result.answer}")
print(f"Reasoning: {result.rationale}")
# Output
#   Question: What color is the sky?
#   Options: {'A': 'Red', 'B': 'Blue', 'C': 'Black', 'D': 'Yellow'}
#   Selected Answer: B
#   Reasoning: The sky is typically blue during the day due to Rayleigh scattering, which causes shorter wavelengths of light to scatter more than longer wavelengths. Other colors listed, such as red, black, and yellow, do not represent the color of the sky under normal daylight conditions.

In this example, we first generate three different completions using the Predict module, then use MultiChainComparison to compare them and select the best answer. The module adds a rationale field to explain why it selected that particular answer.

Each of these core module types serves a different purpose and is suited to different types of tasks. By understanding their capabilities and how to use them, you can select the right module for each part of your AI system.

DSPy modules can be configured in various ways to control their behavior. These configuration options allow you to fine-tune how the module interacts with the language model and processes inputs and outputs.

Let's explore some common configuration options using the ChainOfThought module as an example:

# Configure the module to generate multiple completions
classify = dspy.ChainOfThought('question -> answer', n=5)

question = "What's something great about the ColBERT retrieval model?"
response = classify(question=question)

# Access the multiple completions
for i, answer in enumerate(response.completions.answer):
    print(f"Completion {i+1}: {answer}")

# Access the selected completion
print(f"\nSelected reasoning: {response.reasoning}")
print(f"Selected answer: {response.answer}")

In this example, we configure the ChainOfThought module to generate five different completions by setting n=5. This is useful when you want to explore different possible answers to a question. The module will generate five different reasoning paths and answers and then select one as the final output.

The output might look something like this:

Completion 1: One great thing about the ColBERT retrieval model is its superior efficiency and effectiveness compared to other models.
Completion 2: Its ability to efficiently retrieve relevant information from large document collections.
Completion 3: One great thing about the ColBERT retrieval model is its superior performance compared to other models and its efficient use of pre-trained language models.
Completion 4: One great thing about the ColBERT retrieval model is its superior efficiency and accuracy compared to other models.
Completion 5: One great thing about the ColBERT retrieval model is its ability to incorporate user feedback and support complex queries.

Selected reasoning: We can consider the fact that ColBERT has shown to outperform other state-of-the-art retrieval models in terms of efficiency and effectiveness. It uses contextualized embeddings and performs document retrieval in a way that is both accurate and scalable.
Selected answer: One great thing about the ColBERT retrieval model is its superior efficiency and effectiveness compared to other models.

You can also configure modules with reasoning-effort settings and token limits:

# Configure the module with specific LM parameters
math = dspy.ChainOfThought(
    "question -> answer: float",
    reasoning_effort="low",
    max_tokens=500
)

A lower reasoning effort can be appropriate for simpler tasks, while more demanding tasks may benefit from a higher effort setting. Different modules may have their own specific configuration options. For example, the MultiChainComparison module has an M parameter that controls how many reasoning paths to compare:

# Configure MultiChainComparison to compare 5 reasoning paths
mc_solver = dspy.MultiChainComparison(MultipleChoice, M=5)

Configuration options give you fine-grained control over how modules behave, allowing you to adapt them to your specific needs. As you become more familiar with DSPy, you'll develop an intuition for which configuration options to use in different situations.

One of the most powerful features of DSPy is the ability to compose modules into larger programs. Just as you can combine PyTorch modules to build complex neural networks, you can combine DSPy modules to build sophisticated language model applications.

To create a custom module, you define a new class that inherits from dspy.Module and implements a forward method. Inside this method, you can use other DSPy modules to process inputs and generate outputs.

Here's an example of a custom module that implements multi-hop retrieval, a technique for finding information across multiple sources:

class Hop(dspy.Module):
    def __init__(self, num_docs=10, num_hops=4):
        self.num_docs, self.num_hops = num_docs, num_hops
        self.generate_query = dspy.ChainOfThought('claim, notes -> query')
        self.append_notes = dspy.ChainOfThought('claim, notes, context -> new_notes: list[str], titles: list[str]')

    def forward(self, claim: str) -> list[str]:
        notes = []
        titles = []

        for _ in range(self.num_hops):
            query = self.generate_query(claim=claim, notes=notes).query
            context = search(query, k=self.num_docs)
            prediction = self.append_notes(claim=claim, notes=notes, context=context)
            notes.extend(prediction.new_notes)
            titles.extend(prediction.titles)

        return dspy.Prediction(notes=notes, titles=list(set(titles)))

This module uses two ChainOfThought modules: one to generate search queries based on a claim and accumulated notes, and another to extract new notes and titles from the search results. It performs multiple "hops" of retrieval, using the information gathered in each hop to inform the next one.

By composing modules in this way, you can create complex workflows that leverage the strengths of different prompting techniques. This modular approach makes it easy to experiment with different strategies and iterate on your designs.

In this lesson, we've explored the DSPy modules, the building blocks of DSPy programming. We've learned about the core module types, how to use and configure them, and how to compose them into larger programs. By understanding these concepts, you're now equipped to build sophisticated language model applications that can tackle a wide range of tasks.

As you continue your journey in DSPy programming, remember the consistent three-step pattern for using modules: declare with a signature, call with input arguments, and access the outputs. This pattern will guide you as you explore new modules and develop your own custom solutions.

With these tools at your disposal, you're ready to create powerful AI systems that can understand and generate natural language with precision and creativity. Happy coding!