Chain of Thought with Backtracking
Last Chapter
Welcome to the last chapter of Advanced Techniques in Prompt Engineering, titled "Chain of Thought with Backtracking". This is another advanced prompt engineering method that enables us to help LLMs find solutions in highly complex situations. Let's start by understanding the core idea behind it.
Understanding Chain of Thought with Backtracking
The core idea behind "Chain of Thought with Backtracking" is to help the LLM think backward after analyzing the situation instead of being only a one-way train. As you might remember, LLMs are designed to be next-word-prediction machines, and in case problems require a lookback, they naturally tend to become less and less effective. Our goal as Prompt Engineers is to help them navigate this challenge by creating specific instructions on how to backtrack after getting to a certain point.
Simple Example
Let's take a look at a simple example prompt that includes Chain of Thought and nothing more.
Sample output:
As you can see the LLM gives it a good attempt but gets the result completely wrong. The reason is that this is the kind of problem where working backward from the end is much easier than the other way around.
Introducing Backtracking
While progressing linearly through a problem is effective by using the standard Chain Of Thought method, there are instances where you might need to revisit and revise earlier steps. This is where "Backtracking" comes into play. It involves going back after the initial analysis to come up with the final answer. Let's enhance the prompt above to see an example:
Notice here a key component do not suggest an answer - this is done to avoid biasing the LLM. If the LLM produces an answer while doing the forward analysis (aka Chain of Thought), it will be more likely to stick to that answer even after Backtracking, even though it's likely to be incorrect.
Sample output:
This time the answer is absolutely correct and you can see the overall depth of the analysis is much stronger.
