Model Selection and Routing
Introduction: Price Is Not the Whole Decision
Welcome to Model Selection and Routing in Python, and congratulations on reaching the first unit of this course. In the previous course, we built the measurement tools: we read provider-reported usage, split it into billing categories, and turned those numbers into dollars. Measurement told us what a request did cost. It never told us whether that was the right amount to spend.
That is the question we take up now. When a provider offers a cheap tier and a premium tier separated by a fivefold price difference, which one does a given task actually need? Answering that requires looking at two things at once: the money spent and whether the answer is good enough to use.
In this lesson, we will:
- Define a price sheet that covers two model tiers at once;
- Write a single cost function that prices either tier without branching;
- Send one identical prompt to both tiers;
- Judge the two answers against the cost gap using a written-down standard.
By the end, we will have a repeatable comparison procedure and a clear sense of why "always pick the strongest model" is a decision worth questioning.
Two Tiers, Two Price Sheets
Everything starts with naming the two models and writing down what each one charges. We keep the model names in constants and use those constants as dictionary keys, so each model name is spelled out in exactly one place.
The structure is model → category → per-token rate, and it uses the same four billing categories we already worked with: fresh input, cache writes, cached input, and output. Reading the two blocks side by side, the premium tier costs exactly 5x as much as the cheap tier in every single category: $1.00 versus $5.00 per million fresh input tokens, $6.00 versus $30.00 per million output tokens, and the same ratio for both cache categories. That uniform multiplier is what makes the comparison clean: nothing is hidden in an unusual rate.
Pricing Any Tier With One Function
With the table in place, one function can price a response from either model. Notice that model is a parameter, not an assumption baked into the body.
As you may recall from the previous course, usage.input_tokens is a total that already contains the cached and cache-write tokens, so we recover the fresh portion by subtracting both. Each category is then multiplied by its matching rate, and the four products are summed.
The design point matters more than the arithmetic: because rates comes from the table lookup, there is no if model == ... branch anywhere. Adding a third tier would mean adding a table entry, not editing this function.
Sending One Identical Prompt to Both Tiers
Now we run the experiment. The prompt is defined once, outside the loop, and every call receives that same string.
Holding the prompt constant is the entire point of the setup: the model name is the only thing that changes between iterations, so any difference in cost or answer text is attributable to the tier and nothing else. For each model, we print three things: the model name, the cost computed by cost_for() to six decimal places (these amounts are fractions of a cent), and the answer itself through response.output_text. Printing the cost and answer together is deliberate, since we cannot judge one without the other.
Reading the Two Runs: Cost Side
Here is what the program prints. Let us look at the two cost lines first and postpone the answers for a moment.
The premium call costs $0.001775 versus $0.000535, making it roughly 3x more expensive for one tiny question. Fractions of a cent feel harmless, but multiply that $0.00124 difference by 5,000 requests a day, and the premium default adds about $6.20 daily, or nearly $2,250 a year, for this one request shape. Notice also that the observed gap is below the 5x rate ratio: the cheap answer was longer, so it billed more output tokens, and output volume affects real costs just as much as rates do.
Reading the Two Runs: Quality Side
Cost only matters once we know both answers are usable, so we need a standard written down before we read them. For this prompt, an acceptable answer must state the mechanism (a separate structure that avoids scanning the whole table) and at least one tradeoff. Both answers clear that bar: each names a B-tree structure that avoids a full scan, and each gives a tradeoff, with the cheap answer covering both extra storage and slower writes.
So the premium spend bought nothing here, which leads to the rule we will carry through this course:
- Quality is a constraint that cost optimization must satisfy, not a number we trade away for savings;
- A cheaper model is the right answer only when it clears the bar we defined in advance;
- The verdict can flip on harder work: multi-step reasoning, long documents, or strict output formatting are exactly where premium tiers earn their price.
One honest caveat: a single comparison on a single prompt is evidence, not proof. Real routing decisions rest on many samples across the task types an application actually serves.
Conclusion and Next Steps
We now have a small but complete comparison workflow: a per-model price table keyed by model name, one tier-agnostic cost function driven by that table, one identical prompt sent through both tiers, and an explicit quality bar that gives the cost difference meaning. The important shift is in how we frame the decision: we no longer ask "what did this cost?" but "did this task need what we paid for?"
This comparison is the manual version of a judgment we will soon hand over to code. In the practices ahead, we will build the billing-category split, the table-driven cost function, and the two-tier comparison ourselves, then read the results with the same critical eye. Let us head into the exercises and put this tradeoff to the test.
