Predicting LLM Request Costs
Introduction: Why Predict Cost Before Sending?
Welcome back to Measuring and Understanding LLM Costs in Python. With Unit 1 behind us, we can already read a provider's usage object and turn it into an exact dollar amount. That skill has one limitation: it only works after the money has been spent.
Plenty of real decisions have to happen earlier. Should we reject a prompt that carries a 200-page document? Should this request go to a cheaper model? Should we warn a user that their query will be expensive under our configured rates? Every one of those choices needs a number before the request leaves our machine.
In this unit, we will count raw prompt tokens locally, assume an output length, convert both into a predicted cost, and then compare that prediction with the provider's actual usage.
Counting Raw Prompt Tokens Locally with tiktoken
Tokenization is not magic, and it is not a network service: it is a deterministic algorithm. The tiktoken library lets us count raw prompt text on our own machine, instantly and at zero cost. On CodeSignal, it is already available.
encoding.encode(text) returns a list of integer token IDs, one per token. We never look at the IDs themselves; we only need len(), which is the token count. A detail worth remembering: different model families use different encodings, so the encoding name must match the family we are calling, or our counts will be quietly wrong.
For intuition, the raw prompt text "Give three practical ways to reduce the cost of an LLM request." becomes 14 tokens: common words map to single tokens, while punctuation and unusual strings like LLM may split into several.
Local Prompt Counts and Provider-Reported Input
Our estimate has two halves, and they are fundamentally different. The tokens in the raw prompt text are deterministic: the string is sitting right there in a variable, and the tokenizer will split that same string the same way every time.
Complete billed request input can include more than that raw string: message serialization, instructions, tool schemas, images, and provider-side scaffolding may all add tokens. That means local raw-text counts are best treated as estimates of total billed request input, while provider-reported usage remains the source of truth after the call.
In other words, a local prompt count is useful for planning, but it is not a promise that the provider will report the same input total. In this unit’s illustrative run, the local tokenizer counts 14 raw prompt tokens while the provider reports 20 input tokens for the processed request.
Output tokens are unknowable until the model finishes answering, so we must supply an assumption. That assumption should cover the provider-reported output_tokens total, which can include hidden reasoning tokens as well as visible answer text.
In practice, that assumption comes from several places, depending on what kind of estimate we need:
- A historical average measured from past calls of the same prompt type, for expected-cost planning.
- A percentile such as p90 or p95 from past calls, for a high-side planning estimate.
- A
max_output_tokenscap set on the request, for a worst-case bound. - A deliberate budget: "we are willing to pay for about 80 tokens of an answer here."
If we present a cap-based worst-case estimate, we should enforce the same ceiling in the actual request by passing the corresponding max_output_tokens value to the provider call.
This guessed half usually dominates the total. With our rates from Unit 1, output costs $6.00 per million against $1.00 for input, so a single output token is worth six input tokens. Being off by 60 output tokens hurts far more than being off by 60 input tokens.

