The first AI invoice is a surprise for most teams. Not because the price per token is high, but because nobody was watching how many tokens a single feature actually sends, and it turns out to be far more than anyone guessed.
The useful part is that most of that spend is design rather than destiny. A handful of decisions about what you send, how often and to which model usually takes the bill down a long way without touching the quality of what comes back.
You're paying for the same context over and over
Most applications resend a large, unchanging block of text on every single call: a system prompt, a set of tool definitions, a style guide, a reference document. None of it has changed since yesterday, and you pay for all of it every time.
Prompt caching fixes that, and it costs you nothing but discipline. Put the stable content first, the volatile user input last, and keep the order consistent so the provider can reuse what it already processed. Cached input is billed at a fraction of fresh input.
Output costs more per token than input, and it's also the part your user waits for. Setting a hard limit and asking for structured output instead of prose is the cheapest change available to you.
The other quiet leak is chat history. Twenty turns in, you're resending the entire conversation with every message. Summarize the old turns and keep the recent ones, and cost stops growing with the session.
Once the obvious waste is gone, three structural moves do the rest.
Route by difficulty: run the cheaper model first and escalate only when a check fails, because most requests in a real workload are easy and never needed your best model. Batch anything that isn't waiting on a person, since classification, enrichment and evaluation jobs are usually fine finishing within the hour and batch endpoints are priced for exactly that. And retrieve rather than stuff: pulling the three relevant paragraphs into the prompt beats sending the whole document and hoping, which costs less and often answers better because the model has less to ignore. Put a semantic cache in front of all of it if your traffic repeats, which most support and search traffic does.
One rule holds the whole thing together. Build an evaluation set before you start cutting, because every technique here can quietly make output worse, and a cheaper answer that's wrong is the most expensive result on the list.
Measure tokens per request before you optimize anything
Cost is an engineering property
We treat token spend the way we treat a query plan: something you measure, attribute to a feature, then fix at the source. It's the same discipline behind getting an AI pilot into production, where the bill is often what stops one.
If nobody on your team can say what a single request costs, that's the place to start rather than a pricing conversation with your provider. Instrument it, attribute it per feature, then work the list above from the top.
Most teams find the first two changes cover the bulk of the waste and the rest is tuning. That's the kind of work we do across our stack, whether it lands in Python services or data work.