How to Reduce the Unit Cost of AI Work
The gap between a naive AI implementation and a tuned one is wide enough to decide whether a product makes money. Repeated content can be billed at a fraction of full price, work nobody is waiting on can run at a discount, and much of what an agent re-reads on every step was never needed. This guide works those levers to make the work itself cheaper, without swapping models: you keep the model and cut what the work costs to run.
Developing
Start here. Build the foundation.- 1
When a prompt repeats a fixed block across calls, such as a system prompt, tool definitions, or a reference document, structure the request so that block is cached and billed at the reduced rate, and keep the variable content below the stable part so you do not break the cache. This lever is close to free. You know it worked when cache hits show up in usage data and the fixed portion stops being charged in full every time.
- 2
When no user is waiting on the result, such as bulk classification, evaluation runs, or overnight enrichment, send the work through the asynchronous batch path instead of the real-time one. The trap is defaulting everything to the interactive endpoint just because it is the one you built first. You are done when that work runs on the discounted path and still meets its deadline.
Proficient
Build consistency and rhythm.- 3
When a workload stuffs documents, history, or tool output into the prompt, measure how much of it actually changes the answer and cut the rest. Inside an agent loop this is the strongest lever you have, because every step re-reads the context, so anything carried needlessly is paid for again and again. You know you got it right when input size drops and quality holds on a test set.
- 4
After a cost change ships, ask the people who depend on the output whether anything got worse before you claim the saving. You are done when they report no loss they can identify, or the loss is named and accepted as a deliberate trade. A saving measured only on the bill is how you find out weeks later that quality slipped and took a business result with it.
Mastered
Operate at the highest level.- 5
Turn the changes that worked into patterns others can apply without redoing the investigation: what the lever was, what it saved, what it cost in latency or quality, and when not to use it. You know it landed when another team applies one and reports the saving back. A pattern nobody has adopted is just a retrospective.
Common Pitfalls
Avoid the common failure modes.- Putting variable content near the top of a prompt, which silently invalidates the cache and returns the workload to full price.
- Running everything through the real-time endpoint because it is already built, and paying the interactive premium on work nobody reads until morning.
- Declaring a saving from the bill alone, without checking whether the people who use the output noticed a drop.