The first surprising AI invoice usually arrives about six weeks after something works. A team builds an assistant that drafts quotes or triages support tickets, everyone loves it, usage spreads, and nobody notices that cost scales with adoption rather than headcount. By the time finance asks what this line item is, the honest answer is often "we're not sure."
That's not a modeling problem or a vendor problem. It's a measurement problem. Most companies track AI spend the way they track a SaaS subscription — one number, once a month — when the real economics live at the level of a single task.
Pick the Unit of Work First
Before you can control cost, you need to name what you're buying. Not "AI spend" but "one quote drafted," "one invoice coded," "one inbound lead researched and routed," "one support ticket summarized and tagged."
This sounds pedantic until you try it. Teams often discover their AI feature doesn't have a clean unit at all — it's a chat window where people do undefined work of varying value. That's fine for exploration and terrible for budgeting. If you can't define the unit, you can't tell whether the system is cheap, expensive, or simply busy.
Once you have a unit, three numbers tell you almost everything:
- Cost per attempt — what one run costs in model calls, retrieval, and tooling
- Attempts per good result — including retries, regenerations, and the ones a human throws away
- Human minutes saved per good result — measured against how the step actually ran before, not an idealized version
Multiply the first two and you have true cost per outcome. Compare it to the third and you have an answer you can defend in a budget meeting.
Follow the Tokens, Not the Model Name
When per-task cost comes in higher than expected, the cause is rarely the model tier. It's usually one of a few familiar culprits.
Context bloat is the most common. Someone stuffs an entire policy manual, a long conversation history, and a generous set of retrieved documents into every call because it improved quality once, in testing. Nobody goes back to find out how much of it the model actually needed.
Agent loops are the second. A system that can call tools will happily call them repeatedly, re-reading the same record, second-guessing itself, and burning through steps on a task a human would finish in one pass. Without a step cap, a single edge case can cost many times what a typical run costs.
Silent retries are the third. Error handling that quietly re-runs failed calls is good engineering and a budget hazard when nothing logs how often it fires.
You can't see any of this without per-task logging: input size, output size, tool calls, retries, latency, and outcome, tied to a task ID. That instrumentation is cheap to add at the start and painful to retrofit once the feature is in production.
Pull the Cheap Levers Before Switching Models
Downgrading to a smaller model is the obvious move, and often the wrong first one, because it trades cost for quality in a way users notice immediately. Try the structural levers first.
Trim retrieval to what the task needs. Cache system prompts and reference material that doesn't change between calls. Summarize conversation history instead of replaying it. Cap the number of steps an agent can take and fail loudly when it hits the ceiling. Move anything that doesn't need an instant answer into a batch job that runs overnight.
Then route. Most workloads contain a long tail of simple cases and a short head of hard ones. A classifier or a simple rule can send routine work to a smaller model and escalate only the ambiguous cases, with a human review path for the ones that stay ambiguous. Standardizing on one expensive model for everything is the AI equivalent of flying every package overnight.
Put a Ceiling on It Before You Need One
Every AI feature in production should have hard limits: a per-user rate limit, a per-day budget cap, and a kill switch someone is authorized to flip without a meeting. Spend alerts should fire at a threshold that still leaves room to react, and they should go to a person, not a channel.
This isn't pessimism. It's the same discipline you'd apply to any variable-cost resource. The difference is that AI spend scales with enthusiasm, and enthusiasm is exactly what you want in the early months.
Decide What "Worth It" Actually Means
The benchmark isn't zero. It's what the step costs today — fully loaded, including the rework, the delay, and the senior person who gets interrupted to answer the same question each week. Some tasks will clear that bar easily. Others won't, and the right call is to stop running them through a model and fix the process instead.
Good AI economics come from the same place good operations do: clear units of work, honest measurement, and limits set before the pressure arrives.
If you're scaling an AI feature past its pilot and want a cost model that holds up — or a second opinion on whether the workload belongs in a model at all — we'd be glad to talk it through. Reach out to Infraxio and tell us what you're running.