Most AI pilots don't fail on accuracy. They fail on arithmetic. The demo classified fifty support emails beautifully, everyone applauded, and then someone asked what happens at five thousand a week — and nobody had an answer that held up.
The reason is simple. A pilot runs on generous conditions: low volume, a curated sample, and a smart person watching every output for free because they're excited about the project. Production runs on messy inputs, real volume, and a team that already has a job. If you want a defensible decision about whether to scale, you need a cost-per-task figure that includes all of that.
What the pilot bill leaves out
The model API charge is usually the smallest line in the eventual total, and it's the only one most pilots measure. Here's what tends to hide underneath it.
Retries and failures. Real inputs include scanned PDFs, forwarded threads with six replies, and records with fields nobody filled in. Each one may take two or three passes, or a fallback to a more expensive model. Your effective cost per successful task is higher than your cost per attempt.
Retrieval and context. If the system pulls from your documentation, ERP, or CRM to ground its answers, you're paying for that lookup and for every token of context you stuff into the prompt. Generous context is the easiest way to make a pilot accurate and the easiest way to make production expensive.
Human review time. This is the big one. If a person spends ninety seconds verifying each output, that's a real labor cost and it doesn't shrink with volume. Price it at the loaded hourly rate of whoever actually does it.
Integration and maintenance. Someone owns the connection to your systems, the monitoring, the prompt updates when a process changes, and the model migration when a provider deprecates a version. Amortize that over the tasks you expect to run.
Build the number, then build the comparison
Write the equation out plainly: inference cost per attempt, times average attempts per successful task, plus retrieval cost, plus review minutes at a loaded rate, plus a monthly maintenance allocation divided by monthly task volume. You'll have a per-task figure inside an hour.
Then do the harder half: cost the current process the same way. Not the idealized version — the real one, including the rework loop when something is entered wrong, the time spent chasing missing information, and the delay before a customer gets an answer. Teams often skip this step and end up comparing a fully-loaded AI cost against an imaginary free baseline. That comparison always loses, and it shouldn't.
Levers that actually move the number
When the math comes back unfavorable, the instinct is to abandon the use case. Usually the design is just untuned. A few things reliably help:
- Route by difficulty. Send the easy majority to a small, cheap model and escalate only the ambiguous cases to a larger one.
- Shrink the context. Retrieve the three relevant paragraphs instead of the whole document. Precision beats volume for both cost and quality.
- Cache the repeats. Many workflows ask near-identical questions all day. Answer once.
- Batch what isn't urgent. Overnight processing is cheaper than real-time and most back-office tasks don't need real-time.
- Narrow the scope. A system that handles the four most common request types well and routes the rest to a human is cheaper and better than one that attempts everything.
Don't forget the failure tax
Cost per task is only half the picture. You also need the expected cost of being wrong. An AI that drafts an internal summary and gets it slightly off costs a few minutes. An AI that posts to your general ledger, prices a quote, or emails a customer directly can cost you a correction, a credit, or a relationship.
So weight your estimate by consequence. High-consequence tasks justify heavier review, which raises cost per task — and that's fine, as long as you knew it going in. Low-consequence, high-volume tasks are where thin review and aggressive cost tuning pay off. The mistake is applying the same design to both.
Set the threshold before you look at the result
Decide in advance what per-task cost makes this worth scaling, and what makes it a no. Writing that number down first is what keeps a promising pilot from turning into a project that ships because momentum demands it rather than because the economics work.
And if the answer is no, that's a useful outcome. You spent a small amount to learn where the ceiling is, and you can redirect toward a use case with better margins — usually one with higher volume, lower stakes, and cleaner input data.
We do this modeling with operators regularly: sizing the real cost of an AI workflow, tuning the architecture until it clears the bar, and connecting it to the ERP, CRM, and web systems where the work actually lives. If you have a pilot that looked great in the demo and you're not sure what it costs at scale, get in touch — we'll help you run the numbers before you commit to the build.