Most AI teams think of their API bill as the cost of intelligence. It isn't. It's mostly the cost of repetition.
We analyzed token spend across 12 enterprise AI deployments over 90 days. In every case, the majority of API cost came from a small number of task types - structurally identical calls that ran hundreds or thousands of times per day with minor input variations.
The pattern was consistent enough that we started calling it the repetition tax.
What the data showed
Across the 12 deployments, the top 3 task types by call volume accounted for an average of 71% of total token spend. These weren't the most complex tasks. They were the most frequent ones:
- Extracting structured fields from documents
- Classifying support tickets into routing categories
- Generating standardized summaries from call transcripts
None of these tasks require frontier-model intelligence. They require consistent, reliable execution of a well-defined pattern. But because they were built on a general-purpose API, they paid frontier-model prices.
Why teams end up here
The path is predictable. A team ships an AI feature using a capable foundation model. It works well in testing. They scale it. The API bill grows linearly with usage. By the time the cost is painful, the dependency is baked in.
The alternative - fine-tuning a smaller model - sounds expensive and slow. It requires labeled data, training infrastructure, evaluation pipelines, and ongoing maintenance. For a team that just wants to ship, it feels like a distraction.
This is the trap. The upfront cost of a specialized model feels higher than staying on a general API. The ongoing cost of the general API feels like a fixed overhead. But it's not fixed - it scales with every user, every transaction, every document processed.
The break-even math
For a task running 10,000 times per day on a frontier model at $15/M input tokens and $60/M output tokens, with an average of 2,000 input tokens and 500 output tokens per call:
- Daily cost: roughly $600
- Monthly cost: roughly $18,000
A fine-tuned 7B parameter model running on dedicated inference costs roughly $800–1,200/month at that volume, plus a one-time training cost that typically lands between $5,000–15,000 depending on dataset size and iteration cycles.
Break-even: 1–2 months.
After that, the specialized model is 90%+ cheaper per call, faster (smaller models have lower latency), and more reliable on the narrow task it was trained for.
What this requires
The bottleneck isn't the training - it's the labeled data and the evaluation infrastructure. You need:
1. A representative sample of production inputs (500–2,000 examples is usually sufficient for a focused task)
2. Ground-truth labels for those examples
3. An eval set you trust to measure whether the model is actually doing the job
Getting these right is harder than it sounds, especially the evals. Most teams underinvest in evaluation infrastructure and discover the model is wrong in ways they didn't anticipate. The solution is to build the eval set from real production failures, not from synthetic examples - which is part of what the trace capture layer exists to support.
The repetition tax is optional. Most teams just don't know they're paying it.