Easy to Start, Hard to Stop: The Token Economics That Decide Whether an AI SaaS Survives

After a few years of building with LLMs non-stop, at Telos Labs for clients and at SimpleDocs as the person responsible for the technology, I’ve come to a conclusion that feels obvious in hindsight and is almost never acted on in time: for an LLM-dependent SaaS business, token economics are not a line item. They are a survival question.

The lesson here is not about the cost of agentic engineering or what you pay for coding tools. I want to dive into what happens to a software business when a meaningful share of its recurring revenue is consumed by the tokens needed to serve its own features.

Here’s how concrete that gets. At SimpleDocs, we priced out what it would cost to run our AI functionality at full production scale: one frontier provider, standard ingestion approach. For the contract we modeled it against, the projected one-time ingestion cost alone came out at more than ten times the annual contract value because of the scale of their data, before serving a single ongoing query. We ran that model during architecture, and before those choices were locked in, which is the only time running it is cheap. For most teams, they run it when the bill arrives. We were able to change the LLM approach and the token economics meaningfully, thereby avoiding a costly mistake.

Model providers can eat your margins

Mechanics decide your unit economics long before you notice them operating.

Frontier providers are running the oldest playbook in platform economics: subsidize adoption, become the default, let switching costs accumulate, then collect through pricing that scales with your success. The token top-ups and usage resets filling your feed are not generosity. They are discounts calibrated to keep your workloads on-platform while your prompts, evals, and integrations quietly specialize around one model family. Every month you stay single-provider, migration gets more expensive.

Your token bill is their revenue line, so every incentive on their side points toward more usage at higher effective prices, not toward protecting your margin. With per-token pricing, your heaviest customers generate their biggest invoices, and the only party with any reason to cap that spend is you.

Lock-in also has a quality cost that gets less attention than the invoice. Building on a single model lineage means inheriting that lineage’s blind spots along with its strengths, and you can’t see them from inside: only an eval against a different family shows you what your model reliably gets wrong.

Why everyone builds into this trap

Early on, you build for capability. A pilot should over-index on frontier models: they’re fast to build on, the quality ceiling is high, and validation work would slow you down before you know what you are validating. At pilot scale, the per-token cost is invisible.

The problem is that pilots run on small datasets, so the structural exposure only becomes visible when you model full-scale production. By then, the architectural choices are baked in. A single-provider assumption that works at 10,000 documents can be catastrophic at 500,000. This is the general failure mode of irreversible decisions: they get made while the commitment is high and the learning is low, and by the time you have the information, you no longer have the option.

The right time to model your full-scale cost is during initial architecture. Not after you’ve shipped, scaled, and discovered that your largest customers are your least profitable ones.

What we do about it

The solution to this problem is a technology setup that mixes and matches models based on performance, cost, and latency, backed by the two capabilities that make it safe: routing and evals. This is easier said than done, and it results from practices that are individually unglamorous. If you’re the one buying rather than building, these same practices are what the five questions to ask before you approve an AI project are designed to surface.

1. Scope every AI action to the current view by default.

Most AI systems default to the worst possible scope: everything. A user asks a reasonable question in a chat interface, the system fans out across the entire document estate, and a single message generates a five-figure compute bill. The fix here is product discipline: default to the current folder or view, require an explicit choice to go broader, and show the blast radius before a wide-scope action runs. This is also just better behavior. When someone asks about a clause type, they almost always mean within this matter, not across everything. The cheap default and the correct default are usually the same default.

2. Enforce hard caps server-side, not advisory limits.

Advisory limits are suggestions, and power users treat them that way. Your most active users are your most expensive users, and a warning that says “this will touch 50,000 documents, are you sure?” does not stop someone who is sure. A server-side cap that refuses to execute above a threshold does. One of these approaches informs users, while the other controls cost. Build both, and don’t confuse them.

3. Test lower-cost models against every task, and pay for frontier when needed.

Instead of assuming frontier models for everything, we test where open-source and lower-cost models could meet our quality bar, and price the architecture around that blend. On the ingestion contract we modeled, routing high-volume, well-defined work to cheaper models cut the projected cost by roughly 60 percent.

Frontier models earn their keep on complex reasoning, novel formats, and tasks where generalization matters more than cost. High-volume, well-defined work is where cheaper models get tested first: bulk ingestion, standard field extraction, classification on known categories. Once a model is validated on your domain, it runs at a fraction of the per-token cost, and on some infrastructure configurations at no per-token cost at all.

In practice, a mature application tends toward a blend. Pay the premium where it adds value. Everywhere else, use the cheapest capable approach.

4. Pay the validation toll, once.

None of the above is safe without evals. Routing a task to a cheaper model on vibes is how you trade a cost problem for a quality problem. You have to prove the model performs on your task before you trust it in production, and you have to keep that proof current as models and prompts change.

At SimpleDocs, this is a built system, not a policy. Every production prompt change is scored against a suite of LLM-judged quality criteria, dozens of eval packs and well over a hundred test scenarios at last count, run on golden documents with known terms: a real governing law to cite, a real notice period to catch, a real liability cap to respect. Each criterion carries its own passing threshold, and the judge’s written reasoning is stored alongside every score, so a failing case tells you what to fix rather than just that something broke. The same machinery fans out across multiple models at once, scoring a candidate against the identical suite the incumbent passed. That is what turns switching providers into a data decision rather than a vibe check, and it’s what makes routing to cheaper models safe rather than reckless.

That validation work is a genuine upfront cost, but also a one-time cost. The per-token savings are permanent. This trade, fixed validation cost against unbounded usage cost, is the whole game.

The unit economics you end up with

All of this pushes toward one structural goal: moving cost off an unbounded, usage-driven axis and onto a bounded, predictable one.

A system whose inference costs scale with query intensity cannot be priced confidently, because query intensity is controlled by your users, not by you. A system whose actions are scoped, capped, and routed to the cheapest capable model scales with data volume instead, which is the same axis your pricing charges on. When cost and price move together, margins are a design property rather than a hope.

There is an inversion available here that most SaaS businesses never see. Every task that migrates from a frontier model to a validated cheaper one permanently leaves the token bill. Done right, the system gets less expensive to serve as it matures. This is cost efficiency that improves with scale instead of degrading under it.

None of this requires rearchitecting from scratch. It requires making deliberate choices early, while they’re still cheap to make. The pricing model is easy to start and hard to stop, so start accordingly.

Frequently asked questions

How much does it cost to run LLMs at production scale?

Far more than pilot-stage numbers suggest. Per-token pricing looks negligible on small datasets, but costs scale with usage intensity rather than customer count. A document platform ingesting hundreds of thousands of agreements on a single frontier provider can face six-figure one-time costs before any recurring usage begins. The pattern to watch for: any workload where one user action can fan out into thousands of AI calls. That's where token bills outrun contract values.

Why shouldn't you build an AI product on just one LLM provider?

Two reasons. First, pricing exposure: when your entire inference load sits with one provider, you have no leverage and no fallback when prices change or rate limits tighten. Second, quality blind spots: every model lineage has characteristic weaknesses, and you can't detect them without comparing output against a different model family. Multi-provider routing turns both problems into design choices you control instead of risks you absorb.

Are open-source models good enough to replace frontier models in production?

For well-defined, repetitive tasks on a constrained domain, frequently yes. Extraction, classification, and structured output on contract language are examples where a validated open-source model often matches frontier quality at a fraction of the cost. The critical word is validated: the migration is only safe when the candidate model passes the same evaluation suite the frontier model passed, judged against the same quality thresholds on representative production documents.

What's the first step to fixing AI costs in a product that's already live?

Check the default scope of your AI actions. Most cost blowouts trace back to a single design choice: user actions that run against the entire dataset by default rather than the current folder or view. Fixing the default scope and adding server-side caps costs nothing to implement and immediately eliminates the most expensive accidental queries. Do that before evaluating any model migration, because it addresses the structural exposure rather than just discounting it.