LLM Unit Economics: Cost per Successful Task, Not per Token

What you need to know

Three claims sit underneath everything that follows, and if you accept them the rest is arithmetic.

First, the per-token price is an input cost, not a unit economic. Your customer does not buy tokens. They buy a summarised contract, a triaged support ticket, a reviewed pull request. The number of tokens that takes is a function of your engineering, not of the rate card, and it moves by multiples between a naive implementation and a careful one.

Second, inference sits in cost of goods sold, and that changes the shape of the business. Traditional software has a marginal cost close to zero, which is where the 80 per cent gross margins came from. An AI product pays real money for every request served, forever. Published benchmarks for 2026 put AI product gross margins in the region of 50 to 60 per cent against roughly 70 to 90 per cent for mature SaaS. Sources vary and definitions of COGS differ, so read those as directional — but the direction is not in dispute.

Third, the average hides the thing that will hurt you. Blended cost per customer is a comforting number that conceals a distribution in which a minority of accounts consume a majority of inference. Under flat pricing, some of those accounts are losing you money right now, and you will not see it in the mean.

The only cost formula that matters

Start from the outcome and work backwards. For a given task type — one summarisation, one ticket resolution, one code review — the true cost is:

cost_per_successful_task
  = (  attempts_per_task
     * (input_tokens  * input_price
        + output_tokens * output_price)
     + retrieval_cost          # embeddings, vector search
     + tool_cost               # external API calls the task makes
     + orchestration_cost      # compute running the loop itself
    )
    / success_rate

Two terms in that expression do the damage, and both are usually missing from the spreadsheet.

attempts_per_task counts every model call made in service of one outcome. A single-shot completion has a value of one. An agent loop that plans, calls three tools, evaluates its own output and revises once has a value of seven or eight, each with its own growing context. Teams that quote a per-task cost based on one call routinely underestimate by an order of magnitude.

Dividing by success_rate is the term people find uncomfortable, and it is the most important one. If your pipeline produces an acceptable result 80 per cent of the time and the other 20 per cent is retried, escalated to a stronger model, or handed to a human, then the cost of a delivered outcome includes the cost of the failures. At an 80 per cent success rate your true cost per outcome is 1.25 times the cost per attempt before you count the escalation itself.

This is also why the cheap model is not always the cheap model. Consider the same task through two configurations:

Configuration	Cost per attempt	Success rate	Escalation cost	True cost per outcome
Small model only	$0.004	72%	Human review at $0.60	$0.172
Small model, escalate to large on validator failure	$0.004	72% then 94%	$0.045 per escalation	$0.021
Large model only	$0.045	94%	Human review at $0.60	$0.083

The figures are illustrative, but the ordering is the point and it is robust: the escalation ladder beats both pure strategies, because it pays the small price on the majority of requests and the large price only where it is needed. Note also that the small-model-only row is the most expensive, because its failures land on the most expensive resource you have, which is a person. Cheap models with no validator are frequently a false economy, and only a cost-per-outcome model reveals it. The mechanics of building that ladder are covered in our guide to cache, route and compress.

Getting the data: tag at the call site

You cannot compute any of this without attribution, and attribution has to be added at the point of the call. Retrofitting it from provider invoices is not possible, because the invoice is one number.

The discipline is a single wrapper that every model call in your codebase goes through, which records a structured event per call.

@dataclass
class LLMCallRecord:
    ts: datetime
    tenant_id: str          # which customer
    surface: str            # which product feature
    task_id: str            # groups all calls for one outcome
    attempt: int            # 1, 2, 3...
    model: str              # as RETURNED by the provider, not requested
    input_tokens: int       # from the response usage block
    output_tokens: int
    cached_tokens: int      # cache reads are priced differently
    latency_ms: int
    outcome: str            # "ok" | "validator_failed" | "error" | "escalated"

def call_model(prompt, *, tenant_id, surface, task_id, attempt=1, **kw):
    t0 = time.monotonic()
    resp = provider.complete(prompt, **kw)
    emit(LLMCallRecord(
        ts=now(), tenant_id=tenant_id, surface=surface,
        task_id=task_id, attempt=attempt,
        model=resp.model,
        input_tokens=resp.usage.input_tokens,
        output_tokens=resp.usage.output_tokens,
        cached_tokens=getattr(resp.usage, "cached_input_tokens", 0),
        latency_ms=int((time.monotonic() - t0) * 1000),
        outcome="ok",
    ))
    return resp

Three details in that record are worth defending in review, because they are the ones people cut.

Record the model the provider returned, not the one you asked for. Routing layers, fallbacks and provider-side aliasing all mean the model that served your request is sometimes not the one your code named. If you price against the requested model you will be quietly wrong.

Record cached tokens separately. Cache reads are priced very differently from fresh input, and a cost model that treats all input tokens identically will misstate the effect of your caching work in both directions. If you have invested in caching, this is the field that proves it paid.

Carry task_id through the whole loop. This is the field that converts per-call data into per-outcome data, and it is the one most often omitted. Without it you can report cost per API call, which nobody cares about.

From cost per task to margin per customer

Once calls are tagged, margin per account is a query rather than a project. The structure is:

-- Monthly gross margin by tenant
SELECT
  t.tenant_id,
  r.mrr,
  SUM(c.input_tokens  * p.input_price / 1e6)
    + SUM(c.output_tokens * p.output_price / 1e6)
    + SUM(c.cached_tokens * p.cached_price / 1e6)   AS inference_cost,
  r.mrr - (/* inference_cost */ ... ) - t.other_cogs AS gross_profit
FROM llm_calls c
JOIN model_prices p
  ON  p.model = c.model
  AND c.ts >= p.effective_from
  AND c.ts <  COALESCE(p.effective_to, 'infinity')
JOIN tenants t ON t.tenant_id = c.tenant_id
JOIN revenue  r ON r.tenant_id = c.tenant_id AND r.month = date_trunc('month', c.ts)
GROUP BY 1, 2;

Then plot the distribution, not the average. The chart worth having on a wall is gross margin per account, sorted ascending. In most AI products it looks the same: a left tail of accounts at negative margin, a long flat middle, and a right tail of light users subsidising everyone else.

What you do about the left tail is a business decision with four options, and it is worth being explicit about which one you are choosing rather than drifting.

Option	When it fits	Risk
Engineer the cost down	Heavy accounts use a feature you can optimise	Engineering time is not free either
Introduce usage limits or metering above an allowance	Usage varies widely and customers accept metering	Buyers dislike unpredictable bills
Reprice the segment	Heavy users cluster in an identifiable segment	Churn among your most engaged accounts
Accept the loss deliberately	Strategic logos, or genuine expansion potential	Only defensible if it is a decision, not an accident

All four are legitimate. The failure mode is not choosing one — carrying loss-making accounts because nobody has computed which ones they are.

The metrics your investors will ask for

Fundraising expectations have caught up with the cost structure. Whether you are raising in London or Bengaluru, expect the diligence to include some version of these, and it is far better to bring them than to be asked.

• Gross margin, with COGS defined explicitly. State what you have included. Inference, vector database, embedding generation and the compute running your orchestration all belong in COGS. If you have excluded something, say so — an unexplained 78 per cent invites more scepticism than an explained 54 per cent.
• Inference cost as a percentage of revenue. Reported figures for 2026 vary considerably by company stage and product type, with published estimates ranging from single digits at some public SaaS companies disclosing the line separately to substantially higher shares at scaling-stage AI-native businesses. Know your own number and its trend.
• Margin trend over time, at constant model prices. This is the one that distinguishes a team with a handle on its costs. Falling per-token prices flatter everyone; holding prices constant and showing your margin still improving demonstrates that the improvement came from your engineering.
• Cost per successful task, by task type. Along with the success rate that produced it. This shows you understand your product as a system rather than as an invoice.
• Margin distribution across accounts. With a statement of what you intend to do about the left tail.

Where the money usually is

Once you can see the numbers, the same handful of findings turn up across most teams. In rough order of how often they are the largest single item:

Context that nobody reads. Retrieval configured to return twelve passages when four would do, chat history carried in full for the entire session, a system prompt that grew by accretion and is now three thousand tokens on every call. This is usually the biggest and cheapest win, and it needs no model change.

Uncached repeated prefixes. If the same long instruction block leads every request and you are not using provider-side caching, you are paying full input price for identical tokens thousands of times a day.

Retries that nobody counted. Look at the distribution of attempt in your call records. Teams are routinely surprised by how much of their volume is second and third attempts, because the retry logic sits in a library and the failures never surfaced.

The wrong model on the easy majority. Most task distributions are bimodal: a large volume of straightforward requests and a tail of genuinely hard ones. Sending everything to the model sized for the tail is the most common overspend, and it is what the escalation ladder above exists to fix.

Answers you already gave. In question-answering and support workloads a meaningful proportion of incoming requests are semantically equivalent to something already handled. Serving those from a cache instead of the model is covered in our guide to semantic caching for LLM apps.

Forecasting: the model that does not embarrass you

Once you can measure cost, someone will ask you to project it. The naive approach — take last month's inference bill and grow it by the revenue growth rate — is wrong in a specific and predictable way, and it is worth understanding why before you present a number to a board.

Inference cost is driven by three multiplicative factors that grow at different rates, and collapsing them into one growth rate destroys the information you need.

monthly_inference_cost
  =  active_accounts               # grows with sales
   * tasks_per_account_per_month   # grows with adoption and feature launches
   * cost_per_successful_task      # falls with your engineering, and with vendor prices

The first term tracks your commercial motion. The second is the one that surprises people: within a cohort of existing customers, usage per account typically rises over the first several months as the product gets adopted more deeply and as you ship features that generate more calls. That means a business with flat customer numbers can still see inference costs climb steadily. The third term is the only one you directly control, and it is the only one that can move in your favour.

Forecast each separately, from your own cohort data rather than from a blended average. Take accounts that joined six months ago and look at how their monthly task volume has moved since; that curve, not your overall growth rate, is what predicts the second term. Then run at least a downside case in which vendor prices do not fall at all, because a plan that only works if someone else cuts their prices is not a plan.

Three mistakes that recur

Counting only the generation model. Embedding calls, reranking, safety classification, a small model used for routing and any judge model in your evaluation loop all cost money and all scale with traffic. In retrieval-heavy products the supporting calls can approach the cost of the main generation, and they are usually invisible because they sit in a different code path.

Forgetting internal usage. Your own evaluation runs, regression suites, staging traffic and the team using the product internally all appear on the same invoice. Tag them with a reserved tenant identifier so they can be excluded from customer margin calculations and tracked as the research-and-development expense they actually are. Teams that skip this routinely misattribute a significant share of spend to customers who did not incur it.

Assuming a price cut is a margin improvement. When a provider drops prices, the saving is only banked if your usage does not expand to fill it. Track whether your task volume rises after a price cut — very often it does, because the cheap tier makes previously uneconomic features viable and someone ships them. That may be an excellent outcome, but it is a decision to spend the saving on capability rather than on margin, and it should be made explicitly rather than discovered in the next invoice.

A note on currency and region

One thing that rarely appears in cost-model templates and matters for teams outside the United States: your inference costs are dollar-denominated and your revenue may not be. A team in Chennai billing in rupees, or in Manchester billing in pounds, carries exchange-rate movement directly in gross margin, and a few percentage points of currency drift is the same magnitude as a decent optimisation programme.

Two practical responses. Track margin in your reporting currency and in dollars, so you can tell a currency effect from an operational one. And when you set prices, decide explicitly whether you are absorbing currency risk or passing it through — because doing so by default means absorbing it. Teams with an India presence should also check whether subsidised compute programmes change the maths for self-hosted workloads; we looked at that in our comparison of IndiaAI GPU subsidies against UK equivalents.

Where to start

If you have none of this today, the order that gets you furthest fastest is short. Put the logging wrapper in front of every model call and backfill nothing — a fortnight of clean data beats a year of reconstructed guesses. Build the price table and the pricing query. Compute cost per successful task for your top three surfaces, including retries and failures. Plot margin per account and look at the left tail. Then pick the single largest line and work on it.

That sequence takes a week or two of engineering time and it changes the conversation permanently. The difference between a team that can say "this feature costs eleven paise per completed summary and the margin on our heaviest account is nine per cent" and one that can say "our bill was about forty thousand dollars last month" is not sophistication. It is whether anyone decided to measure. More cost engineering guides are collected in our tips section.