The Trilemma Is Not a Curve But a Surface
The familiar framing treats latency and cost as a single dial: pay more for faster responses. That intuition is wrong because quality is the third axis, and the three quantities are coupled through the same underlying lever — how much computation you spend per token generated. Larger models, longer reasoning chains, retrieval over more documents, and higher sampling budgets all raise quality, and all of them raise both latency and cost in lockstep. You are not choosing a point on a line. You are choosing a point on a surface, and the surface is curved such that improvements on any one axis past a threshold demand disproportionate sacrifice on the other two. The job of the system architect is to locate the operating point that the product requires, not the one the model permits.
The Mechanism: Compute Per Request Is the Shared Currency
Every quality gain in a generative system is purchased with floating-point operations, and those operations have a fixed physical cost in time and money. A frontier model with ten times the parameters produces better answers and is roughly ten times more expensive per token and meaningfully slower to first token. Chain-of-thought reasoning multiplies output length, which is the dominant driver of both wall-clock latency and per-call cost because generation is sequential and autoregressive. Retrieval-augmented generation improves grounding but inflates the prompt, and prompt length governs the prefill stage that the user waits through before a single output token appears. The architect who does not hold this shared currency in mind will optimize one axis and discover the bill — in dollars or in milliseconds — has migrated rather than vanished.
Failure Modes Cluster at the Corners
Systems fail not in the interior of the surface but at its corners, where one axis has been maximized and the neglected axes collapse. Pushing quality without latency discipline yields an assistant that is correct but abandoned, because users interpret a multi-second blank screen as a broken product regardless of what eventually arrives. Pushing latency and cost without a quality floor yields confident, fast hallucination — the most dangerous corner, because it degrades trust invisibly until a single wrong answer reaches a customer or a regulator. The instructive cases:
- A support bot that streams instantly but routes 8% of refund requests on fabricated policy is cheaper per ticket and catastrophic per incident.
- A research tool that reasons for ninety seconds is brilliant in a demo and unused in production, because no one waits ninety seconds twice.
- A summarizer scaled to the largest model for prestige burns margin on a task a model a tenth the size handles indistinguishably.
The Decision: Set the Operating Point Per Task, Not Per Platform
The controlling implication is that the operating point belongs to the task, not to the organization's default model choice. A single product contains many inference calls with radically different tolerances: autocomplete must answer in under a hundred milliseconds and can be wrong cheaply; a legal-clause analysis can take ten seconds and must never be wrong cheaply. Treating these with one model and one budget guarantees you are overpaying on the easy calls and under-serving the hard ones. The mature architecture routes — classifying each request and dispatching it to the smallest, fastest, cheapest configuration that clears the quality bar that particular task demands.
This reframes the build. The deliverable is not a model but a policy: an explicit, measured mapping from request classes to operating points, with a quality floor defined and monitored per class rather than assumed globally. Latency budgets and cost ceilings become product requirements written down before model selection, not consequences discovered after launch. The organizations that win the inference economy are not those with access to the best model; access is increasingly uniform. They are those who have decided, deliberately and per task, exactly how much quality each decision is worth — and have built the routing to spend not one floating-point operation more.