Jargonautica
Cost, Speed and Service Levels

Cost Per Task

Also called: Unit Economics

What it actually costs to do one useful unit of work, as opposed to the headline price per million tokens. A single support ticket resolved might involve retrieval, several model calls, a reranking step and a retry, and the number that matters to whoever signs off the project is the total for that ticket. Working it out early kills a lot of otherwise appealing ideas, which is a service.

Technical view

Build it up from token counts per stage, including the system prompt and retrieved context that are resent on every call, multiplied by input and output prices, plus embedding, storage and any human review time. Retries, agent loops and long conversations dominate the tail, so measure the distribution rather than the mean. Levers include prompt caching for static prefixes, routing simple traffic to smaller models, tightening retrieval so fewer tokens are sent, and capping iterations. Compare against the cost of the process being replaced rather than against zero.


Part of AI Terminology Index, section: Cost, Speed and Service Levels.

How it connects

Cost Per Task links to 22 other terms, and the 12 closest are drawn here. The faint lines are links between those terms. See it in the full graph.

InferenceInferenceAgentAgentRate Limits and QuotasRate Limits and Quo…Multi-agent SystemsMulti-agent SystemsWorkflow Versus AgentWorkflow Versus Age…Small Language ModelSmall Language ModelBudget CapsBudget CapsTokensTokensCachingCachingModel RouterModel RouterReturn on InvestmentReturn on InvestmentSub-agentSub-agentCost Per Task