Cost-per-Token Economics 2026 — Frontier vs Open-Weights

What it is

Cost-per-token is the unit economics that determines whether an LLM-powered feature is a rounding error on the cloud bill or the line item that gets a finance director asking pointed questions in a budget review. Every production language-model workload, whatever the use case, ultimately reduces to a small number of variables multiplied together: tokens consumed per call, calls per day, and price per token — and each of those three variables has moved by roughly an order of magnitude at various points over the last few years, in directions that are not always intuitive.

Why it matters

• Reasoning models generate 5-20× more internal tokens than they emit, billed at output rates — a deep-research call returning 500 visible tokens can consume 8,000 internal reasoning tokens, costing $0.20 on Opus 4.7 at its $25/M output list price.
• Prompt caching is the biggest lever on repeated-context cost, cutting 80-90% off input cost for repeated system prompts in high-volume RAG applications.
• Self-hosting Llama-3.1-70B on a single H100 at €3/hour hits cost parity with the Claude Sonnet API at roughly 50% utilization — below that, the API stays cheaper.

Key points

• Frontier list prices checked 2026-09-22 (per M tokens, in/out): GPT-5 $1.25/$10 (GPT-5.5 $5/$30); Claude Opus 5.5 $4/$20, Opus 5 and Opus 4.7 $5/$25, Sonnet 5 $2/$10, Haiku 4.5 $1/$5; Gemini 2.5 Pro $1.25/$10; cached input up to 90% off.
• Mid-tier (GPT-5 mini, Claude Sonnet, Gemini Flash) at $0.15-$3/M input — the right default for 60-80% of production calls.
• Open-weights self-hosted at $0.20-$1.50/M effective; managed endpoints (Together/Fireworks/Groq) $0.20-$0.90/M.
• Reasoning-model internal tokens (extended thinking, reasoning tokens) bill at output rates — 8,000 internal tokens for a 500-token answer = $0.20/call on Opus 4.7.
• Prompt caching is the single biggest API cost lever: 80-90% reduction on repeated system prompts; use for any RAG system prompt repeated across calls.
• Self-host break-even vs API at ~50% utilisation on a single H100 serving 70B at TP=8; below that, API is cheaper per effective token.
• Model routing (70% small / 30% frontier) cuts total bill 60-80% with negligible quality loss when the router is calibrated on a held-out eval set.
• SAP AI Core consumption model abstracts provider list prices; the four cost drivers (prompt size, output length, model tier, utilisation) still apply to internal cost allocation.
• Per-request cost attribution (tag by workflow + user + tenant, ledger write, drift alert) is the FinOps minimum viable discipline before a 10k-user rollout.
• A 10k-user Joule deployment at 200 calls/user/day on frontier without caching: ~$13M/year at Opus 4.7 list price ($25/M output); caching plus routing to Sonnet 5 cuts that several-fold.

Terms used on this page

Prompt caching

API feature where repeated identical input prefixes (system prompts, RAG context) are stored server-side and billed at 10-25% of the standard input rate; Anthropic's implementation offers ~90% discount, OpenAI's ~75%.

Reasoning tokens

Internal chain-of-thought tokens generated by thinking models (Claude extended thinking, GPT-5 reasoning, DeepSeek-R1) — billed at output rates but not returned in the visible answer; can multiply effective output cost 5-20×.

Model router

Service that dispatches each request to the smallest model expected to satisfy quality requirements; the dominant cost-reduction pattern in production LLM stacks — a well-calibrated router cuts 60-80% of frontier spend with negligible quality loss.

Cost-per-1k-tokens

Industry-standard unit price; commonly expressed per M tokens (=1000 × per-1k) on vendor pricing pages since 2024 as prices fell below $0.01/1k.

SAP AI Core consumption unit

SAP's abstraction layer over underlying model API costs on BTP; enterprise customers see SAP-blended unit costs rather than Anthropic/OpenAI list prices, but prompt size, output length, model tier, and utilisation still drive the internal bill.

FinOps for LLM

Operational discipline of tagging every LLM API call with workflow, user, and tenant IDs; writing usage to a billing ledger; setting per-tenant cost alerts; and implementing model routing and prompt caching as first-class architectural choices before go-live.

Batch inference pricing

Most providers offer 50% discount for asynchronous batch jobs (no real-time SLA); suitable for overnight analytics enrichment, document classification, and bulk summarisation — not for interactive Joule queries.

Token budget controller

System-prompt directive or API parameter that caps the maximum tokens a model uses for internal reasoning (e.g. Anthropic's effort parameter on current Claude models, `budget_tokens` on older ones); critical for cost predictability on Opus-class workloads.

Sources

1. Anthropic — Claude API pricing (checked 2026-09-22)
2. OpenAI — API pricing (checked 2026-09-22)
3. Anthropic — Pricing
4. Google AI — Gemini API Pricing
5. Together AI — Open Model Pricing
6. SAP AI Core — Consumption-Based Pricing
7. Anthropic — Prompt caching documentation (Claude Platform Docs)
8. Groq — official pricing page
9. Fireworks AI — official pricing page
10. Databricks Docs — Mosaic AI Gateway
11. OpenAI — Reasoning models guide (reasoning tokens are additional to input/output tokens and drive cost; vendor documentation)
12. OpenAI — Prompt caching guide (discounted cached input tokens for repeated prefixes; vendor documentation)
13. Anthropic — Extended thinking (thinking tokens billed as output tokens; vendor documentation)
14. Epoch AI — LLM inference price trends (independent research on how price per token has fallen by task level)
15. OpenAI — Batch API guide (discounted asynchronous processing as a cost lever; vendor documentation)