Concept card

Cost-per-Token Economics 2026 — Frontier vs Open-Weights

As of 2026-07-24T15:00:00Z

Edited by Cédric-Olivier Mâry, sole editor and publication director — 27 years in the SAP ecosystem. How these pages are produced, and where automation is used.

What is Cost-per-Token Economics 2026 — Frontier vs Open-Weights?

Cost-per-token pricing spans two orders of magnitude in May 2026 — Gemini 2.5 Pro at $1.25/M input to Claude Opus 4.7 at $75/M output — and the biggest lever is routing 70% of calls to a small model.

Cost-per-token is the unit economics that determines whether an LLM-powered feature is a rounding error on the cloud bill or the line item that gets a finance director asking pointed questions in a budget review. Every production language-model workload, whatever the use case, ultimately reduces to a small number of variables multiplied together: tokens consumed per call, calls per day, and price per token — and each of those three variables has moved by roughly an order of magnitude at various points over the last few years, in directions that are not always intuitive.

The market has stratified, not just fallen

Why it matters

Reasoning models generate 5-20× more internal tokens than they emit, billed at output rates — a deep-research call returning 500 visible tokens can consume 8,000 internal reasoning tokens, costing $0.60 on Opus 4.7.
Prompt caching is the biggest lever on repeated-context cost, cutting 80-90% off input cost for repeated system prompts in high-volume RAG applications.
Self-hosting Llama-3.1-70B on a single H100 at €3/hour hits cost parity with the Claude Sonnet API at roughly 50% utilization — below that, the API stays cheaper.

Key points

May 2026 frontier list prices: GPT-5 $5/$15 per M in/out; Claude Opus 4.7 $15/$75; Gemini 2.5 Pro $1.25/$10; cached-input tiers 25-90% off.
Mid-tier (GPT-5 mini, Claude Sonnet, Gemini Flash) at $0.15-$3/M input — the right default for 60-80% of production calls.
Open-weights self-hosted at $0.20-$1.50/M effective; managed endpoints (Together/Fireworks/Groq) $0.20-$0.90/M.
Reasoning-model internal tokens (extended thinking, reasoning tokens) bill at output rates — 8,000 internal tokens for a 500-token answer = $0.60/call on Opus 4.7.
Prompt caching is the single biggest API cost lever: 80-90% reduction on repeated system prompts; use for any RAG system prompt repeated across calls.
Self-host break-even vs API at ~50% utilisation on a single H100 serving 70B at TP=8; below that, API is cheaper per effective token.
Model routing (70% small / 30% frontier) cuts total bill 60-80% with negligible quality loss when the router is calibrated on a held-out eval set.
SAP AI Core consumption model abstracts provider list prices; the four cost drivers (prompt size, output length, model tier, utilisation) still apply to internal cost allocation.
Per-request cost attribution (tag by workflow + user + tenant, ledger write, drift alert) is the FinOps minimum viable discipline before a 10k-user rollout.
A 10k-user Joule deployment at 200 calls/user/day on frontier without caching: ~$40M/year at Opus 4.7 list price; with caching + Sonnet routing: ~$2-4M/year — 10-20× swing.

Terms used on this page

Prompt caching: API feature where repeated identical input prefixes (system prompts, RAG context) are stored server-side and billed at 10-25% of the standard input rate; Anthropic's implementation offers ~90% discount, OpenAI's ~75%.
Reasoning tokens: Internal chain-of-thought tokens generated by thinking models (Claude extended thinking, GPT-5 reasoning, DeepSeek-R1) — billed at output rates but not returned in the visible answer; can multiply effective output cost 5-20×.
Model router: Service that dispatches each request to the smallest model expected to satisfy quality requirements; the dominant cost-reduction pattern in production LLM stacks — a well-calibrated router cuts 60-80% of frontier spend with negligible quality loss.
Cost-per-1k-tokens: Industry-standard unit price; commonly expressed per M tokens (=1000 × per-1k) on vendor pricing pages since 2024 as prices fell below $0.01/1k.
SAP AI Core consumption unit: SAP's abstraction layer over underlying model API costs on BTP; enterprise customers see SAP-blended unit costs rather than Anthropic/OpenAI list prices, but prompt size, output length, model tier, and utilisation still drive the internal bill.
FinOps for LLM: Operational discipline of tagging every LLM API call with workflow, user, and tenant IDs; writing usage to a billing ledger; setting per-tenant cost alerts; and implementing model routing and prompt caching as first-class architectural choices before go-live.
Batch inference pricing: Most providers offer 50% discount for asynchronous batch jobs (no real-time SLA); suitable for overnight analytics enrichment, document classification, and bulk summarisation — not for interactive Joule queries.
Token budget controller: System-prompt directive or API parameter that caps the maximum tokens a model uses for internal reasoning (e.g. Anthropic's `budget_tokens` in extended thinking); critical for cost predictability on Opus-class workloads.

Sources

Full card available to members. What the full card adds: the full decision framework · the SAP vs Snowflake / Databricks / Fabric comparison · the common pitfalls and their fix · the cheat sheet · the architecture schemas · the code blocks · the facts worth quoting.

Open in the app →