Concept card

AI Cost Attribution — Model Gateway Billing per Scenario

As of 2026-07-24T14:00:00Z

Edited by Cédric-Olivier Mâry, sole editor and publication director — 27 years in the SAP ecosystem. How these pages are produced, and where automation is used.

What is AI Cost Attribution — Model Gateway Billing per Scenario?

A single 'summarize my meeting' interaction can quietly trigger three chained model calls totaling 8,500-23,600 tokens — negligible at 100 users, but €290k-875k a year at 10,000 users running 20 interactions daily.

What it is

SAP AI Core's Model Gateway is the managed inference layer that every Joule request, every generative-AI flow in SAP Build, and every custom large-language-model call passes through on its way to a model. It exists partly for convenience — one authentication surface, one retry and rate-limiting layer instead of each application managing its own — and partly for control: the Gateway is also the single metering point SAP uses to bill AI consumption. Understanding how that billing works, at the level of individual model calls and business scenarios, is the difference between an AI programme that stays inside its budget and one that triggers an emergency finance review in month three, after the invoice arrives and nobody can explain which feature caused it.

How the Gateway actually meters usage. It sits between the calling application — Joule, the Generative AI Hub inside Build, a custom BTP extension — and the underlying model, whether that is a hosted GPT-class model, a Claude model, or an open-weights model running on SAP's own infrastructure. The caller never holds credentials to the underlying model directly; the Gateway owns authentication, rate limiting, and usage tracking. The billing unit is the token: input tokens, meaning the prompt plus system instructions plus any retrieved Knowledge Graph context, and output tokens, meaning what the model generates. Different model tiers carry different per-token prices, and enterprise contracts typically bundle a monthly token allowance across all models with overage charges once that allowance is exhausted.

Why it matters

One user query can decompose into a RAG call, a summarization call and a formatting call — three billed inferences, not one
The per-interaction cost (€0.004-0.012) looks trivial until multiplied by scale: 10,000 users × 20 interactions/day = €800-2,400/day
Understanding this scenario decomposition is what separates an AI programme that stays in budget from one triggering an emergency review in month three

Key points

Model Gateway is the SAP AI Core managed inference router — all Joule, Build GenAI, and custom LLM calls are metered here in tokens (input + output).
A single Joule interaction triggers 3–4 model calls (RAG retrieval + summarisation + formatting); typical token range 8,500–23,600 tokens per interaction.
At 10,000 users × 20 interactions/day, mid-tier model pricing implies €290k–€875k/year — cost modelling before go-live is non-negotiable.
Top 4 cost levers: prompt engineering (move static context to KG), chunk count reduction via re-ranking, model tier selection (small model for classification), semantic caching.
Use AI Core `resource-group` header to attribute token consumption by business domain — without it, the monthly AI bill is attributable to no one.
Agentic multi-step chains cost 15–40× single-turn RAG; set max-turns cap (4–6 for enterprise workflows) at the AI Core orchestration layer.
Input-length guards prevent long-document ingestion spikes (50-page contract = 40,000–80,000 tokens in one call).
AI Cost Attribution — Model Gateway Billing per Scenario is mastered only when it changes a named buyer decision.
Start with the semantic contract and control model before demonstrating the tool.
Use current SAP, analyst, study, KG, and news signals as evidence, not decoration.

Terms used on this page

Model Gateway: SAP AI Core's centralised inference router — handles auth, rate limiting, retry, and token metering for all LLM calls from Joule and BTP applications.
Token: The billing unit for LLM inference — roughly 4 characters or 0.75 words. A prompt of 1,000 words ≈ 1,333 input tokens.
RAG (Retrieval-Augmented Generation): Pattern that retrieves relevant document chunks from the Knowledge Graph and passes them as context to the LLM — grounds responses in enterprise data, adds input token cost.
Semantic caching: Technique that stores LLM responses keyed by embedding similarity — on a cosine-similar subsequent query, returns the cached response without a live LLM call.
Cross-encoder re-ranker: Lightweight model that scores the relevance of retrieved RAG chunks before passing them to the main LLM — reduces input token count by 50–70% with minimal quality loss.
resource-group: AI Core API optional header that tags token consumption by a named logical group (e.g. 'finance', 'hr') — enables BTP cost management to slice AI spend by business domain.
Decision owner: The accountable person who accepts the trade-off and funds the next action.
Semantic contract: The shared definition of business terms, metrics, entities, and access rules used by tools and teams.

Sources

Full card available to members. What the full card adds: the full decision framework · the SAP vs Snowflake / Databricks / Fabric comparison · the common pitfalls and their fix · the cheat sheet · the architecture schemas · the code blocks · the facts worth quoting.

Open in the app →

Guides that answer with this page

These guides cite this page as one of the sources their answer rests on.

SAP AI Core pricing: what is public, what is not