Analytics Legends The knowledge platform for SAP Analytics
Academy module

LLM Cost Engineering — AI Units, Token Budgets, Caching

LLM Cost Engineering — AI Units, Token Budgets, Caching — Analytics Legends section illustration for the SAP Analytics knowledge base (concepts, studies, Academy)

As of 2026-09-25

Edited by , sole editor and publication director — 27 years in the SAP ecosystem. How these pages are produced, and where automation is used.

SAP prices generative AI through two currencies that must never be mixed: capacity units, converted from tokens via a model-specific GenAI-token ratio published in SAP Note 3437766, and AI Units, SAP's single virtual currency for Premium AI — autonomous agents at 0.02 AI Units per action, SAP Joule for Consultants at 35 AI Units and up to 22,900 pooled requests per user per month. This module works from SAP's own AI Core rate-limit and prompt-caching documentation to cover the mitigation sequence for a 429 rate-limit error, explicit cache_control placement and its differences between Claude and Nova, the usage fields (cached_tokens, cache_creation_tokens) that prove a caching strategy actually worked, and the governance discipline of a per-feature token budget set from measured usage with a meaningful alert threshold. It closes with a five-step sequence for building a dated, sourced cost case instead of an anxious guess. Three exercises and a self-assessment gate the move to M366.

What you will learn

  • Distinguish SAP's two cost currencies — capacity units for the generative AI hub and AI Units for premium Business AI — and classify a workload into the correct one without mixing them
  • Cite SAP's own published AI Units figures (0.02 AI Units per agent action; 35 AI Units and up to 22,900 pooled requests per user per month for Joule for Consultants) accurately and dated
  • Explain how generative-AI-hub rate limits work (RPM, resource-group vs tenant scope, the quota-increase process) and apply the correct mitigation sequence when one is hit
  • Design an explicit prompt-caching strategy — correct cache_control placement, TTL choice, model-family differences between Claude and Nova — and verify its savings from cached_tokens and cache_creation_tokens
  • Set and govern a token budget per feature, based on measured usage with an alert threshold, not a tenant-wide guess
  • Build a dated, sourced cost case for a client engagement that never conflates capacity units with AI Units

Who this is for. M333 introduced SAP's metering chain — tokens to GenAI tokens to capacity units — and warned against quoting SAP's fictitious worked example as a price. This module goes further: it separates SAP's two genuinely different cost currencies (capacity units for the generative AI hub, AI Units for premium Business AI), and gives you the three levers that actually move a cost line — rate limits, prompt caching, and knowing which currency you are even spending.

1. Two currencies, not one, and mixing them is the single most common estimate error

SAP prices generative AI through two separate commercial models, and a cost estimate that mixes them is wrong by construction, not by a small margin. Capacity units (CUs) are SAP BTP's general billing metric; generative AI hub consumption converts into them through the chain M333 introduced — tokens counted per request, converted to GenAI tokens at a model-specific ratio published in SAP Note 3437766, then GenAI tokens converted to CUs by a model-specific factor. This is the currency of raw model calls through orchestration: your own prompts, your own retrieval, your own agent loop, billed by what you actually send and receive.

Prerequisites

  • M333 (AI & LLM Fundamentals) — this module builds directly on its tokens-to-GenAI-tokens-to-capacity-units chain and SAP Note 3437766
  • Basic familiarity with the orchestration service's request structure (module_configurations, prompt_templating)
  • Optional: access to the generative AI hub to run E1-E2 against real usage data instead of on paper

Outcomes

  • Classify any generative-AI workload correctly into capacity units or AI Units before estimating its cost
  • Produce a rate-limit-aware deployment plan that names the mitigation sequence and the quota-increase lead time
  • Design and verify a prompt-caching configuration that measurably reduces cost, not just a config that looks right
  • Set a per-feature token budget with a measured baseline and a meaningful alert threshold

Full module available to members. The full module adds: the decision framework · the end-to-end scenario walkthrough · the KPI scorecard · the anti-patterns · the code blocks · the knowledge check · the diagrams.

Open in the app →

Next in this track