AI & LLM Fundamentals for SAP Consultants — Transformers, Tokens, Embeddings, Context
As of 2026-11-01
First step of the SAP AI path for senior SAP data consultants new to AI. It explains, exactly and without equations you will not use, what the rest of the path assumes: next-token prediction and the transformer's attention; tokens as the unit of limits and cost, and SAP's metering chain tokens → GenAI tokens → capacity units (SAP Note 3437766); embeddings and cosine similarity in the HANA Cloud Vector Engine; the context window and context rot; temperature and non-determinism; hallucination, grounding and RAG; in-context learning vs RAG vs fine-tuning; reasoning models; tool calling, agents, MCP and A2A; SAP-RPT tabular models vs LLMs; and evaluation. Three exercises (count tokens, compare embeddings, choose RAG or fine-tuning for three SAP cases) and a self-assessment gate the move to M325.
What you will learn
- Estimate the tokens of a German SAP ticket prompt, measure them with getTokenUsage() in the SAP Cloud SDK for AI, and explain the gap
- Convert measured tokens into GenAI tokens and capacity units with SAP's metering method, labelling fictitious rates as method only
- Compute COSINE_SIMILARITY with VECTOR_EMBEDDING in SAP HANA Cloud and state why one vector column must use one embedding model
- Choose between in-context learning, RAG, fine-tuning, a reasoning model and a tabular model for a given SAP case, and justify the choice with a metric
- Specify a tool definition (name, description, input_schema) and explain who executes the call and under which identity
- Build a 30-case golden set with thresholds and a named sign-off before a demo is presented as a commitment
Module overview
Who this is for. You have built BW models, Datasphere spaces or S/4HANA reports for years, and project meetings now use token, embedding, grounding and agent. After this module you can do five things on a client project: estimate what a prompt will cost before anyone builds it, check that a vector search compares like with like, pick between prompting, retrieval and fine-tuning with a written reason, explain who executes a tool call, and run a 30-question test before a demo becomes a commitment. Two of them are measurable with the labs in chapter 4 (30 minutes). Next modules assume this vocabulary: M325 (Generative AI Hub hands-on), M326 (Joule agents), M329 (SAP-RPT and TabPFN), M330 (HANA Cloud vectors) and M331 (the C_AIG certification).
Source: SAP Help — HANA Cloud Vector Engine: similarity measures
Prerequisites
- Working knowledge of at least one SAP data product (BW/4HANA, Datasphere, S/4HANA embedded analytics or SAP Analytics Cloud)
- Comfort reading JSON and simple SQL
- Optional for the hands-on variants of E1 and E2: access to the generative AI hub (or its 30-day trial) and a HANA Cloud instance with the NLP feature enabled
Outcomes
- Explain to a client, in plain language, why an LLM can be fluent and wrong, and what grounding changes.
- Produce a token and capacity-unit estimate for an SAP use case using SAP's documented method, clearly separated from any price.
- Design the retrieval side of a RAG use case at concept level: embedding model, similarity measure, one model per column, freshness and authorisations.
- Recommend in-context learning, RAG, fine-tuning, a reasoning model or a tabular model for a given SAP requirement, with the evaluation that would confirm it.
Open missions
151 live roles and contracts in SAP AI & Joule right now — see the SAP AI & Joule missions · freelance SAP contracts.