AI & LLM Fundamentals for SAP Consultants — Transformers, Tokens, Embeddings, Context
As of 2026-09-23
First step of the SAP AI path for senior SAP data consultants new to AI. It explains, exactly and without equations you will not use, what the rest of the path assumes: next-token prediction and the transformer's attention; tokens as the unit of limits and cost, and SAP's metering chain tokens → GenAI tokens → capacity units (SAP Note 3437766); embeddings and cosine similarity in the HANA Cloud Vector Engine; the context window and context rot; temperature and non-determinism; hallucination, grounding and RAG; in-context learning vs RAG vs fine-tuning; reasoning models; tool calling, agents, MCP and A2A; SAP-RPT tabular models vs LLMs; and evaluation. Three exercises (count tokens, compare embeddings, choose RAG or fine-tuning for three SAP cases) and a self-assessment gate the move to M325.
What you will learn
- Explain next-token prediction, the transformer and attention well enough to predict how prompt length drives cost, latency and answer quality
- Estimate and then measure the tokens of an SAP prompt, and convert tokens into GenAI tokens and capacity units with SAP's metering method (SAP Note 3437766) without quoting fictitious rates as prices
- Describe what an embedding is, compute cosine similarity in the HANA Cloud Vector Engine, and state why one vector column must use one embedding model
- Use the context window, temperature and structured output deliberately, knowing that temperature 0 is not deterministic and long contexts degrade recall
- Choose between in-context learning, RAG and fine-tuning for an SAP use case and justify the choice with an evaluation metric
- Explain tool calling, agents, MCP and A2A, including who executes an action in S/4HANA, and when a tabular model such as SAP-RPT beats an LLM
Who this is for. You have built BW models, Datasphere spaces or S/4HANA reports for years; now the meetings are about tokens, embeddings, grounding and agents. This module is the first step of the SAP AI path on this platform. It gives you the working knowledge — exact, not mystical — that the next modules assume: M325 (generative AI hub hands-on), M326 (Joule agents), M329 (SAP-RPT and TabPFN), M330 (HANA Cloud vector and knowledge graph) and M331 (the C_AIG certification). Each section ends with the consequence for an SAP project; the three exercises work with a trial tenant or on paper.
1. What a large language model actually does
A large language model (LLM) is a neural network trained to predict the next piece of text given the text before it. Anthropic's glossary describes pretraining exactly this way: autoregressive models "are pretrained to predict the next word, given the previous context". Answering, writing ABAP or summarising a ticket is that one operation repeated: read the input, score every possible next token, pick one, append it, repeat until a stop condition.
Prerequisites
- Working knowledge of at least one SAP data product (BW/4HANA, Datasphere, S/4HANA embedded analytics or SAP Analytics Cloud)
- Comfort reading JSON and simple SQL
- Optional for the hands-on variants of E1 and E2: access to the generative AI hub (or its 30-day trial) and a HANA Cloud instance with the NLP feature enabled
Outcomes
- Explain to a client, in plain language, why an LLM can be fluent and wrong, and what grounding changes.
- Produce a token and capacity-unit estimate for an SAP use case using SAP's documented method, clearly separated from any price.
- Design the retrieval side of a RAG use case at concept level: embedding model, similarity measure, one model per column, freshness and authorisations.
- Recommend in-context learning, RAG, fine-tuning, a reasoning model or a tabular model for a given SAP requirement, with the evaluation that would confirm it.
Full module available to members. The full module adds: the decision framework · the end-to-end scenario walkthrough · the KPI scorecard · the anti-patterns · the code blocks · the knowledge check · the diagrams.