Concept card

Grounding & RAG (Retrieval-Augmented Generation)

As of 2026-10-06

Edited by Cédric Mary, sole editor and publication director — 27 years in the SAP ecosystem. How these pages are produced, and where automation is used.

What is Grounding & RAG (Retrieval-Augmented Generation)?

Grounding feeds a model the enterprise facts it was never trained on, at request time, so answers are current and traceable. In SAP it has three concrete channels: the grounding module of the orchestration service in SAP AI Core (documents chunked, embedded and retrieved via Pipelines, Vector and Retrieval APIs), document grounding for Joule and Joule agents, and structured grounding through BDC data products and SAP Knowledge Graph. Choosing the channel is the first design decision.

Grounding and RAG in one paragraph

A large language model knows what was in its training data and nothing about your purchasing policy, last night's stock levels or the clause in a supplier contract. Grounding is the umbrella term for supplying that missing context at request time; retrieval-augmented generation (RAG) is its most common technique: index the knowledge in advance, retrieve the relevant pieces when a question arrives, and pass them to the model with the prompt so it composes its answer from them. SAP's AI Core documentation describes the grounding module in exactly those terms: connect models to external data sources, retrieve relevant information at runtime, and supply it as context for generation.

Why it matters

Grounding is the difference between a model that sounds right and one whose answer can be traced to a document or governed record.
SAP offers three distinct channels — orchestration grounding for documents, Joule document grounding, structured grounding via BDC and SAP Knowledge Graph — and each fails differently when misused.
Managed grounding pipelines have hard limits (daily refresh, 8,000 documents per pipeline) that decide architecture before any prompt is written.

Key points

Grounding = supplying enterprise context at request time; RAG = index, retrieve, inject into the prompt.
Orchestration grounding module (SAP AI Core): Pipelines API + Vector API to index, Retrieval API to search.
Repositories: SharePoint, AWS S3, SFTP, SAP Build Work Zone, SAP Document Management service, ServiceNow, Google Drive.
Managed pipelines: plain-text extraction, daily refresh, max 8,000 documents per pipeline.
Prerequisites per SAP tutorial: extended AI Core plan, resource group enabled for document grounding, generic secrets, orchestration deployment.
Joule document grounding: standard path via SAP Build Control Tower + Object Store; alternatives via AI Core grounding management.
Structured facts come from BDC data products/Datasphere/HANA; SAP Knowledge Graph supplies the semantics.
Orchestration workflow V1 is deprecated in favour of V2; grounding configuration is the same in both.

Terms used on this page

Grounding: Supplying external, domain-specific or current data to a generative model at request time so its answer is based on it.
RAG: Retrieval-augmented generation: index knowledge, retrieve relevant chunks per question, inject them into the prompt.
Pipelines API: AI Core grounding API that ingests documents from a repository, chunks and embeds them into a vector store.
Vector API: AI Core grounding API to push your own pre-chunked content into the vector store.
Retrieval API: AI Core grounding API that searches the indexed repositories for content similar to a query.
max_chunk_count: Orchestration grounding search setting limiting how many chunks are returned into the prompt.
Structured grounding: Grounding on governed records (BDC data products, Datasphere, HANA) rather than on document chunks.

Sources

Full card available to members. What the full card adds: the full decision framework · the SAP vs Snowflake / Databricks / Fabric comparison · the common pitfalls and their fix · the cheat sheet · the architecture schemas · the code blocks · the facts worth quoting.

Open in the app →

Guides that answer with this page

These guides cite this page as one of the sources their answer rests on.

Freelance SAP LLM consultant: scope, vetting, rates