AI & Analytics Legends The knowledge platform for SAP Analytics
Academy module

LLMs on Enterprise Data: Generative AI Hub, BDC & SAP Databricks

LLMs on Enterprise Data grounding architecture: SAP sources flow through chunking and embedding, hybrid retrieval, a grounded prompt to the LLM, and an output validation gate, logged to a governance record that feeds back into chunking and access-control sync. — architecture diagram for LLMs on Enterprise Data: Generative AI Hub, BDC & SAP Databricks, Analytics Legends Academy module M136

As of 2026-10-10

Edited by , sole editor and publication director — 27 years in the SAP ecosystem. How these pages are produced, and where automation is used.

LLMs on SAP data are an architecture problem, not a model choice. SAP ships four routes: the generative AI hub in SAP AI Core (extended plan; model library plus an orchestration service chaining templating, grounding, data masking, content filtering, translation and structured output); SAP Business Data Cloud with SAP HANA Cloud (vector and knowledge graph engines, 'the AI database' of BDC) and SAP Databricks (data products shared zero-copy via Delta Sharing); Joule and Joule Studio agents for end users; and table-native models (SAP-RPT-1.6, TabPFN-3.5 Plus) for predictions an LLM should not make. Production quality comes from semantic metadata, tool calls for exact values, citations, SQL sandboxing, masking and a business-owned golden set.

What you will learn

  • Assign six real AI use cases to the right SAP route (generative AI hub, BDC with SAP HANA Cloud or SAP Databricks, Joule/Joule Studio, SAP-RPT or TabPFN) with a written five-line rationale each.
  • Build a generative AI hub orchestration configuration combining templating, document grounding, SAP Data Privacy Integration masking and content filtering, and prove masking works on a test prompt.
  • Choose the data path per question — tool call, document grounding, HANA Cloud retrieval or SAP Databricks batch — and explain the authorisation consequence of each.
  • Implement output controls for natural-language-to-SQL (sandbox, object whitelist, result check) and for generated narrative (number-by-number fact check).
  • Define a business-owned golden set and the regression rule that re-runs it on every change of model, prompt, retrieval or data.

Module overview

Putting a large language model (LLM) on top of SAP data is not a model-selection exercise. Every serious deployment has to answer the same four questions: where does the model run and under which contract, how does it get the right business context at the right moment, what stops it from leaking or inventing data, and who checks the output before a decision is made. SAP now ships a concrete answer to each question, spread across the generative AI hub in SAP AI Core, SAP Business Data Cloud (BDC) with SAP HANA Cloud and SAP Databricks, and Joule. This module maps those routes, shows how to build on each, and teaches the controls that turn a demo into a production system.

Prerequisites

  • Hands-on experience with SAP BTP and one of SAP Datasphere or SAP Business Data Cloud
  • Review core concepts first: C027, C203, C087
  • Recommended: module M053 (SAP AI Core & AI Launchpad)

Outcomes

  • Explain to a sponsor why data path, controls and evaluation come before model choice.
  • Operate the generative AI hub and its orchestration modules through SAP AI Launchpad or the SAP Cloud SDK for AI.
  • Position SAP Databricks, SAP HANA Cloud and the generative AI hub as complementary parts of one architecture.
  • Route tabular prediction to SAP-RPT or TabPFN and use the LLM only for explanation.
  • Hand over an evaluation and operating model signed by the DPO and the business owner.

Full module available to members. The full module adds: the decision framework · the end-to-end scenario walkthrough · the KPI scorecard · the anti-patterns · the code blocks · the knowledge check · the diagrams.

Open in the app →

Next in this track