Concept card

Databricks-in-BDC Lakehouse Pattern

As of 2026-07-24T14:00:00Z

Edited by Cédric-Olivier Mâry, sole editor and publication director — 27 years in the SAP ecosystem. How these pages are produced, and where automation is used.

What is Databricks-in-BDC Lakehouse Pattern?

SAP's $250M Databricks tie-up gives zero-copy data sharing via Delta Sharing — fits Python-heavy data science or existing Databricks use, not SAP-anchored modeling without a data-engineering team.

Business Data Cloud, SAP's open managed data platform, is built in large part on a strategic partnership with Databricks — a two hundred and fifty million dollar commitment announced in 2024, with zero-copy, bidirectional data sharing as its central technical promise. In practice this means SAP applications such as S/4HANA, SuccessFactors, Ariba, and BW/4HANA expose their data to the Databricks lakehouse through Delta Sharing, the open protocol for sharing Delta Lake tables without physically moving or duplicating the underlying data, while Databricks workloads can write enriched or derived datasets back into SAP-native storage through the same protocol. For an SAP analytics architect, this is the pattern to reach for when an engagement combines Python-heavy data science workloads, large raw-data ingestion volumes, or an existing Databricks investment with a need for governed SAP business data — and it is the wrong pattern, adding unnecessary platform complexity, when the engagement is SAP-anchored business-user modeling with no data-engineering team to operate the Databricks side.

The Problem It Solves

Why it matters

BDC Connect registers Databricks tables in the BDC catalog with row-level security inherited directly from SAP's own Data Access Controls — no duplicate governance layer to maintain.
The pattern explicitly solves the 'data gravity trap': trusted SAP data locked in SAP's compute boundary while ML pipelines live in Databricks, previously bridged only by ETL and schema mapping.
The engagement-fit criterion is stated directly — this is not a universal pattern, it's conditional on the client's existing data science footprint.

Key points

BDC = SAP's open data platform anchored in $250M Databricks partnership; core protocol = Delta Sharing (zero-copy, bidirectional).
BDC Connect registers Databricks (and Snowflake, S3/Iceberg, BigQuery) in the BDC data catalog — no custom connectors, uses Unity Catalog Delta Sharing endpoint.
Unity Catalog row-level security and SAP DAC rules must be aligned — misalignment means a Databricks notebook can see rows a SAP user in the same role cannot.
Use when: existing Databricks investment + Python-native data team + Spark-scale ML (>500M rows) + non-SAP data in Delta Lake.
Do not use when: SAP-only estate + business-user modeling + no Databricks skills + latency <500ms required.
Write-back pattern: Databricks ML outputs (scores, forecasts) → Delta table in Unity Catalog → Delta Sharing → Datasphere remote table → SAC KPI or Joule grounding.
Dremio (announced May 2026, not yet GA) adds a query-time federation tier between Datasphere (hot, <500ms) and Databricks batch (cold, 10–30s) — does not replace Databricks-in-BDC.
Databricks-in-BDC Lakehouse Pattern is mastered only when it changes a named buyer decision.
Start with the semantic contract and control model before demonstrating the tool.
Use current SAP, analyst, study, KG, and news signals as evidence, not decoration.

Terms used on this page

Delta Sharing: Open Apache-licensed protocol from Databricks for sharing Delta Lake tables without copying data — readers access signed-URL Parquet files at the source storage location.
Unity Catalog: Databricks' unified governance layer for data and AI assets across all Databricks workspaces — enforces row-level security, column masking, and audit logging.
BDC Connect: SAP's catalog-time federation component in Business Data Cloud — registers external sources (Databricks, Snowflake, S3/Iceberg, BigQuery) in the BDC data catalog without data movement.
Delta Lake: Open-source ACID-compliant storage layer from Databricks that brings reliability to data lakes — tables stored as Parquet + transaction log (Delta Log).
Data Access Control (DAC): SAP Datasphere's row-level security framework — filters data at query time based on the requesting user's authorisation, enforced on Analytic Models and remote tables.
Zero-copy integration: Data sharing architecture where no physical data copy is created — the consumer reads from the producer's storage location via metadata and signed URLs, eliminating ETL pipelines and storage duplication.
Decision owner: The accountable person who accepts the trade-off and funds the next action.
Semantic contract: The shared definition of business terms, metrics, entities, and access rules used by tools and teams.

Sources

Full card available to members. What the full card adds: the full decision framework · the SAP vs Snowflake / Databricks / Fabric comparison · the common pitfalls and their fix · the cheat sheet · the architecture schemas · the code blocks · the facts worth quoting.

Open in the app →

Guides that answer with this page

These guides cite this page as one of the sources their answer rests on.