Concept card

AI Center of Excellence — Operating Model Design

As of 2026-10-10

Edited by Cédric Mary, sole editor and publication director — 27 years in the SAP ecosystem. How these pages are produced, and where automation is used.

What is AI Center of Excellence?

A federated hub-and-spoke AI CoE — lean central team, domain-owned use-case backlogs — is the only operating model that avoids both the 6-month approval bottleneck and the shadow-ChatGPT sprawl that fails an EU AI Act audit.

What it is

An AI Center of Excellence (AI CoE) is the organisational unit — or federated network of units — that owns the standards, tooling, talent, and governance for AI deployment across an SAP estate. In 2026, most large SAP clients have an analytics CoE that owns Datasphere and SAC; the AI CoE is the evolution of that structure to absorb generative AI (Joule agents, SAP AI Core, large language model integration) without losing the data governance discipline the analytics CoE took years to build.

The problem is that generative AI creates two opposing failure modes: over-centralisation (an AI CoE that is a bottleneck — every use case queues for 6 months of central approval while business units shadow-deploy ChatGPT on uncontrolled data) and under-centralisation (every business unit deploys its own models, creates its own training data outside Datasphere, and the organisation accrues dozens of ungoverned AI systems that will fail the EU AI Act audit). The operating model that avoids both is the federated hub-and-spoke: a central CoE owns standards, tooling, the AI Foundation model register, and EU AI Act compliance documentation; each business domain owns its own AI use-case backlog and deploys within the guardrails the CoE defines.

Why it matters

Names the two failure modes explicitly (over-centralisation vs. under-centralisation) — a diagnostic tool for reading a client's current state in one conversation.
Five structural components (platform layer, use-case registry, model governance, talent spine, literacy programme) give a concrete design checklist, not just a philosophy.
Threshold given: hub-and-spoke fits organisations with >5 business domains and >50 SAP power users — below that, simpler governance suffices.

Key points

An AI CoE owns standards, tooling, talent and governance for AI across the SAP estate — usually the evolution of the analytics CoE that already owns Datasphere and SAC.
Two opposite failure modes: over-centralisation (every use case queues for months of central approval while shadow AI spreads) and under-centralisation (each business unit trains or deploys models outside Datasphere's governance).
A federated hub-and-spoke model — a lean central team, domain-owned use-case backlogs — is the pattern that avoids both; calibrate the crossover point from the client's actual approval-queue length, not a borrowed headcount rule.
The CoE's first deliverable is an inventory of AI use cases and their EU AI Act risk classification, not a tooling choice.
A dashboard-style governance board (build-time review only) is not sufficient for an agent that keeps making judgement calls every time it runs — agent oversight needs an added monitoring plan (drift, cost, incident rate) attached before go-live.
An AI use-case registry only functions as a control when backed by a technical inventory of deployed agents and MCP servers, not a voluntary self-reporting form.
Five structural components: an AI platform layer (AI Core, Datasphere feature store, BDC-Databricks), a use-case registry with risk classification, a model governance layer (model cards, bias testing, monitoring thresholds), a talent spine (C150), and an AI literacy programme for business users.
Ask where each business unit's AI training data actually lives before designing the CoE — data living outside Datasphere means the under-centralisation failure mode is already under way.

Terms used on this page

AI Center of Excellence (AI CoE): The organisational unit, or federated network of units, owning standards, tooling, talent and governance for AI deployment across an SAP estate.
Hub-and-spoke operating model: A federated CoE design where a lean central team owns standards, the model register and compliance documentation while each business domain owns its own use-case backlog within those guardrails.
Shadow AI: An AI deployment — a chatbot subscription, a Joule Studio agent, a generative AI hub trial — stood up using a legitimate individual entitlement without ever passing through the CoE's intake process.
AI use-case registry: The CoE's inventory mapping every AI use case to an EU AI Act risk tier before engineering starts; only functions as a control when backed by a technical inventory of deployed agents, not a voluntary form.
Model governance layer: The CoE function covering model cards (EU AI Act Art. 11), bias-testing cadence and performance-monitoring alert thresholds for every deployed model or agent.
AI literacy programme: The CoE's training track for business users, covering what Joule agents can and cannot do — the practical alternative to users circumventing an AI system they do not understand.
Talent spine: The named AI CoE roles (explored in C150) that give the operating model continuity independent of any single hire; the crisis point most CoE designs underweight.

Sources

Full card available to members. What the full card adds: the full decision framework · the SAP vs Snowflake / Databricks / Fabric comparison · the common pitfalls and their fix · the cheat sheet · the architecture schemas · the code blocks · the facts worth quoting.

Open in the app →