Concept card

AI Center of Excellence — Operating Model Design

As of 2026-07-23

Edited by Cédric-Olivier Mâry, sole editor and publication director — 27 years in the SAP ecosystem. How these pages are produced, and where automation is used.

What is AI Center of Excellence — Operating Model Design?

A federated hub-and-spoke AI CoE — lean central team, domain-owned use-case backlogs — is the only operating model that avoids both the 6-month approval bottleneck and the shadow-ChatGPT sprawl that fails an EU AI Act audit.

What it is

An AI Center of Excellence (AI CoE) is the organisational unit — or federated network of units — that owns the standards, tooling, talent, and governance for AI deployment across an SAP estate. In 2026, most large SAP clients have an analytics CoE that owns Datasphere and SAC; the AI CoE is the evolution of that structure to absorb generative AI (Joule agents, SAP AI Core, large language model integration) without losing the data governance discipline the analytics CoE took years to build.

The problem is that generative AI creates two opposing failure modes: over-centralisation (an AI CoE that is a bottleneck — every use case queues for 6 months of central approval while business units shadow-deploy ChatGPT on uncontrolled data) and under-centralisation (every business unit deploys its own models, creates its own training data outside Datasphere, and the organisation accrues dozens of ungoverned AI systems that will fail the EU AI Act audit). The operating model that avoids both is the federated hub-and-spoke: a central CoE owns standards, tooling, the AI Foundation model register, and EU AI Act compliance documentation; each business domain owns its own AI use-case backlog and deploys within the guardrails the CoE defines.

Why it matters

Names the two failure modes explicitly (over-centralisation vs. under-centralisation) — a diagnostic tool for reading a client's current state in one conversation.
Five structural components (platform layer, use-case registry, model governance, talent spine, literacy programme) give a concrete design checklist, not just a philosophy.
Threshold given: hub-and-spoke fits organisations with >5 business domains and >50 SAP power users — below that, simpler governance suffices.

Key points

AI Center of Excellence — Operating Model Design is mastered only when it changes a named buyer decision.
Start with the semantic contract and control model before demonstrating the tool.
Use current SAP, analyst, study, KG, and news signals as evidence, not decoration.
Separate verified facts from directional trends and modeled assumptions.
Define owner, metric, threshold, support path, and rollback before scaling.
For AI use cases, measure reliability, cost, latency, safety, and human validation.
Leave a reusable operating asset: memo, checklist, control table, and exception log.
A premium answer is short, trade-off explicit, and defensible in a steering committee.

Terms used on this page

Decision owner: The accountable person who accepts the trade-off and funds the next action.
Semantic contract: The shared definition of business terms, metrics, entities, and access rules used by tools and teams.
Control plane: The layer that applies policy, access, lineage, monitoring, and escalation across the operating model.
Evidence grade: A label that separates verified fact, directional signal, modeled assumption, and field observation.
Adoption metric: The measurable behavior proving that the concept changed actual work after go-live.
Agent reliability: The consistency, cost, safety, and policy compliance of an agent across repeated runs.
Decision owner: The accountable person who accepts the trade-off and funds the next action.
Semantic contract: The shared definition of business terms, metrics, entities, and access rules used by tools and teams.

Sources

Full card available to members. What the full card adds: the full decision framework · the SAP vs Snowflake / Databricks / Fabric comparison · the common pitfalls and their fix · the cheat sheet · the architecture schemas · the code blocks.

Open in the app →