AI Red-Teaming for SAP — Joule, AI Core and Agents
As of 2026-10-04
Red-teaming an SAP AI feature is systems testing, not prompt trivia: you attack the composition — prompt template, grounding corpus, tools and MCP servers, identity propagation, budget limits, logging — not just the model's refusals. Map the catalogue to the OWASP Top 10 for LLM Applications (2025) and the OWASP Top 10 for Agentic Applications (December 2025), test the SAP controls you rely on (orchestration content filters with prompt shield and Llama Guard 3, data masking, principal propagation, AI Agent Hub, inference observability) instead of assuming them, run every attack as an entitled and a non-entitled user, and promote every finding into a regression suite triggered by template, model, corpus and tool changes. The May 2026 Five Eyes guidance on agentic AI and EU AI Act Article 15 make this evidence expected, not optional.
What you will learn
- Map an SAP AI feature to the OWASP Top 10 for LLM Applications 2025, the OWASP Top 10 for Agentic Applications and seven SAP-specific surfaces, and derive a catalogue from harm statements
- Test — not assume — the SAP controls in play: orchestration input/output filters (Azure AI Content Safety with prompt shield, Llama Guard 3), data masking, principal propagation, AI Agent Hub, inference observability
- Run a two-identity red-team protocol with three-state scoring and severity tied to harm statements
- Turn findings into an event-triggered regression suite with a release gate and a signed exception path
Module overview
Red-teaming is adversarial testing of an AI feature before real users — or real attackers — find its failure modes. For SAP AI work the useful question is not "could someone break the model?" but "what business consequence does the most likely failure of this composition produce, and have we shown that the worst realistic case is acceptable?" A Joule skill, a Joule Studio agent or an application built on the generative AI hub in SAP AI Core is a chain: a prompt template, a model, a grounding corpus, tools and APIs, an identity under which calls run, a budget, and logs. Most serious findings live at the joins.
Prerequisites
- Intermediate hands-on experience on SAP AI or analytics projects
- Review core concepts first: C130, C261, C120
Outcomes
- Explain to a CISO why red-teaming targets the composition (prompt, grounding, tools, identity, budget, logs) rather than the model alone
- Design attacks for indirect injection through a grounding corpus and for authorisation bypass on a skill's backend path
- Specify agent-specific tests: tool scoping, irreversible-action confirmation, untrusted A2A/MCP messages, loop and cost ceilings
- Cite the external expectations accurately: AI Act Articles 15 and 55, the Five Eyes guidance of 1 May 2026, OWASP lists and MITRE ATLAS
Full module available to members. The full module adds: the decision framework · the end-to-end scenario walkthrough · the KPI scorecard · the anti-patterns · the code blocks · the knowledge check · the diagrams.