LLMOps on SAP BTP — Monitoring, Logging and Drift for Production AI Core Workloads
As of 2026-09-25
Expert-level LLMOps module that builds production monitoring, logging and drift detection for SAP AI Core generative AI workloads from documented parts rather than a single product: resource groups (isolation, the 50-group tenant quota, and the finest native cost grain), Inference Observability (exact headers, the 16-label/64-character limits, the extended-plan and S3-only requirements, and the documented caution that payloads are stored unmasked), forwarding application logs to SAP Cloud Logging, the auditing-and-logging feed of platform security events (including the four greppable authentication-failure strings), the metrics extension, and a drift-detection pattern built from periodic M366 evaluation re-runs plus SHA256 prompt-hash version tracking. Closes with a field-tested three-layer observability pattern for production agents and cost as an operating signal, sourced from SAP AI Core documentation and named SAP practitioners' published field guides.
What you will learn
- Explain why SAP AI Core observability is composed from documented parts (resource groups, Inference Observability, application logging, audit logs, metrics, Evaluations) rather than one product
- Design a resource-group landscape for isolation, cost tracking and security, and state the 50-resource-group tenant quota
- Configure Inference Observability correctly (persistence mode, labels, object store secret) and explain why it must be paired with data masking
- Forward AI Core application logs to SAP Cloud Logging and read the auditing-and-logging feed for platform and security events
- Build a drift-detection pattern from periodic evaluation re-runs, prompt-hash version tracking and tracked metrics, in the absence of a native drift alert
- Design a three-layer observability stack (step trace, business-decision audit table, infrastructure logging) for a production agent, including cost as an operating signal
Module overview
Who this is for. You have shipped a generative AI use case on SAP AI Core — an orchestration deployment, maybe an agent — and the question just changed from "does it work" to "how do we know if it stops working, quietly, at 3 a.m." This module builds the answer from what SAP AI Core actually documents (resource groups, Inference Observability, application logging, audit logs, the metrics extension) plus field-tested patterns from SAP practitioners for the parts SAP does not ship as a single product: drift detection and cross-layer observability. It assumes M325, M333 and, ideally, M366 (the evaluation harness this module reuses for drift).
Prerequisites
- M333 (AI & LLM Fundamentals for SAP Consultants) and M325 (SAP Generative AI Hub hands-on)
- M366 (Evaluating Generative AI on SAP Data) strongly recommended — this module reuses its evaluation harness for drift detection
- Comfort with SAP AI Core's resource-group and multitenancy model, REST APIs, and reading JSON logs
- Basic familiarity with a log-analysis tool (OpenSearch, Kibana or equivalent) is helpful but not required
Outcomes
- Stand up a resource-group landscape that separates dev/QA/production and explain its dual role as a cost and security boundary.
- Configure Inference Observability on a production endpoint correctly, with labels and persistence mode matched to the actual need.
- Produce a drift-detection runbook that a production support team can run without the original consultant present.
- Defend, to an operations stakeholder, why no single SAP product provides LLM monitoring end to end, and what compensates for that.
Full module available to members. The full module adds: the decision framework · the end-to-end scenario walkthrough · the KPI scorecard · the anti-patterns · the code blocks · the knowledge check · the diagrams.