LLMOps on SAP BTP — Monitoring, Logging and Drift for Production AI Core Workloads

What you will learn

• Explain why SAP AI Core observability is composed from documented parts (resource groups, Inference Observability, application logging, audit logs, metrics, Evaluations) rather than one product
• Design a resource-group landscape for isolation, cost tracking and security, and state the 50-resource-group tenant quota
• Configure Inference Observability correctly (persistence mode, labels, object store secret) and explain why it must be paired with data masking
• Forward AI Core application logs to SAP Cloud Logging and read the auditing-and-logging feed for platform and security events
• Build a drift-detection pattern from periodic evaluation re-runs, prompt-hash version tracking and tracked metrics, in the absence of a native drift alert
• Design a three-layer observability stack (step trace, business-decision audit table, infrastructure logging) for a production agent, including cost as an operating signal

Module overview

Who this is for. You have shipped a generative AI use case on SAP AI Core (an orchestration deployment, maybe an agent) and the question just changed from "does it work" to "how do we know if it stops working, quietly, at 3 a.m." This module builds the answer from what SAP AI Core actually documents (resource groups, Inference Observability, application logging, audit logs, the metrics extension) plus field-tested patterns from SAP practitioners for the parts SAP does not ship as a single product: drift detection and cross-layer observability. It assumes “SAP Generative AI Hub (Hands-on: Orchestration, Grounding, Masking, Filtering)”, “AI & LLM Fundamentals for SAP Consultants (Transformers, Tokens, Embeddings, Context)” and, ideally, the evaluation harness this module reuses for drift.

Prerequisites

• AI & LLM Fundamentals for SAP Consultants and SAP Generative AI Hub hands-on
• Evaluating Generative AI on SAP Data strongly recommended — this module reuses its evaluation harness for drift detection
• Comfort with SAP AI Core's resource-group and multitenancy model, REST APIs, and reading JSON logs
• Basic familiarity with a log-analysis tool (OpenSearch, Kibana or equivalent) is helpful but not required

Outcomes

• Stand up a resource-group landscape that separates dev/QA/production and explain its dual role as a cost and security boundary.
• Configure Inference Observability on a production endpoint correctly, with labels and persistence mode matched to the actual need.
• Produce a drift-detection runbook that a production support team can run without the original consultant present.
• Defend, to an operations stakeholder, why no single SAP product provides LLM monitoring end to end, and what compensates for that.