Generative AI Hub orchestration pipeline — the six modules in order

The pipeline in one sentence

The orchestration service of the generative AI hub (C119) combines modules into a pipeline executed by one API call, POST {deploymentUrl}/v2/completion with the AI-Resource-Group header. SAP fixes the order centrally; you configure each module or omit the optional ones in config.modules. The hands-on build of such a call is module M325; this card explains what each stage does, in which order, and how it fails.

The real order of execution

SAP's workflow documentation lists the stages in this order, and the intermediate_results of a full V2 response come back in the same sequence:

1. Grounding (optional) — retrieves chunks from your data repositories for the placeholder you designate and writes them into another placeholder. If masking is configured with mask_grounding_input, the grounding query is masked before retrieval.
2. Templating (mandatory) — builds the messages from the template and the placeholder values, including the grounding output.
3. Input translation (optional) — translates the templated prompt, via SAP Translation Hub's Document Translation service.
4. Data masking (optional) — anonymizes or pseudonymizes personal data in the prompt.
5. Input filtering (optional) — content safety checks; if any configured filter flags the prompt, the model is never called.
6. Model call (mandatory) — the LLM, through the harmonized API.
7. Output filtering (optional) — content safety checks on each returned choice.
8. Unmasking — pseudonymized values are restored in the answer (pseudonymization only).
9. Output translation (optional) — SAP documents that it runs after unmasking, on the unmasked output.

Why it matters

• Knowing that input filtering runs after masking and translation explains why a filter can miss or over-flag content: it scores the translated, masked prompt, not what the user typed.
• A rejected input costs a filter call but no model tokens; an output block returns HTTP 200 with finish_reason content_filter — client code that only checks HTTP status will show empty answers.
• The V1 endpoint disappears on 31 October 2026 and masking_providers passed its removal date on 15 September 2026: payloads copied from older blogs or slides fail today.

Key points

• Order: grounding → templating → input translation → masking → input filtering → model → output filtering → unmasking → output translation.
• Mandatory: prompt_templating with its model block. Everything else is optional and can be omitted.
• The six-box slide is correct in sequence but omits both translation steps and unmasking; model access is not a separate module in V2.
• Masking: anonymization is irreversible; pseudonymization is restored in the answer and in tool-call arguments; mask_grounding_input masks the retrieval query.
• Filtering: Azure Content Safety (thresholds 0/2/4/6, Prompt Shield, protected code) and Llama Guard 3 (14 boolean categories); any violation rejects.
• Fallbacks: config.modules as a list; switch on unsupported model, and on 408/429/5xx for non-streaming calls.
• Deadlines: masking_providers removal date 15 Sep 2026; V1 /completion decommissioned 31 Oct 2026.
• Translation uses SAP Translation Hub (sap_document_translation), before masking on input and after unmasking on output.

Terms used on this page

Orchestration config

The JSON under config.modules that configures each pipeline stage; can be stored and versioned in the prompt registry.

Placeholder

A {{?name}} variable in a template, filled from placeholder_values or, for grounding output, by the grounding module.

Pseudonymization

Masking method that replaces personal data with numbered placeholders and restores the originals in the answer.

Anonymization

Masking method that replaces personal data irreversibly; the model and the answer only see placeholders.

Prompt Shield

Azure Content Safety prompt-attack detection on input; runs before harm classification and ignores system and developer messages.

Llama Guard 3

Content classifier (llama_guard_3_8b) with 14 hazard categories configured as booleans; only requested categories are enforced.

intermediate_results

Per-stage outputs returned with the answer (grounding, templating, masking, filtering, llm…); may contain sensitive data.

Fallback configuration

An ordered list of module configurations; orchestration moves to the next one on unsupported models or, without streaming, on 408, 429 and 5xx.

Sources

1. SAP AI Core docs (SAP-docs GitHub, 14 Sep 2026) — Orchestration Workflow V2: endpoint, modules, full example
2. SAP AI Core docs — Orchestration Workflow V1 (deprecated, decommissioning 31 Oct 2026) with the module order
3. SAP AI Core docs — Templating (placeholders, defaults, template_ref, timeout, max_retries)
4. SAP AI Core docs — Grounding (repositories, 8,000 documents per pipeline, Pipelines/Vector APIs)
5. SAP AI Core docs — Translation module (runs before input masking, after output unmasking)
6. SAP AI Core docs — Data Masking (anonymization vs pseudonymization, entity list)
7. SAP AI Core docs — Enhancing Model Consumption with Data Masking (providers, allowlist, mask_grounding_input)
8. SAP AI Core docs — Content Filtering (Azure Content Safety, Prompt Shield, Llama Guard 3)
9. SAP AI Core docs — Enhancing Model Consumption with Input Filtering
10. SAP AI Core docs — Orchestration with Fallbacks (408, 429, 5xx, unsupported model)
11. SAP AI Core docs — Orchestration Error Handling (V1 vs V2 error shape)
12. SAP AI Core docs — Orchestration overview (global scenario orchestration, harmonized API)
13. SAP AI Core — What's New (SAP-docs GitHub): RPT-1.5/1.6, Tabular AI Orchestration, masking_providers removal, Prompt Shield change
14. SAP AI Core docs — Enhancing Model Consumption with Output Filtering (finish_reason content_filter)
15. SAP AI Core docs — Streaming (chunk_size default 100, overlap, accuracy of filtering and unmasking)
16. SAP AI Core docs — Model Configuration (timeout 1–600 s, max_retries 0–5)