Prompt Engineering & Prompting Techniques for SAP AI

What prompt engineering is, and what it is not

Prompt engineering is the discipline of designing the text, structure and examples you send to a language model so that it returns output that is accurate, usable and repeatable. It changes only the input, which makes it the cheapest and fastest lever you have, and also one with a ceiling. Anthropic's own prompt engineering overview (accessed 18 October 2026) is explicit that you should have clear success criteria, a way to test against them and a first draft before you start tuning, and that not every failing evaluation is best solved with a prompt: sometimes a different model fixes latency or cost more easily. In the generative AI hub of SAP AI Core, in Joule Studio agents and in orchestration pipelines, the prompt is a versioned, reviewed and tested artefact, not a chat message typed once.

Why it matters

• Prompts are now deployable configuration in the generative AI hub, Joule Studio and orchestration pipelines, so quality, versioning and testing are delivery topics, not chat habits.
• Choosing prompt engineering, RAG or fine-tuning in the right order is the cheapest way to avoid an over-built solution on a client project.
• Prompt injection turns every untrusted e-mail, document or web page into a possible instruction, so prompt design and security design must be done together.

Key points

• Six elements: instruction, context, input data, output format, constraints, tone and audience.
• Few-shot: Anthropic suggests 3-5 relevant, diverse examples (accessed 18 Oct 2026).
• Zero-shot-CoT trigger lifted one 2022 model on MultiArith from 17.7% to 78.7% (arXiv 2205.11916); reasoning models now think by built-in design.
• response_format accepts text, json_object or json_schema; a schema is API-enforced, wording is not.
• Orchestration chains templating (mandatory), grounding, translation, content filtering, data masking in one call.
• Prompt registry: versioned templates with {{?placeholders}}, fetched by ID (immutable) or by name, scenario, version.
• Evaluations score prompts with metrics such as exact match and JSON schema match; compare prompt versions with one run each.
• Prompt injection (OWASP LLM01): direct or indirect; control it with architecture, not wording.

Terms used on this page

Prompt engineering

Designing the text, structure and examples sent to a model to get accurate, repeatable output.

Zero-shot / few-shot

Prompting with no examples / with a handful of worked examples.

Chain-of-thought

Eliciting intermediate reasoning steps before the final answer.

System prompt

Standing instructions and role that frame every turn of a conversation.

Structured output

Output constrained to a format, for example a JSON schema.

Prompt template

Fixed prompt text with placeholders filled at runtime.

Prompt registry

Generative AI hub service storing versioned, reusable prompt templates.

Orchestration service

Generative AI hub service chaining templating, grounding, filtering and masking in one call.

Sources

1. SAP Learning — Managing Prompts with the Prompt Registry and Templates (templates, versions, {{?placeholders}}, imperative and declarative APIs; accessed 18 Oct 2026)
2. SAP Developers tutorial — Generative AI hub evaluation (datasets, prompt template by ID, metrics, win rate; accessed 18 Oct 2026)
3. SAP Developers tutorial — Prompt optimization in the generative AI hub (model-specific optimisation, Extended plan; accessed 18 Oct 2026)
4. SAP Developers tutorial — Prompt optimization, tool calling and response formatting (response_format text/json_object/json_schema; accessed 18 Oct 2026)
5. SAP Developers tutorial — Prompt registry in the generative AI hub (found via search; snippet only, accessed 18 Oct 2026)
6. SAP Help Portal / SDK reference — Prompt Registry API (found via search; not read in full, accessed 18 Oct 2026)
7. SAP Learning — Discovering the Orchestration Service (module list as summarised by search; accessed 18 Oct 2026)
8. SAP Learning — Introduction to Orchestration (templating mandatory, modules chained in one call; search summary, accessed 18 Oct 2026)
9. SAP Developers tutorial — Create a Joule Studio agent (expertise vs instructions, simulator; search summary, accessed 18 Oct 2026)
10. SAP Learning — Building Joule agents with Joule Studio (search summary, accessed 18 Oct 2026)
11. Anthropic — Prompt engineering overview (success criteria first; vendor documentation, accessed 18 Oct 2026)
12. Anthropic — Prompting best practices (3-5 examples, XML tags, role, long documents first, adaptive thinking; vendor documentation, accessed 18 Oct 2026)
13. Anthropic — Prompt caching (5-minute default, 1.25x write, 0.1x read, exact prefix; vendor documentation, accessed 18 Oct 2026)
14. arXiv 2005.14165 — Language Models are Few-Shot Learners (28 May 2020)
15. arXiv 2201.11903 — Chain-of-Thought Prompting Elicits Reasoning in Large Language Models (28 Jan 2022)
16. arXiv 2205.11916 — Large Language Models are Zero-Shot Reasoners (24 May 2022)
17. OWASP GenAI Security Project — LLM01 Prompt Injection (definition, direct vs indirect, mitigations; accessed 18 Oct 2026)