Prompt templates and the prompt registry

In a prototype, the prompt lives in code. In production, the prompt is a configuration item that changes more often than the code around it, and every change can alter behavior. SAP's prompt registry is the generative AI hub's answer: it manages "the life cycle of your prompts, from design to runtime" for both prompt templates and complete orchestration configurations.

Templates and placeholders

A prompt template is a set of messages with placeholders, for example a system message with the role and constraints and a user message with {{?idoc_status}}, {{?segment_extract}} and {{?runbook_context}}. At run time the application supplies the placeholder values and orchestration fills them. Templates can carry defaults, and SAP AI Launchpad's prompt editor lets designers try a template before it is stored.

Two interfaces: imperative and declarative

Orchestration configurations can be stored too, as reusable, versioned assets with history and import and export. A stored orchestration config can reference a prompt template by ID or by name, scenario and version, and completion requests can reference the stored config instead of sending the whole configuration inline. SAP notes that orchestration configs are available through the imperative API and are managed at tenant level. Since August 2026, templates and configs can be listed in pages, which helps once a tenant holds many of them.

Testing a prompt change

A prompt change is a code change without a compiler. Treat it like one: version it, review it, test it against a fixed regression set, then release it. Contoso keeps 40 representative failed IDocs (synthetic data built from CMQ test cases) with an expected classification and the key facts the explanation must contain. A change passes only if classification accuracy doesn't drop and every expected fact still appears. Changing the model version triggers the same run.

Prompt optimization

The hub can also improve a prompt for you. Prompt optimization takes a template from the prompt registry and a JSON dataset of desirable responses in an object store, and optimizes the prompt to maximize a metric you choose; metrics are tracked in the ML Tracking Service, and the optimized prompt is saved back to the registry. SAP's caveats are worth repeating: it is model-specific, runs can take "from minutes to multiple hours", they submit a large number of requests to the target model (with costs), and Mistral and DeepSeek models are not supported. Custom metrics are supported as LLM-as-a-judge metrics with numeric or Boolean output.