Lesson 6 of 6
Deploying, monitoring and costing agents (AI Units)
9 min
Listen to this lesson
The Fleet Order Status agent passed its gate. Now it has to reach clerks, stay healthy and stay affordable. This lesson covers environments and transports, what to monitor in production, and how to estimate and watch AI Unit consumption.
Environments and the path to production
In the classic edition, a project is released as a version and deployed to an environment. Each developer can activate a private environment in Control Tower for isolated tests. SAP's architecture guidance describes a standalone environment for testing that is not connected to production Joule, and the shared environment that deploys to unified Joule across SAP applications, so users reach the agent from SAP S/4HANA, SAP Build Work Zone and other Joule-enabled apps. Contoso maps these to its BTP landscape: private and test in cmc-ext-dev and cmc-ext-test, shared in cmc-ext-prod, moved along the transport path CMC_DEV, CMC_TEST and CMC_PROD with Priya forwarding after test sign-off and Luis running the scheduled production import.
What to monitor
| Signal | Where to look | Action threshold (Contoso) |
|---|---|---|
| Skill and action failures | Joule Studio monitoring for deployed skills; destination and Cloud Connector logs | More than 2% failed calls in a day |
| Escalation rate | Agent conversation analysis; process logs | Sudden rise or fall of 50% week on week |
| Answer quality | Weekly sample of 20 conversations scored against the case rules | Any control miss; accuracy below 95% |
| Grounding freshness | Document grounding status; grounding register | Any document past review date |
| AI Unit consumption | SAP for Me consumption per premium feature | Above 120% of the weekly forecast |
| Expedite requests | SAP Build Process Automation monitor | Requests stuck without decision for 24 hours |
Production monitoring for the Fleet Order Status agent. Tool names vary by edition; the new Joule Studio adds observability in its managed runtime.
Costing the agent
Custom agents are premium AI and consume AI Units. SAP Learning's commercial material describes consumption for customer-built agents per agent step, with tiers (it lists basic, standard and advanced agents at 0.005, 0.01 and 0.025 AI Units per step); SAP product pages for the agent builder described it as a free promotion until December 2026, after which it becomes a premium AI feature. Treat these figures as a dated snapshot and confirm rates and the counting rules with your SAP account team. Separately, SAP's May 2026 announcement gives free design-time access to the new Joule Studio under fair use through the end of 2026.
| Input | Value (illustrative) | Note |
|---|---|---|
| Conversations per working day | 300 | From the clerk call log baseline |
| Average agent steps per conversation | 6 | Measured in the evaluation runs |
| Rate per step | 0.01 AI Units (standard tier) | Confirm tier and rate in your contract |
| Daily consumption | 300 × 6 × 0.01 = 18 AI Units | Plus retries; add 20% headroom |
| Yearly consumption (250 days) | About 4,500 AI Units, 5,400 with headroom | Compare with time saved per conversation |
An illustrative AI Unit estimate. The method matters more than the numbers: measure steps in testing, multiply by the contracted rate, add headroom.
What would you do?
Two weeks into the pilot, AI Unit consumption is 180% of forecast. Answer quality is fine. What should Grace and Elena do first?
Hypercare and retirement
- Day 1Production smoke testFive evaluation cases in production with Maria watching
- Week 1Daily reviewFailures, escalations and consumption every morning
- Weeks 2 to 4Weekly sampling20 conversations scored against the case rules
- Month 2Scale decisionScorecard: accuracy, time saved, cost per case
- Every releaseRe-evaluationFull set after any change to tools, documents, model or instructions
Register the agent in Contoso’s AI inventory and, where Contoso uses it, SAP AI Agent Hub, with its owner, tools, data and model. Plan its end as well as its start. If an SAP-delivered agent later covers the same need, compare both on the same evaluation set before switching, then undeploy the custom agent, remove its destinations from Control Tower, and archive the release records. An agent nobody owns is a risk that keeps consuming AI Units.
- Release versions and deploy to private, test (standalone) and shared environments; follow the normal BTP transport path and gates.
- Monitor skill failures, escalations, sampled quality, grounding freshness, AI Units and process backlogs.
- Estimate cost as conversations × steps × rate, with headroom; confirm rates and tiers with SAP.
- Treat a rise in steps per conversation as a regression, just like a wrong answer.
Check your understanding · not graded
Lesson check
1. What does deploying to the shared environment do?
2. How do you estimate an agent's AI Unit consumption?
3. Consumption jumped after an instruction change. What should the evaluation runs have measured?
4. Who forwards Contoso's agent from CMC_TEST to production?