Lesson 5 of 5
Reporting program health honestly
9 min
A status report has one job: to help decision-makers act in time. The most common failure in SAP programs is the watermelon report: green on the outside, red on the inside. It is rarely caused by lying. It comes from vague status definitions, metrics that measure activity instead of outcomes, and a culture where amber feels like failure. This lesson covers how to make honest reporting the easy path.
Define the colors
| Status | Contoso definition |
|---|---|
| Green | On track for the next gate and go-live with current plan and resources; no action needed from steering |
| Amber | At risk: the next gate or go-live is threatened unless the named actions succeed; steering is informed and may need to decide |
| Red | Will miss the next gate or go-live without a decision from steering, such as scope, date or resources |
Status colors defined by what they ask of decision-makers, not by how people feel.
Defining colors by the action they require changes behavior. Amber stops being a confession and becomes a request for attention. Red becomes a request for a decision, which is exactly what a steering committee exists for.
Measure outcomes, not activity
| Activity metric (misleading alone) | Outcome metric (pair it with) | Source at Contoso |
|---|---|---|
| Test cases written | In-scope processes and requirements covered by test cases | Cloud ALM solution process and requirement traceability |
| Tests executed | Tests passed with evidence, by process and priority | Test execution analysis |
| Defects closed this week | Open defects by severity and age, with trend | Defect reporting |
| Stories completed | Requirements with passed tests linked | Requirement traceability and burn-up |
| Workshops held | Processes validated with approved backlog | Scope status and requirement approvals |
A report structure that works
Leading indicators
Lagging indicators tell you what already happened, such as a missed gate. Leading indicators warn you early. For quality, the best leading indicators are upstream: requirements without acceptance criteria, stories Done without linked test evidence, test cases written but not linked to processes in scope, unvalidated assumptions about data and environments, and entry criteria for the next test cycle that are not yet met. Grace tracks a handful of these every week, because they turn into missed gates six weeks later.
- Week 1Leading signalThirty percent of requirements still lack acceptance criteria
- Week 3Leading signalStories Done but only half with linked passed tests
- Week 5Lagging signalSIT entry criteria not met; cycle start slips
- Week 8Lagging signalRealize gate at risk; steering asked to decide on scope or date
Culture
Honest reporting needs safety. If an amber status leads to blame, people will report green until the day it turns red. Grace thanks people who raise amber early, asks "what do you need?" rather than "whose fault is it?", and makes sure the steering committee sees an early amber as good management. She also compares self-reported status with the data in Cloud ALM; when they disagree, the data starts the conversation.
What would you do?
Contoso's weekly report has been green for eight weeks. Priya's data shows SIT pass rate at 62 percent, 14 open severity 2 defects aging over two weeks, and two interfaces not yet available. Workstream leads say "we will catch up". What should Grace report?
- Watermelon reports come from vague definitions, activity metrics and fear of amber.
- Define status colors by what they ask of decision-makers.
- Pair activity metrics with outcome metrics drawn from Cloud ALM traceability, test and defect data.
- Track leading quality indicators and make early amber safe and valued.
Check your understanding · not graded
Lesson check
1. What is a watermelon status report?
2. How should amber be defined?
3. Which is an outcome metric?
4. Which is a leading indicator of quality trouble?