Lesson 1 of 6
Warning signs: leading quality indicators
12 min
SAP programs rarely fail suddenly. They drift: a design decision left open, a test cycle that starts with half its entry criteria met, a data mock that "nearly" reconciles. By the time the lagging indicators turn red, such as a missed go-live or a flood of production incidents, the options for recovery are expensive. This course is about seeing trouble early, diagnosing it honestly and leading the program back to a go-live that can be defended. It starts with the signals.
Leading versus lagging
A lagging indicator tells you what has already happened: missed milestones, budget overrun, post-go-live incidents, audit findings. A leading indicator moves before the outcome and gives you time to act. In SAP programs, the most reliable leading indicators are about quality and readiness, not cost.
| Area | Leading indicator | Lagging indicator |
|---|---|---|
| Design | Open design decisions and change requests still rising late in Realize | Rework in UAT; scope dispute at go/no-go |
| Test readiness | Test cycles starting late with few entry criteria met | Missed test exit dates |
| Defects | Arrival rate above closure rate; average age rising; reopen rate above about 10% | Go-live with open critical defects |
| Data | Mock reconciliation differences unexplained; error rates not falling between mocks | Wrong balances after go-live |
| Business engagement | Key users missing workshops and test sessions | Low adoption; high how-to ticket volume |
| Environments | Test systems unavailable or out of sync with production-bound transports | Blocked test days; failed cutover steps |
| Reporting | Status green while detailed metrics deteriorate | Sudden red at the gate |
Signals in the test and defect data
Test and defect data is the most honest data a program has, because it is produced by execution rather than by opinion. Three patterns deserve immediate attention. First, a widening gap between defect arrival and closure: the backlog grows and every retest waits. Second, rising defect age: old defects are the hard ones, often design problems disguised as bugs. Third, pass rates that look good but cover little: 95% passed of the 40% executed says nothing about the 60% not run.
- Open design decisions lateBuild proceeds on assumptions
- Test entry criteria waivedSIT starts with unstable build and incomplete data
- Defects outpace fixesBacklog grows; retest blocks progress
- Cycles mergeSIT and UAT overlap; regression dropped
- Status stays greenSteering committee unaware until the gate
The watermelon problem
A watermelon status report is green outside and red inside. It happens when status is reported by self-assessment against milestones ("on track"), rather than measured from evidence. It is rarely dishonest by intent: every workstream lead believes they can recover their part. But the sum of many small optimisms is a large surprise. In the what-if, Contoso's weekly report showed SIT as green for five weeks, while the defect backlog tripled and only a third of the end-to-end cases had run.
Soft signals
Some of the earliest signals are not in any report. Key users stop attending. Workshops end without decisions. The same issue appears in three steering committee packs with a new date each time. Partner staff rotate faster than planned. People start saying "we will fix it in hypercare". Experienced program leaders treat these as data: write them down, look for a measurable counterpart, and raise them before they harden into a crisis.
- Decisions reopened that everyone thought were closed.
- Workarounds accepted in SIT that nobody has put in front of the process owner.
- Test cases being rewritten to match what the system does.
- Cutover rehearsals postponed to give build more time.
- Steering committee packs that grow longer while saying less.
What would you do?
Grace notices that SIT has been reported green for five weeks, but the defect backlog has tripled and only a third of end-to-end cases have run. What should she do first?
- Leading indicators move before outcomes; in SAP programs they are mostly about quality and readiness.
- Watch defect arrival versus closure, defect age, reopen rate, coverage, entry criteria and mock reconciliation.
- A watermelon report is green outside and red inside; compare reported status with measured evidence.
- Soft signals such as reopened decisions and falling attendance are data too.
Check your understanding · not graded
Lesson check
1. Which is a LEADING indicator of trouble?
2. What does a widening gap between defect arrival and closure indicate?
3. A "watermelon" status report is:
4. Why can 95% passed be misleading?