Lesson 2 of 5
Test readiness evidence
10 min
Status reports are optimistic by nature. Teams report green because they believe they are nearly done, and nobody wants to be the one turning a dashboard red. The process owner's job at go/no-go is to look behind the color and check that the evidence supports it.
What to ask for
- Execution: scenarios planned, executed and passed for each process in scope, for UAT and for the final regression or mock cutover tests.
- Open defects: by severity, with status, owner and, for any deferred defect, a signed risk acceptance.
- Coverage: which scenarios were descoped or not run, and why. A 100% pass rate on 60% of the scope is not 100%.
- Evidence: a sample of executed scenarios with document flow and journal entries, chosen by you.
- Retests: defects closed in the last two weeks, and whether their retests and related regression passed.
- Was every in-scope scenario executed?Yes → check results · No → ask what was descoped, who agreed and what the risk is
- Do results meet the exit criteria?Yes → check open defects · No → no-go for this process unless risk is formally accepted
- Is every open high or critical defect fixed or formally accepted?Yes → sample evidence · No → condition or no-go
- Does a sample of evidence hold up?Yes → test readiness is green · No → status is not reliable; widen the sample
Spotting a green that is not green
| Warning sign | What it often means | Question to ask |
|---|---|---|
| Pass rate jumped from 70% to 98% in the last week | Tests were re-scoped or passed without full evidence | Which scenarios changed status, and can I see three of them? |
| Many defects closed as "works as designed" or "duplicate" recently | Defects were closed to meet exit criteria | Who agreed these closures? Do the process owners accept them? |
| "Executed" counts include blocked or not-applicable tests | Execution looks complete when it is not | How many passed with evidence, not just executed? |
| No defects in a complex area such as pricing or intercompany | Area not tested deeply, or defects tracked elsewhere | What scenarios covered it, and with what data? |
| Fixes transported in the last days with no regression | New risk introduced after testing | What regression ran after the last transports? |
Warning signs behind a green status.
None of these signs proves a problem. Each is a reason to ask one more question and look at one more piece of evidence. A team that is genuinely ready will answer easily.
Ask the same questions of the final regression run after the last transports. Many programs freeze changes some days before cutover and allow only emergency fixes after that. Every emergency fix should come with its own retest and a note of which processes it could affect, so that the process owner knows whether the evidence reviewed last week still describes the system that will go live.
Training and role readiness
Test readiness is not only scripts. The go/no-go also needs evidence that users can work: training completion by role, not just headcount, and confirmation that each user has been assigned the production business roles they were tested with. A process that passed UAT with key users can still fail on day one if warehouse staff in Chicago cannot post goods issue because a role was never assigned.
| Evidence | Weak | Strong |
|---|---|---|
| Test results | Dashboard showing 98% green | Scenario list with pass/fail, evidence links and a sampled review |
| Defects | Count of open defects | Open defects by severity with owners, dates and signed risk acceptances |
| Training | "Training complete" | Completion by role and site, with gaps named and a plan |
| Roles | "Roles assigned" | Role assignment report checked against the user list per site |
What would you do?
Contoso's O2C status is green: 98% passed. Dan samples three scenarios and finds that two have only success-message screenshots, and one billing document has no journal entry. What should he recommend?
- Ask for execution, open defects, coverage, evidence and retest status per process.
- Watch for warning signs: sudden pass-rate jumps, mass defect closures, descoped areas, late fixes without regression.
- Include training by role and role assignment as test readiness evidence.
- Choose your own evidence sample, including the most complex area.
Check your understanding · not graded
Lesson check
1. Why is a 100% pass rate not necessarily 100% ready?
2. Which is a warning sign behind a green status?
3. Who should choose the evidence sample at go/no-go?
4. Why does role assignment belong in test readiness?