Lesson 3 of 6
Exit criteria: are we done?
8 min
Listen to this lesson
Exit criteria define when a test phase can be considered complete. ISTQB calls them the conditions for officially completing an activity; in agile terms, they play a role similar to a definition of done. They turn "are we finished?" from a feeling into a check. On SAP programs, they are the basis of every test phase sign-off.
Typical exit criteria
| Area | Exit criterion | Contoso example for UAT |
|---|---|---|
| Coverage | All planned test cases executed; all in-scope requirements traced to at least one executed test | 100% of UAT cases executed; requirement traceability shows no untested requirement |
| Results | Pass rate at or above an agreed threshold, with every failure explained | At least 95% passed; each failure linked to a defect |
| Open defects | No open critical defects; no open high defects without an agreed workaround and owner | Zero critical; two high with documented workarounds accepted by Dan |
| Retests and regression | All closed defects retested; regression run after the last fix transport | Regression pack run on the final build |
| Risks | Remaining risks documented and accepted by the right owners | Risk acceptance records signed for deferred items |
| Evidence | Results, defects and test summary report available for review | Test summary report published; evidence attached in SAP Cloud ALM |
Exit criteria by area. Thresholds are examples; each program sets its own.
Thresholds and their traps
Pass-rate thresholds are useful but dangerous on their own. A 95% pass rate can hide a failed period-end close or a failed tax scenario that matters more than all the passes together. That is why good exit criteria combine a pass rate with severity-based limits on open defects and with coverage of business-critical scenarios: "all scenarios marked critical by the process owner passed" is stronger than any percentage.
| Entry criteria | Exit criteria | |
|---|---|---|
| Question | Are we ready to start testing? | Are we done testing this phase? |
| Checked | Before the phase starts | Before the phase is signed off |
| Typical items | Environment, build, data, approved test cases, trained testers with access | Coverage, pass rate, open defects by severity, retests, accepted risks, report |
| Contoso examples | QAS client 100 refreshed; build 7 imported | No open critical defects; all critical O2C and finance scenarios passed |
| If not met | Delay or partial start with waivers | Extend, or sign with conditions and accepted risks |
Agree them before execution
Exit criteria must be agreed and published before the phase starts, by the same people who will sign it off. Once results are known, there is strong pressure to redefine done: lower the pass threshold, reclassify a high defect as medium, or call a failed scenario out of scope. Criteria agreed in advance give the test lead something firm to point at, and they protect process owners from being pressured into accepting what they never agreed to.
That does not mean criteria can never change. Sometimes a program learns something that genuinely changes what matters. The rule is that changes are made openly, by the people who own the sign-off, with the reason recorded, never quietly by whoever writes the status report.
The test summary report
The evidence for exit is usually brought together in a test summary report (ISO/IEC/IEEE 29119-3 calls it a test completion report). It states what was tested, what was not and why, results against each exit criterion, open defects with their status and workarounds, risks, and a recommendation. It is the single document a process owner should read before signing a test phase.
What would you do?
Contoso's integration cycle ends with 97% passed, which meets the agreed 95% threshold. However, one open defect stops the period-end close for company code 1710, which Dan marked as a critical scenario. The PMO says the exit criteria are met. Are they?
- Exit criteria define when a phase is done: coverage, results, open defects by severity, retests and regression, accepted risks, evidence.
- Pass rates alone mislead; combine them with severity limits and critical scenario coverage.
- Agree exit criteria before execution, with the people who will sign.
- Change criteria only openly, by the owners, with reasons recorded.
- The test summary report presents results against each criterion.
Check your understanding · not graded
Lesson check
1. Which is a typical exit criterion for SAP integration testing?
2. When should exit criteria be agreed?
3. Why is a pass rate alone a weak exit criterion?
4. What does ISO/IEC/IEEE 29119-3 call the report that summarizes testing at the end of a phase?