Lesson 5 of 5
Coverage at quality gates
10 min
Listen to this lesson
At a quality gate, the steering committee asks one question in many forms: is everything we agreed to build, built and tested? Requirement coverage is the answer, provided it is reported honestly. This lesson shows how Contoso turns traceability into gate evidence and how a process owner reads it before signing.
Gates in the Contoso plan
- End of ExploreScope gateAll workshop requirements captured, approved or rejected; acceptance criteria present
- Each sprint reviewBuild checkpointStories linked; features created for approved requirements
- SIT entryTest readinessEvery requirement with a feature has linked test cases in the SIT plan
- SIT exitIntegration evidenceLinked tests executed; failures dispositioned; no open critical defects
- UAT exitBusiness acceptanceProcess owners confirm requirements; residual risks recorded
- Go/no-goCutover decisionCoverage, defects and mock conversion results reviewed by Grace's steering group
Cloud ALM supports gates directly. SAP has delivered quality gate management with checklists, and its roadmap material describes an optional constraint so that a gate can be set to Accepted or Conditionally Accepted only when every checklist item has been checked. Contoso uses a checklist item per evidence type: requirement coverage, defect status, interface tests, role tests and data migration results for Wave 2 later.
Three numbers, by process
Report requirement coverage as three separate figures, each per process, as the gate course in this track explains: design coverage (requirements with at least one linked test case), execution coverage (linked tests run in the current plan) and pass coverage (requirements whose linked tests all passed in the latest run). Add one more line that only requirements can provide: the count of approved requirements with no feature or story, because unbuilt scope is a release risk even when every test that exists has passed.
| Process | Approved reqs | Design coverage | Execution coverage | Pass coverage | Unbuilt |
|---|---|---|---|---|---|
| Order to cash | 64 | 100% | 97% | 91% | 0 |
| Procure to pay | 51 | 94% | 100% | 96% | 2 |
| Record to report | 38 | 100% | 82% | 79% | 0 |
| Plan to produce | 29 | 97% | 93% | 93% | 1 |
Read the table line by line. Procure to pay has executed everything that exists, but three requirements have no test, and two approved requirements have nothing built. Record to report is behind on execution, a schedule problem. Each line leads to a different action, and a single blended percentage would hide all of them.
Turning gaps into decisions
Gates rarely pass with zero gaps. What matters is that every gap has a decision. For each untested or unbuilt requirement, the gate record should say one of: fixed before the gate, accepted with a condition and a date, deferred to a later release with the process owner's agreement, or rejected. Record the decision on the requirement itself or the gate checklist so the next gate can check it was honored.
- Pull the viewsRequirement and solution process traceability, filtered to in-scope processes
- Compute three figuresDesign, execution and pass coverage per process, plus unbuilt count
- List exceptionsEvery requirement that is not built, not tested or not passed
- Decide each oneFix, condition, defer or reject, with an owner and date
- Owner reviewDan and Maria trace one high-risk requirement end to end
- Record the gateChecklist items checked; gate accepted or conditionally accepted
Keeping the numbers honest
Three habits keep coverage trustworthy. First, agree how test cases linked to stories count toward their requirement, and print that rule on the report. Second, freeze the requirement scope for the gate: requirements approved after the cut-off go to the next cycle, so the denominator does not move during the meeting. Third, never "fix" coverage by linking an existing test to a requirement it does not exercise. A test lead who does that once loses the steering committee's trust in every later number.
What would you do?
Two days before Contoso's SIT exit gate, procure-to-pay design coverage is 94 percent. A consultant offers to link the generic PO creation test to the three uncovered requirements so the figure reaches 100 percent. What should Priya do?
- Gates ask whether everything agreed was built and tested; requirement coverage answers it.
- Report design, execution and pass coverage per process, plus approved requirements with nothing built.
- Every gap needs a decision: fix, condition, defer or reject, with owner and date.
- Never raise coverage by linking tests that do not exercise the requirement.
Check your understanding · not graded
Lesson check
1. Which figure shows the share of requirements with at least one linked test case?
2. Why add a count of approved requirements with nothing built to the gate report?
3. What should happen to each gap at a gate?
4. A consultant wants to link a generic test to uncovered requirements. Why refuse?