Lesson 4 of 5
The evidence pack
12 min
Listen to this lesson
At go/no-go, decision makers sign something that will be judged later, sometimes by auditors. The evidence pack is the compact, traceable set of documents that shows what was tested, what was found, what remains and who accepted it. It turns "the test lead says we are ready" into "here is the evidence; you can check it".
What goes in
| Section | Contents | Contoso example |
|---|---|---|
| 1. Scope and approach | What was in and out of scope, test levels and cycles run, exit criteria | SIT cycles 1–3, UAT, regression; scope by value stream; exit criteria from the test strategy |
| 2. Coverage and traceability | Requirements, process steps and high risks linked to executed tests and results | Traceability from processes and requirements to test cases, runs and defects |
| 3. Execution results | Results by cycle and value stream; final cycle in detail | Executed, passed, failed, blocked per stream; final-cycle pass and execution rates |
| 4. Open defects | Every open critical and high defect with severity, process, workaround, owner and target date; summary of the rest | 0 critical, 4 high with workarounds and owners; 23 medium/low summarized |
| 5. Risk register and residual risk | High and medium risks with residual status and acceptances | R-07 partly mitigated, accepted by Dan until R3 |
| 6. Exit criteria status | Each criterion met, not met or met with deviation, with reason | All met except one: two UAT scripts in plan to produce deferred, accepted |
| 7. Sign-offs | Process owner and key user sign-offs per value stream, with dates and conditions | Maria for order to cash, Dan for record to report, others |
| 8. Related readiness inputs | References to data migration validation, cutover rehearsal, performance and security role tests | Mock 3 migration sign-off; cutover rehearsal report from Luis |
Contents of a go/no-go evidence pack.
Traceability is the backbone
Traceability means anyone can follow a line from a requirement or process step to the tests that cover it, to the results of their latest execution, to any defects raised and their status. In SAP Cloud ALM, test cases are related to processes, requirements and user stories, and test execution analytics include requirements traceability, which makes this largely a reporting step if the links were maintained during the project. If links were not maintained, building them at the end is slow and unconvincing; that is why the test strategy should require them from day one.
- Requirement or process stepOrder to cash: price fleet order for Great Lakes Fleet Services
- Test casesFleet rebate pricing, five variants; linked risk R-03
- Latest executionCycle 3: 5 of 5 executed, 4 passed, 1 failed then passed on retest
- DefectsOne high defect, fixed and confirmed, closed
- Sign-offMaria signed order to cash with no conditions on this step
Open defects: the section everyone reads
Decision makers go straight to open defects. For every open critical and high defect show: a business-language description (not just the technical title), the affected process and organizational scope, the severity, the workaround and who performs it, the owner, the planned fix date or release, and the residual risk if the workaround fails. "Credit memos for export customers post to the domestic revenue account; finance reclassifies daily; fix in R3; owner Dan" is a decision-ready line. "DEF-1182 – account determination error – open" is not.
- Hand overGive the pack to an internal auditor or PMO analyst
- TraceThree high risks and two requirements, start to finish
- Watch for gapsNote every place they need to ask the test team
- FixClose missing links, results or sign-offs
- ReadyThey can trace every line unaided
Keep it compact
An evidence pack is not every artifact the project produced. It is a summary with links. Ten to twenty pages plus a traceability export is typical for a large SAP program. Put the detailed results, logs and test evidence in the test management tool and reference them; the pack should let a reader understand the position in half an hour and check any line in a few minutes more.
What would you do?
The day before Contoso's go/no-go, Priya's draft pack has full execution results and a complete defect list exported from the tool, 400 rows. Grace asks for the pack to be "decision-ready". What change matters most?
State what was tested, where
Results only mean something for the system they were obtained on. Include a short statement of the test basis: the system and client (for Contoso, QAS client 100), the date of the last data refresh, the transport or build level tested, and any transports imported after the final regression with how they were checked. If a late transport reached production without passing through the final regression, the pack must say so, together with the targeted test that covered it. Decision makers are rarely told this, and it is exactly the kind of gap that causes a surprise in the first week after go-live.
- The pack covers scope, coverage and traceability, results, open defects, residual risk, exit criteria, sign-offs and related readiness inputs.
- Traceability from requirement to test to result to defect is the backbone; maintain links from day one.
- Describe open critical and high defects in business language with workaround, owner and fix date.
- Keep it compact with links; test it on someone outside the team.
Check your understanding · not graded
Lesson check
1. What is the backbone of an evidence pack?
2. Which item is essential for each open high defect?
3. Why should traceability links be required from day one?
4. How can you test whether a pack is ready?