Lesson 4 of 7
Evidence standards
9 min
Listen to this lesson
If every team decides for itself what counts as proof, gate reviews become arguments about screenshots. An evidence standard defines, for each functional area and test type, what must be captured, in what form and where. It is the most practical part of the framework and the one testers will use every day.
Qualities of good evidence
These qualities echo long-standing data integrity ideas used in regulated industries: records should be attributable, legible, contemporaneous, original and accurate. You do not need a regulated program to benefit from them.
Evidence by functional area
| Area | Minimum evidence for an end-to-end test | Contoso example |
|---|---|---|
| Order to cash | Document flow from order to payment with numbers; pricing and tax checked against expected; output sent; accounting document reviewed | Great Lakes order for CMC-ALT-100 through delivery from shipping point 1710, billing and incoming payment |
| Procure to pay | PR and PO with approval path; goods receipt; invoice with three-way match; payment | PO for $31,000 to Lakeside Components triggers approval (over $25,000); evidence shows the approver and date |
| Record to report | Period-end steps executed; key reports reconciled; ledger 0L and 2L values compared | Month-end close in 1710 with trial balance before and after; IFRS adjustments in 2L |
| Integration | Message status in the monitor and the posted business document checked against the source | Inbound ORDERS IDoc in status 53 and the sales order compared with the customer's message |
| Data migration | Counts, totals and samples at legacy, extract and target; difference log | Wave 2 open AR for 1910 matches MXN 48,615,320.40 by customer and currency |
| Security and roles | Positive and negative tests per role: allowed actions work, forbidden ones are blocked | Ana (AP clerk) can post invoices but cannot release payments |
Minimum evidence by area. Templates make these the default, not a negotiation.
| Weak evidence | Why it fails | Strong evidence |
|---|---|---|
| "Tested OK" in a spreadsheet cell | No document, no data, no comparison | Step results in SAP Cloud ALM with document numbers and expected vs actual |
| A cropped screenshot | No system, client, user or time visible | Screenshot or export showing system, client, user and time, attached to the step |
| An email from the key user | Not linked to tests; cannot be traced | Recommendation recorded on the test plan with a reference to results |
| A dashboard of pass rates | Hides which processes failed | Pass rates by process and criticality, with failures listed |
How much is enough?
Enough evidence is evidence that would convince a skeptical reviewer who was not there, proportionate to risk. A low-risk report layout needs one execution with a result. A critical financial posting needs the posting checked against an expected value, in both ledgers where relevant, with the accounting document number recorded. Write this proportionality into the standard as levels.
| Level | Applies to | Required |
|---|---|---|
| Level 1 | Low-risk, non-financial steps | Step result and tester; documents named |
| Level 2 | Standard business processes | Level 1 plus expected vs actual values and attachments for key steps |
| Level 3 | Critical processes, financial postings, integrations, migration | Level 2 plus independent review of evidence before acceptance |
Evidence levels. Assign a level to each process in the test scope.
What would you do?
Kenji's team submits UAT evidence for a custom pricing enhancement: 40 screenshots in a shared folder, named by date, with no test case references. What should Priya do?
- Good evidence is attributable, specific, compared, original and linked.
- Define minimum evidence per functional area, such as document flows for O2C and three-way match for P2P.
- Scale evidence to risk with levels; Level 3 adds independent review.
- Process owners sample evidence before they sign.
Check your understanding · not graded
Lesson check
1. Which quality is missing from a cropped screenshot with no system or user visible?
2. What extra requirement does Level 3 evidence add at Contoso?
3. Which is minimum evidence for an integration test?
4. A process owner opens three random test cases before signing. What is she checking?