Lesson 5 of 5
Reports as test evidence
7 min
Reports are not only for running the business. In testing, they are one of the best ways to prove that a process changed data correctly, because a report reads what the system actually stored, not what the tester thinks happened. A saved variant turns that into a repeatable check.
Before and after
Many test scenarios change a total. Posting a goods receipt of 500 CMC-RTR-200 rotors should raise stock in plant 1710 by 500. Billing an order for Great Lakes Fleet Services should raise open receivables by the invoice amount. Running the same saved report before and after the test step and comparing the two results proves the change, and proves nothing else moved.
- BeforeRun the saved variantRecord count and totals, for example stock of CMC-RTR-200 in plant 1710 = 1,200 PC
- Test stepExecute the scenarioPost goods receipt of 500 PC against the purchase order from Lakeside Components
- AfterRun the same variant againStock = 1,700 PC; the difference matches the receipt
- EvidenceStore both resultsBefore and after figures, variant name, system, client and times
Why the variant matters
If the "before" and "after" runs use different filters, the comparison is meaningless. A named variant guarantees both runs select the same data. It also lets another tester, or an auditor, repeat the check later. At Contoso, Priya's team keeps a short list of public evidence variants per process area, such as "UAT stock 1710 rotor" and "UAT open AR 1710", so testers do not invent their own each time.
| Test scenario | Report to run before and after | Expected difference |
|---|---|---|
| Goods receipt for purchase order | Stock list for CMC-RTR-200, plant 1710 | + received quantity |
| Goods issue for delivery | Stock list for CMC-ALT-100, plant 1710 | − delivered quantity |
| Customer invoice | Open customer line items, Great Lakes Fleet Services | + invoice amount |
| Incoming payment | Open customer line items, Great Lakes Fleet Services | − paid amount; item cleared |
| Supplier invoice | Open supplier line items, Lakeside Components | + invoice amount |
Before-and-after report checks for common scenarios.
Testing the report itself
Sometimes the report is the thing under test: a new analytical app, a custom query, or a standard report after an upgrade. Then the approach flips. You compare the report against an independent source: a set of documents you created with known values, another report that should agree, or the journal entries behind the totals. The Cross-Process track covers analytics testing in more depth; the basic idea is that a total is only trustworthy if you can trace it back to the records that make it up.
- Is the report the object under test?Yes → compare it with independent data you control · No → use it as evidence of a process step
- Do before and after runs use the same saved variant?Yes → compare totals · No → rerun with one variant before drawing conclusions
- Does the difference equal the expected change exactly?Yes → record as evidence · No → look for other postings in the same period or a defect
- Could other testers have posted to the same data?Yes → narrow the variant to your test documents or a dedicated test product · No → result stands
What would you do?
A Contoso tester posts a customer invoice for 12,500.00 USD. The open receivables report for Great Lakes Fleet Services rises by 13,700.00 USD between the before and after runs. What should he do next?
- Run the same saved variant before and after a test step to prove a change in totals.
- Named, shared variants make evidence repeatable by others.
- When the report itself is under test, compare it with independent data you control.
- Shared test data can distort differences; use dedicated data or narrow the selection.
Check your understanding · not graded
Lesson check
1. Why use the same saved variant before and after a test step?
2. A goods receipt of 500 PC is posted. What should a before-and-after stock report show?
3. When a report itself is under test, what should you compare it with?
4. The difference is larger than expected in a shared test client. What is a likely non-defect cause?