Lesson 1 of 6
Why test scope drifts from reality
10 min
Listen to this lesson
Every SAP regression set is a theory about how the business uses the system. It was written from a blueprint, a process list and the memory of key users, usually years ago. Since then the business added customers, changed terms, invented workarounds and stopped doing some things altogether. The test set did not follow. Process mining reads the system's own records and shows how work really flows, which is the evidence a test lead needs to correct the theory.
Where test scope usually comes from
| Source | Strength | Blind spot |
|---|---|---|
| Business process hierarchy or solution documentation | Complete list of designed processes | Says nothing about frequency or workarounds |
| Key-user interviews | Knows what matters and what hurts | Remembers the usual path and the last incident, not the long tail |
| Previous project's test set | Ready to run | Inherits old scope, including processes nobody runs any more |
| Change impact analysis (course sap-tst-303) | Ties scope to what changed | Ranks objects, not how often the business uses them |
| Process mining | Shows real variants, volumes and exceptions | Only sees what leaves a trace in the data |
Each scope source has a blind spot; mining fills the frequency and variant gap.
Two kinds of drift
Untested variants. Real paths the business runs often that no test case covers. At Contoso (illustrative course data), Maria's regression set covers the standard order for Great Lakes Fleet Services: order, delivery from shipping point 1710, goods issue, invoice. Mining twelve months of order-to-cash in EP1 shows that about one in eight orders passes through a credit block and release before delivery, and one in fifteen has its price changed after the order is created. Neither path was in the regression set.
Dead scenarios. Test cases for paths that no longer happen. Contoso still had six test cases for consignment fill-up and issue from a customer program that ended two years ago. Mining showed zero consignment cases in twelve months. Every regression cycle executed them, and every failure in them was triaged by someone who could have been testing the credit-block path instead.
How drift happens
| Cause | Contoso example (illustrative) | Effect on test scope |
|---|---|---|
| New customers or channels | Great Lakes Fleet Services moves more orders to EDI | Inbound IDoc path gains weight; manual order entry loses it |
| Workarounds | Price corrected after order because a condition record was missing | A rework path that no test case describes |
| Retired programs | Consignment program ended two years ago | Dead tests keep running |
| Organizational change | Aftermarket orders move to plant 1720 Chicago | Profit center CMC-AFTMKT paths need more data rows |
| The system change itself | SAP S/4HANA credit management replaces the classic credit check | Activity names and paths change after go-live |
Typical causes of drift between test scope and real usage.
Mining also has limits, and a test lead should state them up front. It shows what happened, not what should happen: a frequent path can be frequent because users work around a defect. It shows only what left a record. And it says nothing about processes that have not started yet, such as a new business model planned for next year. That is why usage is one lens of three, not a replacement for design and change.
Why it matters for sign-off
A go/no-go pack that says "412 of 420 regression tests passed" sounds convincing. It does not say how much of the business those 412 tests represent. If the untested variants carry 20% of order volume, the pass rate overstates confidence. Process owners such as Dan and the order-to-cash signers need a second number: how much of real usage, by volume and by value, the passed tests cover. That number is the subject of lesson 5; this course builds up to it.
- Scope by design answers "did we test what we built?"
- Scope by change answers "did we test what we changed?" (courses sap-tst-302 and sap-tst-303).
- Scope by usage answers "did we test what the business does?" This is what mining adds.
- A good regression scope answers all three; none of them alone is enough.
What would you do?
Priya presents a regression pass rate of 98% to Grace's decision group. Dan asks: "What share of our real order volume do those tests cover?" Nobody can answer. What should Priya do before the next readiness review?
- Test scope is a theory of business usage that drifts over time.
- Mining exposes untested high-volume variants and dead scenarios.
- Scope by design, by change and by usage answer different questions; use all three.
- A pass rate means more when paired with coverage of real usage.
Check your understanding · not graded
Lesson check
1. What does process mining add to test scoping?
2. What is a dead scenario?
3. Which question does scope by usage answer?
4. Why can a 98% pass rate overstate confidence?