Lesson 4 of 6
Risk-based selection
8 min
No company can re-test everything for every change, and none should. Risk-based selection puts test effort where a failure would hurt most and where change is most likely to cause one. The output is two things: a core regression pack run for every release, and a targeted scope chosen for each specific change.
Scoring scenarios
Score each business scenario on a few factors, using a simple 1 to 3 scale. Keep it simple enough that key users can do it in a workshop.
| Factor | Question | High (3) at Contoso |
|---|---|---|
| Business criticality | What happens if this scenario fails for a day? | Cannot ship or invoice fleet customers; cannot pay Lakeside Components |
| Financial and compliance impact | Does a failure misstate financials, tax or regulatory reports? | Revenue, tax or inventory valuation wrong |
| Usage | How often and by how many people is it run? | Daily, by many users |
| Change impact | Does this release touch it? | Mapped to items in the release information or transports |
| Failure history | Has it broken before? | Defects in past releases or hypercare |
| Complexity and integration | How many components and interfaces does it cross? | Crosses sales, logistics, finance and an external interface |
Scoring factors for regression selection.
| Scenario | Criticality | Financial | Usage | Change impact | History | Complexity | Result |
|---|---|---|---|---|---|---|---|
| Sell from stock CMC-ALT-100 to Great Lakes Fleet Services | 3 | 3 | 3 | 2 | 2 | 3 | Core pack plus deep tests |
| Procure CMC-RTR-200 from Lakeside Components | 3 | 3 | 3 | 1 | 1 | 2 | Core pack |
| Stock transfer 1710 to 1720 | 2 | 2 | 2 | 3 | 1 | 2 | Targeted this release |
| Customer returns with credit memo | 2 | 3 | 1 | 1 | 3 | 2 | Core pack (failure history and financial impact) |
| Display product short texts in Spanish | 1 | 1 | 1 | 1 | 1 | 1 | Monitor |
Scoring extract for a Contoso feature pack. Impact factors: criticality, financial, usage; likelihood factors: change impact, history, complexity.
Scores are a conversation aid, not arithmetic to obey blindly. If Dan says a low-scoring scenario is politically sensitive, for example because an auditor asked about it last year, move it up. Record the reason so next year's review understands the choice.
The core regression pack
The core pack is the set of end-to-end scenarios that must pass for any release to reach production. For Contoso it includes sell from stock for CMC-ALT-100 through billing and payment, procurement of CMC-RTR-200 from Lakeside Components through invoice and payment, production of alternators in plant 1710, the stock transfer between plants 1710 and 1720, period-end closing steps, and the critical reports. Keep it small enough to run in the shortest test window you face, which for Public Edition is the gap between test and production upgrades.
The targeted scope
For each change, add scenarios mapped from release information and transports, plus their neighbors: same process with other data, upstream and downstream steps, and integrations. Include the roles: run at least one scenario per affected business role with that role's test user, because authorization side effects only show up with real roles.
- Start with the changeRelease items, transports, role changes
- Find direct scenariosScenarios that execute the changed objects
- Add neighborsUpstream, downstream, other data variants, integrations
- Add role checksOne scenario per affected role with its real test user
- Add the core packAlways included
Writing regression scenarios that find things
A regression scenario earns its place only if it can fail when something important changes. Write expected results as business outcomes, not "document saved": the invoice amount for 50 CMC-ALT-100 to Great Lakes Fleet Services, the revenue account and profit center on the journal entry, the stock left in storage location 171A, the open item on the customer. Include at least one negative check per critical scenario, such as an order above the credit limit that must be blocked. A pack of scenarios that only check for success messages can pass while the business result is wrong.
Keep test data stable and documented. A regression test that depends on whatever stock happens to be in the test system will fail for the wrong reasons. Agree standard test data per scenario, and a way to restore it before each cycle.
Keeping the pack healthy
- Add scenarios from hypercare and production incidents: every escaped defect is a missing test.
- Retire scenarios for processes no longer used.
- Update test data and expected results when configuration changes on purpose.
- Review scores once a year with the process owners.
What would you do?
Contoso's next feature pack touches pricing. Priya proposes testing all 340 scenarios in the library. The window is three weeks with six key users. What is the better approach?
- Score scenarios on criticality, financial impact, usage, change impact, failure history and complexity.
- High-impact scenarios form the core pack run every release.
- Targeted scope adds scenarios touched by the change, their neighbors and role checks.
- Automate the core pack with SAP tools; keep manual effort for judgment and new areas.
- Keep the pack healthy: add escaped defects, retire unused scenarios, update data.
Check your understanding · not graded
Lesson check
1. Which scenarios belong in the core regression pack?
2. What does change impact mean in risk scoring?
3. Why include one scenario per affected role with its real test user?
4. A defect escaped to production. What should happen to the regression pack?