Lesson 5 of 5
Pruning the suite
9 min
Listen to this lesson
Suites grow. Every project adds tests, every defect adds a regression test, and nobody removes anything. After a few releases, the suite takes too long to run, maintenance swallows the team, and many tests protect processes nobody uses any more. Pruning is the deliberate, data-driven retirement of tests whose cost exceeds their value. The ISTQB syllabus includes it in continuous improvement: evaluate existing scripts for revision or elimination, decompose complex and slow tests, and target tests that run rarely or never.
Measure value and cost
| Value signals | Cost signals |
|---|---|
| Covers a critical or high-volume business process | Maintenance hours per release |
| Has found defects, or guards a control auditors rely on | Flip rate and time spent analyzing false failures |
| Covers changed areas often (high change frequency) | Runtime and environment or data demands |
| Unique coverage: no other test checks this | Overlap with other tests on the same steps |
Signals for judging an automated test's worth.
Usage data from production helps judge process value. On the SAP side, transaction and app usage statistics show which processes users actually run; a test for a transaction that nobody has used for a year protects little. Combine that with your own run history: defects found, failures that were false alarms, and hours spent on each test.
Options short of deletion
- Split a long end-to-end test into smaller tests so one failure does not hide the rest.
- Move down a level: replace UI checks of business rules with API or ABAP Unit tests, and keep one UI path.
- Consolidate tests that repeat the same steps into reusable functions or one parameterized test.
- Reduce frequency: run low-risk tests weekly instead of nightly.
- Retire: remove from the active suite, archive in version control, and record the coverage gap.
A retirement process
- Candidate listEach quarter, rank tests by cost and value from run history and usage data.
- ReviewPriya and the automation engineers propose actions: keep, redesign, split, reduce, retire.
- Process owner agreementDan, Maria or the relevant owner confirms the business risk of each retirement.
- Retire and archiveRemove from the active suite; archive with a tag and the reason.
- Record the gapNote which risk is now covered manually, at a lower level, or accepted.
At Contoso, the first pruning review cut the nightly suite from 250 to 205 tests. Thirty-one duplicates created by different project teams and 22 tests of legacy transactions retired by the conversion were removed, and four slow end-to-end tests were split into twelve smaller ones (250 − 53 + 8 = 205). The nightly runtime fell by about a third and maintenance time per feature pack stack dropped noticeably, while coverage of the critical processes did not change.
What would you do?
A 45-minute end-to-end test covering procure-to-pay has flipped 12 times in 60 runs and found no defects in a year. It is the only automated check of purchase order approval over $25,000. What should Priya propose?
- Suites grow without pruning; cost eventually exceeds value.
- Judge tests by value (criticality, unique coverage, defects found) and cost (maintenance, flakiness, runtime).
- Prefer split, move down a level, consolidate or reduce frequency before retiring.
- Retire with process owner agreement, archive with a reason and record the coverage gap.
- Protect tests that guard controls; redesign them instead of removing them.
Check your understanding · not graded
Lesson check
1. Which test is the strongest candidate for retirement?
2. Who should agree before a test of a business process is retired?
3. What does "move down a level" mean when redesigning a costly UI test?
4. Why are some slow tests marked "protected" in the pruning review?