Lesson 2 of 6
Building a tiered suite
7 min
Listen to this lesson
A regression suite is a curated set of tests that represents the business processes that must keep working. Curated is the key word. A suite is not every test case ever written on the project; it is the smallest set that gives the business confidence, organized so you can run the right amount for each change. The most practical structure is three tiers: smoke, core and extended. Each deeper tier adds to the one before it, and each has a target run time that the team defends. The run time matters as much as the content: a tier is only useful if it fits the window the change allows. If smoke takes half a day, nobody will run it after every import; if core takes a week, it will be cut short at every stack.
| Tier | Purpose | Contoso contents (examples) | Triggered by |
|---|---|---|---|
| Smoke | Prove the system is usable after any import or technical change | Log on to client 100; open Manage Sales Orders; create a standard order for CMC-ALT-100 in sales area 1710 / 10 / 00; post a goods receipt for CMC-RTR-200 in plant 1710; display a journal entry; confirm key background jobs and the main interfaces ran | Every transport import to QAS and production; system refresh; kernel or database patch |
| Core | Prove the critical processes still produce the right business result | Sell from stock from plant 1710 through billing and journal entry; buy rotors from Lakeside Components through invoice and payment proposal; production order for CMC-ALT-100 consuming CMC-RTR-200; a standard cost check ($150 Detroit, $156 Chicago); a manual journal entry and a simple close step | Support package or feature package stack; significant configuration release; custom code release |
| Extended | Cover variants, exceptions and rarely run processes | Partial deliveries, returns and credit memos; intercompany between 1710 and 1720; output forms; month-end and year-end steps; interfaces with error handling; authorization checks by role; customer Great Lakes Fleet Services special pricing | Release upgrade; large stacks; changes to period-end or cross-company processes |
A tiered suite for Contoso. Tests in a deeper tier are run in addition to the shallower ones.
Notice what the core tier is built from: business processes, not transactions. A test that only proves a sales order can be saved belongs in smoke. A core test follows the process to its business result, with the document flow and journal entry compared against expected values. That is the evidence a process owner can sign. The extended tier, in turn, is where the variants live: the partial delivery, the return, the second plant, the customer with special pricing. These are the cases that most often break quietly, because nobody runs them every day, so they deserve a place in the suite even though they do not run for every change.
Build from the process hierarchy
Start the suite from your solution documentation: the list of end-to-end processes and process steps that Contoso actually runs. In SAP Cloud ALM that is the solution process structure; in SAP Solution Manager it is the solution documentation. Map each test to the process step it proves. Two things follow. You can show coverage (which processes have no regression test), and when change impact analysis later tells you a process step is affected, you can find its tests immediately. Most of the suite's raw material already exists. The project's system integration test and user acceptance test cases are the natural starting point, but they need trimming. Project tests were written to prove new design, so they often cover one variant in great detail and repeat set-up steps. Regression tests should be shorter, should reuse stable test data such as Contoso's fixed customer Great Lakes Fleet Services and supplier Lakeside Components, and should end in a checkable result. Write each one so that someone other than its author can run it and reach the same verdict.
- List processesTake the end-to-end processes and steps from the solution documentation.
- Harvest project testsStart from the system integration and user acceptance test cases.
- Trim and stabilizeShorten set-up, reuse fixed data such as Great Lakes Fleet Services, end in a checkable result.
- Map to process stepsLink every test to the step it proves, so gaps and impacted tests show up.
- Assign a tierPlace each test in smoke, core or extended and check each tier fits its run time.
Keeping the suite lean
Suites grow by default. Every defect adds a test, every project adds a set, and nobody removes anything. A suite that takes three weeks will be skipped, which is worse than a smaller suite that is run. Treat the suite like a product with an owner, usually the test lead, and review it after every release. Look at which tests found defects, which ones failed for reasons unrelated to the change (bad data, outdated expected results), and which processes changed or disappeared. Then apply a few rules:
- Retire tests for processes, plants or document types the business no longer uses.
- Merge tests that repeat the same path with trivial data differences.
- Move down a tier a test that has not found a defect in a long time and covers a low-criticality, rarely changed area.
- Move up a tier a test whose area keeps changing or keeps producing defects.
- Refresh expected results when a release legitimately changes behavior, so the suite does not report false failures.
- Is the process, plant or document type still in use?No → retire the test · Yes → next question
- Does another test follow the same path with trivially different data?Yes → merge them · No → next question
- Does the area keep changing or producing defects?Yes → move the test up a tier · No → next question
- Low criticality, rarely changed, no defect found in a long time?Yes → move it down a tier · No → keep it where it is
- Did this release legitimately change the expected result?Yes → refresh the expected result before the next run · No → done
- A regression suite is a curated, process-based set of tests, not every test ever written.
- Smoke (under an hour), core (one to two days) and extended (one to two weeks) tiers let you match depth to the size of the change.
- Build the suite from the process hierarchy so you can show coverage and react to impact analysis.
- Keep it lean: retire, merge, move tests between tiers and refresh expected results after every release.
Check your understanding · not graded
Lesson check
1. What is the purpose of the smoke tier?
2. Which test belongs in the core tier rather than smoke?
3. Why build the suite from the process hierarchy?
4. A test has not failed in two years and covers a rarely changed, low-criticality report. What should you do?