Lesson 5 of 6
Running the cycle
8 min
A regression cycle is a small project with a fixed window. It needs a plan, named roles, clear entry and exit criteria, and a way to handle defects found late. Contoso runs its cycles in SAP Cloud ALM: test plans contain the selected test cases, testers execute and record results with evidence, and defects link back to the failed steps.
Entry criteria
Entry criteria protect testers from wasting time on a system that is not ready. Do not start the cycle until:
- The change is applied to the test system and technical checks are complete.
- Smoke checks pass: users can log on, core apps open, a document can be posted.
- Test data is available: master data, stock in plants 1710 and 1720, open orders for the scenarios.
- Roles and test users are adjusted for the release.
- The test plan is approved: scope, schedule, testers and exit criteria.
- Known issues from release information are listed so testers do not report them as new.
Check the test system itself. Regression results only predict production if the test system matches production in configuration, roles and relevant data. If Contoso's quality system has drifted, for example because fixes were transported to production directly in an emergency, the cycle tests the wrong thing. Before a major cycle, confirm with the technical team that transports are in sync and decide whether a refresh of the test system from production is needed.
Schedule and roles
- Day 0Upgrade of test systemTechnical team applies the change; smoke checks
- Days 1 to 2Core packAutomated and manual core scenarios; first results
- Days 3 to 7Targeted regressionKey users test changed areas, neighbors and role checks
- Days 8 to 10Fix and retestDefects fixed, retested; affected core scenarios re-run
- Day 10Exit reviewResults against exit criteria; recommendation to process owners
| Role | Contoso | Responsibility |
|---|---|---|
| Test lead | Priya | Plan, scope, daily status, exit review |
| Key users | Maria and area key users | Execute business scenarios, record evidence, raise defects |
| Technical team | Basis and developers | Apply change, fix defects, adjust custom code and roles |
| Process owners | Dan and peers | Decide on open risks and sign off |
Roles in a regression cycle.
Daily control
In a short window, a day lost is a tenth of the cycle. Priya runs a fifteen-minute daily status: scenarios executed, passed, failed and blocked, new defects by severity, and anything stopping testers. Blocked tests deserve the most attention. A test blocked by missing stock in storage location 172A or a locked test user is not a product problem, but it consumes the window just the same. Prepare test data and users before day 0, and name someone in the technical team who can unblock within hours.
Testers should record evidence as they go: document numbers, the business result compared with the expected one, and a short note for anything unusual. Evidence reconstructed on the last day is weaker and slower than evidence captured during execution.
Exit criteria
Exit criteria say when the cycle is finished and the change can move on. Agree them before the cycle starts, for example:
- 100% of core pack scenarios executed; all passed or with an accepted workaround.
- At least 95% of targeted scenarios executed; untested ones listed with a reason.
- No open critical or high defects; medium defects with workarounds accepted by the process owner.
- Critical reports reconciled for at least one period.
- Evidence attached for each executed scenario.
Defects found late
The hardest moment in a cycle is a serious defect on day 9 of 10. The decision depends on severity, available fixes and whether you control the timing.
- Is the defect critical for a core process?Yes → next question · No → fix in the next release train, with a workaround if needed
- Can a fix be built and retested before the window closes?Yes → fix, retest the defect and re-run affected core scenarios · No → next question
- Do you control the production timing (on-premise or Private Edition)?Yes → postpone the production move until fixed · No (Public Edition) → prepare a workaround, raise an SAP incident if the cause is SAP standard, and alert the process owner
What would you do?
On day 9 of Contoso's Public Edition regression window, Maria finds that invoices for partial deliveries post revenue to the wrong account. The production upgrade is in five days and cannot be moved. What should happen?
- Entry criteria: change applied, smoke checks passed, data and roles ready, plan approved, known issues listed.
- A short cycle runs core pack first, then targeted tests, then fix and retest.
- Exit criteria are agreed before the cycle and include execution, defects, reports and evidence.
- Late critical defects: fix if possible; postpone if you control timing; otherwise workaround and escalate.
- After a fix, re-run surrounding and core scenarios, not just the failed step.
Check your understanding · not graded
Lesson check
1. Which is an entry criterion for a regression cycle?
2. When should exit criteria be agreed?
3. What is the difference between retest and regression after a fix?
4. A critical defect is found late, and Contoso controls the timing in Private Edition. What can it do?