Now accepting guest editorial from SAP, Oracle, Workday and ServiceNow practitioners.Pitch a guide
The Sign-Off Table

Lesson 1 of 6

Why regression matters

8 min

Go-live is the start of change, not the end. After Contoso Motor Co. goes live on SAP S/4HANA, it will receive SAP releases, feature packs, support packages and SAP Notes, and its own project team will transport fixes and enhancements every few weeks. Each change is meant to improve something. Regression testing checks that it did not break something else that used to work.

In an integrated ERP system, "something else" is often far away from the change. SAP S/4HANA shares master data, configuration and the Universal Journal across processes. A change to pricing can alter revenue postings. A change to a business role can remove an app a billing clerk needs. A new version of a delivered CDS view can change a report's totals. The person who made the change rarely sees the side effect; the key user who runs the process does.

Where upgrade issues come from

CategoryWhat changesContoso example of a side effect
Changed standard behaviorSAP adjusts a process, default or message in a new releaseA delivery date check that used to warn now blocks the order
Deprecated or replaced appsAn app, catalog or transaction is replaced by a successorBilling clerks lose a tile because its catalog was removed from the role
Custom code and extensionsSAP objects your code uses change or are no longer releasedA custom credit check for fleet customers stops being called
Interfaces and APIsFields, versions or behavior of APIs and messages changeCarrier label interface rejects deliveries from shipping point 1720
AuthorizationsNew authorization checks or changed default valuesPlant controllers cannot display stock values after the upgrade
Output and formsOutput management or form templates changeInvoices print without the customer's PO number
AnalyticsDelivered views and queries change or are replacedMargin KPI tile shows a different value for the same period
Typical categories of upgrade and release side effects.
The Sign-Off Table · thesignofftable.com/academy/sap-bku-301

The cost of an escaped regression is usually far higher than the cost of the test that would have caught it. An invoice error found in testing costs an hour. The same error found by a fleet customer after three weeks of wrong invoices costs credit memos, re-billing, a revenue correction at month-end and some of the customer's trust. That asymmetry is the business case for regression testing, and process owners should use it when asking for key user time.

What regression testing is, and is not

Regression testing re-runs existing scenarios that passed before, to show they still pass after a change. It is different from testing the new feature itself, which needs new test cases. A good release test therefore has two parts: tests of what was meant to change, and a regression pack for what was meant to stay the same.

New feature testsProve the change does what it should: new app, new field, fixed defect
Targeted regressionScenarios near the change: same process, same data, same integration
Core regression packBusiness-critical end-to-end scenarios run for every release
Smoke checksA short list confirming the system is usable at all after the technical upgrade
Layers of a release test. Each layer answers a different question.
The Sign-Off Table · thesignofftable.com/academy/sap-bku-301

Regression testing is also a permanent capability, not a project activity. The project team that built Contoso's system will shrink after go-live, but changes will keep coming for as long as the system runs. Keeping scenarios, test data, test users and expected results current is part of running SAP, in the same way as month-end close.

Why key users and process owners own it

Technical teams can confirm that an upgrade completed and the system starts. Only the business can confirm that order to cash still ends in correct invoices and correct postings. Key users design and run the business regression scenarios; process owners decide whether the results are good enough to move the change to production. Priya, Contoso's test lead, provides the test management framework, SAP Cloud ALM, and coordinates the cycle.

What would you do?

A support package is planned for Contoso's private cloud system. The technical team says "SAP only fixed bugs in this package, so no business testing is needed." Dan asks Maria for her view. What should she say?

Key points
  • Regression testing proves that what worked before still works after a change.
  • In an integrated system, side effects appear far from the change.
  • Typical issue categories: behavior, deprecations, custom code, interfaces, authorizations, output, analytics.
  • A release test combines new-feature tests, targeted regression, a core pack and smoke checks.
  • Key users run business regression; process owners decide whether results are good enough.

Check your understanding · not graded

Lesson check

  1. 1. What does regression testing check?

  2. 2. Why is SAP's own testing not enough for your upgrade?

  3. 3. A billing clerk loses a tile after an upgrade. Which category is this?

  4. 4. Which layer confirms the system is usable at all after a technical upgrade?