Upgrade test streams

An upgrade test plan is not one regression run. It is a set of streams, each answering a different question, owned by different people and running at different points of the update. Missing a stream is how upgrades fail: the business processes pass, but the nightly interface to the bank stopped because a job definition changed, or month-end close takes three times as long.

The streams

Technical verificationDid the update complete correctly? Logs, follow-up activities, jobs, Fiori content, front ends.

Custom code and modificationsDid our code survive? Adjustments in SPDD, SPAU and SPAU_ENH; custom code findings fixed; developer tests green.

Functional regressionDo our business processes still work end to end?

Integration and interfacesDo inbound and outbound interfaces, IDocs, APIs and partner files still flow?

Security and rolesCan users still do their jobs, and only their jobs, with new and changed apps?

PerformanceDo critical transactions, jobs and period-end runs still meet their time windows?

Data validation and business sign-offIs business data the same before and after, and do process owners accept the result?

Test streams for an SAP S/4HANA upgrade.

The Sign-Off Table · thesignofftable.com/academy/sap-tst-304

Technical verification

Some technical work is done by SAP ECS under RISE, and some is yours. The upgrade guide lists follow-up activities such as front-end updates (ECS upgrades the SAP Fiori front-end server as part of the request), task lists to activate and update Fiori content, and application-specific activities identified by the checks. It also notes that application data can be migrated after the upgrade during business operations by the Silent Data Migration Infrastructure, visible in transaction SDM_MON. Contoso's technical checklist includes: background jobs rescheduled and running, Fiori launchpad content and app descriptors working, SAP GUI on a compatible version, output management working, RFC destinations and certificates valid, and silent data migrations completed.

Custom code and modifications

Modifications to SAP objects appear for adjustment during the update: dictionary objects in SPDD, repository objects in SPAU, and enhancements in SPAU_ENH, which the guide says to process after SPDD and SPAU. The guide also says to test the modification adjustment in a development system with the same release and the same modifications as production. Adjustments are recorded in transports so that later systems receive them without repeating the work. The test stream here verifies three things: every listed object was adjusted, the adjusted code works (developer tests, ABAP Unit, ATC with simplification checks), and the functions that use it pass functional tests.

Modifications are adjusted once in development, in the order SPDD, SPAU, SPAU_ENH, and the recorded transports carry that work into later systems.

The Sign-Off Table · thesignofftable.com/academy/sap-tst-304

AnalyseRun custom code checks against the target release; fix findings before the update where possible.
AdjustDuring the update in CMD: SPDD, then SPAU, then SPAU_ENH; record in transports.
Unit testRun ABAP Unit and ATC on adjusted objects; Kenji's team signs off technically.
Carry forwardAdjustment transports are offered to later systems during their update.
Process testFunctional regression covers every process that uses adjusted objects.

The custom code and modification stream.

The Sign-Off Table · thesignofftable.com/academy/sap-tst-304

Functional, integration and security

Functional regression is the stream most teams know. In an upgrade, focus it on three kinds of scope: critical processes always (order to cash for Great Lakes Fleet Services, procure to pay with approvals for purchase orders over $25,000, period-end close), processes touched by simplification items and changed apps, and processes using custom code or modifications. Integration testing runs the real interfaces end to end in CMQ, because technical changes often surface there first. Security testing checks that business roles still contain the apps and authorizations users need, since new releases can replace apps or change catalogs, and that no role suddenly grants more than intended.

Hands-on lab · ACCELQ

Simulate a changed screen and review self-healing

After an update, screens change in small ways, and a regression test can fail on a broken element instead of a real defect. ACCELQ's Self-Healing tries to recover when an element's identification breaks and reports what it did. Reconciling a saved element fixes it once for every Action that uses it.

Practice on SAPUI5 Shopping Cart demo app (public, no login)

If you haven't set up ACCELQ yet, do the setup lab first.
Record a short Action on the Shopping Cart with the Web Recorder that opens a category and adds a product to the cart. When you click the button that adds it, use the locator options to save it as a Smart Locator with a clear name, so it goes into the Element Repository.
Build a Scenario with Initiate Browser Session and this Action, run it, and confirm it passes. This is your before-update baseline.
Now simulate the update. Open the saved element in the Element Explorer and change its identification criteria until the match count drops to zero. Write down what you changed.
Click Run, check that self-healing is on under Smart Reconcile in the Run modal (it's on by default), and run the Scenario again.
Open the Test Report. If self-healing recovered the element, the step passes and the report compares the designed properties with the self-healed ones. If it didn't, the step fails on that element. Record which one happened.
Fix the element at the source by restoring its identification criteria in the Element Explorer, or by reconciling it to the right element on the View. Rerun and confirm a clean pass with no healing.

Done when You have three Test Reports: a clean baseline, a run against the broken element that shows either a self-heal or a failure, and a clean rerun after your fix. Keep the middle one. In an upgrade, every self-heal is a changed screen that someone should review before sign-off, not a silent pass.

Learn more in ACCELQ’s docs: Self-Healing: Comprehensive approach to adaptive element identification · Setting up Element Identification with Element Explorer · Locator-Free and Smart Locator

Start the free ACCELQ trial Matt Angerer, who runs The Sign-Off Table, works at ACCELQ. Disclosure

Performance, data validation and sign-off

Performance testing compares critical transactions and jobs before and after on comparable data: billing runs, MRP, payment runs, period-end steps. Data validation compares business data before and after the update. SAP provides a tool for exactly this, Data Transition Validation (DTV), described in the upgrade guide: define the comparison reports with the lines of business and auditors during preparation, extract data from the source release during ramp-down, and validate after the upgrade. The guide states it is available as part of the standard license.

Data Transition Validation compares the same agreed reports before and after the update, and every difference must be explained or raised as a defect.

The Sign-Off Table · thesignofftable.com/academy/sap-tst-304

Stream	Owner at Contoso	Main evidence
Technical verification	Sam (Basis) with SAP ECS	Checklist with results, job overview, SDM_MON status
Custom code and modifications	Kenji	Adjustment lists complete, ATC and ABAP Unit results
Functional regression	Priya	Executed test plan in SAP Cloud ALM, defects resolved
Integration	Integration lead	Interface run log, partner confirmations
Security and roles	Security lead	Role test results, critical authorization check
Performance	Sam with Priya	Before-and-after timings for critical runs
Data validation and sign-off	Dan, the sales process owner and other process owners	DTV results, signed acceptance

Stream owners and evidence.

What would you do?

In CMQ after a feature pack stack, all functional regression tests pass, but the data warehouse team reports that delta extraction stopped overnight. Which stream should have caught this?

Key points

Plan seven streams: technical, custom code and modifications, functional, integration, security, performance, data validation and sign-off.
Use the upgrade guide's follow-up activities as a technical test checklist.
Adjust modifications in SPDD, SPAU, then SPAU_ENH, in a matching development system; carry them forward by transport.
Focus regression on critical processes, simplification-affected areas and custom code.
Use Data Transition Validation to compare business data before and after the upgrade.

The streams

Technical verification

Custom code and modifications

Functional, integration and security

Performance, data validation and sign-off

Lesson check