Lesson 4 of 5
Versioning tests with the solution
11 min
Listen to this lesson
A test describes how the solution should behave at a point in time. When the solution changes, the test must change in step, and when two versions of the solution exist side by side, two versions of the test may be needed. Without versioning, teams end up with tests that work in CMD but fail in CMQ because the transport has not moved yet, or tests updated for the new release that can no longer support emergency fixes on the old one.
Testware is configuration too
The ISTQB Test Automation Engineering syllabus treats configuration management of testware as essential: environment configuration, test data and test suites must be managed across environments and versions of the system under test. It describes two ways of matching tests to releases: feature toggles in the testware that switch test suites on per release or environment, and releasing the testware with the same version as the system, using tags or branches in a configuration management system.
| Approach | How it works | Fits when |
|---|---|---|
| Single version with feature toggles | One test codebase; a configuration per environment says which features are active | Changes flow continuously and environments differ only briefly |
| Version tags per release | Testware tagged with the solution release it matches | Releases are discrete, such as feature pack stacks |
| Branches during parallel lines | A maintenance branch for the old version and a main line for the new one | Two solution versions must be supported at once |
Follow the transport, not the calendar
In an ABAP landscape, a change reaches each system when its transport is imported. Tests should follow the same path. If a test change depends on a transport, it should be activated in an environment only when that transport is there. Several practical patterns help.
- Link each test change to the requirement, user story or feature that caused it, so you can see which tests a change affects.
- In SAP Cloud ALM, features bundle transports and move them through the landscape; record on the feature that automation was updated, and run the affected tests when the feature reaches quality.
- Let tests read an environment configuration that states what is deployed, rather than hard-coding behavior per system.
- Release the test change to the shared suite when the transport is imported into quality, not when the developer finishes coding.
- Requirement or user storyThe change is defined and linked to the processes it touches.
- Feature with transportsKenji's team builds the change in CMD; transports are assigned to the feature.
- Test change on a branchThe automation engineer updates the affected tests and links them to the feature.
- Import into CMQThe feature is deployed to quality; the test change is merged and the affected tests run.
- Ready for productionTest evidence is attached; the feature is approved and deployed to CMP.
Parallel versions during a conversion or upgrade
Contoso met the hardest case during the conversion. The ECC production system EP1 still needed emergency fixes through ED1 and EQ1, managed by the support team with ChaRM in SAP Solution Manager 7.2, while the S/4HANA program built and tested in CMD and CMQ with SAP Cloud ALM. The upgrade guide for Private Edition describes the same pattern for upgrades: during the upgrade the development system is locked, so a temporary copy of development can supply production with emergency corrections, and changes must then be made in both, which is dual maintenance.
- Before conversionOne lineTests on main match ECC (ED1, EQ1, EP1).
- Program startBranchCreate an ECC maintenance branch; main moves to S/4HANA (CMD, CMQ).
- During programDual maintenanceAn ECC emergency fix changes a process: update the ECC branch, then assess the S/4HANA version.
- Go-liveSwitchProduction runs on S/4HANA; the S/4HANA line becomes the only active suite.
- After hypercareArchiveTag and archive the ECC branch; stop running it.
Version everything a test needs
A test is more than a script. Keep its locator maps, data factory code, environment configuration and expected results in the same repository with the same version, so a tagged release recreates exactly what ran. Record the solution version on every result: system, client, release and stack, and the test version. A failure report that cannot say which test version ran against which solution version cannot be trusted for sign-off.
What would you do?
Kenji's team changes the credit check logic in CMD. The automation engineer immediately updates the credit tests in the shared suite. The next nightly run in CMQ fails, because the transport is still waiting for approval. What should the team change?
- Testware needs configuration management: environment settings, data and suites.
- Match tests to releases with feature toggles, version tags or branches.
- Test changes follow the transport and feature path, not the developer's calendar.
- During conversions and upgrades, dual maintenance applies to tests as well as code.
- Record test version and solution version on every result used for sign-off.
Check your understanding · not graded
Lesson check
1. When should a test change that depends on a transport be activated in the CMQ suite?
2. Which ISTQB option ties testware exactly to a system release?
3. Why does the Private Edition upgrade guide suggest a temporary copy of the development system?
4. Which pair of facts should every result used for sign-off record?