Environment orchestration

A continuous testing architecture fails first on environments. Tests are ready at 19:00, but CMQ is being restarted, an RFC destination still points to the old carrier endpoint, the BTP test subaccount runs last week's build and the job that processes IDocs is not scheduled. Each of these produces failures that look like defects. Environment orchestration is the discipline of knowing, and proving, that every environment is ready before tests run, and of coordinating changes to environments with the teams that own them, Basis above all.

Know your environments

Environment	Purpose	Typical tests	Owner
CMD (development)	Build and unit test	ABAP Unit, ATC, developer checks	Kenji's team
BTP dev and test subaccounts	Build and deploy extensions	Pipeline unit, API and acceptance tests	Extension team
CMQ (quality), client 100	Integrated testing	Regression, end-to-end, UAT, automation suites	Priya for use; Sam for platform
CMR (pre-production), client 100	Release regression after go-live	Full regression before each CMP import, release UAT, weekend performance checks; paired with BTP subaccount cmc-ext-preprod	Priya for bookings; Sam for refreshes
Partner test endpoints	Integration testing	EDI, bank, carrier flows where partners offer test systems	Integration team
CMP (production)	Business use	Smoke checks and monitoring only	Operations

Contoso's test environments and their owners.

The Sign-Off Table · thesignofftable.com/academy/sap-tau-401

Under RISE with SAP, SAP Enterprise Cloud Services operates the Private Edition systems, and some activities, such as running the Software Update Manager for upgrades, are requested through service requests rather than performed by the customer. System copies and refreshes are typically handled the same way. Check your contract's service description for which environment operations you can do yourself and which need a request and lead time; that lead time belongs in your test calendar.

Under RISE, a refresh is a service request with lead time, followed by Basis post-copy steps and the automation re-seed.

The Sign-Off Table · thesignofftable.com/academy/sap-tau-401

Readiness checks before every run

Every scheduled run should start with an automated readiness check that stops the run with one clear message instead of producing hundreds of confusing failures. The check is itself a small test suite.

AvailabilityLog on to CMQ and the BTP test app with the technical users; check response times.
VersionsRead the deployed build, the last imported transport or feature, and the release and stack; compare with what the suite expects.
ConnectionsCheck that key destinations reach their test endpoints or virtual services, not production.
Background processingConfirm the jobs the processes need, such as output and IDoc processing, are scheduled and running.
Data and usersCheck pool levels, test users, roles and certificates.
Go or stopAll green: run. Any red: stop, report the failed check and notify the owner.

An automated environment readiness check.

The Sign-Off Table · thesignofftable.com/academy/sap-tau-401

Hands-on lab · ACCELQ

Run a readiness Scenario first, on a Local Agent inside your network

A Local Agent runs inside your network and pulls jobs from ACCELQ, so no inbound ports need opening. That is how it reaches internal systems like CMQ. You'll build a readiness Scenario and put it first in a Static Test Suite, so every run starts by proving the environment.

Practice on TripPin OData V4 reference service (public, no login; not SAP, but the same OData protocol as SAP's released APIs) and the SAPUI5 Shopping Cart demo app (public, no login)

If you haven't set up ACCELQ yet, do the setup lab first.
Open Resources > Local Agents, confirm your Agent is running and note its visibility: one user, one project or the whole tenant. Decide which a shared agent for a quality system would need.
Create a Library Action (non UI) Readiness: OData service. Insert a REST call GET https://services.odata.org/V4/TripPinService/$metadata and assert Status 200.
Create a Context Cart Home with the cart URL as entry page and Context Synchronization on the categories list. Add an Action Readiness: cart UI that, in Intent mode, checks the list with Verify Element Exists.
Create a Scenario Readiness check with Initiate Browser Session and both readiness Actions.
Click + > Test Suite, choose Static, and add Readiness check first, then two other Scenarios. A Static suite keeps that order and doesn't run in parallel.
Run the suite on your Local Agent and open the Test Report.
Simulate a broken connection: change the readiness URL to .../TripPinServiceX/$metadata (it returns 404) and run again. Read the report top-down and classify later failures as environment first. Then restore the URL.

Done when Two Test Reports: one with readiness green and the suite following it, and one where readiness is red. Write down the agent visibility you would choose for a shared quality system, and why.

Learn more in ACCELQ’s docs: Securely execute tests behind firewall using Local Agent · Test Suites in ACCELQ · Setting up Context Synchronization

Start the free ACCELQ trial Matt Angerer, who runs The Sign-Off Table, works at ACCELQ. Disclosure

Coordinate change with Basis

Environments change constantly: kernel patches, support package or feature pack stack updates, refreshes, certificate renewals, new integrations. A continuous testing architecture needs a shared calendar and a handshake. Sam's Basis team publishes planned changes; the automation team pauses or adjusts schedules; after the change, the readiness check and a smoke suite confirm the environment before full suites resume.

WeeklyChange calendarBasis, integration and automation teams review planned environment changes for the next two weeks.
Before changePauseScheduled suites for the affected environment are paused; the reason is shown on the dashboard.
After changeReadiness and smokeReadiness check and a 15-minute smoke suite run automatically.
ResumeFull suitesNightly and end-to-end suites resume; results are compared with the last good run.
MonthlyReviewEnvironment-caused failures are counted and their causes fixed at the source.

A rhythm for coordinating environment change with testing.

The Sign-Off Table · thesignofftable.com/academy/sap-tau-401

Treat BTP environments as code

On SAP BTP, much of the environment can be created and configured by script: subaccounts, entitlements, service instances, destinations and role collections, using SAP's BTP command line interface or infrastructure-as-code tooling. That makes short-lived test environments possible: the pipeline creates a fresh test space, deploys the build, runs the acceptance tests against virtual or mocked dependencies and removes the space afterwards. The S/4HANA systems themselves are long-lived and shared, so the orchestration there is about readiness and coordination rather than creation.

On BTP the pipeline can create its own test space and throw it away; the S/4HANA systems stay shared and long-lived.

The Sign-Off Table · thesignofftable.com/academy/sap-tau-401

Ephemeral per buildBTP test spaces created and destroyed by the pipeline.

Long-lived, ownedA team's own sandbox system for experiments; low coordination cost.

Shared, short slotsBooked windows on a performance or preproduction environment.

Shared, long-livedCMQ: needs a calendar, readiness checks and clear ownership.

Orchestration needs differ by environment lifetime and sharing.

The Sign-Off Table · thesignofftable.com/academy/sap-tau-401

Hands-on lab · ACCELQ

Add disposable execution capacity and run a suite in parallel

Execution capacity can be scripted like a short-lived test space. ACCELQ supports Dockerized agents alongside installed Local Agents, and a Dynamic Test Suite can use Parallel Execution, which a Static suite can't. Parallel runs only work when each Scenario owns its data.

Practice on SAPUI5 Shopping Cart demo app (public, no login) and the TripPin OData V4 reference service (both public, no login)

If you haven't set up ACCELQ yet, do the setup lab first. The self-contained REST data lab gives you a Scenario that owns its data.
Add a second agent: either run a Dockerized agent by following the Dockerized Agents article, or install another Local Agent from Resources > Local Agents > Download Agent on a second machine with JDK 8, 11, 17, 19 or 21.
Confirm both agents appear in Resources > Local Agents.
Click + > Test Suite, choose Dynamic, and set criteria that select four or more of your Scenarios from earlier labs, including the self-contained one.
Click Run, turn off Parallel Execution, choose an Agent and run. Note the duration.
Run again with Parallel Execution on and compare the duration.
In each Test Report, check the Agent's CPU and memory panel. That is your sizing evidence for how many sessions one agent can carry.
Check every Scenario in the suite: any that rely on a shared, remembered record must be redesigned before you scale up.

Done when Two Test Reports of the same Dynamic suite, sequential and parallel, with their durations and the Agent load noted, plus a short list of Scenarios that are not yet safe to run in parallel.

Learn more in ACCELQ’s docs: Using Dockerized Agents for ACCELQ test execution · Installing Local Agent using Install wizard · Test Suites in ACCELQ

Start the free ACCELQ trial Matt Angerer, who runs The Sign-Off Table, works at ACCELQ. Disclosure

What would you do?

At 02:00, 140 of 300 nightly tests fail in CMQ with connection errors to the carrier tracking service. The readiness check has no connection checks. What should the automation architect change first?

Key points

Environment failures look like defects; orchestration proves readiness before tests run.
Under RISE, some environment operations are service requests; plan their lead time.
Start every run with checks for availability, versions, connections, jobs, data and users.
Prove test systems cannot reach production partners, especially after refreshes.
Coordinate change with Basis through a shared calendar; script short-lived BTP environments.

Know your environments

Readiness checks before every run

Coordinate change with Basis

Treat BTP environments as code

Lesson check