Now accepting guest editorial from SAP, Oracle, Workday and ServiceNow practitioners.Pitch a guide
The Sign-Off Table

Lesson 6 of 6

Pipeline gates, evidence and sign-off

9 min

Listen to this lesson

0:00 / 7:41

Tests only protect releases if they run every time and their results decide what happens next. Contoso builds and deploys the fleet app with a pipeline (SAP offers the SAP Continuous Integration and Delivery service on SAP BTP for this) and moves it between subaccounts with SAP Cloud Transport Management, as described in course sap-rel-203. Each step has a quality gate: a condition that must be met before the release candidate moves on.

  1. On every commitBuild and fast testsLint, CDS compile, service tests with cds.test, QUnit and OPA5 against mock data, stored-spec contract checks
  2. On mergeDeploy to cmc-ext-devMTA deployed; a short smoke test against CMD
  3. Release candidateTransport to cmc-ext-testImported by SAP Cloud Transport Management; integration smoke tests against CMQ client 100
  4. Test cycleRegression, end-to-end and UATAutomated regression plus the end-to-end scenarios with key users; results in SAP Cloud ALM
  5. Go decisionTransport to cmc-ext-prodAfter sign-off; post-deployment smoke test with production monitoring
The fleet app's path from commit to production, with tests at each step.
The Sign-Off Table · thesignofftable.com/academy/sap-btp-301

Gates that mean something

  • All fast tests green before anything is deployed; no "known failures" left in the suite.
  • Contract checks green against the stored specs, and against CMQ's live metadata at its current support package.
  • Authorization matrix green locally, plus deployed negative tests per role in cmc-ext-test.
  • Integration smoke green after each transport, one call per destination and one event per subscription.
  • No open critical or high defects, and end-to-end scenarios signed off by key users.

Flaky tests

A test that passes and fails without code changes destroys trust in the whole suite. Contoso's rule: a flaky test is a defect. It is quarantined (moved out of the gate, visibly, with an owner and a date) and fixed or deleted within a sprint. Common causes in CAP and BTP projects are shared data between tests, fixed waits for asynchronous steps, dependence on test order, and UI locators based on generated IDs. Rerunning until green is not allowed, because it hides real timing and data problems.

A flaky test is a defect: it leaves the gate visibly, with an owner and a date, and is fixed or deleted within a sprint.
The Sign-Off Table · thesignofftable.com/academy/sap-btp-301
  1. Are all gate tests green for this exact version?Yes → continue · No → stop; fix or formally waive with the test lead
  2. Do contract checks pass against CMQ at the support package CMP will have?Yes → continue · No → stop; adapt the app or plan the SAP S/4HANA change
  3. Are end-to-end scenarios signed off with evidence in SAP Cloud ALM?Yes → continue · No → stop; complete or reschedule
  4. Are open defects within the agreed exit criteria?Yes → recommend go · No → stop; escalate to Grace with options
Priya's release readiness decision for a BTP extension.
The Sign-Off Table · thesignofftable.com/academy/sap-btp-301

The evidence package

EvidenceSource
App version and transport IDs testedPipeline and SAP Cloud Transport Management
Automated test results per level, with datesPipeline reports attached to the test plan
Contract check report against CMQ's support packageScheduled contract job
Destination and role collection exports for cmc-ext-test and cmc-ext-prodSAP BTP cockpit
End-to-end scenario results and key-user sign-offSAP Cloud ALM test plan
Open defects with agreed dispositionsSAP Cloud ALM defects

What Priya files before recommending a go for the fleet app.

The evidence package is only valid for the exact version going live, so any change after the release candidate, even text, starts a new candidate.
The Sign-Off Table · thesignofftable.com/academy/sap-btp-301

What would you do?

The day before go-live, a developer fixes a typo in a handler message and wants to transport the change straight to cmc-ext-prod, "since it is only text". The tested release candidate was the previous version. What should Priya decide?

Key points
  • Automate the path from commit to production and give each step a meaningful quality gate.
  • Gates cover fast tests, contracts, authorization, integration smoke, defects and end-to-end sign-off.
  • Treat flaky tests as defects: quarantine visibly and fix quickly; never rerun until green.
  • The evidence package must refer to the exact version going live.
  • Recommend go only with gates passed or formally waived, and hypercare monitoring in place.

Check your understanding · not graded

Lesson check

  1. 1. Which SAP service moves the fleet app between subaccounts in Contoso's pipeline?

  2. 2. How should a flaky test be handled?

  3. 3. Why must the evidence package refer to the exact version going live?

  4. 4. Which check belongs to the fast tests run on every commit?