Where AI helps the TCoE

AI arrives in testing from two directions at once. SAP is adding assistants and agents to SAP Cloud ALM and Joule, and test tools are adding AI features of their own. Meanwhile testers use general-purpose chat assistants whether or not there is a policy. The TCoE's job is not to adopt everything or to ban everything. It is to decide where AI improves the outcomes it is measured on, at a risk it can control.

Map AI to the test process

Test activity	What AI can do	Main risk	Human role
Scoping and impact analysis	Suggest affected processes from transports, release notes, custom code and process models	Missed impacts presented confidently	Release assessment owner confirms scope
Requirements and test design	Draft requirements, user stories and test cases from workshop notes or process models	Plausible but wrong expected results	Process test lead reviews every case before it enters the pack
Test data	Identify the data a test needs; suggest records	Generic example values used as if real	Data lead confirms values exist in the target system
Automation build and maintenance	Record and generate steps; identify elements without fixed locators; self-heal after UI changes	A wrong element accepted silently	Automation engineer reviews healing events
Execution triage	Cluster failures, flag flaky tests, suggest likely causes	Real defects dismissed as noise	Triage lead confirms classification
Defects and reporting	Summarize defects, find duplicates, draft status narratives	Summaries that change meaning	Author checks before sending

Where AI can help across the SAP test process, with the risk and the human role for each.

Two kinds of AI sit in this table. Machine learning features such as self-healing locators or flaky-test detection learn patterns from execution data and give the same answer to the same input. Generative AI features, built on large language models, produce new text, test cases or code, and may give a different answer each time. Generative outputs need more review, because they can be fluent and wrong at once. ISTQB's Testing with Generative AI syllabus (CT-GenAI v1.1) lists the main risks as hallucinations, reasoning errors and biases, data privacy and security, and even energy consumption, and devotes a chapter to rolling GenAI out in a test organization.

Rate value and risk

High value, high riskGenerated test cases for critical processes: pilot with full review

High value, low riskFailure clustering and flaky-test flags: adopt early

Low value, high riskFully autonomous scoping without review: avoid

Low value, low riskDefect summaries and report drafts: allow with spot checks

Rating AI use cases by value and risk. Start where value is high and a wrong answer is cheap to catch.

The Sign-Off Table · thesignofftable.com/academy/sap-tcoe-408

For each candidate use case, write a one-page card: what goes in, what comes out, who reviews it, what a wrong output costs, how sensitive the input data is, and which KPI from sap-tcoe-406 should move. A use case that cannot name its KPI is a demo, not a roadmap item.

Screening AI use cases before a pilot. Every pilot has a KPI, a privacy decision and a review step that can catch errors.

The Sign-Off Table · thesignofftable.com/academy/sap-tcoe-408

Contoso's shortlist

Use case	KPI it should move	Risk rating	Decision
Failure clustering and flaky-test detection in regression	Regression cycle time; automation stability	Low	Adopt in the automation platform
Draft manual test cases for enhancements from approved requirements	Enhancement testing hours (30 h → 22 h per change)	Medium	Pilot with process test lead review
Identify test data parameters from manual test cases	Data request lead time; data-caused failures	Medium	Pilot when available to Contoso
Release impact suggestions for feature pack stacks	Critical process coverage; escapes	High	Pilot as a second opinion only
Defect summaries for the quality council pack	Council preparation effort	Low	Allow with author check
Autonomous regression scope selection	Cycle time	High	Not now: no reliable way to catch misses

Contoso's AI shortlist. Every item names the KPI it should move; the last one is consciously rejected for now.

Hands-on lab · ACCELQ

Generate Test Cases with Autopilot and review them like any AI output

This lesson treats AI-generated test design as useful but untrusted until reviewed. ACCELQ can generate data-driven Test Cases for a Scenario with Generate > Autopilot, which uses AI on the Scenario's logic. You generate a set, then apply the TCoE review step before any of it counts as an asset.

Practice on The pre-configured sample project in your ACCELQ trial (a Scenario with input parameters), or a Scenario you parameterized on the SAPUI5 Shopping Cart demo

If you haven't set up ACCELQ yet, do the setup lab first.
Open a Scenario whose steps have input parameters and go to its Test Cases tab. Note how many Test Cases exist and export them to Excel as your "before" record.
Choose Generate > Autopilot and let ACCELQ create Test Cases from the Scenario's logic (the label may differ slightly in your version).
Review each generated row as a process test lead would: is the data meaningful, is it a duplicate of an existing case, does it cover a boundary or a negative path the old set missed? Mark each row keep, change or reject in a simple log.
Delete the rejected rows and edit the ones you changed in Edit Mode, then click Save Changes.
Add a Tag or custom field value such as "AI-generated, reviewed" to the kept Test Cases, so traceability shows where they came from.
Run the Scenario with the reviewed set and open the Test Report.
Write down three numbers for the pilot card: generated rows, accepted rows, and minutes spent reviewing. The acceptance rate and review time are what the TCoE pilot measures.

Done when A review log with keep, change or reject for every generated Test Case, the tagged accepted set, one Test Report, and the three pilot numbers. Keep the before and after exports with the log.

Learn more in ACCELQ’s docs: Working with Test Cases in ACCELQ · Concept of Parameterization and Test Cases

Start the free ACCELQ trial Matt Angerer, who runs The Sign-Off Table, works at ACCELQ. Disclosure

What would you do?

A partner demonstrates an AI feature that drafts 200 test cases for Contoso's order-to-cash process in ten minutes. Grace wants to replace the order-to-cash regression pack with the drafts before the next feature pack stack update. What should Priya propose?

Key points

Map AI to each test activity with its risk and the human role that catches errors.
Treat generative outputs as drafts: fluent and wrong is the typical failure.
Rate use cases by value to TCoE KPIs and by the cost of a wrong output; start high value, low risk.
Every use case names its KPI, its data sensitivity and its review step, or it stays parked.

Map AI to the test process

Rate value and risk

Contoso's shortlist

Lesson check