Now accepting guest editorial from SAP, Oracle, Workday and ServiceNow practitioners.Pitch a guide
The Sign-Off Table

Lesson 5 of 6

Traceability and review

17 min

Listen to this lesson

0:00 / 6:56

This course discusses ACCELQ by name. Matt Angerer, who runs The Sign-Off Table, works at ACCELQ. Disclosure

An AI-drafted test case that nobody can trace or show was reviewed is a liability at sign-off. Auditors and process owners ask the same two questions about every test: what requirement does it prove, and who decided it was right? AI doesn't change the questions. It adds one: where did this test come from?

The trace chain

Traceability in SAP Cloud ALM runs from requirement to process, test case, plan, run and defect, and defects can lead back to requirements.
The Sign-Off Table · thesignofftable.com/academy/sap-aiq-203

In SAP Cloud ALM, test cases link to solution processes and requirements, test plans group them per cycle, and runs and defects hang off the plan (see sap-alm-203 and sap-alm-204). Joule-generated test cases arrive with traceability links to their source requirements, user stories and processes already set. Test cases drafted outside Cloud ALM have none: you create them in Test Preparation or upload them, and you must link them yourself. A draft that can't be linked to a requirement is either testing something nobody asked for or exposing a requirement nobody wrote down; both are worth a conversation.

Record where it came from

What to recordWhyWhere at Contoso
Origin: AI-drafted, and with which toolReviewers and auditors can weigh it; trends show where AI helpsA tag or field on the test case; Joule marks its own as Manually created with AI
Sources usedShows what the draft could and couldn't knowRequirement IDs and document versions in the test case description
Prompt or prompt template referenceLets the team improve the prompt, not just the outputThe team's prompt library, referenced by ID
Reviewer and dateProves a person approved itThe status change from In Preparation to Prepared, by the reviewer
Changes made in reviewShows the review was realA short review note on the test case

Provenance fields that make AI-drafted tests audit-ready.

Review as a status gate

Make review a status change only the reviewer makes. Contoso's rule: an AI-drafted test case stays In Preparation until the key user for the process reviews it against the checklist and sets it to Prepared. The person who drafted it, or prompted the AI to, can't approve it. That is the same separation you'd want for any test case, made explicit because AI makes drafts cheap and approval is where the cost now sits.

An AI-drafted test case moves to Prepared only through the key user's review; untraceable drafts are rejected.
The Sign-Off Table · thesignofftable.com/academy/sap-aiq-203
TraceabilityLinked to at least one requirement or user story and to the solution process.
ExistenceEvery app, transaction, field and message checked in the test system for this release.
CoverageHappy, error and edge cases for the requirement; both sides of each threshold.
Expected resultsEach one visible in the system: document, status, message or workflow item.
DataValidated against the test client and recorded in the data plan.
ProvenanceOrigin, sources and reviewer recorded; AI marking left in place.
Contoso's six-layer review checklist for AI-drafted test cases.
The Sign-Off Table · thesignofftable.com/academy/sap-aiq-203

From reviewed manual case to automation

SAP's Cloud ALM generator produces manual test cases only, so turning reviewed cases into automation is a separate step on a separate tool. Test platforms are adding AI to that step too. ACCELQ, the platform used in this course's labs (the Academy's host works there), describes several AI-assisted authoring features. ACCELQ says Test Cases can be generated from its Data Types and Data Lists, or with an AI option called Autopilot. ACCELQ says its default element identification, Locator-Free, uses an AI-detected label in each statement instead of a stored locator. Its SAP marketing page also uses terms such as "Agentic AI" and "SAP Test Discovery" that we could not match to its documentation, so treat those as claims to verify in a trial.

Apply the same rules you use for Joule. Generated Test Cases are drafts until reviewed against the process; AI-identified elements need the same stability checks as any locator; and the reviewed manual test case in Cloud ALM stays the reference the automation must match, with its ID in the automated test's name so the trace survives the move.

What would you do?

During an audit dry run, Contoso's internal auditor picks five test cases that supported the procure-to-pay sign-off. Two were drafted with AI outside Cloud ALM. One has no requirement link, and its status was set to Prepared by the tester who prompted the AI. What should Priya do?

Key points
  • Every AI-drafted test case must trace to a requirement; Joule-generated cases arrive linked, externally drafted ones must be linked by hand.
  • Record origin, sources, prompt reference, reviewer and review changes.
  • Make review a status gate: only a reviewer other than the author moves a case from In Preparation to Prepared.
  • Automation and AI authoring features on test platforms follow the same rule: drafts until reviewed, with the reviewed case ID carried through.

Check your understanding · not graded

Lesson check

  1. 1. What traceability do Joule-generated test cases in Cloud ALM have when they are created?

  2. 2. Who should move an AI-drafted test case from In Preparation to Prepared at Contoso?

  3. 3. An AI-drafted test case can't be linked to any requirement. What does that suggest?

  4. 4. How should ACCELQ's AI-assisted authoring claims be treated, according to this lesson?