Lesson 5 of 6
Review and approval rules for AI output
9 min
Listen to this lesson
AI features don't change who is accountable. A requirement still needs an owner who agrees it describes the business need; a test case still needs a key user who agrees it proves the requirement; a process model still needs a process owner. What changes is the volume and speed of drafts, and the risk that people rubber-stamp text that "looks complete." Your job as test lead or PMO lead is to set rules that keep review real and leave a record.
Who reviews what
| AI output | Drafted by | Reviewer | Approval recorded as |
|---|---|---|---|
| Requirement | Requirement generation or the Requirement and User Story Agent | Process owner or key user for the area (Lucía for Monterrey O2C, Dan for finance) | Ready for Approval → Approved by the Requirement Approver, logged in history |
| User story | Requirement and User Story Agent | Product owner of the workstream | Story accepted into the sprint backlog |
| Process model | AI-assisted process modeler | Process owner | Model approved and published in SAP Signavio |
| Test scope | Test Scoping Agent | Test lead (Priya) | Scope Summary confirmed in Joule, noted in the test plan |
| Manual test case | Test Preparation Agent | Key user plus a tester | Status changed from In Preparation to Prepared |
| Test data list | Test Data Management Agent | Tester and data owner | Data sheet with real Wave 2 values attached to the test case |
Contoso's review matrix for AI output in Wave 2.
The review checklist
- Source check: every statement traces to a transcript, decision log or model. Anything else is marked ASSUMED and resolved.
- Value check: org units, thresholds, codes and dates match Contoso's decisions (company code 1910, sales area 1910 / 10 / 00, payment term 0004).
- Completeness check: exceptions and negative paths are present, not just the happy path.
- Testability check: acceptance criteria and expected results are specific enough to pass or fail.
- Sensitivity check: no personal data, prices or remarks from the transcript leaked into the artifact.
Traceability is the evidence
An auditor or a steering group will eventually ask, "how do you know this requirement reflects what the business agreed?" With AI in the loop, the answer has three parts: the source documents the draft was generated from, the named reviewer who approved it, and the downstream links that show where it went. SAP Cloud ALM gives you most of this if you use it consistently. Keep the transcript in the Documents app, approve through the approval states rather than by editing a status field, and let generated test cases keep their traceability links to requirements, user stories and processes. Don't copy generated text into spreadsheets for review and paste it back, because that breaks the chain and the AI flag along with it.
For Wave 2, Contoso adds one more element: each workshop transcript gets a short decision log written by the facilitator, listing the decisions and who made them. Reviewers check generated requirements against the decision log first and the full transcript only when something is unclear. That halves review time without lowering the bar, because the decision log is the authoritative summary the business already signed.
Keeping review honest
Rubber-stamping is the main risk. Three habits help. First, sample audits: each two-week sprint, Grace's PMO picks ten approved AI-generated items at random and rechecks them against the checklist; findings go back to the reviewer, not into a blame report. Second, keep the flag: don't clear "Manually created with AI" or similar markers, because they let you report on AI-generated content and audit it later. Third, limit batch size: a reviewer facing 200 generated test cases on Friday afternoon will skim. Generate per process and per sprint.
| Practice | Without it | With it |
|---|---|---|
| Named reviewer per artifact | Everyone assumes someone else checked | One person is accountable and the history shows who |
| Approval locks the content | Text changes after approval without trace | Changes need a new approval cycle |
| AI flag kept | AI drafts can't be found or audited later | Reports show AI share and defect rates |
| Small batches | Reviews degrade into skimming | Reviewers keep their attention |
What would you do?
Halfway through Wave 2, Priya notices that test cases generated by the Test Preparation Agent are being set to Prepared within minutes of generation, by the same tester, 60 at a time. What should she do?
- AI changes the speed of drafting, not who is accountable for approval.
- Assign a named reviewer per artifact type and record approval in the tool (locked and logged).
- Use a five-point checklist: source, values, completeness, testability, sensitivity.
- Keep AI flags, review in small batches, and run PMO sample audits each sprint.
Check your understanding · not graded
Lesson check
1. Who should review AI-generated manual test cases before they are set to Prepared?
2. Why keep the "Manually created with AI" flag on test cases after review?
3. What does a PMO sample audit check each sprint?
4. Which review check catches a transcript remark about a named employee appearing in a requirement?