Lesson 5 of 6
The skills shift
8 min
Listen to this lesson
AI changes what testers spend their time on more than whether testers are needed. Writing routine test steps, maintaining locators, triaging obvious noise and drafting reports shrink. Judging whether a test proves the right thing, reviewing AI outputs, designing data, understanding risk and talking to process owners grow. A TCoE that plans this shift keeps its people and gains capacity; one that ignores it ends up with tools nobody trusts and a team that feels replaced.
How roles change
| Role | Shrinking work | Growing work | New skills |
|---|---|---|---|
| Test analyst | Writing routine steps; formatting test cases | Reviewing generated cases; designing negative and boundary tests; exploratory testing | Evaluating AI output; test design techniques at a higher level |
| Automation engineer (Ravi's team) | Fixing locators; recording repetitive steps | Automation architecture; reviewing self-healing; operating AI features and agents | Agent permissions, evaluation sets, integration with SAP Cloud ALM |
| Test data and environment lead (Wei) | Manual data creation | Recipe design; masking rules; validating AI-identified data needs | Data engineering; privacy by design |
| Process test lead | Status chasing | Risk ranking; review gates; coaching business testers | Explaining AI-assisted evidence to process owners |
| Business tester | Keying test data and steps | Acceptance judgment; exploratory checks of real scenarios | Recognizing when a result is plausible but wrong |
| TCoE lead (Priya) | Manual reporting | Portfolio decisions; AI governance; partner and vendor management | AI risk management; value measurement |
How AI shifts work in Contoso's TCoE. Business and process knowledge become more valuable, not less.
A new specialist role also appears: someone who tests AI features themselves. As SAP adds Joule agents to business processes, a TCoE will be asked to test non-deterministic features, such as an agent that proposes a payment block or drafts a supplier email. That needs evaluation sets, tolerance for variation and new acceptance criteria. The Academy's course on testing Joule agents and AI features (sap-aiq-202) covers this in depth; for the roadmap, the point is to grow one or two people into it before the first such feature reaches Contoso's release calendar.
A skills plan, not just a training budget
Extend the skills matrix from sap-tst-310 with the new skills and assess people on real tasks, not self-ratings: review ten AI-drafted test cases and find the planted errors; explain a self-healing event; write a prompt that drafts a test case from a requirement without inventing data. Then plan development per person.
- Months 0 to 3AI literacy for everyoneShort course on limits, data rules and the guardrails; required before tool access
- Months 3 to 9Reviewer skillsAnalysts and process test leads practice on the golden set; CT-GenAI for two people
- Months 6 to 12Automation and agentsRavi's team: evaluation sets, agent permissions, self-healing review
- Months 12 to 18AI quality specialistTwo people trained to test AI features; first evaluation of a Joule feature
- Months 18 to 24Assessment skillsTwo people prepared for TMMi v2.0 assessment work
Partners and business testers
The shift applies to partners too. Contoso's managed service contract (sap-tcoe-407) now names AI literacy and the guardrails as obligations for partner staff, and the partner's use of any AI tool on Contoso assets must be declared and follow the same review rules. For business testers, the message is reassuring and true: their judgment is the part AI cannot supply. Training for them focuses on spotting plausible but wrong results, such as a generated test that expects the wrong price for CMC-ALT-100 or the wrong payment term, and on exploratory checks of the scenarios they know best.
What would you do?
After the first AI pilot, two test analysts ask whether their jobs will disappear. What should Priya do?
- AI shrinks routine steps and maintenance and grows judgment, review, data design and risk work.
- Grow an AI quality specialist role before Joule features reach the release calendar.
- Assess new skills with real tasks and plan development per person over 24 months.
- Extend obligations to partners, reassure business testers with the truth, and keep some design work manual.
Check your understanding · not graded
Lesson check
1. Which skill becomes more valuable for business testers as AI drafts tests?
2. How should new AI-related skills be assessed?
3. What is the risk of reviewers who only ever review AI drafts?
4. Why should Contoso grow an AI quality specialist before the first Joule feature reaches its release calendar?