Lesson 3 of 5
Assessment workshops
12 min
Listen to this lesson
Risk scores are only credible if the people who own the consequences agreed to them. That means a short, well-run workshop with process owners and key users, not a spreadsheet the test lead fills in alone. This lesson shows how to run one that ends with a usable ranking instead of a wall of red.
Before the workshop
- Draft the candidate risks yourself, using the template from lesson 1 and the inputs from lesson 2. Owners react to a draft far faster than to a blank page.
- Attach evidence to each risk: the transports or settings involved, usage figures, defect history.
- Agree the scales in advance. A 1–3 or 1–5 scale for likelihood and for impact, each level with a written definition.
- Invite the right people: for Contoso, Dan for record to report, Maria for order to cash, a procurement key user, a developer or functional consultant who knows the change, and Priya as facilitator.
- Time-box: 60–90 minutes per process area is usually enough.
- Draft risksTestable statements using the lesson 1 template
- Attach evidenceTransports, usage figures, defect history
- Agree scalesWritten definitions for each impact and likelihood level
- Score separatelyOwners score impact; people who know the change score likelihood
- Publish and confirmRegister out within a day; owners confirm in writing
| Level | Impact definition (Contoso) | Likelihood definition (Contoso) |
|---|---|---|
| 3 – High | Misstated financials, stopped shipments or payments, compliance breach; no practical workaround | Directly changed, complex or novel; or broke in a recent release |
| 2 – Medium | Customer or user disruption with a workaround; limited financial effect, noticed quickly | Indirectly affected by the change, or moderate complexity |
| 1 – Low | Inconvenience; easy workaround; no financial or compliance effect | Not changed and stable; or changed but not used in production |
A written three-level scale. Definitions stop "high" from meaning "important to me".
Running the scoring
Score impact and likelihood separately. Ask the process owner for impact, because they own the consequence. Ask the consultant or developer for likelihood, because they know the change. Then multiply (or look up the combination in a matrix) to get the risk level. Discuss disagreements briefly and record the reasoning in one line; that line is what an auditor or steering committee will read later.
Stopping everything from being rated high
The classic failure is the meeting where every owner rates every process as high impact. Some countermeasures that work:
- Force a distribution. Agree up front that no more than roughly a quarter of risks may be scored high impact. Owners then have to choose.
- Anchor with consequences, not labels. Ask "what exactly happens on day one if this is wrong?" A real answer separates "trucks stop" from "a report looks odd".
- Separate the two scores. Owners tend to inflate likelihood out of anxiety. Let the people who know the change score it.
- Use evidence. Put usage numbers and defect history on the screen. "This transaction ran 14 times last quarter" ends many debates.
- Compare pairs. When stuck, ask "if we could only test one of these two deeply, which one?"
| Final rating | Count | Why it landed there |
|---|---|---|
| High | 4 | Day-one consequence: billing revenue account, payment run, GR/IR clearing, intercompany postings |
| Medium | 5 | Month-end effect that a controlled first close with extra checks can catch |
| Low | 3 | Reports with an easy workaround |
After the workshop
Publish the scored list within a day, as the first version of the risk register. Ask each owner to confirm their scores in writing (an email or a status in your test management tool is enough). From now on, test planning refers to risk IDs, so every test can say which risk it reduces, and every high risk can show which tests cover it.
What would you do?
In Contoso's procure-to-pay workshop, the procurement key user insists that a cosmetic change to a purchase order printout layout is "high impact, high likelihood" because suppliers will see it. The functional consultant says only the logo position changed. What should Priya do as facilitator?
Lightweight or heavyweight?
The testing literature describes heavyweight risk techniques, such as failure mode and effects analysis (FMEA) or fault tree analysis, and lightweight ones driven by stakeholder workshops like the one described here. For most SAP releases a lightweight workshop is the right tool: it is fast, it involves the people who will later accept the residual risk, and its output is good enough to set priorities. Reserve heavyweight analysis for areas where failure has safety, regulatory or very large financial consequences, such as a pharmaceutical batch release process or a payment interface to a bank, and be prepared to involve specialists.
| Lightweight (stakeholder workshop) | Heavyweight (FMEA, fault tree analysis) | |
|---|---|---|
| Speed | Hours per process area | Days or weeks |
| Who takes part | Process owners, key users, consultants | Specialists plus owners |
| Best for | Most SAP releases and quarterly change | Safety, regulatory or very large financial consequences |
| Contoso use | Release R2 risk register | Not needed; a bank payment interface could justify it |
| Output | Ranked list with score, reason, owner | Detailed failure analysis, summarized for sign-off |
Whichever style you use, keep the output simple: a ranked list with a score, a reason and an owner per risk. A sophisticated analysis that nobody can read at sign-off does not help anyone decide.
- Draft risks and gather evidence before the workshop; agree written scales in advance.
- Process owners score impact; people who know the change score likelihood.
- Counter "everything is high" with forced distributions, consequence questions, evidence and pairwise comparison.
- Publish the scored list as the first risk register and get owners to confirm it.
Check your understanding · not graded
Lesson check
1. Who should normally score the impact of a risk?
2. What is the purpose of written definitions for each score level?
3. Which technique helps when owners rate everything high?
4. What should happen right after the workshop?