Lesson 4 of 5
Assessing defects by revenue and customer impact
12 min
Not every defect must be fixed before go-live, and not every "minor" defect is minor. In order to cash, two questions decide how serious a defect is: what does it do to revenue and cash? and what does the customer experience? Technical severity (does the transaction dump?) matters less than business impact.
Rating guide
| Defect | Revenue / cash impact | Customer impact | Rating |
|---|---|---|---|
| Contract price not applied; list price billed | High: overbilling and disputes, delayed cash | High: wrong invoices | Must fix |
| Invoice emails sent to the wrong contact | Medium: invoices not received, late payment | High: confidential data to the wrong person | Must fix |
| Revenue posted without profit center | Medium: margin reporting wrong | None | Fix before go-live or first close |
| Order confirmation shows wrong logo | None | Low | Fix after go-live |
| Credit block not triggered above limit | High: credit exposure | None directly | Must fix |
| Delivery due list slow for large selections | Low if workaround exists | None | Workaround, fix after go-live |
Examples of order-to-cash defect ratings at Contoso.
Questions that decide the rating
- Does the defect produce a wrong amount for the customer or in the books?Yes → high revenue impact · No → continue
- Will the customer see it, or receive something wrong or late?Yes → high customer impact · No → continue
- Does it expose personal or confidential data?Yes → treat as must fix regardless of volume · No → continue
- Is there a workaround that is safe, documented and sustainable at volume?Yes → may be accepted with an owner and a fix date · No → must fix before go-live
Volume changes everything
A defect affecting one order in a thousand looks small in a test cycle. At Contoso's volume of roughly 400 orders a day it is several wrong orders every week, each one a call to customer service and possibly a credit memo. When rating, estimate the production frequency: which customers, products or channels trigger it, and what share of orders they represent. A workaround that takes two minutes per order is fine for five orders a week and impossible for five hundred.
Workarounds also need owners. A documented workaround says who does it, when, how it is checked, and when it ends. "Billing team will fix invoices manually" is not a workaround; "Billing clerk checks the daily list of invoices for customer group 02 and corrects the freight condition before release, until fix release 1.2 on August 15" is.
Retesting fixed defects
A must-fix defect is closed only when the fix has been retested with evidence, and the scenarios around it have been regression tested. A pricing fix for aftermarket customers can break fleet pricing; a change to output determination can stop order confirmations. For each fix, Priya's team reruns the failing scenario and at least one neighboring scenario that uses the same configuration, and attaches both results to the defect.
What would you do?
Late in UAT, Priya's team finds that invoices for aftermarket customers are sent to the sales contact instead of the accounts payable contact. The developer rates it "low: the invoice is correct, only the recipient is wrong". Maria disagrees. Who is right, and why?
- Rate O2C defects by revenue or cash impact and customer impact, not technical severity.
- Wrong amounts customers see, wrong recipients and missing credit controls are must-fix defects.
- Estimate production frequency; small test-cycle counts can mean many incidents at volume.
- Accepted workarounds need an owner, a check and an end date.
- No open must-fix defects at sign-off.
Check your understanding · not graded
Lesson check
1. Which two questions decide an O2C defect's seriousness?
2. A defect sends invoices to the wrong email address. How should it be rated?
3. What makes a workaround acceptable?
4. Why estimate production frequency for a defect seen once in testing?