Lesson 4 of 5
Data quality metrics across mocks
9 min
Mock loads are rehearsals, and rehearsals are only useful if you measure them. Data quality metrics tell the program whether each mock was better than the last, which objects are lagging and whether the exit criteria for the next mock are realistic. They also give the steering committee something more honest than "migration is green".
A small set of useful metrics
| Metric | How it is calculated | What it tells you |
|---|---|---|
| Open findings by dimension | Count of open profiling findings per object and dimension | Where source quality still needs work |
| Rule pass rate | Rules whose targeted tests passed ÷ rules in the specification | Whether the build matches the specification |
| Load error rate | Records rejected ÷ records in scope, per object | How clean the data is against target checks |
| In-scope count stability | Records in scope now vs the previous mock, with reasons | Whether errors are being fixed or quietly excluded |
| Reconciliation differences | Number and value of unexplained differences per object | Completeness and correctness in bulk |
| Validation findings | Data owner findings per object, open and closed | Whether data is right and usable for the business |
Six metrics that cover source quality, build quality, load, reconciliation and business validation.
Measure per object and per dimension, not as one program-wide number. A 0.5% error rate across all objects can hide a 9% rate on open purchase orders, which is the object that will stop goods receipts on day one. Report the numbers that matter for the next decision, with the owner next to each one.
Watching the trend
- Mock 1Technical baselineBusiness partner load error rate 6.8%; 41 open findings; rule pass rate 72%
- Mock 2Full volumeError rate 1.2%; 18 open findings; rule pass rate 94%; in-scope count stable
- Mock 3Quality and timingError rate 0.1%; 3 open findings, all low impact with owner decisions
- Dress rehearsalGo-live conditionsNo critical findings; metrics repeat Mock 3 within the cutover window
Agree target values per mock in advance and put them into the exit criteria, so that the decision to move on is not argued after the fact. A typical progression: in Mock 1 every object is attempted and the main problems are known; in Mock 2 full volume loads with errors explained; in Mock 3 error rates are near zero for critical objects and reconciliation differences are all explained; the dress rehearsal repeats Mock 3 under go-live conditions.
Prioritizing what to fix
Between mocks there is never time to fix everything. Rank open findings by business impact and by the number of records affected. The matrix below is how Priya and the data owners agree priorities in the Wave 2 mock reviews.
Making metrics trustworthy
- Use the same profiling and reconciliation scripts on every extract, versioned with the specification.
- Record the extract key date for each mock so numbers are comparable.
- Count rejects from the Migration Cockpit message log per object, grouped by cause, rather than from memory.
- Separate "valid for SAP" (load errors) from "right for the business" (owner findings); both must reach their targets.
- Show trends across mocks on one page per object, with the owner and the next action.
What would you do?
In Mock 3 the open purchase order load error rate drops from 4.1% to 0.2%. The migration specialist reports a strong improvement. Priya notices that the in-scope count also fell from 2,940 to 2,610 items. What should she do?
- Measure each mock: open findings, rule pass rate, load error rate, scope stability, reconciliation and validation findings.
- Report per object and dimension with owners, not one program-wide number.
- Set metric targets per mock in the exit criteria before the mock runs.
- Always pair error rates with in-scope counts to catch silent exclusions.
Check your understanding · not graded
Lesson check
1. Why report load error rates per object rather than one program-wide figure?
2. What does in-scope count stability help detect?
3. A finding has high business impact and affects many records. What is its priority?
4. When should metric targets for the next mock be agreed?