Lesson 3 of 5
Sampling for attribute validation
10 min
Some attributes cannot be compared automatically: a bank account that must be confirmed against a supplier letter, a price that must be checked against a contract, an address that must be judged "right" by the person who knows the customer. For those, you check a sample. The question is how many records, which ones, and what a clean sample lets you conclude. A sample chosen by convenience proves very little; a sample designed on purpose can support a defensible statement.
Three ways to choose a sample
| Approach | How records are chosen | What it supports | Wave 2 use |
|---|---|---|---|
| Statistical (random) | Random selection from the whole population, with a recorded seed | A confidence statement about the error rate in the population | Random sample of migrated business partners |
| Stratified | Population split into strata, each sampled separately | Statements per stratum; better coverage of small but important groups | Customers by balance band; products by type |
| Risk-based (judgmental) | Records chosen for value, complexity or known problems | Assurance on the records that matter most; no statistical statement | Top 50 customers by revenue; every merged duplicate |
How big a random sample?
For attribute checks, the useful question is: "If we find no errors in the sample, how confident are we that the error rate in the whole population is below some limit?" A handy approximation for this zero-error case is the rule of three: if you check n randomly chosen records and find no errors, you can be about 95% confident that the true error rate is below 3 ÷ n. Turned around, to be about 95% confident that the error rate is below a tolerable rate p, check about 3 ÷ p records and find none.
| Tolerable error rate | Approximate sample size (95% confidence, zero errors found) |
|---|---|
| 10% | 30 |
| 5% | 60 |
| 2% | 150 |
| 1% | 300 |
| 0.5% | 600 |
Rule-of-three sample sizes. For small populations the exact number is a little lower; use a statistics reference or tool for exact figures.
Notice what drives the size: the tolerable error rate, not the population size. Checking 300 records gives roughly the same assurance whether there are 4,000 or 400,000 customers. Notice also the cost: every halving of the tolerable rate doubles the sample. Agree the tolerable rate per attribute with the data owner, because it is a business judgment: a 2% error rate in name capitalization may be acceptable, while any error in supplier bank details is not, which is why bank details are usually checked in full by automated comparison or by a separate verification process rather than sampled.
Combining approaches
When the sample finds errors
If a random sample finds errors, the clean-sample conclusion no longer holds. Do not fix the sampled records and declare victory: the error rate in the population is probably similar to the sample. Find the cause (a mapping rule, a source data problem, a cleansing rule), fix it for the whole population, and then check a new sample after the next load. Record the sample, its seed or selection method, every result and every finding, because the sample is evidence.
- Can the attribute be compared automatically for every record?Yes → full automated comparison; no sample needed · No → next question
- Is any error in this attribute unacceptable, such as bank details?Yes → check every record or use a separate verification process · No → next question
- Are there high-value or complex-rule records?Yes → include them all as risk-based picks · No → next question
- What error rate can the owner tolerate?Size the random sample from it, for example about 300 records for 1%
What would you do?
In Wave 2 Mock 2, a random sample of 150 suppliers finds four with the wrong payment terms. The migration specialist corrects the four records in the target and proposes to close the finding. What should Priya do?
- Use full automated comparison where possible; sample only what cannot be compared automatically.
- Combine risk-based picks, stratified samples and a random top-up.
- Rule of three: zero errors in n random records gives about 95% confidence the error rate is below 3 ÷ n.
- Errors in a sample mean errors in the population: fix the cause and resample.
Check your understanding · not graded
Lesson check
1. Using the rule of three, about how many random records with zero errors give 95% confidence of an error rate below 5%?
2. What mainly drives the size of a random attribute sample?
3. Which sample approach gives assurance on the most important records but no statistical statement?
4. A random sample finds errors. What is the right response?