Lesson 1 of 5
Estimating test effort
11 min
Listen to this lesson
Every SAP test plan eventually turns into a question from the program office: how many people, for how long? A test lead who answers with a single number and no reasoning will be overruled. A test lead who answers with two independent estimates, their assumptions and a range will be listened to. This lesson gives you both techniques and a way to combine them.
Two families of technique
Test management standards such as the ISTQB syllabi group estimation techniques into two families. Metrics-based techniques use data: counts of what you must test multiplied by measured rates from past work, or ratios such as test effort as a share of build effort. Expert-based techniques use the judgment of people who have done the work: Wideband Delphi and its agile cousin planning poker, where experts estimate independently, compare and converge, and three-point estimation, where each expert gives an optimistic, most likely and pessimistic figure.
| Technique | Family | Good for | Watch out for |
|---|---|---|---|
| Unit rate × count | Metrics-based | Test cycles with a known test case inventory | Rates from a different kind of program |
| Ratio to build effort | Metrics-based | Early sizing before test cases exist | Hides SAP specifics such as data and integration |
| Wideband Delphi / planning poker | Expert-based | New or unusual scope; building team buy-in | Anchoring on the loudest voice |
| Three-point (a + 4m + b) ÷ 6 | Expert-based | Expressing uncertainty as a range | Pessimistic case set too low |
Building a metrics-based estimate for an SAP cycle
In SAP programs the natural unit of scope is the business process, often expressed as SAP Best Practices scope items (for example BD9 Sell from Stock or J45 Procurement of Direct Materials) plus custom extensions and interfaces. Count the test cases each scope item needs, including variants such as plants, order types and partner functions, then apply complexity-based rates.
| Element | Contoso SIT cycle 2 assumption |
|---|---|
| Test case inventory | 420 end-to-end test cases: 180 simple, 170 medium, 70 complex |
| Execution rate | Simple 1.5 h, medium 3 h, complex 6 h (from cycle 1 actuals) |
| Base execution | 180 × 1.5 + 170 × 3 + 70 × 6 = 1,200 hours |
| Retest allowance | 25% of base for failed and blocked tests, from cycle 1 defect rate |
| Preparation and data | 15% for test data set-up, evidence review and status reporting |
| Total | 1,200 × 1.40 = 1,680 hours |
A unit-rate estimate. Every rate and allowance is an assumption you can show and defend.
Note what makes SAP different from a typical application estimate: test data (each order-to-cash test needs a customer, material with stock, pricing and credit settings), integration (interfaces to banks, carriers and plant systems often need their own test windows), authorizations (tests run under business roles, not super-users) and business tester availability. Estimate those explicitly rather than burying them in a contingency figure.
Building an expert-based estimate
Bring the process test leads for order to cash, procure to pay, record to report and plan to produce together. Give each the scope list and ask for a three-point estimate per process, written down independently before discussion. Reveal, discuss the outliers, and re-estimate once. For Contoso, the experts' combined three-point result for SIT cycle 2 came to roughly 1,500 to 2,100 hours with an expected value near 1,750.
- Prepare the scope listScope items, variants, interfaces and custom objects
- Independent estimatesEach expert writes optimistic, most likely and pessimistic
- Reveal and discussExplore the biggest differences, not the average
- Re-estimateOne more independent round
- Compare with metricsExplain any large gap before you publish
- Publish a rangeExpected value, range and the assumptions behind it
Re-forecasting during the cycle
An estimate is a forecast, not a promise. Re-forecast weekly during execution using three numbers: tests remaining, actual hours per test by complexity so far, and the current failure rate (which drives retest effort). If Contoso's cycle 2 shows a 35% failure rate instead of the planned 25%, the retest allowance grows and the end date moves. Report that early and with the arithmetic visible; a test lead who forecasts a slip in week two is managing, one who announces it on the last day is not.
What would you do?
Grace, Contoso's PMO lead, asks Priya for UAT effort "by Friday, one number." No UAT test cases exist yet, but SIT actuals are available. What should Priya provide?
- Use at least two techniques: one metrics-based, one expert-based.
- Count scope by process and variant, and apply rates measured on your own program wherever possible.
- Estimate SAP-specific effort (data, integration, authorizations, tester availability) explicitly.
- Publish a range with assumptions, then re-forecast from actuals.
Check your understanding · not graded
Lesson check
1. Which is an expert-based estimation technique?
2. In three-point estimation, how is the expected value usually calculated?
3. Why should test data effort be estimated explicitly in SAP programs?
4. What is the best source of execution rates for cycle 2?