Now accepting guest editorial from SAP, Oracle, Workday and ServiceNow practitioners.Pitch a guide
The Sign-Off Table

Lesson 5 of 5

Tracking estimates against actuals

8 min

An estimate is a forecast, and forecasts are only useful if you check them. Tracking estimates against actuals tells you, early, whether the plan is still true, and it builds the history that makes your next estimate better. This lesson covers what to measure, how to re-forecast, and how to feed the results back.

  1. PlanBaseline estimate with drivers and ranges
  2. MeasureActual hours, cases executed, defects, load times, validation hours
  3. CompareVariance per driver, not just in total
  4. Re-forecastEstimate to complete from actual rates
  5. DecideUse contingency, change scope or move dates
  6. CalibrateUpdate rates for the next cycle and release
The tracking loop. Each cycle boundary is a natural point to run it.
The Sign-Off Table · thesignofftable.com/academy/sap-pmo-303

What to measure

Track the drivers you estimated, not just total effort. A total that is on plan can hide one driver far over and another under, and only the drivers tell you what to fix.

AreaMeasureWhy it matters
Test executionCases executed and passed per week against planShows pace and quality together
ReworkShare of cases failing first time; retests per caseTests the retest assumption directly
DefectsArrival rate, closure rate, open backlog by priority, average ageShows whether fix capacity keeps up
Blocked testsCases waiting on environment, data or a defectReveals waste the estimate did not plan
MigrationLoad duration and error rate per object per mockFeeds cutover timing and cleansing re-estimates
Business timeValidation and UAT hours actually given against the demand planEarly warning of resource gaps

Driver-level measures for testing and migration.

In SAP Cloud ALM, the Test Execution Analysis app shows execution status of test cases across test plans, and defects raised from test runs are tracked with their status and priority, so much of this can be read from the tool rather than collected in spreadsheets. Effort hours usually come from the program's time recording; agree early how testers book time so that test execution, retest and preparation can be separated.

Re-forecasting from actual rates

After SIT cycle 1 week 2, Priya sees 30% of cycle 1 cases executed against a plan of 50%, and a first-time failure rate of 34% against the 25% estimated. Rather than reporting "behind plan", she re-forecasts: at the measured pace, cycle 1 needs four weeks instead of three; at the measured failure rate, retest effort rises by about 50 hours. She shows the forecast with the drivers that changed, and the contingency available to absorb it.

  1. Is the variance inside the agreed tolerance (for example ±10%)?Yes → note it and continue · No → re-forecast and find the driver
  2. Is the driver temporary (an environment outage) or structural (higher defect density)?Temporary → absorb in contingency · Structural → re-forecast all remaining cycles
  3. Does the forecast still fit inside contingency?Yes → release contingency per the agreed trigger · No → take options to the steering committee
Deciding what to do with a variance. Structural variances change every remaining cycle, not just the current one.
The Sign-Off Table · thesignofftable.com/academy/sap-pmo-303

Reporting without spin

Report the forecast, not just the status. "Cycle 1 is 20 points behind" invites reassurance; "at current rates cycle 1 ends one week late and uses 60% of SIT contingency" invites a decision. Show trends over several weeks so a single good or bad week does not drive the conversation, and keep metric definitions stable so last month's chart and this month's chart mean the same thing.

Calibrating for next time

After each cycle and each mock, update the rates: hours per test case by complexity, retest rate, validation hours per object, load time per thousand records. Keep them in a simple estimation model owned by the PMO. Contoso's first post-go-live release will be estimated from these actuals, not from the original bid, and that is how an organization stops repeating the same underestimate.

Scope analysis tools can sharpen later estimates further. Organizations still on SAP Solution Manager 7.2 can use its Scope and Effort Analyzer to predict which used business processes a planned SAP maintenance change affects, which indicates what to test before the change is installed. Whatever tool you use, combine its scope view with your own calibrated rates.

What would you do?

At the end of Mock 2, Contoso's total migration validation hours are exactly on plan. But complex objects took 60% more hours than estimated and simple objects took half. What should Grace conclude?

Key points
  • Track the drivers you estimated, not just total effort.
  • Re-forecast from actual rates at each cycle and mock boundary, and say which driver changed.
  • Report forecasts and contingency burn, with stable definitions and trends.
  • Calibrate your estimation model with actuals so the next release is estimated from evidence.

Check your understanding · not graded

Lesson check

  1. 1. Why track estimates at the level of drivers rather than total effort?

  2. 2. Which SAP Cloud ALM capability helps you see execution status of test cases across test plans?

  3. 3. What is a structural variance?

  4. 4. What should happen to measured rates after each cycle and mock?