Now accepting guest editorial from SAP, Oracle, Workday and ServiceNow practitioners.Pitch a guide
The Sign-Off Table

Lesson 4 of 5

Testing KPI definitions

8 min

A KPI is a measure judged against a target. "On-time delivery 94%" means nothing until you know what counts as on time, which deliveries are included, which period is measured, and what the target is. Most KPI disputes on SAP projects are not calculation bugs. They are definition gaps: business and IT built different KPIs under the same name.

Write the definition first

Before testing a KPI, write a definition both business and IT sign. Keep it to one page:

ElementContoso example: on-time delivery
Business questionDo fleet customers get alternators when we promised?
FormulaDelivery items with goods issue on or before the first confirmed date ÷ all delivery items with goods issue in the period
Unit and grainPercentage of delivery items, not of orders or quantities
PopulationShipping points 1710 and 1720; excludes returns and free-of-charge deliveries
PeriodBy goods issue date, calendar month
Target and thresholdsTarget 95%; warning below 92%; critical below 88%
Data sourceDelivered analytical query or a custom one built on released CDS views, named
OwnerMaria for the definition; logistics process owner for the target

A KPI definition card.

How SAP S/4HANA builds KPIs

In SAP S/4HANA, KPIs for the launchpad are typically defined with the Manage KPIs and Reports app (the analytics specialist business role gives access). A KPI is based on a measure from an analytical query. An evaluation adds the filters, the target, and the thresholds that color the tile, and the goal type that says whether higher or lower is better. A tile and a drill-down report then present the evaluation to users. SAP also delivers many predefined KPIs for common business areas.

  1. Analytical queryProvides the measure, such as delivered items and on-time items
  2. KPINames the measure and says whether higher or lower is better
  3. EvaluationAdds filters (shipping point, period), target and thresholds
  4. Tile and drill-downShows the value with a color; opens the detail behind it
How a KPI tile is assembled. Test each layer.
The Sign-Off Table · thesignofftable.com/academy/sap-xpr-305

What to test

  • Formula: build a small controlled data set (for example, 20 deliveries of CMC-ALT-100, 18 shipped on time) and confirm the KPI shows 90.0%.
  • Population: add a return delivery and a free-of-charge delivery; the value must not change.
  • Period: post a goods issue on the last day of the month and one on the first day of the next; each lands in the right month.
  • Thresholds and goal type: values at 95%, 92%, 91.9% and 87.9% show the expected colors; boundary values are where errors hide.
  • Drill-down consistency: the tile value equals the total of the drill-down report with the same filters.
  • Restriction: a user restricted to shipping point 1720 sees a KPI for 1720 only.
Controlled data testClear formula, small data: build a known set and compare exactly
Sample and reconcileClear formula, large data: reconcile against documents for a narrow selection
Definition workshop firstAmbiguous formula, small data: settle the definition with the owner, then test
Definition plus parallel runAmbiguous formula, large data: agree the definition, then compare a month against the legacy figure and explain the gap
Choosing how to test a KPI.
The Sign-Off Table · thesignofftable.com/academy/sap-xpr-305

Targets that move

Targets can be fixed, such as 95%, or relative, such as last year's value or a budget figure held in another measure. A relative target adds a second data source to test: confirm the comparison value is the right year, the right version of the plan and the right organizational scope. A KPI that compares June actuals for plant 1710 with the full-year budget for both plants will be colored red all year and nobody will trust it.

Check the trend and unit too. A KPI shown as "94" with no unit invites the question "94 what?". A trend arrow based on the previous period must use the same definition for both periods; if the definition changed at go-live, the first trend after go-live compares different things and should be suppressed or footnoted.

When the KPI does not match the legacy figure

Contoso has tracked on-time delivery in ECC for years. The new tile shows 3 points lower. That is not automatically a defect: the new definition may be more honest. Run one month in parallel, list the deliveries counted differently, and let the owner decide which definition stands. Then communicate the change before go-live, so the first board meeting does not discover it.

What would you do?

Contoso's on-time delivery tile shows 91% for June. Maria's controlled count of the same deliveries gives 94%. The difference is entirely from orders with several schedule lines. What is the most useful next step?

Key points
  • Most KPI problems are definition gaps, not calculation bugs.
  • Write a one-page definition: question, formula, grain, population, period, target, thresholds, source, owner.
  • In SAP S/4HANA, an evaluation holds a KPI's filters, target and thresholds; tiles and drill-downs present it.
  • Test formula, population, period boundaries, threshold boundaries, drill-down consistency and restrictions.
  • Differences from legacy KPIs need a parallel run and an owner's decision.

Check your understanding · not graded

Lesson check

  1. 1. What most often causes KPI disputes on SAP projects?

  2. 2. Where are a KPI's target and thresholds held in SAP Smart Business-style KPIs?

  3. 3. Why test threshold values such as 91.9% and 92%?

  4. 4. The new KPI is 3 points lower than the ECC figure. What should happen?