Now accepting guest editorial from SAP, Oracle, Workday and ServiceNow practitioners.Pitch a guide
The Sign-Off Table

Lesson 4 of 5

Choosing what to automate

10 min

Listen to this lesson

0:00 / 5:35

No SAP team can automate everything, and trying is the most common way automation programs fail. The skill is choosing: picking the tests where automation pays back soonest and lasts longest, and leaving the rest to people. That takes explicit criteria, not enthusiasm.

Four core criteria

CriterionQuestion to askGood candidatePoor candidate
FrequencyHow often will this test run?Every transport batch and every upgradeOnce, for a one-off migration
StabilityHow often do the process and its screens change?Settled standard process, stable appA process being redesigned next quarter
DeterminismIs there one clear, machine-checkable expected result?List price = $240.00 per unit; document posted"The layout looks right"
Business value and riskWhat happens if this breaks unnoticed?Invoicing, payments, tax, pricingA rarely used display report

The four core criteria for automation candidates.

Two practical conditions sit alongside the core four. Data availability: can the test create or reset its own data, or does it depend on something consumed or hard to rebuild? And technical feasibility: can the UI or interface be driven reliably, with stable identifiers and observable results? A test that scores well on value but cannot get data is not ready yet.

A high score is not enough: a candidate must also have reproducible data and identifiable screens before it is automated.
The Sign-Off Table · thesignofftable.com/academy/sap-tau-101
Automate firstHigh risk, run often: billing, pricing, payments, goods movements
Automate selectivelyHigh risk, run rarely: year-end close steps; consider a semi-automated checklist
Cheap winsLow risk, run often: smoke checks such as logon and key app launch
Leave manualLow risk, run rarely: one-off reports, retiring processes
A first-pass prioritization matrix. Filter the result by stability, determinism and data.
The Sign-Off Table · thesignofftable.com/academy/sap-tau-101

A scoring approach

Many teams turn the criteria into a simple score so that choices can be explained and revisited. Score each candidate from 1 to 3 on frequency, stability, determinism, business risk and data availability. Multiply nothing, weight little, and keep the scoring sheet with the test inventory in SAP Cloud ALM or wherever your test cases live, so the next person knows why a test is or is not automated.

  1. Start from the business process list and its risk ratings, not from whatever is easiest to record.
  2. Score each candidate; discuss borderline cases with the process owner and the automation engineer together.
  3. Check data and feasibility for the top candidates with a short spike before committing.
  4. Automate in small batches, put each batch into regular runs, and only then start the next.
  5. Re-score after each release: processes stabilize, risks shift and some tests retire.
  1. InventoryPriya lists the 140 business-critical regression test cases with their processes
  2. ScoreFrequency, stability, determinism, risk, data: 1 to 3 each
  3. SpikeTwo days to prove data setup and element identification for the top ten
  4. Build batch 1Order-to-cash core: order, delivery, billing, accounting check
  5. Run and learnWeekly runs in CMQ; failures triaged; then batch 2
How Contoso selected and built its first automation batch.
The Sign-Off Table · thesignofftable.com/academy/sap-tau-101

What would you do?

The finance team asks Priya to automate the asset year-end closing test, which runs once a year and was redesigned this quarter. The billing regression test runs weekly and has been stable for six months. Which should go first?

Retiring tests is part of selection

Selection is not only about adding tests. Processes are retired, apps are replaced and some checks stop finding anything because a lower-level test now covers the same rule. Each release, Priya's team reviews the suite for tests that cover a retired process, tests that duplicate a unit or API check, and tests whose failures were never real defects. Removing them lowers maintenance and run time without lowering coverage. A smaller suite that the team trusts is worth more than a large one that people have learned to ignore.

Every test has a life cycle: it is re-scored each release and retired when its process goes or a lower test covers it.
The Sign-Off Table · thesignofftable.com/academy/sap-tau-101
Key points
  • Core criteria: frequency, stability, determinism, business value and risk.
  • Check data availability and technical feasibility before committing.
  • Use a simple, recorded score and revisit it after each release.
  • Build in small batches and get each into regular runs before starting the next.

Check your understanding · not graded

Lesson check

  1. 1. Which test is the strongest automation candidate?

  2. 2. What does "determinism" mean as an automation criterion?

  3. 3. Why keep the candidate scoring sheet with the test inventory?

  4. 4. A high-value test cannot get reliable data. What is the right status?