Lesson 6 of 6
Automation candidates
7 min
Listen to this lesson
Regression is repetitive by definition, which makes it the natural home for test automation. But automation is an investment, not a switch. An automated test costs time to build, needs stable data to run, and needs maintenance every time the process or the screen changes. It pays back only when it is run often enough, on a process stable enough, to recover those costs. The skill is choosing which tests to automate and which to leave manual. A rough way to think about it: if building an automated test costs as much as running it by hand ten times, and maintaining it costs as much as two manual runs a year, then a test run four times a year takes years to pay back, while a test run every week pays back within a quarter. The numbers vary by tool and team, but the shape of the calculation does not. A good candidate has five traits:
| Runs per year | Manual runs saved per year | Less maintenance (2 runs) | Time to repay a build of 10 runs |
|---|---|---|---|
| 4 (quarterly) | 4 | 2 net | About 5 years |
| 12 (monthly) | 12 | 10 net | About 1 year |
| 52 (weekly) | 52 | 50 net | About 10 weeks |
- High repetition. The test runs many times a year: every transport, every stack, every hotfix cycle.
- Stable process. The steps and screens will not be redesigned next quarter.
- Deterministic data. The test can create or reliably find its own data, such as a fixed customer, product and plant, and gets the same expected result each time.
- Checkable result. Pass or fail can be decided by comparing values: a status, a quantity, a journal entry amount, a document flow.
- Clear ownership. Somebody is responsible for keeping the automated test working after changes.
- Will the test run several times a year or more?Yes → next question · No → keep it manual; one-off checks rarely repay the build
- Is the process stable for the next few releases?Yes → next question · No → wait until the redesign settles
- Can the data be created or reset reliably?Yes → next question · No → fix the test data first, or automation will fail randomly
- Can the expected result be checked by values, not judgment?Yes → next question · No → keep the judgment step manual and automate the set-up around it
- Is there an owner for maintenance?Yes → automate · No → agree ownership before building
| Test | Runs per year | Decision | Reason |
|---|---|---|---|
| Order-to-cash smoke test in plant 1710 | Daily in QAS, after every import | Automate first | Highest repetition, stable process, fixed data |
| Core procure-to-pay for CMC-RTR-200 from Lakeside Components | Every release and stack | Automate | Stable, critical, result checkable through PO history and journal entries |
| Month-end close checks | 12 | Automate the checks, keep the review manual | Stable and repeatable, but some steps need an accountant's judgment |
| One-off data migration reconciliation | 1 | Manual | Will never be run again |
| New returns process being redesigned next quarter | Unknown | Not yet | Unstable; the automated test would be rebuilt immediately |
| Usability review of a new Fiori app | Per release | Manual | The result is a human judgment |
Automation decisions for a sample of Contoso tests.
Start with the smoke tier. It runs most often, it is short, and a reliable automated smoke test changes how a team works: every import to QAS gets a health check within the hour, without waiting for key users. Next come the critical, stable core tests, which protect the processes that matter most and run at every stack. Leave the extended tier mostly manual until the first two layers are reliable; a small automated suite that always works is worth more than a large one that fails for reasons nobody has time to investigate. Data deserves early attention too. Automated tests at Contoso use dedicated test products and a fixed customer and supplier in client 100, and each test creates its own documents rather than depending on documents left over from an earlier run.
- FirstAutomate the smoke tierRuns most often and is short: every QAS import gets a health check within the hour.
- NextCritical, stable core testsOrder to cash from plant 1710 and procure to pay for CMC-RTR-200, run at every stack.
- Only when both are reliableSelected extended testsAdd variants one by one; keep the rest manual.
- ThroughoutOwn the test dataDedicated products, fixed customer and supplier in client 100; each test creates its own documents.
SAP's own options
For SAP S/4HANA Cloud Public Edition, SAP provides the Test Automation Tool. It comes with standard automated tests that align with SAP Best Practices test scripts, and SAP offers post-upgrade tests for standard processes that have not been changed, run with the customer's consent after each upgrade. For processes the customer has configured differently or extended, the customer must adapt or build the automated tests and keep them current. Results can be reviewed together with manual test results in SAP Cloud ALM.
For SAP S/4HANA in the private cloud or on premise, SAP's long-standing options include eCATT (extended Computer Aided Test Tool, transaction SECATT), which suits transaction-level tests in SAP GUI, and component-based test automation in SAP Solution Manager. SAP Cloud ALM can orchestrate automated tests from integrated automation providers alongside manual tests. Whichever tool you use, the selection criteria above do not change. Neither does the need to manage automated and manual tests together: a regression run is one run, whether a test was executed by a person or a script, and its results, defects and evidence belong in the same test plan so the sign-off sees the whole picture.
The cost nobody budgets is maintenance. Automated tests break when the system changes legitimately: a new mandatory field, a renamed button in a Fiori app, a changed default after a feature package stack. After each stack or upgrade, budget time to repair automated tests before you trust their results. A suite with many false failures teaches people to ignore red results, which is worse than having no automation. Plan the repair work into the regression calendar from Lesson 4: when a feature package stack lands in QAS, the first day goes to fixing automated tests broken by legitimate changes, and only then do their results count toward sign-off.
What would you do?
Priya has budget to automate about 20 tests this quarter. Which set should she choose first?
- Automate tests that run often, on stable processes, with deterministic data and checkable results.
- Do not automate one-off checks, unstable processes or judgment calls.
- Start with the smoke tier, then critical, stable core tests.
- Budget maintenance after every stack or upgrade, and make automated tests capture real evidence.
Check your understanding · not graded
Lesson check
1. Which test is the weakest automation candidate?
2. Why should a process under redesign not be automated yet?
3. For Public Edition, who adapts automated tests for customized processes?
4. What is the hidden cost of an automated regression suite?