Lesson 1 of 5
Why data breaks automation
10 min
Listen to this lesson
Ask any team that has run SAP regression automation for a year what makes their suite fail, and the honest answer is rarely "the application". It is data. The sales order test fails because the customer is blocked for posting. The goods issue test fails because the material has no unrestricted stock in storage location 171B. The invoice test fails because somebody else already cleared the open item it was looking for. None of these are defects, yet each one costs an analyst twenty minutes and chips away at the trust the business places in the suite.
This course is about designing data so that automated SAP tests fail only when something is genuinely wrong. It builds on the test data foundations in Test Data for SAP (sap-tst-206) and the service-level techniques in API and Service Test Automation (sap-tau-203). We will follow Contoso Motor Co.: Priya, the test lead, runs a nightly regression suite in the quality system CMQ, client 100, ahead of the S/4HANA conversion go-live, and Kenji's ABAP team owns the data helpers.
Five ways data breaks a run
Data failures cluster into a handful of patterns. Learn to recognize them and most of the triage work becomes routine.
| Pattern | What it looks like at Contoso | Root cause |
|---|---|---|
| Consumed data | Goods issue for CMC-ALT-100 fails with a stock shortage after three nights of runs | The test uses stock but never puts it back |
| Changed state | Sales order creation for Great Lakes Fleet Services stops with a credit block | Another test, or a person, changed the customer or its exposure |
| Refresh drift | Every test that hard-codes order 0000012345 fails on Monday | CMQ was refreshed from production over the weekend |
| Collisions | Two parallel runs pick the same open item and one finds it already cleared | Shared data with no reservation or ownership |
| Time and period | Postings fail in the first days of a month | The posting period is closed or the test data has aged past a date rule |
Common data failure patterns in SAP regression automation.
- Hard-coded keyThe script expects a customer, material or document number that someone typed in months ago.
- Data changesA refresh, a parallel run, a person or a previous test changes or consumes it.
- Test failsThe failure message is about the data, not the process under test.
- Analyst investigatesTwenty minutes later the verdict is "not a defect".
- Trust erodesAfter enough false alarms, people stop reading the report.
Why SAP makes this harder than most systems
SAP data is deeply connected. A sales order needs a customer with sales area data for 1710 / 10 / 00, a material extended to plant 1710 and the sales organization, pricing condition records, a valid shipping point, credit settings and an open posting period. Any one of those can be missing or changed, and the error appears at the order, far from the cause. Documents also change state as the process moves: an order becomes a delivery, the delivery is picked and goods-issued, a billing document is created and posted. A test that depends on finding a document in a particular state is fragile, because every other process that touches that document can move it on.
Finally, SAP landscapes are refreshed. Basis copies production into quality to give testers realistic data, and that copy replaces everything the automation team built. The number ranges in the copy are production's, so document numbers your scripts remember may now point to different documents or to nothing at all.
The strategy in one picture
No single technique covers everything. Creating a business partner inside every test is cheap; creating a full plant with costing and MRP settings inside every test is not. The skill is choosing, per data object, the cheapest approach that makes the test repeatable. The rest of the course gives you the options and the trade-offs.
What would you do?
Priya's nightly suite in CMQ has 180 tests. Over two weeks, 31 failures were investigated and 26 turned out to be data problems. The steering committee asks whether automation is worth keeping. What should Priya do first?
- Most automated SAP test failures are caused by data, not by defects or scripts.
- Typical patterns: consumed data, changed state, refresh drift, collisions and period or date problems.
- SAP data is connected and documents change state, so errors surface far from their cause.
- Data problems cause false passes as well as false failures; every test should know which record it uses.
- A strategy combines self-contained creation, pools, runtime discovery, privacy and diagnostics.
Check your understanding · not graded
Lesson check
1. A goods issue test for CMC-ALT-100 passed for three nights and now fails with a stock shortage. No code changed. Which data pattern is this most likely?
2. Why can data problems cause a false pass?
3. After a weekend refresh of CMQ from production, every test that remembers a document number fails. What is the root cause?
4. Postings in Priya's suite fail only in the first days of each month. Which data pattern fits best?