Lesson 5 of 5
Privacy and diagnostics
11 min
Listen to this lesson
Two topics close the course. First, privacy: automated tests read, create and log data at a scale no human tester does, and when the test system holds a production copy, much of that data is personal. Second, diagnostics: when a test fails, the team must decide quickly whether the cause is data, environment, the test itself or a real defect. Both depend on deliberate data design.
Personal data in automation
A production copy of an S/4HANA system contains business partners who are people, employee data, bank details, addresses and contact persons. Under the EU General Data Protection Regulation and similar laws, copying that data to a test system is still processing personal data, and pseudonymized data is still personal data. Automation adds exposure in places people forget: screenshots, recorded videos, request and response logs, exported result files and defect attachments.
Controls that work
- Prefer synthetic data for anything the suite creates. A test customer named "AUTO Fleet 0412" with a placeholder address carries no personal data.
- Mask production copies before testers and automation use them. SAP has offered scrambling as part of its test data tooling, and many teams run masking as a post-copy step; check what is licensed and supported in your landscape.
- Minimize evidence: capture what proves the result (document number, status, amounts) and avoid full-screen captures of address or bank data.
- Restrict access to evidence stores with the same care as the test system itself.
- Set retention for logs and screenshots and delete on schedule.
Classifying failures
When a test fails, the first job is classification. The ISTQB Test Automation Engineering syllabus points out that the defect can be in the test case, the system under test, the automation framework, the hardware or the network, and recommends inspecting logs, setup and teardown, and running isolated tests. For SAP teams a four-way classification is practical.
| Class | Typical evidence | Who acts |
|---|---|---|
| Data | Missing stock, blocked customer, record consumed, pool below minimum | Automation team fixes data design or pools |
| Environment | RFC destination down, system unavailable, background job not running, CMQ being refreshed | Basis or integration team |
| Test | Locator changed, wrong assertion, timing issue in the script | Automation engineer |
| Defect | Process gives a wrong result with valid data in a healthy system | Defect logged for development with full evidence |
- Was the environment healthy during the run?No → environment · Yes → continue
- Did the setup log show the expected data with the expected attributes?No → data · Yes → continue
- Does the failure reproduce manually with the same data?No → test (script or timing) · Yes → continue
- Is the result wrong against the agreed expected result?Yes → defect · No → fix the expected result in the test
Good data design makes this fast. If every test logs what it created, what it reserved and what it found at run time, the analyst can answer the data question in a minute. If the run starts with environment checks and pool checks, many environment and data failures never reach individual tests at all, because the run stops early with one clear message.
What would you do?
A payment run test fails in CMQ. The log shows the open supplier item was reserved correctly, the RFC check at run start passed, and the same steps done manually with the same item give the same wrong discount amount. How should the failure be classified?
- Production copies carry personal data; pseudonymized data is still personal data.
- Prefer synthetic data, mask copies, minimize evidence, restrict access and set retention.
- Masking rules must keep processes working; test them after each masking run.
- Classify every failure as data, environment, test or defect before routing it.
- Run-start checks and setup logs make classification fast and keep only real defects with developers.
Check your understanding · not graded
Lesson check
1. Where does automation most often leak personal data that teams forget about?
2. A test fails because the RFC destination to the warehouse system was down during the run. How should it be classified?
3. Why must masking rules be tested with the key processes?
4. What should automated evidence for a sales order test capture, given the privacy controls in this lesson?