Lesson 5 of 5
Test types and change-related testing
9 min
Listen to this lesson
Test levels say where in delivery a test belongs. Test types say what quality it checks. The two are independent: you can run a functional test at unit level or at acceptance level. The ISTQB Foundation syllabus groups types as functional, non-functional, black-box and white-box testing, and separately describes change-related testing: confirmation and regression.
Functional and non-functional
Functional testing checks what the system does: does a sales order for Great Lakes Fleet Services price correctly, does goods receipt for rotors CMC-RTR-200 post inventory in plant 1710? Most SAP testing is functional.
Non-functional testing checks how well it does it. On SAP programs the important ones are performance (can billing for 5,000 deliveries finish overnight; does MRP complete in its window), security and authorization (can a sales clerk post a journal entry? they should not), usability (can Maria's team enter an order quickly in the Fiori app), and reliability and recoverability (does an interface resume cleanly after an outage). These are often planned as separate cycles with specialist owners.
| Quality checked | The question | Contoso example |
|---|---|---|
| Performance | Does it finish in time at real volumes? | Billing for 5,000 deliveries completes overnight; MRP finishes in its window |
| Security and authorization | Can users do only what their role allows? | A sales clerk cannot post a journal entry |
| Usability | Can people work quickly and correctly? | Maria's team enters a fleet order in the Fiori app without workarounds |
| Reliability and recoverability | Does it recover cleanly from failure? | An interface resumes after an outage without losing or duplicating messages |
Black-box and white-box
Black-box tests are derived from behavior and specifications without looking inside; nearly all business-process tests are black-box. White-box tests are derived from internal structure, such as the branches of a custom ABAP enhancement. Developers use ABAP Unit for this, and SAP's ABAP Test Cockpit runs static checks on custom code. A tester rarely writes white-box tests but should ask whether custom code was covered by them.
Confirmation versus regression
When a defect is fixed, two different things need checking. Confirmation testing (re-testing) runs the failed test again, after the fix transport is imported, to prove the fix works. Regression testing checks that the change did not break anything else that used to work. A fix to Contoso's revenue account determination needs a confirmation test of the failed billing case, and regression of other billing scenarios, because they share the same configuration.
- Defect foundBilling posts revenue to the wrong G/L account
- Fix importedFix transport imported into QAS client 100
- Confirmation testRe-run the failed billing case; it must now pass
- Impact analysisFind what else shares the changed configuration
- Regression testRun the other billing scenarios that use it
- CloseBoth results recorded with the transport number
- Does the test check behavior or results?Yes → functional · No → go on
- Does it check speed, load, security, usability or recovery?Yes → non-functional · No → go on
- Is it re-running a test that failed, after the fix?Yes → confirmation testing · No → go on
- Is it checking unchanged areas still work after a change?Yes → regression testing · No → review the test's purpose
Why change-related testing dominates the run phase
After go-live, most SAP testing is about change. Projects and support teams import transports with fixes and enhancements every week. SAP ships corrections as SAP Notes and as support package and feature package stacks, and new S/4HANA releases periodically. Interfaces and connected cloud services change on their own schedules. In SAP S/4HANA Cloud Public Edition the cadence is fixed by SAP: two major upgrades a year, applied to the test system first so customers can run regression tests before production follows.
| Change | Confirmation test | Regression scope |
|---|---|---|
| Fix transport for a pricing defect | Re-run the failed pricing case | Other pricing and billing scenarios for the same sales area |
| Support package stack | Not applicable (no specific defect) | Critical end-to-end scenarios across modules |
| New output form for invoices | Re-run any failed output tests | Billing output for all customer groups and channels |
| Public Edition release upgrade | Not applicable | Key business processes, extensions and integrations in the test system |
Typical run-phase changes and the testing they need.
Regression packs grow, so they become the natural target for automation. SAP offers its own options, including the Test Automation Tool in SAP S/4HANA Cloud Public Edition and orchestration of automated tests from SAP Cloud ALM, and the classic eCATT in on-premise systems. Automation is covered in its own track; the principle here is that regression scope should be chosen by risk and impact analysis, not by running everything every time.
What would you do?
A fix transport for Contoso's wrong revenue account is imported into QAS client 100. Priya has one day. What testing should she plan?
- Test types are independent of levels: functional, non-functional, black-box, white-box.
- Non-functional SAP testing covers performance, authorizations and security, usability and recoverability.
- Confirmation testing proves a fix works; regression testing proves nothing else broke.
- In the run phase, change-related testing dominates; scope regression by risk and impact.
Check your understanding · not graded
Lesson check
1. Re-running a failed test after the fix transport is imported, to prove the fix works, is called:
2. Checking that a sales clerk role cannot post journal entries is which type of test?
3. Which test is white-box?
4. Why does change-related testing dominate after go-live?