Now accepting guest editorial from SAP, Oracle, Workday and ServiceNow practitioners.Pitch a guide
The Sign-Off Table

Lesson 6 of 6

Maintainability review

9 min

Listen to this lesson

0:00 / 5:14

A framework is software, and the ISTQB syllabus says so plainly: test automation is a software development activity, so design principles and patterns matter as much for test automation engineers as for developers. The best time to find maintainability problems is a design review before the suite grows, and then once per release. This lesson gives you a review method you can apply to Contoso's framework or any other.

Three questions behind every review

CriterionQuestionSymptoms of troubleTypical fix
CouplingIf one thing changes, how many places must change?A Fiori update needs edits in dozens of tests; tests reference locators or system IDsMove details into page objects, components or profiles
DuplicationIs each piece of knowledge in exactly one place?Several versions of Create Sales Order; copied logon code; repeated dataMerge into one component or keyword; parameterize
ReadabilityCan a process owner read a test and say what it proves?Tests full of clicks and field IDs; no expected results; cryptic namesBusiness keywords, explicit expected results, naming rules
Coupling, duplication and readability: the core of a maintainability review.
The Sign-Off Table · thesignofftable.com/academy/sap-tau-301

Design principles in practice

The syllabus lists the four object-oriented principles (encapsulation, abstraction, inheritance and polymorphism) and the SOLID principles, and highlights a few design patterns for automation engineers: the facade, the singleton (often used so that only one driver talks to the system under test), the page object model and the flow model. You do not need to quote them in a review, but they explain most findings. A component that knows about environment URLs violates single responsibility. A test that calls an adapter directly breaks encapsulation of the layers. A copy of a component for one variant is a missed abstraction.

The automation patterns in one chain: tests use a flow facade, flows use page objects, and one shared driver talks to the system.
The Sign-Off Table · thesignofftable.com/academy/sap-tau-301

A review checklist

  1. Do tests call only business keywords or components, never adapters or locators?
  2. Does every component have a documented contract and return business outputs?
  3. Are environment, users, secrets and data all externalized and validated at run start?
  4. Is each locator defined once, in a page object or equivalent, using stable UI5 or GUI identifiers?
  5. Does synchronization wait for conditions, never for fixed time?
  6. Do tests assert on business results with explicit expected values?
  7. Is every automated test mapped to a test case, and do results and defects flow to test management?
  8. Are the framework, tests and data sets versioned together and reviewed like code?
  9. Is there a measured flaky rate, and is anything above the agreed threshold being fixed?

Measure, do not guess

Low riskStable process, well-layered tests: keep as is.
WatchProcess changes often but tests are layered: changes stay local.
Hidden debtTightly coupled tests on a stable area: cheap today, expensive at the next upgrade.
Refactor firstCoupled tests on a frequently changing area: the main source of repair cost.
Where refactoring pays off first.
The Sign-Off Table · thesignofftable.com/academy/sap-tau-301

Use data from the last releases: repair hours per test and per component, number of files touched per UI change, flaky rate, and time to add a new variant. A good framework shows repairs concentrated in a few components and adapters, near-zero edits to test definitions for UI changes, and new variants added as data rows. If the numbers show the opposite, the design is coupled, whatever the diagrams say.

The same UI change costs one edit in a layered framework and dozens in a coupled one; files touched per change measures the difference.
The Sign-Off Table · thesignofftable.com/academy/sap-tau-301

What would you do?

Kenji reviews the framework and suggests adding a separate layer for logging, one for screenshots, one for data, one for configuration and one for reporting. What should Priya answer?

Key points
  • Test automation is software development; apply design principles and patterns.
  • Review coupling, duplication and readability; each has clear symptoms and fixes.
  • Use a checklist covering layering, contracts, configuration, locators, synchronization, assertions, reporting and versioning.
  • Measure repair hours, files touched per change, flaky rate and time to add variants.
  • Refactor first where coupled tests meet frequently changing areas.

Check your understanding · not graded

Lesson check

  1. 1. A Fiori update requires edits in 60 test files. Which maintainability problem does this show?

  2. 2. Which pattern does the ISTQB syllabus mention for ensuring only one driver communicates with the system under test?

  3. 3. Two slightly different Post Goods Receipt components exist. What is the usual fix?

  4. 4. Where does refactoring usually pay off first?