Now accepting guest editorial from SAP, Oracle, Workday and ServiceNow practitioners.Pitch a guide
The Sign-Off Table

Lesson 2 of 5

Solution Manager Test Suite

10 min

Listen to this lesson

0:00 / 6:18

eCATT scripts rarely live alone. In most landscapes they are organized, planned and reported through the Test Suite of SAP Solution Manager 7.2, together with manual test cases and, in many companies, Component-Based Test Automation (CBTA) scripts or partner tool scripts. To migrate test management you need to understand what the Test Suite holds and which of its capabilities your team really uses.

What the Test Suite covers

SAP's Test Suite overview for Solution Manager 7.2 describes a full application for manual and automated testing and change impact analysis, integrated with process management, solution documentation, change request management (ChaRM) and IT service management. Its capabilities group into a few areas:

  1. Solution documentationBusiness processes, process step library and executable library: the scope tests attach to.
  2. Change impact analysisBusiness Process Change Analyzer (BPCA) and Scope and Effort Analyzer (SEA) narrow regression scope.
  3. Test plan managementTest plans, test packages, test sequences and tester assignment.
  4. Test executionManual and automated execution, with defect management.
  5. Test Suite analyticsGap, status and progress reports and dashboards.
The Test Suite as SAP describes it, from scope to analytics.
The Sign-Off Table · thesignofftable.com/academy/sap-tau-205

Plans, packages and sequences

A test plan selects test cases from the solution documentation for a test cycle. Test packages split the plan into work units assigned to testers, and test sequences order packages that depend on each other. Test cases can be manual documents or automated test configurations, including eCATT and CBTA. When you look at an inherited Test Suite, the test plans tell you how the team really tested: which processes were in regression, how often, and who executed what.

How an inherited Test Suite is organized: plans select test cases, packages assign work to testers, sequences order dependent packages.
The Sign-Off Table · thesignofftable.com/academy/sap-tau-205

Test Suite analytics then reports on those plans: SAP lists gap reports, status reports, progress reports and a dashboard. For an assessment, these reports are a quick way to see how testing was really run. A plan whose packages were always closed with every test case passed on the last day of the cycle tells a different story from one with steady progress, defects and retests. Export the status history of the last few regression plans; it shows which test cases were executed manually even though an automated version existed, which is often the clearest sign that the team did not trust its automation.

Automation in the Test Suite

  • eCATT: test configurations assigned as automated test cases.
  • CBTA: SAP's Component-Based Test Automation in Solution Manager, which SAP lists as supporting SAP GUI, Web Dynpro ABAP, CRM Web Client, SAPUI5 and SAP Fiori user interfaces.
  • Test data containers: storage of test data for manual and automated tests.
  • Partner tools: third-party test automation tools integrated through Solution Manager interfaces.

Change impact analysis

Two Test Suite features answer "what should we test?". BPCA analyzes which business processes are affected by a change, using usage data, to optimize regression scope. Scope and Effort Analyzer estimates the impact of support packages or enhancement packages on custom code, modifications and business processes before the physical installation, so test and adjustment effort can be planned. When you migrate, find out whether your team relied on these: their outputs are often the hidden basis of a "risk-based" regression scope.

BPCA and Scope and Effort Analyzer often sit behind a team’s risk-based scope, so find out whether the team relied on them before migrating.
The Sign-Off Table · thesignofftable.com/academy/sap-tau-205
Test Suite artifactTypical content at ContosoQuestion for the migration
Solution documentationECC processes with test documents and executablesIs it current, and does it still match S/4HANA processes?
Test plans and packagesQuarterly SP regression plans, UAT plansWhich plans were actually run in the last two years?
eCATT and CBTA test cases412 eCATT configurations, about 60 CBTA scriptsWhich still pass, and do they test processes that still exist?
BPCA / SEA resultsUsed for the last enhancement packageDoes the team need equivalent impact analysis in the new setup?
Defects and analyticsFive years of historyWhat must be kept for audit, and how?
Inventorying an inherited Test Suite.
The Sign-Off Table · thesignofftable.com/academy/sap-tau-205

What would you do?

Grace asks Priya why she wants a two-week inventory before committing to a Cloud ALM migration plan for test assets. Which answer is strongest?

Key points
  • The Solution Manager 7.2 Test Suite covers solution documentation, impact analysis, test plan management, execution and analytics.
  • Plans contain packages assigned to testers; sequences order dependent packages.
  • Automated test cases can be eCATT, CBTA or partner tool scripts; test data containers serve both manual and automated tests.
  • BPCA and Scope and Effort Analyzer support regression scoping; check whether your team depends on them.
  • Inventory actual usage before planning a migration.

Check your understanding · not graded

Lesson check

  1. 1. In the Solution Manager Test Suite, what splits a test plan into work units assigned to testers?

  2. 2. Which Test Suite feature estimates the impact of support or enhancement packages before installation?

  3. 3. Which UI technologies does SAP list as supported by CBTA?

  4. 4. Why inventory test plan and script usage before planning a migration?