Lesson 2 of 5
Coverage views
9 min
Listen to this lesson
Coverage answers the question "did we test what we said we would build?" It is the heart of most gates in Realize and Deploy. In Cloud ALM, coverage comes from links: requirements and user stories linked to test cases, test cases included in a test plan and executed as test runs. If the links are there, the analytics do the rest. If they are missing, no dashboard can help.
The chain behind a coverage figure
- Scope and solution processesWhat Contoso is implementing, such as order to cash in sales area 1710/10/00
- RequirementsWhat the business needs, assigned to scope and processes
- User storiesBuild items realizing the requirements
- Test casesPrepared in Test Preparation and linked to requirements or user stories
- Test plan and runsTest cases executed in a test plan in Test Execution; each step passed or failed
- Traceability analyticsReadiness of requirements, processes or features based on all of the above
The analytics you will use
Cloud ALM offers several traceability and test analytics. Names and exact features evolve with Cloud ALM's frequent updates, so check the Analytics area of your tenant, but you can expect views along these lines:
| View | What it shows | Gate use |
|---|---|---|
| Requirement traceability | Readiness of requirements based on linked tasks, user stories and test cases and their statuses | Requirement coverage and execution at a glance |
| Solution process traceability | Readiness of scoped solution processes from requirement status and test execution progress | Process-level view for process owners |
| User story and feature traceability | Progress of user stories or features through sub-tasks, tests and deployment | Build and deployment readiness |
| Test execution analysis | Progress of test execution over time, by plan, status and tester | Is the cycle finishing on time? |
Cloud ALM analytics commonly used for coverage at a gate. Confirm what your tenant offers.
Three coverage figures, not one
A single "coverage %" hides more than it reveals. Report three numbers, each by process:
- Design coverage: share of in-scope requirements with at least one linked test case. Missing links mean untested requirements or untraced tests.
- Execution coverage: share of those linked test cases that have been executed in the current cycle's test plan.
- Pass coverage: share of requirements whose linked tests all passed in the latest run.
| Process | Design coverage | Execution coverage | Pass coverage |
|---|---|---|---|
| Order to cash | 100% | 96% | 88% |
| Procure to pay | 92% | 100% | 95% |
| Record to report | 100% | 78% | 74% |
| Plan to produce | 97% | 91% | 90% |
Read Contoso's table carefully. Procure to pay looks perfect on execution, but 8% of its requirements have no test at all. Record to report has full design coverage but is behind on execution, which is a schedule issue, not a design one. Order to cash is executed but has failures to work through. Each needs a different action, and a single blended figure would hide all three.
Coverage beyond functional tests
Requirement traceability naturally covers functional tests. Gates also need evidence for things that are not one requirement: interfaces to the bank and carrier, authorizations by business role, output such as delivery notes and invoices, performance of the month-end close, and data migration reconciliation. Give each of these its own requirements or test cases in Cloud ALM, assigned to the right scope, so they appear in the same views. At Contoso, Priya added interface and role-based test cases to each process, so that "order to cash ready" included the EDI orders from Great Lakes Fleet Services and the billing clerk's role, not just the happy path in a super-user session.
What would you do?
Contoso's requirement traceability shows 100% of procure-to-pay test cases executed, but Priya notices 14 requirements for supplier Lakeside Components' consignment process have no linked test case. What should she report at the gate?
- Coverage depends on links: requirements and user stories to test cases, test cases to test plans and runs.
- Use the traceability and test execution analytics in your tenant; names may evolve with Cloud ALM updates.
- Report design, execution and pass coverage separately, by process.
- Decide and state how indirect links are counted before comparing numbers.
Check your understanding · not graded
Lesson check
1. Gate reports show 100% test completion, but many requirements have no linked test case. What is the risk?
2. Which figure shows the share of requirements with at least one linked test case?
3. Which Cloud ALM view best gives process owners a process-level readiness picture?
4. Why agree how indirect links are counted?