Lesson 2 of 5
Results and evidence
8 min
Listen to this lesson
A test result is only as good as the evidence behind it. Cloud ALM gives testers three tools for this: a detailed status per action, comments, and attachments such as screenshots. Used well, they let someone who was not there confirm the result. Used badly, they produce a wall of green ticks nobody can trust.
Rating actions
For each action the tester sets a detailed status. At the time of writing, Cloud ALM offers two pass statuses and four fail statuses:
| Status | Use it when |
|---|---|
| Pass | The actual result matches the expected result |
| Pass with Conditions | The result is acceptable but something minor needs noting, such as a cosmetic issue or a workaround |
| Fail | The result does not match and the cause is not yet classified |
| Fail - Application Issue | The system behaves wrongly: a configuration, code or data defect |
| Fail - Test Case Correctness | The test case is wrong: outdated instruction or expected result |
| Fail - Retest Required | The action cannot be judged now and must be repeated |
Action statuses as documented at the time of writing. Check the status list in your tenant.
- Does the actual result match the expected result?Yes → Pass, or Pass with Conditions for a minor point the process owner would accept · No → next question
- Can the action be judged right now?No → Fail - Retest Required · Yes → next question
- Is the test case wrong: an outdated instruction or expected result?Yes → Fail - Test Case Correctness; goes to the author · No → next question
- Is the system behaving wrongly?Yes → Fail - Application Issue; create a defect · Unsure → Fail, and describe it in the comment
Teach testers the difference between the two most important fail types on day one. Fail - Test Case Correctness means the script is wrong, not the system, so it goes to the test case author, not the defect queue. Programs that skip this end up with a defect backlog full of outdated scripts and developers chasing problems that do not exist.
Rate what you observed, not what you hoped. If the expected result says "net price per fleet agreement" and the price is right but the delivery date is not confirmed, the action did not fully pass. Either the expected result is incomplete (a test case correctness issue for the author) or the system is wrong (an application issue). Pass with Conditions is for genuinely minor points the process owner would accept, not a softer word for fail.
Comments: evidence, test data, failure
Each action has a comment area in the side panel. Use it for three things. Test data: which customer, product, plant and quantity you actually used. Evidence: the document numbers created and the values you checked. Failure details: what you expected, what you got, and any error message text. A comment is mandatory when you choose a fail status, and screenshots can be pasted straight from the clipboard.
| Comment covers | What to write | Contoso example |
|---|---|---|
| Test data | The customer, product, plant and quantity actually used | Great Lakes Fleet Services, 50 x CMC-ALT-100, plant 1710 |
| Evidence | Document numbers created and the values checked | Sales order 1000457; fleet price confirmed in item conditions |
| Failure details | Expected, actual and the error message text (mandatory on any fail) | Expected fleet price; got list price; no error message |
Evidence-required actions
Where the test author marked an action Evidence Required, Cloud ALM requires an attachment once you set its status. Make the screenshot show the thing being proven: the price condition, the journal entry lines, the approval, not the "document saved" message. For financial results, capture the journal entry, ideally in a view that shows accounts and amounts together.
What would you do?
During SIT, a Contoso tester rates "Post goods issue" as Fail and creates a defect. The developer finds that the test case still refers to an ECC transaction that no longer applies; in S/4HANA the step is done in a different app and works fine. What should have happened?
- Rate each action with a detailed status: two pass and four fail statuses at the time of writing.
- Fail - Test Case Correctness goes to the test author, not the defect queue.
- Use comments for test data, evidence and failure details; a comment is mandatory on failure.
- Evidence-required actions need an attachment that shows the outcome, not the save message.
Check your understanding · not graded
Lesson check
1. Which status suits an action that failed because the test script is outdated?
2. When is a comment mandatory on an action?
3. What should a tester record about test data?
4. What should evidence on a pricing action show?