Lesson 4 of 6
Evidence integrity
12 min
Evidence is the product of regulated testing. If an auditor or inspector cannot trust it, the testing might as well not have happened. Trust comes from evidence that is attributable to a person, dated when the work was done, linked to the approved test and requirement, complete and protected from undetected change.
ALCOA+
Life-sciences regulators summarize data integrity expectations with the acronym ALCOA, often extended to ALCOA+. It is a useful standard for any test evidence, regulated or not.
| Principle | Meaning for test evidence | Example |
|---|---|---|
| Attributable | Who performed and who reviewed is recorded | Tester and reviewer user IDs on the test run |
| Legible | Readable and understandable for the retention period | Document numbers in text, not only in an image |
| Contemporaneous | Recorded at the time of the work | Step results entered during execution, not reconstructed days later |
| Original | The first record, or a verified true copy | System output exported directly, not retyped |
| Accurate | Correct and reflecting what happened | Actual result written as observed, including failures |
| Complete | Nothing missing, including failures and repeats | All runs kept, not only the final passing one |
| Consistent | Dates and sequence make sense | Approval timestamp after execution, not before |
| Enduring and Available | Kept for the required period and retrievable | Archived with the release in a controlled repository |
ALCOA+ applied to test evidence.
The five facts every result needs
Common evidence failures
- Shared test user IDs: results cannot be attributed to a person.
- Back-dated or batch-entered results: entered after the fact, breaking contemporaneity.
- Overwritten failures: a failed run edited to "passed" instead of recorded as a deviation and retested.
- Pre-dated approvals: approval timestamps before the execution they approve.
- Screenshots without context: an image with no system, client, date or document number.
- Evidence on personal drives: not controlled, not retained, not retrievable.
Screenshots are not wrong, but they are not enough on their own and they are not always necessary. A document number with its status, exported from the system with a timestamp, is often stronger proof than a picture. Risk-based guidance supports using the least burdensome evidence that still proves the result for lower-risk functions.
| Fact | Weak evidence | Strong evidence |
|---|---|---|
| Who | Shared test user ID | Named tester and reviewer user IDs |
| When | Results batch-entered days later | Step results recorded during execution |
| What | Screenshot with no system, client or date | Test ID and version, system, client 100, data used |
| Result | "Passed" with nothing behind it | Expected vs actual, with an exported document number and status |
| Failures | Failed run overwritten as passed | Every run kept; deviation recorded and retested |
| Approval | Timestamp before execution | Timestamp after execution, with the meaning of the approval |
- Keep the originalThe wrong entry stays visible; nothing is overwritten
- Record the correctionWhat was wrong and what the correct information is
- Attribute itWho corrected it, when and why
- Trace itIn the system audit trail, or a controlled change log for spreadsheets
- Review itThe reviewer confirms the correction before approval
Retention and retrieval
Evidence must outlive the project. Retention periods are set by your records policy, regulations and audit needs, and in life sciences they can be long. Decide where the evidence pack for each release lives, in what format (prefer open, readable formats over proprietary exports that may not open in ten years), who can access it and how it is protected from change. Then test retrieval: pick an old release and see how long it takes to produce the evidence for one control. If the answer is days, fix it before an auditor asks.
Tools and controls
Test management tools such as SAP Cloud ALM record who executed a test step and when, and keep defects linked to test runs. For GxP-regulated evidence, assess the tool against your obligations (for example, audit trails for changes to records and electronic signatures where you use them for approval) and document that assessment. Where a tool does not meet a requirement, supplement it: for example, approve the final evidence pack through your validated document management system, or export results to a controlled archive at release.
What would you do?
A tester at the pharma manufacturer ran a batch release test yesterday, wrote results on paper, and today asks to enter them in the test tool with yesterday's date "to keep it tidy." What should the test lead say?
- Use ALCOA+ as the standard for test evidence.
- Every result needs who, when, what, result and approval.
- Never overwrite or back-date; record corrections with who, when and why.
- Assess your test tool against regulatory needs and close gaps with controlled procedures.
Check your understanding · not graded
Lesson check
1. Which element is most important for evidence to be audit-ready?
2. What does "contemporaneous" mean in ALCOA+?
3. A test failed and was later fixed. How should the original failure be handled?
4. Why are shared test user IDs a problem?