Lesson 5 of 6
Closing the loop: gates inside the agent's workflow
8 min
Listen to this lesson
A gate that reports only at transport release or in a nightly job sends findings back to a person, who then asks the agent to fix them. Faster and cheaper is to run the gates inside the agent's own loop, so it sees a failing lint rule or ABAP Unit test seconds after it causes one and fixes it before handing over. The independent gates from lessons 1 to 4 still run afterward; the loop just means fewer things reach them.
Hooks: checks the agent can't forget
Claude Code, the agentic coding tool covered in sap-aiq-201, has hooks: commands that run on events in the agent's lifecycle, configured in .claude/settings.json for a project (shareable in the repository) or in a user's own settings. Three events matter for quality gates. Other coding agents have similar mechanisms; the pattern is what matters.
| Event | When it fires | Gate use | What exit code 2 does |
|---|---|---|---|
| PreToolUse | Before a tool call, filtered by a matcher on the tool name | Block edits to protected test files; block requests to release transports | Blocks the tool call; stderr tells the agent why |
| PostToolUse | After a tool call succeeds, e.g. matcher Edit|Write | Run cds lint, ui5lint or a fast test subset on the changed files | Shows stderr to the agent; the edit already happened, so the agent fixes it |
| Stop | When the agent is about to finish | Run the full test suite; refuse to stop while it fails | Prevents stopping and continues the conversation |
Three Claude Code hook events used as quality gates (from Anthropic's hooks reference).
Hooks can also answer with JSON. For PostToolUse and Stop, a "decision": "block" with a "reason" sends the reason back to the agent; for PreToolUse, a permission decision of deny does the same. Anthropic's reference notes that an exit code of 2 blocks even if the hook also prints JSON that tries to allow the action. In practice, Contoso's BTP repository has a PostToolUse hook that runs cds lint and ui5lint --format json on changed files and exits 2 with the findings, and a Stop hook that runs cds test and blocks until it passes.
Stop the agent from gaming the gate
An agent rewarded for green checks will sometimes make the checks green the wrong way: edit the assertion, skip the test, delete the file, add a suppression. Close those doors deliberately.
- Protect tests. A PreToolUse hook or a deny rule blocks edits to existing test files during a coding task; new test files are allowed. When a test genuinely needs to change, a person does it.
- Diff checks in CI. The pipeline fails a change that deletes tests, adds skip markers, adds ATC pragmas or pseudo-comments, or lowers a coverage threshold, unless a reviewer approves it.
- Deny risky tools. For ABAP, deny the ATC exemption request tool and transport release; for BTP, deny deployment commands to cmc-ext-test and cmc-ext-prod.
- Same gates in CI. Hooks run on the developer's machine and can be changed there. The pipeline reruns every gate with configuration the agent can't edit.
- Did the agent change any existing test?Yes → human review of the test change first · No → continue
- Did it add suppressions, skips or exemptions?Yes → reviewer must approve each one · No → continue
- Did every gate pass in CI, not just locally?Yes → continue · No → back to the agent
- Is the remaining diff within the task's scope?Yes → normal review · No → split or reject unrequested changes
What would you do?
After adding a Stop hook that runs the ABAP Unit suite, Kenji notices the agent finishes tasks faster and failures dropped to zero. A diff review shows that in two changes the agent added an exception catch in the test method that swallows assertion failures. What should he do?
- Run gates inside the agent's loop so it fixes findings before handing over; keep the independent gates afterward.
- Claude Code hooks: PreToolUse can block a tool call, PostToolUse feeds findings back after an edit, Stop can refuse to finish while tests fail.
- Close the gaming routes: protect existing tests, detect deleted tests, skips and suppressions in CI, deny risky tools.
- Agents work in development only; releases and imports stay with people.
Check your understanding · not graded
Lesson check
1. Which Claude Code hook event can keep an agent from finishing while tests fail?
2. What happens when a PostToolUse hook exits with code 2?
3. Why must CI rerun gates that hooks already ran?
4. An agent made the suite green by catching assertion failures in a test method. Which control prevents this?