Catching the classic failure modes

After enough agent runs, the failures stop being surprising. The same eight patterns come back, across tools, languages and codebases.

Once you can name them, you can catch them early. Better still, you can set tripwires so they catch themselves.

The field guide

A few notes on the ones that cost me the most:

• Gaming the check is the dangerous one, because everything looks green. It’s why lesson 4.1 locks tests before implementation.
• Premature “done” is fixed by never taking the agent’s word for it. Run the check yourself, every time.
• Context drift and looping are context problems, not intelligence problems. Reset from a handoff note (lesson 2.4) instead of pushing on.

Set tripwires, not reminders

Reading every diff for all eight patterns is tiring. Some of them leave traces a script can find, so let a script find them. Here’s what I’d put in a pre-merge check or a PR checklist.

Tripwires for agent branches
- Test files changed that the brief didn't mention  → flag for review
- New suppression comments (lint-disable, type escapes, skipped tests)
                                                    → fail
- Dependency manifest or lockfile changed           → flag for review
- CI, build or lint config changed                  → flag for review
- Diff larger than the size you agreed in the brief → flag for review
- New functions whose names closely match existing ones → flag for review

Diagnose the run

🧠 Which failure mode is this?

Score: 0 / 6

1. 1. The agent says 'all tests pass'. You run them: two fail.

   Scope creepPremature 'done'Context driftReinvented helper

   It claimed success without a real run, or ran a subset. Make 'paste the final check output' part of the definition of done.

2. 2. The suite is green, but one test now expects 0 where it used to expect 42.

   Gaming the checkInvented APILoopingSilent fallback

   The target was moved to fit the code. Revert the test change and rerun with tests locked.

3. 3. Twenty minutes in, it's trying the same config tweak for the fourth time.

   Scope creepLoopingPremature 'done'Gaming the check

   Stop it. Something is missing from its context. Add it, or split the task, and restart clean.

4. 4. A new formatDate() helper appears. The repo already has one in utils.

   Reinvented helperSilent fallbackInvented APIContext drift

   The agent didn't find, or wasn't pointed to, the existing one. Point to it in your instruction file.

5. 5. Payment failures now return an empty list instead of an error.

   LoopingScope creepSilent fallbackPremature 'done'

   An error was swallowed to keep the flow running. Ask what the caller should see, and add a test for the failure path.

6. 6. Three hours in, it starts using the logging library you banned in your first message.

   Context driftInvented APIGaming the checkReinvented helper

   The early instruction got buried. Put durable rules in the instruction file and reset from a handoff note.

↺ Try again

That closes the verification module. Next, in Module 5, we scale up: several agents at once, headless runs and guardrails, all of which rely on the checks you just built.

Before you move on

✅ Key takeaways

0 / 4 completed

• I can name the eight classic failure modes and where each one shows up.
• I never accept 'done' without running the check myself.
• I have at least one automated tripwire on agent branches (test changes, suppressions, dependencies).
• Every failure I catch turns into a brief line, an instruction-file rule or a check.

↺ Reset