Autonomous coding loops tend to fail the same way. The agent finishes a chunk of work, says "all tests pass, week complete", and the loop believes it. Sometimes that's true. When it isn't, the loop carries on, and week 4 gets built on top of a week 2 that never worked.
I ran into this while building IndicRAG Bench in July. Weeks 1 and 2 ran by hand with the same session workflow each time, and I wanted the rest of the plan to run unattended without losing what made those weeks trustworthy. The design note for weekly-loop still lives in that repo. The idea fits in one sentence: an agent telling you the tests passed is a claim, while a test command exiting 0 in a shell the agent doesn't control is a fact.
So the supervisor never writes code. Each tick it spawns one fresh agent to carry out one week of the plan, then checks the week itself. The gates come from a per-repo config, so they're whatever proves that project works: the test suite, a linter, a smoke run. On top of those it always checks a few things the agent can't talk its way past:
- the working tree is clean afterwards
- the diff since the tick started isn't empty, because a week that changed nothing didn't happen
- the progress file for the week exists and contains the done marker
- the handoff contains the literal next-week marker
- the files the agent said it touched overlap the files git says it touched
That last gate is quiet but useful. An agent that reports a tidy scope while editing half the repo gets caught by it. All the gates run even after the first failure, so the report shows everything that broke, not just the first thing. If something fails, one fix-agent gets the raw gate output and one more try. Still failing means the loop halts.
Inside every week there's also an audit. The week-agent spawns a read-only auditor with a fresh context, which compares the plan, the diff and the docs and writes down the mismatches. That findings file is one of the gates. I'm careful about what it can't do, though. Same-model review can't tell you the approach was right. It's a consistency check, and the README says so.
Two of the review changes are worth telling. The first draft had a gate requiring at least three commits per week. I cut it because it rewards padding. The second came after it shipped: the week-agent was meant to read the audit instructions from the skill folder, and a subagent without access to that folder would quietly skip the audit. The fix was to paste the full audit prompt into the week-agent's prompt, the same way the config already was, so the step can't silently disappear. Same day.
Crash recovery is a small file. The supervisor writes a git-ignored sentinel at the start of each week and clears it only after the gates pass. If the next tick finds one, the last week died halfway, so a recovery agent continues from the partial commits instead of starting over and trampling them. Uncommitted changes before a tick mean a person is mid-flight, and the loop stops instead of committing on top of them.
The one trade I made the config force on you is checkpoint_every. Leave it unset and your review only happens when the loop halts. Set it, and the loop stops every few weeks so you can read what happened. Gates catch regressions and missing files. They can't catch a week that built the wrong thing correctly, with passing tests and honest docs. That one is still on you, which is what the checkpoint is for.