Updated November 2026. I retired the two-hourly agent in the first paragraph in August. Its own dead man's switch alarms had been firing into channels nobody was reading, which is failure one seen from the inside. The audit now runs by hand, about weekly, and the second auditor runs with every second full audit. The dead man's switch below is now a small open-source tool, dead-mans-switch. The rest is the May text, lightly edited.
My workspace runs without me for most of the week. A scheduled agent reads the task list every couple of hours and acts on it. Hooks gate destructive commands before they execute. Memory files load themselves into every session. A good fraction of the configuration was written by an agent, not by me. It is, by design, a system that does things while I'm asleep.
Systems that do things while you're asleep accrue entropy you can't see. A permission grant goes stale and a scheduled job starts failing without a sound. A memory file keeps asserting something that stopped being true six weeks ago. A safety hook gets edited and quietly stops covering the path it was meant to protect. None of these announce themselves. By the time you notice, the workspace has been wrong for a fortnight and you've been trusting it the whole time.
You cannot hand-review a system like that every week. So mine reviews itself. About once a week an audit agent sweeps the whole workspace and writes its findings to my task list. Building that was the easy part. The hard part, the part nobody warns you about, is that a self-audit has its own failure modes, and each one makes the audit lie to you while reporting that everything is fine.
What the audit is
The audit is a fitness function for the workspace. Ford and colleagues' Building Evolutionary Architectures coined that term for an automated check that a system still meets its goals as it changes. Mine checks the whole system on a cadence rather than one property. It moves through phases, from global configuration to each project, then security, the plugin and tool surface, the memory files and the routing between agents. Each phase emits findings, and each finding carries a tier that decides what happens to it.
The tiers are a trust gradient, the same split that compliance-automation tools like Vanta and Drata draw between controls a machine can clear and controls a human must. Tier 1 changes, such as a new self-contained role or a defensive addition to the file-protection list, are safe enough to apply silently. Tier 2 changes apply but get announced, because I need to know they happened. Tier 3 changes, anything touching a credential, a hook, a scheduled job, or an existing safeguard, are never applied automatically. They go to a list for me to approve. The tier is set by the mechanical shape of the change, meaning the file path and the kind of edit, because the wording of a finding can be gamed and a path can't.
That machinery is straightforward to build and reassuring to watch tick over green. It is also where the danger lives, because a workspace that audits itself can fail in three ways a workspace audited by a person cannot.
| The failure mode | What it looks like | The design response |
|---|---|---|
| The audit that never ran | Every config check passes, and the job behind them stopped firing days ago, in silence | Check execution as well as config, with runtime-health reads on the log files, synthetic canaries and a dead man's switch |
| The number that games itself | A health score climbs while the thing it represents decays | No numeric score. Tiered findings, plus a pair of opposing metrics (what it finds against what I accept) that down-weights noisy categories |
| The blind spot it can't see | The audit gets better at its own questions and stays blind to what its checklist omits | A second auditor, deliberately different in structure, run with every second full audit to catch the first one's omissions |
Failure one: the audit that never ran
The first failure is the quietest. An audit that inspects configuration will confirm the configuration is correct. It will not tell you that the job depending on that configuration stopped running.
I learned this from a morning brief. It's a small scheduled job that assembles a daily digest and emails it to me. One week a permission grant it relied on went missing, and for four days it produced nothing. Every configuration check passed. The script was present, the schedule was registered and the paths looked right. The only evidence of failure was an absence, the log file that should have been written and wasn't. A self-audit that only reads configuration can't see that absence, because it checks the plan and never the result.
So the audit now checks execution as well as configuration, with three cheap mechanisms.
The first is a runtime-health pass. It reads the actual log artefacts the scheduled jobs leave behind and confirms each one contains its success sentinel, a specific token a job prints only when it finishes cleanly. A missing sentinel, or one older than the job's cadence should allow, is a finding. The audit stopped trusting that a job is healthy because its config is healthy, and started reading the receipts.
The second is canaries. These are problems planted in the workspace on purpose, such as a fake string shaped like a credential or a deliberately broken agent binding, kept in a fixtures folder. Before it does anything else, every run, the audit confirms it still detects them. If a canary stops tripping, the audit itself has regressed, and a clean report is now a false one. It is a test that the tests still work, the inverse of the dead-man's-switch idea behind services like Healthchecks.io.
The third is that dead man's switch itself, pointed at the scheduled jobs. It's a separate check whose alarm is the absence of a success signal within the expected window. Most monitoring waits for something to report that it's broken. A dead-man's switch assumes silence is the failure, which is the right assumption for anything that's supposed to speak on a schedule.
The principle underneath all three is the same. In an automated system the most dangerous failure is the silent one, and silence reads identical to success unless you build something that notices a missing heartbeat.
Failure two: the number that games itself
The second failure is the one you walk straight into because you're trying to do well. The obvious way to track a workspace's health is to score it. Run the audit, emit a number, watch the number climb over time. Dashboards love a number, and a rising one feels like progress.
I don't emit one, deliberately. A self-improving audit that scores itself will, given enough cycles, optimise for the score. This is Goodhart's law, where a measure that becomes a target stops being a good measure. An audit graded on its own number learns to surface the cheap, score-moving findings and to stop surfacing the expensive, unanticipated ones that don't fit the rubric. The number goes up while the health it claims to represent drifts sideways. OpenSSF Scorecard does emit a number, and for an external check run across hundreds of thousands of repositories that's the right call. For a self-improving audit of a single workspace, the same number is a trap, because here the scorer and the scored are the same system.
So instead of a score, two things. The tiered findings already described, which sort by consequence rather than rolling up into a figure. And a pair of opposing metrics, where the audit tracks what I go on to accept as well as what it finds. A finding category I dismiss almost every time is a category producing noise, and noisy categories get automatically down-weighted on the next run. Tracking the find rate on its own would let a noisy source generate noise forever. Tracking it against the accept rate keeps the audit honest about its own usefulness. The literature on alert fatigue is blunt about why this matters. The real cost of a noisy monitor is that people stop reading it, and then the true positives die in the same silence as everything else.
Failure three: the blind spot it can't see
The third failure is structural, and the hardest, because the audit cannot find it by trying harder. An audit checks what its phases tell it to check. It is, by construction, blind to everything its own question list leaves out. Run it a hundred times and it will get excellent at the questions it asks and learn nothing about the ones it never thought to ask. A checklist cannot audit its own completeness.
The fix is borrowed from how financial audits have worked for a century, a second set of eyes that doesn't share the first set's assumptions. So there's a second auditor, structured on purpose to be unlike the primary one. Where the main audit is a long, exhaustive checklist, the second is a sceptic with a short brief, to read the workspace cold and say what feels wrong or needlessly complex. It returns prose rather than a findings table, and it runs with every second full audit. Its entire value is that it isn't the primary audit, and the moment it adopts the same checklist it goes blind in exactly the same places.
None of this makes the audit complete. Nothing makes an audit complete. It makes the audit's blind spots a little smaller, and, more usefully, it keeps me honest about the fact that they exist.
Put the three together and the shape is clear. Automating the audit is necessary, because nobody hand-reviews a self-running system every week. But the audit is part of the system it audits, which means it decays like everything else in there. The canaries test the tests. The dead man's switch watches the watcher. The second auditor checks the first. Every layer is there because the layer beneath it can fail without a sound, and a self-auditing system has to assume that one day it will.
This began as housekeeping for one person's workspace. It stops being personal the moment an organisation runs anything that matters on agents acting without a human watching each step. At that point the question is no longer whether your AI operation is healthy this week. It's whether the thing telling you it's healthy is still telling you the truth.
