When a team starts using AI, output goes up and the people producing it start to feel they understand the work less. Managers review more than they write, and trust their own sign-offs less. The instinct is to call this a new problem and send people on a course. But it was described in 1983, and later research found that training doesn't prevent it.
Lisanne Bainbridge called it the ironies of automation. Automate the doable parts of a job and the human is left with the two things people are worst at, passively monitoring a process that is mostly right and recovering it when it breaks. Researchers in human factors call the mechanism underneath the substitution myth. You can't hand the machine the doing and leave the understanding job behind unchanged, because the understanding came from the doing. It was a by-product. Take away the doing and you take away the route to the understanding, while the organisation goes on expecting it.
The evidence on the human side is old and peer-reviewed, and it isn't kind to wishful policy. Performance at pure monitoring falls away within twenty to thirty-five minutes. Automation bias, the tendency to accept the system's output as your own conclusion, shows up in experts as well as novices, and Parasuraman and Manzey's 2010 review of the research found it cannot be prevented by training or instructions. Read that twice before writing "staff must critically evaluate all AI output" into a policy document. That sentence asks staff for something the research says instruction can't deliver.
What broke is a pricing assumption. The comprehension your senior people carry was always paid for by immersion, and the immersion came bundled with work you've now automated. Software engineering has started calling the gap comprehension debt, and Werner Vogels, Amazon's CTO, warned about verification debt in his re:Invent keynote last year. Like the financial kind, it builds up out of sight. You find out how much there is on the day something needs deep understanding and nobody has it.
The partial cure is in how the work is set up. Stop verifying by re-reading everything after the fact, and move verification to intent and evidence. Agree the spec before the agent builds. Require output that arrives with its own evidence, such as a test that failed and now passes, a decision log, or a claim that carries its source or an honest unverified tag. My own workspace runs a blunt version of this. Every plan names the document it has to satisfy and up to five things the review must check, before any agent starts. And nothing gets called done on the strength of a summary.
There's a ceiling on all of this, and oversight tooling doesn't raise it. Artefacts carry explicit knowledge. The immersion you lost was tacit, and no summary brings it back. A spec can also be rubber-stamped as easily as a diff, so the cure can turn into the same problem one step earlier. That leaves a deliberate decision about people. Choose where deep comprehension still gets built the slow way, and keep some of the work manual in those seats on purpose. It costs time, and it belongs in the budget as capability.
Three questions for the next leadership discussion. Where in our workflows does verification get paid for, and in whose time? Which outputs arrive carrying evidence, and which arrive carrying only confidence? In which two or three seats are we still willing to buy understanding by having people do the work by hand? Most organisations are running up this debt without having decided to. Answering those three is how you decide.
