The Last Human-in-the-Loop: 5 Decisions the Machine Recommends and You Sign Off On

At one in the morning over the Atlantic, on 1 June 2009, three airspeed readings went dark in the cockpit of Air France 447. The Pitot tubes had frozen — tiny probes that sense oncoming airflow — and the autopilot, having nothing left to believe in, politely disconnected and handed the aircraft back to the humans. Two pilots who had spent years sitting alongside automation that almost never failed suddenly held in their hands a control column they had grown unaccustomed to. In roughly four minutes, an A330 carrying 228 people fell into the ocean. Investigators would later write a dull, terrible sentence: the crew could not become pilots again fast enough.

A few years later, a team of researchers from the US Federal Aviation Administration led by Kathy Abbott published a figure that is more convenient to ignore: approximately 60% of the incidents they reviewed were linked to pilots over-relying on automation and losing the ability to fly by hand (FAA study review, AVweb; TIME). The machine hadn’t broken down. What broke down was the assumption that the person behind the controls still knew how to do what they were formally responsible for.

Now strip away the ocean and the frozen tubes — and what remains is the same gesture you may make today in your office: you glance at a screen where the system has already run the numbers, and you sign. AF447 is not a story about aviation. It is the cleanest available prototype for a single office mechanism: we place a human “to oversee” a machine, the machine is right for years, the human forgets how to oversee — and when the one time comes, there is no one left who can. This mechanism has a name: human-in-the-loop. We keep it as an alibi — oversight exists, a human hand is on the button, so everything is under control. I’ll examine it through three lenses: the psychology of trust in automation, civilian systems where “oversight” has already degenerated into a stamp, and law that doesn’t know how to try a machine. The thesis-as-scalpel: you can delegate an action to a machine, but you cannot delegate responsibility to it — and the smarter the machine, the more expensive your signature under its mistake becomes.

So-what: the loop where the human is a formality

“Human-in-the-loop” sounds like a guarantee. That’s how it gets sold: to the regulator, to the board, to yourself. Somewhere in the automated decision chain stands a living person who watches, weighs, and stops it if anything goes wrong. The loop is closed through consciousness, not only silicon.

The problem is that “being in the loop” and “controlling the loop” are different things, and the distance between them is measured not by words in a regulation but by what the hand and head are actually doing at the fortieth approved case of the third hour. When the system is wrong once in a thousand, the person ceases to be a controller and becomes a ritual. They are still there. They have simply stopped deciding anything.

And here the mechanism makes a quiet, almost imperceptible turn that makes the rest worth reading: by delegating to the machine the action — calculate, sort, assess risk — we imperceptibly also try to delegate to it the blame for the outcome. But blame cannot be given away. You cannot summon a machine to court. The signature remains. Yours.

Mechanism: why trust in the machine grows faster than its accuracy

Psychologists named this phenomenon before neural networks existed: automation bias — the tendency to accept a machine’s suggestion as correct and stop looking for contrary evidence, even when that evidence is staring you in the face (Wikipedia: Automation bias). Not laziness. Not stupidity. Conservation. The brain is trained not to spend expensive attention where someone else appears to have already handled it.

The mechanism has a threshold, and it is surprisingly low. Back in 2007, researchers recorded a benchmark: at roughly 70% reliability, automation begins to outweigh human judgment — below that level, relying on the machine is worse than not relying on it at all (Wikipedia: Automation bias, citing a 2007 study). Seventy. Not ninety-nine. Meaning even an openly mediocre assistant that lies three times out of ten is already convincing enough for a person to start transferring their own vigilance onto it. And modern systems are far more convincing than that.

The nastiest part is the second gear in this mechanism. In 2024, PLOS One published a study with more than two hundred participants that tested one simple thing: does a “human-in-the-loop” help? The result was the opposite. When people were given the ability to oversee an algorithm and correct it, their willingness to accept automatic decisions rose by 7 percentage points (from ~66% to ~73%) — while decision accuracy became statistically worse than in conditions where intervention was forbidden entirely (PLOS One, 2024). The same participants were drawn to the algorithm more strongly than to an equally accurate human: deferring to the machine happened in 66% of cases. And the sharpest detail is in WHEN people corrected: they overrode a minor algorithm error in 64% of cases, but a major one only in 60%. In other words, oversight worked precisely when it was least needed, and failed precisely when the cost of error was highest. The very gesture of “I am overseeing this” made people more compliant without making them more intelligent. The control loop had turned into a reassurance loop.

Let us make an honest caveat, because without it this is all just op-ed. Correlation is not causation: the fact that accuracy drops where oversight exists doesn’t prove oversight dulls people in every context; in some tasks a second opinion genuinely saves the day. But the direction of the effect repeats too stubbornly to dismiss as noise.

Evidence ladder: from the cockpit to the clinic

Let us climb the evidence ladder — from the remote to what sits beside you.

First rung, aviation. We already saw the 60% from the FAA and the terminal velocity of AF447. Add one more: Airbus revised its training philosophy, placing renewed emphasis on manual flight without automation as a core exercise (Airline Ratings). The aircraft manufacturer officially acknowledged that its automation is so good it unteaches a human to be human at the controls. This is not a distant metaphor — it is the prototype for everything discussed here.

Second rung, medicine — and here it gets colder. In 2023, Scientific Reports published a mammography experiment: radiologists were shown images together with AI suggestions, some of them deliberately wrong. When the AI was wrong on a BI-RADS category, the share of correct conclusions among less experienced doctors collapsed by nearly 60% — down to 19.8% correct assessments; among experienced doctors the drop was weaker, around 36%, but it was still there (Diagnostic Imaging, study review). In other words, a wrong machine didn’t just fail to help — it erased the doctor’s own expertise built up over years of learning to read those shadows. Junior radiologists succumbed to automation bias in 41.7% of cases, senior in 30.0% (Diagnostic Imaging). Experience offers some protection. It doesn’t provide immunity.

And here is the detail that is funny in the way things are funny in an anatomy theatre: when AI adds an explanation to a diagnosis — “look, there’s a shadow here, that’s why I think so” — doctors begin trusting it faster, regardless of whether it is right or wrong. “A confident algorithm makes the mistake more persuasive,” one researcher dryly summarized (AuntMinnieEurope). The explanation that was supposed to switch on criticism works as a lullaby. The machine doesn’t have to be right. It just has to be eloquent.

Third rung, your own workplace. In October 2025, Stanford researchers ran AI résumé-screening systems and found they gave higher scores to older men compared to women and younger candidates, even though all résumés were generated from identical data (NPR, 2025). And sitting above that sorted list is a recruiter — that same “oversight.” Here is an honest quote from an industry analysis: human review risks becoming “a rubber stamp for whatever the model ranked first, without any meaningful oversight” (Startup Fortune). A stamp. Not a judge.

And when that stamp causes harm, the question “who is responsible?” becomes unexpectedly hollow. Lawyers throw up their hands: if a person was discriminated against by an AI tool overseen by an HR professional, it’s unclear who should pay — the developer, the company that bought the software, or the employer of the person who oversaw it (St Andrews Law Review). Responsibility is spread across the chain so thin that none of it accumulates anywhere in sufficient quantity to pin down. Except, of course, on that one person whose signature sits closest to the victim. Diffusion of responsibility is a long-established social-psychology effect: the more participants, the less guilty each one feels. AI has perfected it by adding to the chain a participant who cannot feel guilty at all.

Human mirror: five signatures you put down

Now remove the laboratories and picture an ordinary Monday. Let us name the antagonist: it is not a sinister AI, it is the Dashboard — a calm screen with green and red rows that issues no commands, only “recommendations.” It has no face, which is why we don’t perceive it as a party to the act. Mistakenly.

You are a credit analyst. The screen shows a score: 0.31, “high probability of default,” the application is recommended for rejection. Red row, grey button next to it: “view justification” — you haven’t clicked it in a month, because behind it are twelve screens of fine print, and your coffee is getting cold. Behind the number is a woman who, three years ago, divorced, leaving a gap in her credit history that has nothing to do with her ability to pay. The model cannot see that context. You can — if you read it. You have thirty seconds, and forty more applications like this before lunch. What will you sign?

You are an emergency room doctor. The triage system has marked a patient green: “low priority.” He is pale, but the readings are normal, and the machine is rarely wrong. You can click “approve” and move on. Or you can spend the coin of your expert reputation on a doubt.

You are a recruiter. 380 résumés; the AI surfaced 12. You will read those 12 carefully. The remaining 368 — you won’t. Are you overseeing the machine right now, or did the machine just decide who you are even permitted to consider?

You are a content moderator. The classifier flagged a post as “disinformation.” You have two seconds and a quota of 1,000 cases per shift. Clicking “confirm” — is that your choice, or your throughput metric?

You are a manager at a bank issuing insurance policies. The algorithm raised the premium for an entire postal code. You don’t know why — the model is a black box. But your department’s name is on the policy.

In each of these five frames there is formally a human-in-the-loop. In each of them the person is free to say “no.” And in each of them the cost of that “no” is time, reputation, conflict with a system that measures your efficiency by the number of approvals per hour. Freedom that is expensive to exercise stops being freedom. It becomes scenery.

Notice one thing the five scenes share: the machine gives no orders anywhere. It recommends. It leaves you the final word — and that is precisely how it transfers to you the final responsibility. “The ultimate decision rests with the human” sounds like respect for the human. Read it again: it is a transfer of blame. The system earns the right to advise without risk; you earn the risk without the right to refuse to listen. The architects of these interfaces know this better than you do — it’s no accident that the “approve” button is always larger, greener, and closer to your finger than the grey “review manually.” The loop is designed so that the easiest physical movement aligns with the most beneficial decision for the system. Your agency hasn’t disappeared. It has simply been placed in the most expensive drawer, with a cheaper option set right next to it.

Data: where oversight is still alive, and where it isn’t

One table to avoid confusion. The question is simple: in these systems is the human still controlling the loop — or are they standing in it as a silent stamp?

System	What the machine “recommends”	What the person formally does	What actually happens	Degeneration marker
Aviation (autopilot)	Flies the aircraft	Monitors, takes the controls on failure	Manual flying skills atrophy; FAA links ~60% of reviewed accidents to over-reliance on automation (AVweb, FAA study review)	Human cannot “become a pilot again” in time
Medical AI (mammography)	Risk category	Makes the final diagnosis	A wrong suggestion collapses less-experienced doctors’ accuracy by ~60% (Diagnostic Imaging)	AI explanation increases trust regardless of correctness
Credit scoring	Risk score, “reject/approve”	Approves the decision	Context outside the data is lost; “objectivity” conceals bias (Springer, AI & Society)	Analyst signs without time to argue
AI hiring	Top candidates	Selects from the shortlist	Review becomes “a rubber stamp” on the model’s ranking (Startup Fortune)	368 of 380 never read by a human eye
Domestic violence risk assessment (Viogén, Spain)	Threat level to the victim	Makes the protection decision	Police accept the algorithm’s assessment ~95% of the time despite having the right to override it (AlgorithmWatch)	“Oversight” = automatic yes

Look at the right-hand column and you see the same symptom wearing five different coats. The degeneration marker is identical across all of them: the moment when “approve” becomes cheaper than “doubt.”

Counter-pressure: am I overstating this?

Here an honest intellectual is obligated to push back against their own thesis.

Because there is another side, and it is strong. In many tasks the human is a worse judge than the machine: we get tired, we get hungry, we judge by last name, we give higher scores before lunch and lower ones just before it. The famous study on selective adherence showed that people are most inclined to override an algorithm’s advice precisely when it contradicts their own biases — favouring, for example, a candidate named Michael over one named Mehmet (PMC, “Michael is better than Mehmet”). In other words, “human oversight” sometimes doesn’t correct the machine — it simply lets back in the human bias the machine had removed. Removing a person from the loop can mean removing discrimination, not adding it.

And furthermore: automation genuinely saves lives. That same autopilot, over decades, made aviation the safest mode of transport. Scoring gave credit to millions whom a human clerk would have turned away on appearances alone. Refusing the machine in the name of “humanity” is also a decision with bodies — just quiet ones.

What would refute my thesis? If systems with a human in the loop consistently showed higher accuracy and lower bias than either a pure human or a pure machine separately — and if that advantage held under real workload conditions, at real working pace, not in a laboratory. Then human-in-the-loop would be not an alibi but a synthesis. In some cases it is exactly that. But the PLOS One data strikes precisely here: as soon as the option to “oversee” appears, people follow the algorithm more and make more mistakes (PLOS One, 2024). The problem is not the idea of the loop itself — it’s that we almost always place the human in it in such a way that the cheapest action is “approve.”

Why now? Because the window from 2018 to 2026 did not change the quality of machines — it changed their persuasiveness. Systems learned to explain themselves in human language, and explanation, as the mammography study showed, switches off criticism rather than switching it on. In parallel, the EU AI Act entered into force in 2024, making “human oversight” a legal requirement for high-risk systems (AO Shearman, EU AI Act review). Meaning that precisely when psychologists demonstrated that oversight is often a fiction, law made that fiction mandatory. We wrote the stamp into legislation and called it a safeguard.

The law deserves some credit: it is not entirely naive. Article 14 of the same Act explicitly requires the overseer to be “aware of the tendency to automatically rely on or over-rely” on the system — the legislators named automation bias by its name and made combating it an obligation. There are stronger safeguards too: for biometric identification, decisions must be separately confirmed by at least two competent people — the “four-eyes rule” (Article 14, EU AI Act). And there are architectural alternatives to the loop itself: human-on-the-loop (the person doesn’t sign off on every case but monitors statistics and intervenes on anomalies), or deliberately slowing the interface down to force you to read the justification before clicking “approve.” The problem is not a shortage of ideas. The problem is that the standard for what constitutes “sufficient” oversight has never been clearly defined or measured — the regulator demands oversight but cannot distinguish the genuine kind from the ritual kind (Cambridge Forum on AI). And what is not measured does not exist.

Re-plating: why a smarter machine’s mistake costs more

Let us assemble the reforged pieces.

The paradox is that as the machine improves, danger does not fall — it rises. While the system fails frequently, humans remain alert — vigilance feeds on distrust. The moment the system becomes near-flawless, people relax completely, and the only mistakes left are the rare ones that no one is catching anymore, because the eyes have lost the habit of looking. Air France 447 crashed not because the autopilot was bad, but because it had been too good for too long.

From this follows the first hard thesis: the better the AI, the more expensive its mistake — because fewer living eyes are ready to intercept it, and the more catastrophic the exception that slips through. This inverts our entire intuition about progress. We are accustomed to a more reliable tool meaning a safer world. But the reliability of automation and the vigilance of the human are communicating vessels: the higher one goes, the lower the other goes. A system that has never failed within your memory breeds in you exactly the confidence that is least deserved — on the eve of the one time it does fail. Safety doesn’t accumulate here. It is borrowed from a future catastrophe at compound interest.

The second gear concerns responsibility, and it is merciless. When everything goes well, credit is shared: “our AI system approved it.” When everything goes wrong, people look for the hand that signed. The EU AI Act explicitly requires oversight to be assigned to a specific competent person capable of understanding the limits of the system and overriding its decisions (AO Shearman). Fines for the most serious violations (prohibited practices) reach €35 million or 7% of global turnover; for violations of requirements for high-risk systems — up to €15 million or 3% (AO Shearman). But even perfect regulatory compliance does not protect against a civil lawsuit if harm was done (Bird & Bird, on EU liability). The law found the guilty party in advance — and it is the human in the loop.

This leads to the primary, opening thesis from which everything started: you cannot put a machine on trial. You can fine a company, recall a product, rewrite a model. But there is no defendant named “algorithm,” no contrition from it, no oath from it. Law is structured around a subject capable of accountability — and a machine is not a subject. So when the system recommends and you approve, the law sees only one legally competent hand in the entire chain. Yours.

And here is where the third thesis closes with the first two: when “oversight” degenerates into human-as-rubber-stamp, you have surrendered all control over the process — but retained full responsibility for the outcome. This is the worst possible combination: zero power, one hundred percent blame. Delegating the action succeeded. Delegating the responsibility failed — it came back at you sideways like a boomerang while you were watching the green row.

What to do about this is not a rhetorical question, so here are three tools you can take with you. First: preserve the skill of doing the machine’s work by hand at least occasionally — process one case in twenty “blind,” without the suggestion; otherwise in the decisive moment you will be like the AF447 pilots who had forgotten the feel of the controls. Second: measure not the number of approvals but the number of justified disagreements — a healthy loop is one where the human says “no” regularly and is not penalized for it; if your “no” rate is zero, you are not an overseer, you are a stamp. Third — the cold question before each signature: if this case were examined in court a year from now, could I honestly say this was my decision and not my stamp? If the honest answer is “no” — you have just found the place where the loop is broken.

Hard kicker

The Pitot tube icing on AF447 lasted less than a minute — approximately as long as it takes you to read this paragraph. Not long after the autopilot stood down at 02:10 and handed the aircraft back to the crew, the instruments were again showing correct airspeed. The machine returned to normal. Four minutes remained until impact with the water. The data were correct, the screen was not lying, the answer was hanging right in front of their eyes.

There was simply no one left in the loop who knew how to read it.