---
title: The Expensive Comfort of a Human in the Loop
description: Explore the challenges of human oversight in AI decision-making and learn how to ensure effective monitoring as AI adoption grows in banking.
---

[KEDInsights](https://ked-consulting.com/insights)

# [The Expensive Comfort of a Human in the Loop](https://ked-consulting.com/insights/the-expensive-comfort-of-a-human-in-the-loop)

 Written by [KED Consulting](https://ked-consulting.com/insights/author/ked-consulting) | Nov 7, 2026, 1:19:07 PM

In almost every bank's AI proposal there is one slide that settles the room. It usually says some version of the same thing: a human makes the final decision.

It is the right principle. In Indonesia, OJK lists human oversight as one of the core values of its AI governance framework for banks. But the next slide rarely answers a harder question. **What happens to that human when the number of AI decisions grows tenfold and the number of reviewers stays the same?**

Two publications in late October approached this question from different directions. Their answers should unsettle anyone who treats "human in the loop" as a problem already solved.

### **The Checker Who Signs Everything**

Every banker knows the maker-checker principle. One person prepares a transaction and another reviews and approves it. It has anchored internal control for decades, and it works well as long as volumes stay reasonable.

Now picture a maker that is no longer a member of staff but an AI agent. It can prepare hundreds of KYC assessments or suspicious-transaction summaries in an hour. The checker is still one person, with an eight-hour day and a queue that keeps growing.

At some point the check becomes a formality. The reviewer has not stopped caring. The output simply looks tidy, is usually right, and the daily target still has to be met. The FSB names this pattern directly. Human oversight may be weakened by over-reliance and automation bias.

Here lies the paradox. **The control we rely on most is the one that weakens fastest once AI starts to work.**

### **Where the Money Actually Goes**

McKinsey's research on agentic AI in financial-crime prevention, published on 26 October 2026, contains a figure worth pausing on. For one banking customer-service agent task, tokens accounted for only 20 to 25 percent of variable run costs. Human oversight accounted for 70 to 75 percent.

Most AI business cases are built around licences, compute and integration. The McKinsey figure suggests the largest cost line may not appear in them at all.

The authors also found that roughly 60 percent of an agentic task's cost goes into refining answers rather than producing the first one. The logic is simple. If a fixed share of cases goes to human review, oversight cost rises in step with volume. The more successful the adoption, the heavier the burden.

These numbers deserve some care. They come from a set of deployments rather than a representative industry survey. The direction, however, is clear enough to change how we calculate the value of AI.

### **Same Question, Different Answer**

A quieter finding matters just as much. In ten evaluations of a customer onboarding flow at one bank, the sign-off thresholds were at least 80 percent accuracy and 70 percent consistency. Measured consistency ranged from 72 to 100 percent. In the weakest evaluation, roughly one run in four returned a different answer to the same input.

McKinsey notes that this inconsistency clusters around interpretive questions, while checkable facts are handled almost perfectly. That should worry anyone designing a review process. **The parts that are hardest for a human to check are exactly where the agent is most likely to change its mind.**

Adding reviewers does not solve this. What is needed is a systematic way to measure how an agent behaves, consistency included, before and after it reaches production.

### **Regulators Have Quietly Changed the Question**

The FSB's final report on 12 sound practices for AI adoption, published on 15 October 2026, is not a binding rulebook. It is a toolkit that firms can tailor to their own context. Its tone is telling all the same. The FSB states that AI agents pose a distinct challenge for human oversight, given the impracticality of real-time human monitoring of agent decisions as their use scales.

It then points towards more structural answers. Not every agent needs the same autonomy and access. Each agent should carry a unique identifier, so its actions can be traced and told apart from those of human users. Where AI is used to monitor another AI system, the monitor should be independent of what it reviews. The FSB also warns of deskilling, where reliance on AI erodes the very expertise needed to oversee it.

In Indonesia, OJK has said that AI output cannot directly form the basis of material decisions such as credit approval without professional verification. It has also observed the shift from machine learning towards generative and agentic AI. Last month OJK's chair described frontier AI as a new challenge for banking supervision. PERBANAS and IBM figures reported by Katadata put AI adoption in Indonesian banking at 57.9 percent in Q1 2026.

The principle has not moved. Humans remain accountable. What has changed is the practical question of how to make that accountability real rather than a signature.

### **Oversight Is a Design Decision**

Consider a hypothetical scenario. A bank deploys an AI agent to draft first recommendations on AML alerts, and every recommendation still needs an analyst's approval. Three months later volumes have tripled. Review time per case has fallen from more than ten minutes to under two, and the approval rate without changes is close to one hundred percent. On paper the control works perfectly. In reality nobody can say whether it still works at all.

A scenario like this shows that the quality of oversight is a property of system design, not only of individual discipline. The implications reach well beyond the AI team.

**For IT strategy and investment,** the cost of oversight belongs in the business case from day one. **For architecture,** agent identity, logging and behavioural monitoring are core components, not later additions. **For governance and risk,** oversight intensity should match the materiality of the decision and the autonomy of the agent, and the bank should be able to show evidence that oversight is effective.

### **Five Places to Start**

1. **Measure the true cost of oversight.** Take one AI use case already in production and track the human time spent reviewing, correcting and re-running outputs. This number often decides the economics more than the licence fee does.
2. **Tier decisions by materiality and autonomy.** Not every output deserves the same scrutiny. A practical first step is to map use cases into three tiers, such as full review, sample-based review and statistical monitoring, and agree the mapping with risk and compliance.
3. **Make consistency a sign-off criterion.** Before an agent goes live, run the same inputs through it repeatedly. Set a consistency threshold alongside accuracy at every approval gate.
4. **Check whether reviewers are still reviewing.** Track signals such as review time per case and the rate of approvals without changes. Occasionally seed test cases with deliberate errors to see whether they are caught.
5. **Give agents an identity and an audit trail.** Assign unique identifiers, limit access to what each task needs, and make every agent action traceable. This simplifies monitoring and prepares the evidence supervisors will eventually ask for.

Sequence matters. Measurement and tiering should come first, because they show where human judgement is most needed. Architecture and monitoring tools can follow on a much firmer footing.

### **The Human Worth Keeping**

There is nothing wrong with insisting that people stay in control. What deserves a second look is the assumption that placing a person at every step automatically makes a system safe.

In the age of agentic AI, a reviewer's value lies less in how many outputs they approve and more in which decisions genuinely need their judgement. Banks that put people in the right places will capture AI's benefits without losing control.

So the question worth taking to the next committee meeting is no longer "is there a human in the process?" It is **"is that human still really watching?"**

[View full post](https://ked-consulting.com/insights/the-expensive-comfort-of-a-human-in-the-loop)

```json
{
  "@context" : "http://schema.org",
  "@type" : "BlogPosting",
  "author" : {
    "@type" : "Person",
    "name" : "KED Consulting"
  },
  "dateModified" : "2026-11-07T13:19:07.761Z",
  "datePublished" : "2026-11-07T13:19:07Z",
  "headline" : "The Expensive Comfort of a Human in the Loop",
  "image" : {
    "@type" : "ImageObject",
    "height" : 1024,
    "url" : "https://47392781.fs1.hubspotusercontent-na2.net/hubfs/47392781/AI-Generated%20Media/Images/Solitary%20Analyst%20At%20Polished%20Desk%20In%20Modern%20Bank%20Office.png",
    "width" : 1536
  },
  "mainEntityOfPage" : "https://ked-consulting.com/insights/the-expensive-comfort-of-a-human-in-the-loop",
  "publisher" : {
    "@type" : "Organization",
    "logo" : {
      "@type" : "ImageObject",
      "height" : 60,
      "url" : "/hs/hsstatic/content_shared_assets/static-1.4092/img/default-amp-logo.png",
      "width" : 60
    },
    "name" : "KEDInsights"
  }
}
```