Human in the loop

Human in the loop (HITL) describes a design in which a person must approve an action before it takes effect. In an LLM application it usually means the model proposes a tool call — send, pay, delete, publish, grant access — and a human confirms it.

It is worth understanding precisely, because it is simultaneously the most effective control against a compromised agent and the one most often implemented in a way that does nothing.

Why it works

Every other control against prompt injection is probabilistic. Confirmation is not: it inserts a party the attacker cannot reach through the token stream. An injected instruction can make an agent propose an exfiltration; it cannot make the reviewer approve it, provided the reviewer can see what they are approving.

That proviso is the whole control.

The three ways it is rendered useless

• Alert fatigue. A system that asks for confirmation fifty times a day trains its users to approve reflexively. Confirmation is a scarce resource; spend it on the irreversible and the outbound, and design everything else to need no approval.
• Uninspectable prompts. "Allow this agent to run a tool?" tells the reviewer nothing. The prompt must show the actual action and its arguments — the recipient address, the amount, the file, the resource — in a form the attacker cannot control the rendering of.
• Confirming the wrong thing. Approving a plan and then letting the agent execute an arbitrary number of steps under that approval is not human-in-the-loop; it is a delayed blanket authorisation. Bind the approval to the specific call, with its specific arguments, and re-ask if they change.

Where to draw the line

A workable default is to require confirmation for anything that is irreversible, outbound, or privilege-changing, and to require nothing for read-only actions scoped to data the user can already see. That keeps the prompts rare enough to be read.

Related: human oversight obligations under the EU AI Act apply to high-risk systems and are a compliance reason to get this right as well as a security one. See also agent security and excessive agency.