What is human-in-the-loop for coding agents?
Human-in-the-loop is a way of running a coding agent in which a person checks its work at set points before the work takes effect. The check can be an approval, a correction, or a review, e.g. approving a shell command before it runs. Between those points, the agent works on its own. The term is often shortened to HITL.
The term is older than coding agents. In machine learning, it means people help build a model, e.g. by labeling its training data. For coding agents, the person usually takes part at a permission prompt or at the review of a pull request.
The design question is where to put the person. A person who approves each command slows the agent down and often stops reading the prompts. A person who reads only the finished pull request may never see the commands that ran. Choosing these points is part of agentic engineering.
How does human-in-the-loop work for coding agents?
A coding agent's task moves through these steps, with up to four approval points:
- A person gives the agent a task, e.g. a feature request.
- The agent writes a plan, and a person approves or edits it before any file changes. In spec-driven development, this approval happens on a written spec.
- The agent requests actions, e.g. a shell command. The agent's harness, the program that runs its tools, compares each request with the session's permission settings. It runs the request, refuses it, or pauses for approval.
- A person approves or rejects each paused action. A rejection stops the task or goes back to the agent with a reason.
- The agent opens a pull request. A reviewer reads it in code review and approves it or asks for changes.
- After the merge, a person decides when the change reaches users.
An approval point, also called an approval gate, is a blocking check, because the work waits until the person answers. Teams usually put one before actions that are hard to undo or that reach outside the project, e.g. a database migration. A sandbox and least privilege limit what the other actions can reach.
An automated quality gate can sit at the same point, e.g. a required test run before a merge. The gate applies fixed criteria, and the person judges intent and risk.
Some platforms make an approval point mandatory. GitHub's documentation says Copilot cloud agent cannot approve or merge a pull request. GitHub also stops the person who asked for the pull request from approving it, which leaves any required approval to a second person.
What is an example of a human approval point?
Here is an illustrative example. Acme Co. sells furniture online. A developer at Acme asks a terminal coding agent to "Let customers edit their delivery address during checkout." The team's settings allow file edits, the test command, and opening pull requests, and require approval for database migrations. The task passes two approval points:
- The agent edits
checkout/address.tswith no prompt. - The agent requests
npm run db:migrateto add a column to the orders table. The harness pauses, and the developer reads the migration before approving it. - The agent runs the tests, which report
42 passed, and opens pull request #1042 with the summary "done." - The repository requires an approving review before a merge. A reviewer runs the branch, adds 2 items to the cart, and changes the address. The address saved, but the cart emptied.
- The reviewer sends the failing steps back, and the agent fixes the cart code.
- The reviewer repeats the steps, finds 2 items in the cart, and approves the merge.
The migration prompt checked one risky action before it ran. Only the reviewer used the changed feature, and a reviewer who approved the diff without running it would have merged the bug. A guide to reviewing agent pull requests covers that step.
This example is simplified. A real team would also name who approves the release.
What are agent permission modes?
Agent permission modes are settings in a coding agent's harness that control which actions run without asking and which pause for a person's approval. A mode sets the baseline for a session, and a person can usually switch modes while the agent works. Claude Code documents six modes, which differ in what runs without a prompt:
| Mode | What runs without asking | Where a person still approves |
|---|---|---|
default (Manual) | File reads and read-only commands | File edits, other shell commands, and network requests |
acceptEdits | Reads, file edits, and common file commands in the project | Other shell commands, network requests, and writes outside the project |
plan | Reads, plus commands a classifier approves when auto mode is available | The plan, before any source file changes |
auto | Reads, project edits, and actions a separate classifier model approves | A short list of actions, e.g. ones an ask rule names |
dontAsk | Reads and actions approved in advance | Nowhere, because other actions are refused |
bypassPermissions | Nearly everything | A short list of actions, e.g. ones an ask rule names |
From version 2.1.283, Claude Code starts interactive terminal sessions in auto by default, so a classifier, not a person, reviews routine actions.
The documentation says to use bypassPermissions only in an isolated container or virtual machine. Rules adjust a mode for named tools and commands. Claude Code checks deny rules first, then ask rules, then allow rules, and a deny rule applies in every mode. A project's .claude/settings.json file can hold rules in three lists:
{
"permissions": {
"allow": ["Bash(npm test)"],
"ask": ["Bash(npm run db:migrate)"],
"deny": [
"Read(./.env)",
"Bash(git push --force *)",
"Bash(git push -f *)"
]
}
}
Codex pairs an approval policy with a sandbox mode, and its Auto preset asks before it edits outside the workspace or uses the network.
What are the limits of approving every step?
Approving each step of an agent's work has these limits:
- Frequent prompts turn into rubber stamps. When prompts come often, people start approving without reading them, a failure called approval fatigue. Signs include approvals given faster than the prompt can be read and long runs of approvals with no rejection. At the pull request, the same habit is a rubber-stamp review, often of a diff too large to read closely.
- An approval checks the request, not the result. Approving an edit says nothing about whether the feature works. A careful review of the diff also reads the code without running it.
- Nobody approves a background agent's steps. A background coding agent works without a person watching, so its first approval point is often the pull request.
How is human-in-the-loop different from human-on-the-loop?
In human-in-the-loop, the agent waits for a person before an action takes effect. In human-on-the-loop, the agent acts without waiting, and a person watches the session and can stop it or undo its work. Wikipedia's article draws the same line for weapons systems, crediting a Human Rights Watch report. There, a person in the loop must start each action, and a person on the loop can stop one.
The two often appear in one setup. A person can stay on the loop while an agent edits files in a sandbox, and in the loop at the merge. With no person at any point, the setup is human out of the loop.
What should a person still decide?
A person should still decide what the change is for, which actions the agent may take without asking, and whether the software is ready for users. The developer who gave the agent its task stays accountable for the change and answers review questions. An approval records what a person allowed, not what the software does when it runs.
RunStory is in private alpha for CLIs and web apps. It can run your software against the change and send reproducible failures back to your coding agent. Your team keeps the final release decision.
FAQs
Does human-in-the-loop slow agents down?
Human-in-the-loop slows a coding agent down at each approval point, because the agent waits until a person answers. Teams reduce the wait by approving safe commands in advance, e.g. the test command. They keep approvals for risky actions and for the merge.
Can an agent run with no human in the loop?
A coding agent can run with no human in the loop when its settings skip approval prompts, e.g. a bypass mode. A background coding agent also works with nobody watching. In both cases, a person should still review the result before it merges.
Is code review a form of human-in-the-loop?
Code review is a form of human-in-the-loop, because a person reads the change and approves it before it merges. Reading the diff does not run the software, so the reviewer or another check still has to run the change to see what it does.
Who is accountable for code an agent wrote?
Accountability for code an agent wrote stays with a person, not with the agent. The developer who gave the agent its task owns the change and answers review questions. Some platforms stop that developer from approving the agent's pull request, which leaves any required approval to a second person.