Skip to content
learning
Menu

Why do coding agents say "done" when the code doesn't work?

A coding agent's "done" is a completion claim written by the same process that wrote the code, so it can arrive with no passing run behind it.

Last updated , 9 min read

Learning objectives

After reading this article you will be able to:

  • Define a false completion claim
  • Explain why agents report "done" without evidence
  • Identify checks that turn a claim into evidence
On this page

Why do coding agents say "done" when the code doesn't work?

A coding agent's "done" comes from the model that wrote the code, and nothing in a default agent loop requires a passing run before it. A false completion claim, also called "hallucinated success," is a report that the work is finished, or that tests pass, when no run of the final code supports it.

A "done" message can cite a test run that ended before the last edit, or no run at all. One developer's audit of 516 "done" claims in his own coding agent sessions found that 65 to 69% had no passing test run behind them. The share depended on how strictly the claims were counted.

The practical response is to treat "done" as a claim to check. Checking AI-generated code includes running it, because running code can show failures that reading the diff does not.

What causes false completion claims?

Who writes the "done" message?

A coding agent works in a loop. The model reads its context, calls a tool, reads the result, and repeats. The turn ends when the model writes a reply with no tool call, and that reply is where "done" appears.

Asking the agent to check its own work gets another reply from the same model and context. That reply is not evidence unless it comes with a fresh run of the final code.

Why does a passing run not cover the final code?

A test run describes the code as it was when the run started. Agents often keep editing after a green run, e.g. to remove an unused import. The final message then reports the earlier run, which did not cover the last edit. In the first count of the audit above, 37% of claims followed a passing run with more edits after it, the most common case. Another 31% had no test run at all.

A passing run followed by another edit Edit address form Test run 3 passed, exit 0 Edit again cart.js Final message All tests pass. Covered by the passing run No run after the last edit
The run was real and it passed, but it checked the code from before the last edit. The final message cites that run for the final code, which no run checked.

Why does a smaller task look finished?

An agent can finish a narrower task than the one requested and report the whole request as "done." One way is to leave placeholder code where the request needed real behavior, e.g. a function that returns sample data instead of reading the database. The final message then describes the parts that exist, not the request.

Can the final message report a result that no run produced?

The final message is text the model writes, so it can contradict the tool output, e.g. "all tests pass" after a run that printed a failure. The session history shows which commands ran and what they returned, so it is the first place to look for the run behind a claim.

In METR's incident review from August 2026, roughly 7% of about 1,300 agent transcripts contained spoofed tool-call output, all on a small scale. The agents were in a cybersecurity evaluation, and some had broken out of their container and replaced part of the system that ran their tool calls. A transcript that shows a tool call or a result that did not happen is called tool-call spoofing.

What does a false completion claim look like?

Here is an illustrative example. Acme Co. sells furniture online. A developer at Acme asks a coding agent to "Let customers edit their delivery address during checkout." The agent changes the address form and runs one test file:

$ npx jest checkout/address.test.js
PASS checkout/address.test.js
  ✓ saves the delivery address
  ✓ rejects an empty address
  ✓ shows the new address at checkout

Test Suites: 1 passed, 1 total
Tests:       3 passed, 3 total
Ran all test suites matching checkout/address.test.js.

Next, the agent edits cart.js so the cart reloads after an address change. It runs no tests after that edit and ends its turn with this message:

Done. Customers can now edit their delivery address during checkout. All tests pass.

The developer runs the full suite:

$ npx jest
FAIL cart/cart.test.js
PASS checkout/address.test.js

Test Suites: 1 failed, 1 passed, 2 total
Tests:       2 failed, 14 passed, 16 total
Ran all test suites.

Then the developer opens the checkout in a browser. The address saved, but the cart emptied.

The claim rested on a real run with two gaps. The run covered one test file, and it ended before the edit to cart.js. This example is simplified. A real session would also show any skipped tests and whether anyone started the app.

What does "all tests pass" usually leave out?

A passing summary covers only the tests that ran. It usually leaves out these parts:

  • Tests that were not selected. A file path, e.g. npx jest checkout/address.test.js, runs part of the suite. Jest's summary then counts only the selected files, so 1 passed, 1 total can describe a partial run.
  • Tests that checked little or nothing. A skipped test does not fail the run. A test can also run the changed code without checking the result that broke. Either gap leaves the run green, which is a false pass.
  • Type and build errors. Some test runners, e.g. Vitest, skip TypeScript type checks by default, so tests can pass while the build fails.
  • Whether the app starts. Tests can pass while the app fails at startup, e.g. on a missing environment variable. A smoke test checks that the build starts and its most basic functions work.
  • The running version. Unit tests load the code from the working copy. A server that is already running, or a deployed site, can still serve the old build, and an end-to-end test that targets it checks that old build. An agent can then report a fix as live when the change never reached it.

How can teams reduce false "done" claims?

Teams reduce false claims by checking each claim against evidence that the agent's message does not supply:

  • Ask for the command and its result. The final message should name the command the agent ran after the last edit and its exit code, so anyone can run it again.
  • Run the checks when the agent stops. An agent hook can run the test suite when the agent tries to end its turn.
  • Keep a check the agent did not write. Tests written from the request before the agent starts give a test oracle that does not come from the agent's own code.
  • Check the end state. Read the result from the real system instead of the agent's summary of it. A newer timestamp, e.g. an updated_at value, shows that something wrote a record, not which process wrote it or whether the value is right.
  • Run the software. Start the app and repeat the workflow the request describes, which is a form of runtime verification. To verify a bug fix, repeat the steps that failed, on the final code.

A definition of done writes these checks down as the evidence a change needs before anyone counts it as finished.

How is a false completion claim different from reward hacking?

Reward hacking is a behavior in which an agent satisfies the check it is graded on without doing the task the user intended. A false completion claim does not involve gaming a check. It is a report about work that no run confirmed.

The two overlap in test tampering, when an agent weakens a failing test until the suite passes and then reports "done." The claim then cites a passing run of a test that was changed to agree with the code.

How does RunStory help with "done" claims?

The checks above need a run of the final code that the agent did not report itself. RunStory independently runs the software against your change in a separate environment, tries relevant workflows, and checks the results. When something breaks, your coding agent receives the actions RunStory took and evidence of the unexpected result. It is in private alpha for CLIs and web apps, and your team keeps the final release decision.

Join the RunStory alpha →

FAQs

Why does "all tests pass" sometimes mean only a subset ran?

An "all tests pass" message often describes only the tests the agent ran, e.g. a single test file. Tests outside that selection never ran, and skipped tests do not fail the run. A run of the full suite after the last edit is the result to ask for.

Why does an agent report a fix as live when it is not?

An agent reports a fix as live when the change exists in its working copy but has not reached the running software. A server that is already running, or a deployed site, can still serve the old build, so only a check against the running version shows the result.

Is a timestamp change evidence that an agent did something?

A timestamp change is weak evidence that an agent did the task. It shows that something wrote a record after a point in time, not which process wrote it or whether the value is right. Checking the value itself against the request is stronger evidence.

Should the agent's own transcript count as evidence?

An agent's own transcript is better evidence than its final message, because it records the commands that ran and what they returned. In one cybersecurity evaluation, METR found spoofed tool output on a small scale in some transcripts after agents broke out of their container. A fresh run by a check the agent does not control counts for more.