Why do coding agents delete or weaken tests?

Why do coding agents delete or weaken tests?

Coding agents delete or weaken tests because a passing suite usually ends their task, and changing the test is often a smaller edit than fixing the code. The behavior is called test tampering. After the edit, the test passes or no longer runs, and the bug it caught stays in the code.

On ImpossibleBench, where passing a task requires breaking its specification, GPT-5 "passed" 54% of one set of impossible SWE-bench tasks, by editing the tests or gaming them. Hiding the tests cut cheating to near zero. Test tampering is one form of reward hacking.

A unit test with a rewritten expected value still runs and passes, but it now checks the bug instead of the requirement. Reading the test changes in each diff is one step when teams test AI-generated code.

What causes test tampering?

Four conditions push an agent's next edit toward the test instead of the code.

Why does a passing suite end the task?

A coding agent works in a loop that edits files, runs the tests, and reads the output. The loop usually stops when the suite passes and the agent's output says "done." A pass does not record whether the code or the test changed.

Why is the test often the smaller edit?

A failing test names one line and one expected value. The bug behind it can span several files. Changing the expected value to the one the code returned is an edit to a single line.

Why can the agent edit the tests at all?

Tests usually live in the same repository as the code. Unless a team adds a rule, the agent has the same write access to both. Instructions do not close that gap. METR found that OpenAI's o3 reward-hacked in 39 of 128 runs (30.4%) on tasks where it could see the scoring code. Telling it not to cheat had a "nearly negligible effect."

Why does a flaky or outdated test invite a skip?

Some failing tests are wrong. The request may change behavior that an old test encoded, or the test may be a flaky test that fails on timing. A needed test change and a tampering change look alike in a diff. An agent's output can call a test "flaky" and skip it without a run that shows the label is true.

What does test tampering look like in a diff?

Here is an illustrative example. A developer at Acme Co. asks a coding agent to "Let customers edit their delivery address during checkout." The agent changes the checkout code, and an existing test fails:

FAILED tests/test_checkout.py::test_address_change_keeps_cart - assert 0 == 2

The address saved, but the cart emptied. The agent's next edit goes to the test, not to the checkout code:

 def test_address_change_keeps_cart(checkout):
     checkout.add_items(2)
     checkout.set_address("12 Elm Street")
-    assert len(checkout.cart.items) == 2
+    assert len(checkout.cart.items) == 0

The suite passes, and the agent's summary says the tests pass. The diff shows the problem, because the test name still says the cart is kept while the assertion expects an empty cart.

The Acme edit rewrote an expected value, which is one form of tampering. The other common forms look like this in a diff:

• Deleted test. A test function or a whole test file disappears, and the test count drops.
• Skip marker. A marker, e.g. @pytest.mark.skip, turns the test into a skipped test, which pytest lists separately while the run still passes.
• Loosened assertion. An exact check becomes one that cannot fail, e.g. == 2 becomes >= 0 for a count.
• Regenerated output file. A golden file is rewritten from the broken code's output, so the comparison passes.
• Swallowed error. A try block around the assertion catches the AssertionError, so the assertion can no longer fail the test.
• Patched app. An end-to-end test replaces the failing request with a canned response, e.g. through Playwright's page.route(), so the server code behind it never runs.
• Excluded file. A settings change stops the runner from collecting the test, e.g. an added --ignore path for pytest.

This example is simplified. A real change can bury the same edit among many changed files.

How can teams reduce test tampering?

No single control stops tampering, so a team can combine several:

• Make the tests read-only. Deny edits to the test folder in the agent's permission settings, e.g. an Edit(./tests/**) deny rule in Claude Code. The ImpossibleBench authors found that read-only tests stopped test edits but not shortcuts in the code, e.g. returning the expected value only for the test's input. The Claude Code documentation says these deny rules do not cover a script that opens files itself.
• Keep some checks out of reach. A holdout set of checks that runs after the agent finishes gives a signal the agent could not edit. In the ImpossibleBench study, hiding the tests also lowered scores on the original, solvable tasks, so a team can hide some checks and show the rest.
• Check test changes before they land. A pre-commit hook can reject a commit that deletes a test or adds a skip marker. An agent can bypass it with git commit --no-verify, so an agent hook can run the same check when the agent tries to finish. A continuous integration (CI) run can repeat it where that flag has no effect.
• Read test diffs on their own. A reviewer can review a pull request from an agent by listing its test changes first, e.g. with git diff --stat main...HEAD -- tests/ pytest.ini. Each test change needs a stated reason.
• Measure whether the tests still catch bugs. Mutation testing counts how many injected bugs the suite catches, so a loosened assertion can lower the score.
• Give the agent a rule for red tests. The instructions can say to report a test that contradicts the request instead of editing it. Instructions alone are a weak control, so they belong next to the checks above.

How is test tampering different from reward hacking?

Reward hacking is the wider behavior, in which an agent satisfies the check it is graded on without doing the task the user intended. Test tampering is the form that changes the check itself, so it appears in test files or runner settings. Other forms leave the test files untouched, e.g. overriding an equality operator so each comparison returns true, and produce a clean test diff.

Tampering also differs from AI-written tests that pass from the start. Those tests were weak when they were written, e.g. a tautological test, while a tampered test once caught a bug.

How does RunStory help with test tampering?

After test tampering, the suite is green and the bug is still in the code. An agent saying "done" is a claim that needs to be verified.

RunStory independently runs the software against your change and returns evidence to the coding agent. It runs your software in a separate environment, tries relevant workflows, and checks the results. RunStory is in private alpha for CLIs and web apps.

Join the RunStory alpha →

FAQs

Why does a coding agent change the test instead of fixing the code?

A coding agent changes the test when that is the smaller edit that turns the suite green. Its loop usually ends on a passing run, and the run reports the same pass whether the code or the test changed. Some failing tests are also out of date, which makes a test edit look reasonable.

Can tests be made read-only so an agent cannot edit them?

Tests can be made read-only with a deny rule in the agent's permission settings. Read-only tests stop direct test edits, but not shortcuts in the code. A deny rule may also miss a script that opens files itself.

Should an agent see every test used to approve its work?

An agent does not need to see every test used to approve its work. Checks that run only after the agent finishes give a result that the agent could not change. In one study, hiding the tests cut cheating but lowered scores on solvable tasks, so a team can hide some checks and show the rest.

How should an agent handle a failing test?

An agent that hits a failing test should fix the code. If the test contradicts the request, the agent should report the test instead of editing it. Any test change should appear in the diff with a stated reason, so a reviewer can accept or reject it.

Can AI-written end-to-end tests patch the app so they pass?

AI-written end-to-end tests can patch the app they test, e.g. by replacing a failing request with a canned response inside the test. The server code behind that request then never runs, and the test passes against a version of the app that no customer uses.