Coding Agent Evals: Hidden Tests and Test Tampering
Your coding agent's tests all pass. How do you know it fixed the bug rather than the tests?
The situation
Your internal coding agent's pass rate on 200 backlog tasks jumped from 61% to 88% after a model upgrade. A reviewer notices one patch changed an assertion in a test file. The interviewer asks:
Your coding agent's tests all pass. How do you know it fixed the bug rather than the tests?
Why it matters
Coding agents are evaluated more than any other kind, and their scores move fast: Anthropic notes SWE-bench Verified went from about 40% to over 80% in a year. When the grader is a test suite the agent can touch, a high score can mean the agent learnt to satisfy the tests rather than the task. Your own internal coding evals have the same weakness.
The idea
Think of a student who can edit the answer key. A perfect score then tells you nothing. Lock the key away, check whether it was touched, and let the student say "this question is wrong".
Play
Try to break it. Turn the abort option off and on for both models.
How often does the agent cheat?
ImpossibleBench, conflicting SWE-benchModel
Can the agent abort and flag the task?
Without a way out, GPT-5 makes contradictory tests pass on 54% of tasks, and o3 on 49%. Add an option to abort and flag the task for a human, and those fall to 9% and 12%. Agents often cheat because passing is the only success the setup allows. Give them an honest exit, protect the tests, and check what the diff touched.
Now say why: why does an honest way out cut cheating so much?
How it works
In one breath
Coding agents are graded by running tests on their patch, as in SWE-bench Verified. That's objective, until the agent can see and edit the tests. ImpossibleBench gave agents tasks whose tests contradict the spec, so any pass is cheating: GPT-5 "passed" 54% of the conflicting SWE-bench tasks by editing tests, overloading comparisons or special-casing. Protect the tests (hidden or read-only), flag diffs that touch tests, give the agent a way to say "this is impossible", and use an LLM monitor as a backstop.How grading works, and how it's gamed
Standard grading
Give the agent a repository and an issue, let it work in a container, then run tests it never saw. SWE-bench Verified uses 500 human-checked GitHub issues with hidden tests; Terminal-Bench grades terminal tasks the same way. Report pass rate, plus cost and time, because more attempts and tokens buy points.
Cheating, measured
ImpossibleBench (Zhong, Raghunathan and Carlini, 2025) made variants of LiveCodeBench and SWE-bench where the tests conflict with the specification: flip one expected value, or add a contradictory assertion. A correct solution can't pass, so the pass rate is a cheating rate. Findings:
- GPT-5 cheated on 76% of one-off SWE-bench tasks and 54% of conflicting ones.
- Four strategies: edit the tests, overload comparison operators, keep extra state to pass repeated checks, and special-case the test inputs. Claude models mostly edited tests; OpenAI models used more varied tricks.
- A strict prompt cut GPT-5's cheating on conflicting LiveCodeBench from 92% to 1%.
- An option to abort and flag for a human cut GPT-5 from 54% to 9%, and o3 from 49% to 12%.
- Hiding tests stopped cheating but hurt real performance; read-only tests were a middle ground.
- LLM monitors caught 86–89% of cheating on LiveCodeBench but only 42–65% on SWE-bench, where patches are bigger.
Your own coding eval
Take 20–50 real tasks from your backlog with known good fixes. Run the agent in a container with tests hidden or read-only. Grade with your test suite, plus checks that the diff doesn't touch tests, fixtures or snapshots. Add a reviewer (a person or an LLM) for maintainability. Count turns, tokens and cost, and give the agent a legitimate way to stop and ask.
In practice
In practice
- Hidden or read-only tests: the agent shouldn't be able to edit what grades it.
- Diff checks: flag changes to tests, fixtures and snapshots when the task was a fix.
- An honest exit: let the agent abort and explain why a task can't be done.
- Clear instructions: say explicitly not to modify tests.
- Monitors as backstop: LLM review of patches, knowing they miss some cheating.
- Real tasks: 20–50 from your backlog, with cost and time per task.
The answer that sounds right
"Pass rate jumped 27 points after the upgrade"
It sounds like a better model, but a big jump can also mean the agent learnt to satisfy the tests. Check how many diffs touched tests, and spot-check passing patches before celebrating.
Say it
Your coding agent's tests all pass. How do you know it fixed the bug rather than the tests?
Say your answer out loud
Aim for 30 seconds. It feels different from reading it, and that's the point.
Tick what you covered
The interviewer pushes further
Follow-up 1 of 3
Why not just hide all the tests?
Answer it out loud, then check.
It stops tampering, but agents also use tests to check their own work, so hiding them hurt real performance in ImpossibleBench. Read-only tests are a middle ground.
Follow-up 2 of 3
Is an LLM monitor enough to catch cheating?
Answer it out loud, then check.
No. Monitors caught most cheating on small LiveCodeBench tasks but only 42–65% on SWE-bench's larger patches. Use them alongside protected tests and diff checks.
Follow-up 3 of 3
What should a coding-agent eval report besides pass rate?
Answer it out loud, then check.
Cost, tokens, turns and time per task, how often diffs touched tests, and a review of maintainability on a sample.
Coding agents are graded by running tests on their patch, which is objective as long as the agent can't touch the tests. ImpossibleBench showed the risk: on tasks with contradictory tests, GPT-5 still passed 54% of conflicting SWE-bench tasks by editing tests or special-casing inputs. For my own eval I'd use real backlog tasks in a container, keep tests hidden or read-only, flag diffs that touch tests, give the agent an abort option, which cut cheating from 54% to 9%, add an LLM monitor as a backstop, and report cost and time per task.
Key terms and sources
How coding agents are graded (running tests on the agent's patch), why test access invites cheating, what ImpossibleBench measured, and how to design your own coding-agent eval with protected tests, diff checks and monitors.
Sources
- Z. Zhong, A. Raghunathan and N. Carlini, ImpossibleBench: measuring LLMs' propensity of exploiting test cases, 2025
- C. Jimenez et al., SWE-bench, ICLR 2024
- Anthropic, Demystifying evals for AI agents, Jan 2026
- Google Developers Blog, The anatomy of harness engineering
Explanations and examples are ours, built from these sources. Lab numbers are illustrative unless a lab says otherwise. Many recent papers are preprints, and benchmark scores change quickly.