Elegant

AI-generated tests need a human oracle

The Elegant team 5 min readai-and-frontend
AI-generated tests need a human oracle

Ask a model to “write tests for this function” and it will happily produce a dozen — well-structured, nicely named, and quietly worthless, because it derived the expected values from the code you gave it. If the code has a bug, the test asserts the bug is correct. This is the oracle problem: a test needs a source of truth for what the output should be, and that source cannot be the implementation under test. The model is genuinely good at the scaffolding of testing — arranging, mocking, structuring — and genuinely unable to supply the oracle. That part is yours, and confusing “the model wrote tests” with “the behaviour is verified” is how a bug ships with a green suite guarding it.

Deriving expectations from the code enshrines its bugs; the oracle must be external Left: the model reads the code and writes a test asserting the code's current (buggy) output — a circular check. Right: a human supplies the intended output from the spec, catching the bug. model as oracle (circular) code test asserts the bug is correct human oracle spec (intent) test catches the bug
If the expected value comes from the code, the test can only confirm the code equals itself. The oracle — what the output should be — has to come from the spec, i.e. you.

The circular test enshrines the bug

Here is the failure in miniature. The discount function has an off-by-one, and the AI-written test “verifies” it — by reading the buggy output and asserting it:

function applyDiscount(price, pct) { return price - price * pct; }  // pct is 0.1 for 10%… or is it?
// AI test, expectations derived FROM the code:
expect(applyDiscount(100, 10)).toBe(-900);   // asserts the bug (10 read as 1000%) as "correct"

Green suite, shipped bug. The test proved only that the code equals itself.

The oracle comes from intent, not implementation

A useful test encodes what the output should be according to the spec — a value you, the human, decide independently of the code. Write the expectation first, from intent, and the same test now catches the bug:

// oracle from the spec: "10% off 100 is 90"
expect(applyDiscount(100, 0.1)).toBe(90);   // FAILS on the buggy code — exactly right

The discipline is to state the expected value before looking at what the function returns, so the test measures the code against your intent rather than against itself.

Let the model scaffold, you supply the truth

This does not mean writing AI out of testing — it means splitting the work along the line of what it is good at. Let the model generate the structure: the describe blocks, the mocks, the arrange/act boilerplate, the list of cases worth covering (empty, null, large, concurrent — it is good at brainstorming these). Then you fill in every expected value from the spec, and reject any assertion whose expectation was obviously lifted from the implementation. A good tell: an assertion with an oddly specific magic number nobody would choose on purpose (toBe(-900)) is usually the model reading the code. The oracle problem is not an AI quirk to work around; it is a property of what a test is, and the model simply cannot be the oracle for the code it is testing. The harness-skill-eval exercise is exactly about writing the oracle — the eval that says what “correct” means — which is the part that stays human even as everything around it is generated.