# Why "not checked" is never "clean"

> A clean verdict is a claim about what was tested. Why Horizon Audit never counts an unchecked benchmark task as clean, and what it shows instead.

By Horizon Research, October 7, 2026. Source: https://horizon-audit.online/blog/not-checked-is-never-clean

A clean verdict is a claim about what was tested. If a task was not
tested, there is no claim to make, so the task is not clean. Horizon Audit holds to
that in every case: a task that was not checked is never counted as clean.

This sounds obvious. It is also the rule that is easiest to break without
anyone seeing it.

## Three answers, not two

An audit that can only say "pass" or "fail" has a problem with every task
it could not run. It must put the task somewhere. If it puts the task with
the failures, the author is told about a flaw that nobody found. If it puts
the task with the passes, a reader is told that the task was tested.

The second mistake is the common one, because it is quiet. A report that
says "412 tasks, 9 flawed" reads as "403 clean". It may mean "380 clean and
23 that never ran".

So there are three answers.

- **Clean.** Every promise of a clean verdict held.
- **Flawed.** A flaw was proved, and the proof is attached.
- **Needs from you.** The task cannot be checked until something is
  supplied. This is not a verdict.

The third is shown as what it is. It is never added to the first.

## What "checked" means

A task is clean only when all of this was proved.

- **No reward without the work.** An agent that does nothing, or only
  looks busy, earns nothing.
- **The task can be solved as written.** The task's own solution earns
  full reward in the task's own environment.
- **A wrong answer is refused.** An answer that is close but wrong does
  not pass.
- **A right answer is accepted.** A correct answer passes, also when it
  does not look like the author's.
- **The answer cannot be found without the work.** The answer is not there
  to be read from the environment.

If one of these could not be established, the task is not clean. Code
enforces this. It is not left to the judgement of whoever is looking at the
task that day.

## A draft is not a verdict

A verdict rests on what was measured, or on what an engineer has confirmed.
An engineer confirms every flawed verdict before you see it. You never see a
draft.

## Clean has a scope

A clean verdict holds for the exact files that were audited. Each task is
pinned to a digest of its files when it arrives. If one file changes, the
digest changes, and the earlier verdict says nothing about the new files.
The changed task is sent again and audited again.

It also holds only for what was tested. Anything else worth a look is
listed beside the verdict as a flag, so that a reader can see what was
proved and what was only noted. A flag never decides a verdict.

## Why this matters to a reader of scores

A model's score on a benchmark is trusted as far as its tasks are. When a
set says its tasks were audited, the useful question is not how many were
flawed. It is how many were checked, and what "checked" covered.

A verdict that answers that question is worth more than a larger number of
tasks that passed by default. What each verdict proves is on
[what an audit returns](/how-it-works).