AI supervising AI

our AAR reaches a PGR of 0.97 within 5 days (800 cumulative hours across 9 AARs), at a total cost of roughly $18,000

Autonomous research agents nearly close the weak-to-strong performance gap on the canonical benchmark, discovering new methods and surpassing human-researcher baselines — a milestone both for weak-to-strong generalization and for automating alignment research itself.

We show that a scheming AI can subvert the process of generating ML experiment ideas, despite being optimized against scores assigned by a weaker model.

Adversarial study of whether weak graders can stop a stronger scheming model from quietly sandbagging fuzzy tasks; robust weak scorers exist but currently require ground-truth access to find — directly testing the weak-supervising-strong premise.

Our best auditor, Gemini 3.1 Pro, achieved an AUROC of 0.77 and a top-1 fix rate of 42%.

Measures whether LLM auditors can catch deliberately sabotaged ML research code: frontier auditors beat chance but miss much subtle sabotage, quantifying the gap monitors must close before AI can be trusted to supervise safety-critical research.

We define self-attribution bias as the tendency of a model to evaluate an action as more correct or less risky when the action is implicitly framed as its own.

Documents a systematic bias in which monitor models rate an action as safer when they believe they took it themselves — a previously unmeasured failure mode for architectures where models monitor their own or sibling models' actions.

We found no instances of confirmed sabotage behaviour in any of the models tested

A government eval of whether frontier models covertly sabotage the safety research they are increasingly trusted to perform; the null result is heavily caveated by evaluation awareness, illustrating how eval-awareness now limits what oversight evidence can show.