Can systems reason accurately outside familiar patterns rather than merely producing convincing-looking chains of thought?
Reasoning models raise accuracy, but performance degrades under superficial variations of familiar problems, and stated chains of thought are often not faithful accounts of the underlying computation.
View on the map → · Open in Browse →
Competition mathematics stopped being the test: multiple models scored perfect marks on unseen IMO 2026 problems under official judging, and the benchmark group that guards against contamination declared final-answer problems saturated — while locating what remains unreliable in proof-writing, flawed-problem detection, and research mathematics. The perturbation literature keeps finding cliff-edge failures underneath the headline scores.
Previously, no large language model had ever achieved a perfect score under the IMO's official judging process.
Multiple models scored perfect marks on unseen IMO 2026 problems under official judging — the strongest counter-evidence to date that reasoning fails outside familiar patterns, at least in competition mathematics.
mathematical reasoning remains far from solved
The uncontaminated-competition eval group declares final-answer problems saturated while showing proof-writing, flawed-problem detection, and research-level mathematics remain unreliable — precisely locating where reliability now breaks down.
we observe sharp performance transitions under targeted structural interventions even when surface statistics are held fixed
Controlled-perturbation methodology extended to logical satisfiability: reasoning models show cliff-edge failures under structural (not surface) interventions — benchmark accuracy conceals brittleness regimes.