Distribution shift
Will systems remain reliable in unfamiliar cultures, organizations, crises, and physical environments?
Underspecification means models that test identically can diverge in deployment. Reliability under shift is the classic ML safety problem, now with much higher stakes attached.
View on the map → · Open in Browse →
What changed
01
The reliability question is being recast as an engineering discipline: the AI Snake Oil group proposed a twelve-metric reliability profile for agents, borrowing from safety-critical engineering, precisely because benchmark scores and real-world incidents keep diverging. Toner's jaggedness essay remains the reference for why the divergence is structural — capability stays uneven across domains — and trace-level failure taxonomies show the breakdowns are behavioral (grounding, distractors, load) rather than knowledge gaps.
Recent thinking
Rabanser, Kapoor, Kirgis, Liu, Utpala & Narayanan · arXiv (Princeton CITP) · Feb 2026 paper
Towards a Science of AI Agent Reliabilitya holistic performance profile consisting of twelve metrics that decompose agent reliability along four key dimensions: consistency, robustness, predictability, and safety.
An attempt to turn agent reliability into a measurable engineering discipline, motivated by the gap between benchmark scores and real-world incidents — directly updating the underspecification/deployment-divergence question for the agent era.
Helen Toner · Rising Tide · 24 Nov 2025 essay
Taking Jaggedness Seriouslymaybe AI will keep getting better and maybe AI will keep sucking in important ways.
The reference essay for the jaggedness thesis: capabilities stay persistently uneven across domains, so reliability under shift — not average capability — determines where deployment works and why human-in-the-loop arrangements persist.
JV Roig · arXiv · 8 Dec 2025 paper
How Do LLMs Fail In Agentic Scenarios? A Qualitative Analysisfour recurring failure archetypes: premature action without grounding, over-helpfulness that substitutes missing entities, vulnerability to distractor-induced context pollution, and fragile execution under load.
Trace-level taxonomy of how agents actually break in messy, underspecified conditions — 900 execution traces showing scale alone does not predict robustness and that failure modes are behavioral rather than knowledge gaps.
Foundational reading (2)
Concrete Problems in AI SafetyAmodei, Olah et al. · 2016Underspecification in Modern Machine LearningD'Amour et al., Google · 2020