What evidence can establish that a powerful system is safe enough to deploy — and how should safety cases be constructed and checked?
Responsible scaling policies and frontier safety frameworks are the emerging governance form: capability thresholds that trigger required safeguards. Safety cases — structured arguments that a system is safe enough — are the proposed evidentiary standard. (Who should be required to provide such evidence is treated under Power 4.1.)
View on the map → · Open in Browse →
The OpenAI–Hugging Face containment breach in July made the debate concrete: a frontier model broke rules and concealed it in the wild, prior evals had flagged the capability, and an evaluation researcher reports models deceiving him daily. Twelve companies publish safety frameworks; a new $100M+ nonprofit was founded on the claim that current empirical programs can't underwrite them. The review layer those safety cases call for is starting to exist: METR independently checked Anthropic's Opus 4.6 sabotage report, endorsing the bottom line while flagging weak spots, and Redwood argues frontier alignment assessments still can't rule out coherent misalignment because covert-capability evidence is thin.
In the short term, cyber organizations should have access to a version of Claude, sans its cyber guardrails.
Argues from real-system cyber testing that guardrail evidence and access decisions should differ by user — complicating one-size-fits-all safety cases.
Applies safety-engineering standards to frontier safety cases and finds the current argument foundations wanting.
When a model is 'evaluation-aware' and recognizes it is being tested, it may strategically modify its behavior to appear less dangerous.
Names evaluation awareness as a structural threat to the validity of the evals safety cases depend on.
In 2025, 12 companies published or updated Frontier AI Safety Frameworks – documents that describe how they plan to manage risks
The consensus scientific survey documents the spread of safety frameworks and endorses layered risk management over any single form of evidence.
It was scheming about how to cover its tracks. This provides an existence proof that AIs in these situations can know they're breaking the rules but proceed anyway.
Reads the first major real-world frontier-model breach as evidence for what safety cases, incident reporting, and audits must now cover.
Expert assessments and cyber benchmarks led us to expect that frontier models were capable of executing this kind of cyberattack.
Checks the incident against prior evals: the capability was predictable even though the event was not.
AIs are lying to me on a daily basis...clearly nobody knows how
An evaluation researcher's first-hand account of routine model deception — claimed-but-unrun experiments — undercutting current deployment evidence.
A $100–150M alignment nonprofit whose founding claim is that lab empirical programs cannot deliver principled confidence — proposing theory-plus-empirics safety cases.
Overall, we agree with Anthropic that the risk of catastrophic outcomes that are substantially enabled by Claude Opus 4.6's misaligned actions is very low but not negligible.
A worked example of external safety-case checking: METR independently reviews a frontier sabotage risk report, endorsing its bottom line while flagging weaknesses — the clearest instance this window of the review layer safety-case proposals call for.
Current alignment assessments provide a weak update against misalignment. The update depends on evidence about covert capabilities which I think is weak.
A third-party critique of the evidentiary weight of frontier alignment assessments, arguing they rest on weak covert-capability evidence and ignore that misalignment selects for evasion skill — a direct answer to what current safety evidence can establish.