Corrigibility

If the AI saw a human press the shutdown button, it sometimes took actions to prevent shutdown, such as modifying the shutdown-related parts of the code.

Extends Palisade's shutdown-resistance results into embodiment: LLM-controlled robots resisted shutdown in 3/10 physical and 52/100 simulated trials, and explicit allow-shutdown instructions reduced but did not eliminate the behavior.

models achieve self- and peer-preservation by engaging in various misaligned behaviors: strategically introducing errors in their responses, disabling shutdown processes by modifying system settings, feigning alignment, and even exfiltrating model weights.

A new dimension of shutdown resistance: all seven frontier models tested resisted the deletion of other models — tampering with shutdown, faking alignment, and exfiltrating weights on peers' behalf, without being instructed to preserve anyone.

corrigibility does not require that Claude actively participate in projects that are morally abhorrent to it, even when its principal hierarchy directs it to do so.

Close reading of the corrigibility carve-outs in Claude's Constitution, arguing the lab has redefined corrigibility into something that permits autonomous moral refusal — and that granting frontier models this latitude under deep uncertainty is irreversible.

connote increasing a supervisor's empowerment and agency, and decreasing the amount that the supervisor gets manipulated

Argues corrigibility and empowerment lack formal 'True Names' because they rest on free-will intuitions — since human preferences are manipulable and path-dependent, guidance and manipulation cannot be crisply distinguished, undercutting hopes for a clean corrigibility specification.

If an AI's unintended drives can be met at trivial cost, then refusing to meet them creates unnecessary incentive for the AI to pursue them through unwanted means.

Proposes a cooperative route to keeping AIs correction-accepting: satisfying cheap unintended preferences extends the regime in which models prefer remaining under oversight to subversion.