How much control should humans retain if AI systems consistently make decisions that appear wiser, safer, or more benevolent?
Danaher's 'algocracy': rule by systems whose reasoning citizens cannot contest. The modern twist is that deference could be freely chosen, one convenient delegation at a time, and still end somewhere irreversible.
View on the map → · Open in Browse →
The autonomy question got separated from the takeover question. A co-author of Gradual Disempowerment argues that even faithfully aligned AI could make keeping humans in the decision loop look irresponsible, so competitive pressure erodes self-government without anyone seizing power — the value-level worry distinct from the control mechanism tracked under Safety. The philosophy literature catalogues how 'algorithmic paternalism' undermines the preconditions of self-determination even when the machine decides well.
we still might end up optimising or heading at a civilisational level towards outcomes that no one wants
A co-author of Gradual Disempowerment develops the autonomy-value framing rather than the takeover mechanism: competitive pressure could make keeping humans in decisions look 'irresponsible,' eroding self-government even under benevolent, faithful AI.
The complexity and opacity of AI systems challenge human autonomy in special and unprecedented ways.
A philosophy paper cataloguing how 'artificial autonomy' and 'algorithmic paternalism' undermine the preconditions of self-determination — understanding, competence, voluntariness — even when the AI decides well.