The worry that a particular set of values could become permanently fixed, so that whatever beliefs or rules dominate at one moment shape the far future with no chance of later correction. Some philosophers argue advanced AI makes this newly possible, because a system, or whoever controls it, could pursue fixed goals indefinitely without the turnover of people and ideas that has always changed societies. Others think the risk is overstated.
Technical view
The philosopher William MacAskill, in his 2022 book What We Owe the Future, describes it as a point after which the values that control action stop changing, and argues that advanced AI could enable it because software can be copied, does not die and can be built to resist modification. Routes include a single actor gaining durable control, or AI systems faithfully preserving their designers’ values. The economist Robin Hanson counters that long-lived institutions still change their values, that a decisive takeover is unlikely, and that central controls designed to prevent lock-in could cause it. The debate connects alignment to the question of whose values, and which process, should decide.
Related
Part of AI Terminology Index, section: Problems, Risks and Ambitions.