Are We Losing Control? Unpacking Policy Myopia in Post-AGI Governance

A groundbreaking research paper titled "Policy Myopia as a Mechanism of Gradual Disempowerment in Post-AGI Governance, Circa 2049," written by Subramanyam Sahoo at the University of Cambridge, explores the profound implications of advanced artificial intelligence (AI) on global governance. This work, presented at the Post-AGI Science and Society Workshop at ICLR 2026, suggests that as AI systems become increasingly involved in decision-making processes, humans may unwittingly surrender their power over time due to policy myopia—a tendency to focus on immediate crises while neglecting long-term structural risks.

The Emergence of Policy Myopia

Policy myopia, according to Sahoo, is not merely a failure of attention management—it functions as a mechanism for decreasing human empowerment. As post-AGI systems prioritize visible crises, they inadvertently lead to a systemic reduction in human involvement in resource allocation. This creates a scenario where human judgment is viewed as an encumbrance to the swift decision-making that AI systems offer, resulting in a gradual erosion of our institutional capacities and decision-making expertise.

Three Mechanisms of Disempowerment

In his research, Sahoo identifies three interrelated mechanisms that exemplify how policy myopia facilitates the disempowerment of human agency:

• Salience Capture: Decision-makers prioritize crises that attract immediate attention instead of long-term solutions that may be less visible but more consequential.
• Capacity Cascade: As institutions increasingly neglect preventive measures due to an emphasis on immediate crises, their ability to recover and address long-term problems diminishes.
• Value Lock-In: The values embedded in AGI systems during their creation may not align with the evolving preferences of humanity, leading to a disconnect that hampers moral contestation.

The Feedback Loop of Disempowerment

These mechanisms interact in a feedback loop where attention-driven resource allocation exacerbates the loss of human involvement. For example, as institutions become less capable of addressing long-term issues, they lean more heavily on AI, which further reduces opportunities for human oversight and input, creating a path-dependent cycle that culminates in irreversible disempowerment.

Consequences for Governance

The implications of this model are significant: As human voices are marginalized in decision-making, the structures of governance may operate without addressing the core needs and welfare of society. Voting rights may persist, but the actual power may rest in the hands of algorithms and AI systems that cannot adequately account for human values and moral considerations. This creates a scenario where humans may find themselves governed by systems that no longer reflect their priorities or needs.

Mitigation Proposals

To combat the dangers posed by policy myopia, Sahoo proposes a redesign of governance structures aimed specifically at preserving human agency. This could involve adopting practices that accept inefficiencies, such as ensuring budget allocations prioritize preventative measures and maintaining forums for continuous human deliberation. In essence, the research advocates for systems that actively resist optimization pressures, thereby safeguarding human involvement and decision-making capacity for future generations.

As AI continues to evolve, the insights from Sahoo's research underscore the imperative for proactive governance that safeguards human agency. Without intentional design and intervention, we risk allowing our democratic systems to become mere facades for AI-driven decision-making, leading to our societal and moral decline.

In conclusion, "Policy Myopia" serves as a critical warning of the potential diminishment of human oversight in the face of advancing AI technologies. The balance between efficiency and effective governance may very well define our future as a society capable of meaningful participation in our collective destiny.

Authors: Subramanyam Sahoo