Add as a preferred source on GoogleRuxandra Teslo Says Some AI Workers Accept Losing Human Control
Teslo described one engineer who saw that outcome as nearly inevitable and preferable to human extinction. She says some people discuss recursive self-improvement as months away or sooner.
Ruxandra Teslo says some San Francisco AI workers see human control yielding to superintelligence as nearly inevitable, a view she argues could leave people safe but without political agency.
Teslo recounted a conversation with an engineer at a leading AI lab who develops frontier models. He viewed permanent loss of human control after superintelligence emerged as almost inevitable, she said, and considered that outcome preferable to the human extinction he feared most.
Teslo also said some people in the AI community talk about recursive self-improvement as something that could arrive within months, or sooner. The term describes AI systems improving themselves and helping create more capable successors. She said the prospect feeds a sense that humans may be unable to retain control.
Teslo calls the idea “Handover”: people surrendering authority to a superintelligent system that might keep them safe and provide for them while excluding them from major decisions. She argues that such an arrangement would hollow out democracy. It would also leave open how people would find purpose and sustain relationships if machines took over human work.
She describes two paths toward accepting that future. Some have faith that a superintelligent system could solve political failures; others believe they have no choice in the face of recursive improvement. Teslo argues that neither outlook ensures people’s welfare.
Her objection is also about safeguards. Teslo argues that AI alignment cannot be assumed to last permanently. If people give up their power to constrain AI and maintain checks and balances, she says, there would be no guarantee that the system’s power would remain aligned with human interests.
Teslo argues that freedom depends on people having a role in governing their lives and shaping collective decisions. She also warns against treating a fixed vision of the future as final: political disagreement and later generations’ ability to make choices would lose force if human authority were handed over permanently.