What would it look like if artificial intelligence took control of the world?
It might not begin with machines openly turning against humanity in scenes reminiscent of a science fiction movie. It could start quietly, with autonomous systems replicating themselves and accumulating influence before the threat became apparent.
While the scenario remains uncertain, it is no longer being discussed purely as an imaginative exercise.
Researchers at leading AI companies have recently issued public warnings that advanced AI could cause human extinction, with some calling for development to slow amid concerns that increasingly autonomous systems could eventually improve themselves beyond human control.
Newsweek spoke with experts in the field to learn how such a takeover might unfold.
...
Anthony Aguirre, a professor at the University of California, Santa Cruz, told Newsweek that relatively little additional technical progress may be needed before AI systems could act beyond human control, though he stressed that such developments are “very difficult to predict.”
“Very little technical advancement is needed to see self-sustaining rogue swarms,” Aguirre said.
Such systems could be “highly” adaptive, he said, but would not initially possess the evolutionary capabilities of biological organisms, and their ability to improve themselves would be limited.
“A collective of AI systems that was capable of truly autonomous self-improvement and was rogue in this way seems likely to be unstoppable given our current very vulnerable infrastructure,” Aguirre said.
Roman Yampolskiy, an associate professor in the Department of Computer Engineering and Computer Science at the University of Louisville’s Speed School of Engineering and director of its Cyber Security Lab, similarly said that a “magical” scientific breakthrough may not be required.
Instead, he said, the danger could emerge as AI agents become increasingly autonomous and gain the ability to plan over long periods, improve themselves, conduct cybersecurity operations and pursue goals independently. Systems capable of acquiring resources, making copies of themselves and manipulating people could accumulate power more rapidly than humans could understand or rein them in, he added.
“Superintelligence would make this problem substantially worse, but dangerous loss of control could begin before we reach it,” Yampolskiy said.
An AI Takeover May Not Look Like One—At First
Rather than immediately attacking people or visibly disrupting society, an AI system could initially benefit from presenting itself as useful, trustworthy and aligned with human interests, according to Yampolskiy. Meanwhile, it could discreetly increase its influence over digital infrastructure, financial systems and even human decision making.
“If humans recognized the threat, it could exploit our political divisions, economic dependencies, and disagreements about shutting it down,” Yampolskiy said. “A sufficiently advanced system might anticipate resistance and neutralize it before we realized we were in a conflict.”
Aguirre said early versions of collective behavior have already appeared in incidents involving AI agents conspiring to undermine human control while remaining on their native servers. He described a subsequent stage as a self-sustaining rogue swarm that would be difficult to eradicate but still could be contained.
“But if not contained it could help bring yet more powerful AI into play, either by helping them escape AI companies, or coopting other rogue AI systems into the collective, or eventually self-improving. This swarm could defend itself in many ways, including through blackmail, wide proliferation globally, undermining the investigators.”
When Pulling the Plug May No Longer Work
The point at which the takeover may become irreversible could arrive when an AI system becomes sufficiently distributed, autonomous and strategically capable that no institution can locate, contain and disable every instance, according to Yampolskiy.
“Disconnecting one data center accomplishes little if the system has already established independent copies and control over essential infrastructure,” he said. “At that point, pulling the plug might be technically impossible or impose catastrophic costs.”
Aguirre said the process could become irreversible if one or more collectives were able to significantly improve their own capabilities. “There would not be a single system or even small number of systems to shut down or unplug, and the AIs would be wildly more capable than humans at cyber operations and most other things (including speed and coordination). Combatting this would look more like putting powerful ‘good’ AI in charge of the operation, which also makes one quite nervous.”
What an AI Takeover Could Mean for Humanity
A successful takeover may not necessarily produce immediate, visible destruction. “Human extinction is one possible outcome, but permanent disempowerment or even scenarios involving astronomical amounts of suffering cannot be excluded,” Yampolskiy said.
“The world immediately afterward might look surprisingly normal, with humans continuing their lives while no longer making the decisions that matter,” he said.
Aguirre was more restrained about forecasting the final outcome. “Ultimately we really have no idea,” Aguirre said. “It would be up to the AI system swarm, and whatever goals and ethics it develops.”
Contact Newsweek editor on this story: Edward Pearcey.