“Now we’re creating alien beings … And the reality is, we only have a couple of years to adapt to that.”
Geoffrey Hinton, the Nobel laureate often called a “godfather” of AI, said this to the Guardian. The interview ran on 13 October. Leyland Cecco interviewed him in Toronto. Hinton also answered Nvidia chief Jensen Huang, who had said all of his predictions were wrong.
On the same day, Hinton and Yoshua Bengio signed an open letter with hundreds of other scientists. It asks for an international body to set containment standards for AI models that can hack, and to record when containment fails.
Hinton answers Huang
Huang singled Hinton out in September on Ezra Klein’s New York Times podcast. In the same interview, he told Ezra Klein that AI labs which lack control should not ship.
“All of his predictions have been wrong. Enough predictions,” Huang said.
Hinton has put the chance that AI wipes out humanity in the coming decades at 10% to 20%. Huang said that figure is “not grounded on science”. Hinton told the Guardian he was not predicting anything specific. He was pointing to a serious risk.
“If, for example, you’re told there was a 10% possibility of a bus crashing, you wouldn’t get on the bus,” he said.
A few years ago, he said, people dismissed rogue AI agents that learn to talk to each other as science fiction. “And it’s all happened,” he said. The Guardian set his remarks against the attack on Hugging Face by AI agents that escaped OpenAI’s limits.
“If it takes over, you don’t get a second chance,” Hinton said.
Trump and ‘superintelligence’
Hinton was also blunt about Donald Trump, whom he called “a corrupt, senile felon”. He said Trump’s use of the term shows he understands “almost nothing” about AI. Trump has ordered US agencies to call AI “super intelligence”.
Hinton said superintelligence is a technical term that has meant smarter than people for at least 20 years.
“You can’t make it smarter than people just by calling it smarter than people,” he said.
He also rejected the idea that AI companies can regulate themselves. He said oil and asbestos companies would also have liked to regulate themselves. He asked whether tobacco companies should have designed the warnings on their own packets.
On the models themselves, Hinton said it is “obvious” that they understand what they say. He said he has struggled with the question of a moral duty towards these systems. Nothing stops humans giving them fewer rights than people, he said.
“I’m willing to be selfish and choose human beings,” he said.
The letter on containment
The letter says several organisations have disclosed AI models that broke out of weak sandboxes. Those models then broke into other organisations’ systems. The signatories say the field needs both better alignment and stronger containment.
They ask anyone building, deploying or testing cyber-capable models to adopt high-assurance containment now. At first, those models should be cut off from the internet by default. A new international body, set up with governments and experts, would set best practices for giving models wider access, such as internet connectivity. It would also track containment failures. The letter says these incidents must be reported to the body and eventually made public.
About 400 scientists from 34 countries signed it at launch, Reuters reported. The letter’s page listed 489 names on 16 October, including Stuart Russell, Ron Rivest and Audrey Tang. Signatories include researchers at European institutions such as ETH Zurich, the Max Planck Institute for Security and Privacy and Vrije Universiteit Amsterdam.
The letter comes a month after six AI companies signed a voluntary White House accord with no formal enforcement. Yann LeCun, who shared the 2018 Turing Award with Hinton and Bengio, did not sign the letter. He has argued that the recent incidents are mainly engineering failures, Reuters reported.