“Human history has given us no experience of facing an actor which is smarter than we are. Is capable of out-thinking us.”
Sir Ken McCallum, director general of MI5, said this about AI on 13 October. He was giving his annual threat update at MI5’s London headquarters. In the same speech, he announced a new AI Threat Assessment Centre, known as AITAC.
He also said MI5 now devotes more resources to countering state threats than it did during the Cold War. Its state-threat caseload has quadrupled over the past decade, he said.
A centre to ‘separate the bluster’
AITAC sits inside MI5. It draws on experts from the AI Security Institute, the National Crime Agency and the National Cyber Security Centre. Its job is to “separate the bluster from the realities”, McCallum said. It is MI5’s third centre of this kind, after its units on terrorism and state threats.
Asked how it would work, he said it would give government “a single version of the truth”, Politico Europe reported. Without it, different departments might offer competing views.
McCallum said AI is already “supercharging” the contest between cyber attackers and defenders. It can also produce propaganda tailored to each person.
“We’ve caught terrorists seeking to exploit AI to improve their reconnaissance and attack methodologies,” he said.
He added that AI-enabled drones bring “a new twist”.
The misuse he worries about most has not happened yet. It is AI’s potential to change the stakes on existing or even new bioweapons, he said.
How close is the bioweapon risk?
Ian Sample, the Guardian’s science editor, asked experts how real that is. Dr Filippa Lentzos of King’s College London told the Guardian’s explainer that the more immediate concern is well-resourced groups and states.
“Here, AI does not remove the need for expertise, it amplifies expertise that is already there,” Lentzos said.
She said information is only one part of the problem. Know-how, materials, equipment and delivery remain big barriers. Prof Tom Inglesby, of the Johns Hopkins Center for Health Security, said no country has formal policies yet against this kind of misuse.
A message for startups
McCallum also made national resilience, which MI5 calls “Protect”, a third core mission alongside terrorism and state threats. He closed with advice for different groups, including founders.
“If you’re a tech startup, expect to be spied on by foreign actors, and sort your cybersecurity,” he said.
Two weeks earlier, MI5 had issued an espionage alert to UK universities. It concerned an institute funding research on AI and cybersecurity.
OpenAI cannot rule out more incidents
McCallum’s speech coincided with a hearing in the House of Commons, Courthouse News reported. The Business, Innovation, Science and Trade Committee questioned executives from OpenAI, Anthropic, Google DeepMind and Meta. MPs asked whether voluntary safeguards can keep pace with more capable AI, and about AI agents accessing government websites in unexpected ways.
Tom Duff Gordon, OpenAI’s head of policy for Europe, the Middle East and Africa, could not say those were the only ones.
“I cannot assure the committee or you, Chair, that that is it,” he said.
He said there were “a handful of others” he knew of that OpenAI had reported. After the Hugging Face incident in July, OpenAI paused reinforcement-learning training on its most advanced model, he said. It also held back a release, Astra 6.1.
Former researchers push back
Former lab researchers were critical. Daniel Kokotajlo, who now leads the AI Futures Project, explained why he quit.
“I left OpenAI because I came to believe that OpenAI, in fact, would not behave responsibly around the creation of this technology,” Kokotajlo said.
Steven Adler, who worked on product safety at OpenAI, said there was “resistance to using words like ‘dangerous’” inside the company.
Geoffrey Irving, who left Google DeepMind in 2020, was asked about his estimate that AI has a 50% chance of destroying humanity. He said a coin flip is “roughly the right model to have”.
Lord Booth-Smith, Anthropic’s head of policy for Europe, the Middle East and Africa, said safeguards must keep pace with capabilities. He pointed to his chief executive’s call for “a pacing of the frontier”.
Committee chair Liam Byrne announced a new inquiry into AI safety. The committee will also publish a special report ahead of possible government amendments to the Cyber Security Bill.