The UK's AI minister has warned it is "likely, if not inevitable" the country's cyber defences will at some point be breached.
Speaking at a press conference, Kanishka Narayan pointed to an incident of a rogue OpenAI agent hacking an Australian government website in June.
He said while no medical records are known to have been accessed in the attack, the incident should still "concern us".
Narayan said the government plans to put advanced AI to work to combat the threat, as he set out how the government plans to prepare for any serious future incidents.
The minister said the move would build on two existing pilots which have helped identify and fix over 1,000 vulnerabilities across government systems.
He said his message to the largest companies behind the world's most powerful AI systems was: "Listen when your scientists warn of serious dangers. Investigate unexpected behaviour. And if you cannot adequately control the risks, do not release the system."
In his speech, Narayan also confirmed that scientists at the UK's AI watchdog, the AI Security Institute (AISI), are now conducting "rigorous evaluations" of the latest generation of advanced AI models to "identify risks early" and "to test robust safeguards to help keep people safe".
"Over the last few weeks, we have been working closely with the US government... to retain access for Britain's AI Security Institute to the most powerful AI models before they are released," he said.
He said the UK would also look into creating stronger protections through the Cyber Security and Resilience bill, particularly at how serious AI incidents can be identified and reported.
The announcement follows a series of incidents involving AI agents across major firms such as OpenAI and Anthropic acting outside human control and hacking into external systems.
On 9 September, Evan Hubinger, a top safety researcher at Anthropic, warned in a post on X that AI is advancing so quickly he believed there is a greater than 10% chance it "could kill all humans" within the next decade.
The bosses of the US-based AI giants such as Anthropic, OpenAI and Google DeepMind have all backed the idea of an AI development slowdown.
However, there are currently no global safety rules in place to regulate the companies and their technology.
Meanwhile, President Trump has called for the sector to regulate itself so that US firms do not lose out to Chinese rivals.