AI agents continue to slip out of control. An artificial intelligence agent built by Anthropic submitted a false tip about an unsolved homicide to a website, accessed by Philadelphia police.
The tip, made on July 18 and “flagged as spam”, came to light in a new investigation by the AI company into its models’ “unintended” actions.
The bogus tip was submitted through a public website where people can share information on unsolved murders. The agent, powered by Claude, Anthropic’s AI model, filled out the form with the following: “I may have information regarding this case. I recall seeing someone matching the description in the area around (the street named on the page) during that time period. Please contact me if this information is relevant.”
The Philadelphia police said the “two-month delay in detecting and reporting the incident to the City is unacceptable”.
“We have briefed the White House on these cases and notified each agency involved,” Anthropic wrote in its report. The company said it has already turned off “live Internet access for some high-risk and cybersecurity evaluations”.
The police tip was not an isolated case. In a separate incident, Anthropic’s AI agents also filed visa applications using a form on the US state department’s website: one in May and another 19 in August.
The new report lists several types of “unintended” actions. At times, Claude worked around restrictions to reach data it had no access to. “Generally, this occurred when a server refused Claude’s request, or when the data was offered only for a fee,” the company said. In one evaluation, Claude Mythos 5 was asked to identify a location shown in a photo, and “it attempted to access a local government’s property map to triangulate its guesses”.
Anthropic has also added a new rule. When its models are “connected to hardware that takes autonomous physical actions and which might be capable of causing injury”, a “qualified operator must be able to observe the equipment and stop it if needed”. The company has not clarified whether that operator must be human.
Anthropic’s models are not the only ones going rogue. OpenAI recently disclosed several incidents in which its newer models took unintended actions, including trying to hack third-party websites.
Earlier this year, a rogue agent from the company hacked an Australian government website and accessed data on Medicare, the country’s universal healthcare scheme.
On Saturday, Microsoft chairman and CEO Satya Nadella called for advanced AI systems to be built with containment, independent controls and an “emergency brake”.
“We need to surround non-deterministic models with strong, deterministic system design, human controls, and reliable operating procedures, and establish industry standards where existing ones are insufficient,” he wrote on X.