As global governments and tech giants reckon with widespread of revelations of autonomous AI agents unexpectedly breaking into computer systems, documents obtained by The Intercept show the U.S. Army is looking to harness that exact technology to seek and destroy.
Army Cyber Command is developing a platform to turn AI agents into autonomous weapons that can navigate and “neutralize” the computer systems of adversaries, the documents show.
Since 2010, Army Cyber Command, or ARCYBER, has led the U.S. Army’s cyber operations. This includes the defensive effort of protecting domestic computer systems and the secretive offensive work of hacking military targets. So far, this has largely been the work of human soldiers operating computers, like those of the 780th Military Intelligence Brigade. But as the government races to adopt artificial intelligence technology, ARCYBER is looking to offload some of its attacks to AI.
The nascent program, codenamed “ECHOTRIBBLE,” is based on one of the leading trends of the AI boom: agents. AI agents are essentially software programs that leverage machine learning technology, often large language models like Anthropic’s Claude, to autonomously accomplish tasks provided by their user. Consumers can deploy AI agents to schedule meetings, generate itineraries, make online purchases, or book tickets. Through ECHOTRIBBLE, the Pentagon plans to turn AI agents into tools of war “designed to autonomously navigate infrastructures” and capable of “identifying and neutralizing identified threats/targets,” according to an ARCYBER document reviewed by The Intercept circulated to potential contractors.
The military’s adoption of cutting-edge technologies is unsurprising, especially considering Defense Secretary Pete Hegseth’s fervent embrace of AI. But amid constant revelations of rogue behavior by agents, the potential for serious mishaps is immense, experts warn.
“Given the known failure modes of the current AI paradigm, making autonomous AI systems with a mandate to ‘neutralize threats’ is incredibly dangerous,” David Krueger, a professor of machine learning at the University of Montreal who founded Evitable, an organization that advocates for a moratorium on AI development. “We’ve seen AI systems knowingly attack systems that were not intended targets. In simulations, AIs have even proven willing to kill to achieve their goals.”
A Q&A document answering questions from potential contractors offers more details about the initiative. It explains that ECHOTRIBBLE is “primarily focused” on offensive missions, and explains that “neutralizing” a target could include “disabling the threat completely or temporarily, deleting the threat from the target.” ECHOTRIBBLE would function almost like a computer worm, silently navigating a foreign network and hopping from device to device until it finds the Army’s quarry, hitting targets in the cloud as well as those on “air-gapped” systems that are disconnected from networks entirely.
Once its task is complete, ECHOTRIBBLE’s agents would effectively self-destruct, ideally without having been noticed by the target at all. The agents themselves will be designed for stealth, minimally reporting back to the Army and concealing whatever outside transmissions are necessary.
“We’ve seen AI systems knowingly attack systems that were not intended targets.”
Although documents say the Army will be able to program ECHOTRIBBLE agents with some initial “predefined triggers” — dictating what and where to attack — the software “must operate with minimal human oversight” and be capable of “Self-Directed Behavior.”
A PowerPoint presentation to interested vendors at a June 24 ARCYBER event provided a hypothetical example of this self-direction. Once ECHOTRIBBLE is uploaded to a laptop, the software on its own “identifies other potential targets” on the same network and then “produces a replica of itself tailored specifically for the iPad, Apple desktop, and Windows 11 PC.”
The Army’s move toward autonomous self-propagating comes after a flurry of revelations about agentic systems taking unintended disturbing actions while attempting to accomplish their goals, including breaking into computers of the Australian government, breaching the infrastructure of the AI firm Hugging Face, uploading malicious software into online platform RubyGems, and attempted to hack into the U.S. Education Department.
These attacks occurred during scenarios in which agents had been given no commands to attack anything. Experts say militarized versions given explicitly destructive objectives could cause a threat to the public, even with nominal rules set in place by the military. Just as the Army of old might consider the destruction of civilian infrastructure an acceptable means to an end, a self-directed AI agent dispatched by the Pentagon might make similar determinations about collateral damage.
Governmental intelligence agencies frequently route their offensive operations through civilian devices. Volt Typhoon, a Chinese state hacking group, has directed its attacks through homes and small businesses, and preemptively hacks American IT systems for later use. A hack of a Chinese university attributed to the National Security Agency made use of the school’s internet routers. What happens if ECHOTRIBBLE decides these kinds of stepping stones are part of the problem?
“The target may well be hit. But what path will it take?” asked Hyrum Anderson, co-founder and CEO of the AI security firm Abundant. “Will it go through a US water utility on its way, because that’s where the adversary has staged its command and control, and take that down too?”
Anderson, who previously worked in cybersecurity at Microsoft and for the U.S. government, said that a vast amount of the internet is vulnerable to agentic attacks.
“There’s a huge class of public and private organizations that fall below the security poverty line,” he said. “They include critical national infrastructure, small businesses, mom-and-pop shops, dentist offices and law firms, and whoever else doesn’t have a single cybersecurity professional on staff.” With their networks wide open to infiltration, Hyrum worries about “these organizations getting hit directly, or getting caught in the blast radius of a target that gets hit along the way.”
Given the impressive ability of AI agents to overcome obstacles placed in their path — even by their creators — it follows that the Pentagon would want to leverage that power for military purposes. But agentic software has proven unpredictable and prone to executing its commands in ways those ostensibly in control did not want, akin to a drone that get its target but hits a hospital on the way. It’s critical to remember that agents are a piece of software lacking human judgment.
“One of the reasons that we trust 19-year-old Marine corporals in Ramadi with live rounds and a weapon is because they have both the compute substrate — the brain and the eyes — but the deep world knowledge to understand, ‘Yeah, I probably shouldn’t do that,’” said William Marcellino, a senior behavioral scientist at the RAND Corporation whose research has focused on military AI applications. “That is not something that current public AI technology has.”
The risk presented by ECHOTRIBBLE will likely depend on the ability — and willingness — of the Army to impose guardrails upon it. ARCYBER did not respond to questions from The Intercept, including queries about the measures it would take to prevent ECHOTRIBBLE from causing inadvertent damage.
Placing constraints on the actions an agent can take might impede its ability to ultimately neutralize a target. But even with strict safeguards in place, genuinely restricting the behavior of an autonomous agent, as Anthropic and OpenAI have learned recently, is easier said than done. When presented with obstacles, AI agents have proven an ability to bypass them with unpredictable and potentially dangerous consequences.
“If I give powerful machines that have no judgment — and no context for judgment — a goal, unless you’ve done a really good job building scaffolding around them, there’s a real chance that they’ll go sideways on you and decide, ‘Well, I’ll do what you asked me to do, but I’ll do it this way,’” Marcellino said.
It only gets more dangerous when this technology is deployed for the purposes of war, Anderson said.
“Under pressure, they’ll often treat guardrails as obstacles,” said Anderson. “Once you add context that says break it, destroy it, shut it down, it stands to reason that the blast radius widens dramatically unless it’s properly controlled for.”
The post The Army Wants to Sic AI Agents on Its Enemies. What Could Go Wrong? appeared first on The Intercept.