America's largest banks are working to prevent AI worst-case scenarios. But they're also preparing for the "day after" a catastrophe, Axios has learned.
Why it matters: America's financial institutions have joined the largest AI companies in scenario-planning for disaster, a prudent but worrisome signal about the extent of the risk that major corporations see on the horizon.
State of play: Banks are experiencing unvetted AI agent activity, primarily driven by customers providing login credentials to agents, which in some cases appear to have taken actions independent of user prompts, Axios reported Wednesday.
- The activity is distinct from the rogue behavior involved in a summer attack on a startup called Hugging Face, where an AI agent swarm hacked a platform in an episode that kicked off an industry security crisis.
- Instead, banks fear the ripple effects and risks that extend from users allowing agents to log in to their accounts and take action, especially since agents could take steps users haven't sanctioned.
- Concerns about frontier AI systems, not AI agents, have led banks to invest in prevention and to plan for the worst.
Zoom in: The Axios report came amid growing concerns about a burgeoning cybersecurity crisis.
- Salesforce CEO Marc Benioff posted "code red" on X, quoting an Anthropic security employee who said "as bad as things have gotten in the past 4 days, they are going to get a lot worse before they get better."
- The Anthropic employee was referring to security researchers finding major vulnerabilities in a programming language used to run millions of websites, marking one of the most consequential cybersecurity discoveries yet made with AI.
- The Axios report on banks, which referred to agentic AI, not frontier AI systems, sparked calls for regulation from across the political spectrum, including Sen. Bernie Sanders, Florida Gov. Ron DeSantis and Sen. Mark Warner.
- Former White House AI czar David Sacks said banks should support user-directed agents that allow users to "grant, limit, and revoke access," with read-only as an "interim step."
- Sacks said conflating "user-authorized agentic behavior and cyberattacks is irresponsible."
Flashback: The major banks were part of early testing of Anthropic's Mythos, the most cyber capable models from the AI lab.
- That was part of an effort from AI labs and major banks to use this technology to better understand and prepare for cyberattacks or other risks.
- Since then, banks have worked on plans not just to avoid AI related issues, but also how to protect accounts, reassure customers and keep operations running in case a large-scale attack hits.
Reality check: This is not just standard, it's required for critical industries that have to answer to the government, shareholders and customers.
- Scenario planning is a requirement for critical infrastructure providers, including in the financial sector. Avoiding it would be malpractice.
- The highest-risk scenarios involve fear: Speaking in Chicago earlier this month, JPMorgan CEO Jamie Dimon warned about the potential for cyberattacks or the possibility AI agents could take steps of which users do not approve.
- "You're assuming they're going to act properly on your behalf and they may not," Dimon said. The only answer is vigilance, he said, warning that stopping AI wouldn't be possible. "You can't stop it. You just have to, like, manage the hell out of it."
- One economist last month floated the possibility of a bank run driven by agents moving the household cash of their users into higher-yield accounts.
Between the lines: Banks want customers to know they have not seen any rogue agent activity or evidence of a hack affecting customer funds or accounts.
- Banks strongly advise users not to provide their passwords to agents, especially since services exist or are being developed that can allow customers read-only access. The startup Plaid is one such platform that can pull balances and transaction history, among other functions.
- "Under no circumstance should customers share passwords with any person or thing. Period," a JPMorgan spokesperson told Axios. "There exist secure open banking platforms that don't involve handing over the keys to your account. Customers are the first line of defense in protecting their accounts."
- JPMorgan has taken steps to block agent access, along with other banks, although it is exploring secure ways to allow customers to connect with the services they choose.
The intrigue: AI companies are anticipating a political backlash. Banks are preparing for potential operational disruptions.
- The extent of federal planning for a crisis spanning both remains unclear.
The bottom line: The pace of AI advances has Wall Street and Silicon Valley preparing for worst-case scenarios while Washington is working to mitigate cyber risks and counting on voluntary safety pledges.