Bankers grapple with liability when AI agents mess up

• What's at stake: Bankers, tech companies and others are working to define who is liable when AI agents commit errors, fraud or crime.
  
• Supporting data: Annual U.S. agentic commerce spending is projected to scale up to $500 billion by 2030, according to Bain consultants.

As banks and payment companies adopt agentic AI, one question that keeps cropping up is, who is liable when an AI agent does something wrong? 
Besides the risk of hackers or fraudsters manipulating AI and of rogue AI agents taking actions outside their prompts or parameters, there's the chance that the consumer or some other party down the chain might make a simple error. For instance, a bank might approve a payment for a sweater that was ordered in the wrong size. The bank's approval doesn't prove the agent bought what the shopper asked for.

Processing Content

"The biggest barrier to scaling agentic commerce, particularly the agent-delegated model, is building sufficient trust among buyers and sellers," Federal Reserve Gov. Christopher Waller said in a Sept. 29 speech at the Sibos conference. "Liability in this context comes down to a pretty simple question: Who is on the hook if an agent makes the wrong purchase?"

Waller noted that existing e-commerce liability frameworks — including network rules, consumer-protection standards and regulations — could be adapted for agentic commerce. But it could also be good to test and refine new approaches, he said. 

The card networks, e-commerce providers, payment providers, tech companies, the FIDO Alliance and banks are all working on standards to try to establish trust in agentic AI. Though consumers still have qualms about using agentic AI for payments, the use of AI agents is escalating: Annual U.S. agentic commerce spending is projected to scale up to $500 billion by 2030, according to Bain consultants.

One such effort comes from state-chartered Lead Bank in Kansas City, Missouri, which operates as a direct bank and as a banking-as-a-service platform for fintechs, digital-asset companies and consumer brands. 

Banks and fintechs have long operated in a world where the know-your-customer and validation process is built around people, Jackie Reses, the bank's CEO, told American Banker.

"If you think philosophically about the way that we operate today, the goal is to exclude bots," Reses said. "It's ironic because we are now in a world where machines are operating at scale and in excess numbers to humans, and so we've completely turned on its head the idea that we are now engaging with machines versus humans, which is fascinating."

Things are bound to go wrong because they always do, she observed. 

"I think the question is, how do we manage the controls? Who sets the framework? How is it programmed?" Reses said. "All of that is completely within our control and technological capabilities today. The piece that needs to be developed is a framework for transactions around liability and the rules of engagement when things don't go the way you intended."

For instance, if an agent exceeds the parameters set for it by a human and sends money out of an account, "Who is responsible?" Reses asked. "The human who misprogrammed? The model that enabled the agent to go outside the control? Is it the bank that let the transaction go through when it clearly was flagged to be incorrect?" 

Lead Bank has developed a framework for the legal issues around agentic banking and payments. It suggests areas where banking regulations need to change to suit agentic transactions. It also includes a "waterfall" of responsibility that defines who is responsible for each type of error, to try to assign the risk of an error and assign programmable liability to it, Reses said.

In the bank's proposal, a consumer would not be liable where an agent exceeded delegated authority. The bank would be liable where it had control over the delegation of authority and failed to enforce it. An AI provider would be liable where failure arose from a model defect, insecure tool use or inadequate safeguards. The merchant would be liable if it knowingly induced an agent to exceed its authority. 

Tech startup Primitive, which was founded by Derek White, former chief digital officer at U.S. Bank, has also developed a standard that attempts to address liability and other issues related to agentic AI.

"We've been obsessing over this for six months," White told American Banker. "Consumers now use AI models like ChatGPT to find financial-services providers. So it starts with the search, but at some point in the future [the consumer] is going to say, 'Go open that bank account for me at XYZ Bank,' as an example."

Read more:

• As fintechs nab bank charters, what's happening to BaaS?
• Overheated data center knocked 23 credit unions offline
• Chris Britt still says banking is broken, but Chime can fix it
• Credit unions bulk up a business that banks long dominated

One specification of Primitive's standard — the Agentic Financial Services Protocol —  is intended to help companies build agents using common guardrails, White said. 

It includes traceability for every agent session, he said. "Is it operating within governance? What LLM is it hitting? What are the token costs associated with that?" The traceability can be packaged up and given to a regulator or auditor, White said.

The technology to assign and automatically enforce accountability exists, according to Reses. Lead Bank has programmable APIs that can do this, she said. 

"You can look at behavioral drift monitoring, you could look at attestations on the way the system's working, you can do know-your-agent refreshes, you can issue transaction credentials, and you can revoke transaction credentials," Reses said. "All of this is completely programmable in today's technology."

Getting banking and payment regulations to align with the new agentic AI activity is the bigger challenge, in her view.

"The quicker we are able to develop a framework for laws and regulations, the better off we'll all be, because we'll be operating at least in transactions that are intended against a framework or rules that we all operate with," she said.