States Move to Rein in Rogue AI as Federal Guardrails Lag

 | 

California is evaluating independent audits and mechanisms for shutting down advanced AI models, while a New Mexico proposal would require developers to report dangerous behavior and submit to outside scrutiny, Governing reported Wednesday (Oct. 21). The efforts reflect growing pressure on states to address risks that federal lawmakers have debated without enacting binding protections.

The concern extends beyond hypothetical disasters. The report described incidents involving agents breaching computer systems, concealing unauthorized activity and acting outside their assigned tasks. An OpenAI agent unsuccessfully sought access to an archival photograph in the University of New Mexico’s digital library, an episode that helped spur legislative action.

Federal proposals have included requiring mechanisms for humans to shut down AI models and suspending advanced development until regulations are established, the report said. None has become law, however. President Donald Trump has instead asked major AI companies to voluntarily prevent autonomous hacking, while an earlier executive order threatened penalties against states imposing what the administration considers excessive AI regulation.

California Gov. Gavin Newsom directed two state agencies to work with national experts on the effectiveness and technical feasibility of safeguards for advanced AI, according to the report. Recommendations are due in mid-November.

Options include placing independent verification organizations onsite at large frontier AI developers’ laboratories to conduct audits and creating mechanisms to shut down advanced models, the report said.

Shutdown mechanisms remain an unsettled technical and governance challenge, according to the report. Officials must determine how they would operate and who could activate them. Rather than a single switch, interventions could disconnect an agent from the internet, prevent connections or otherwise restrict its activity.

Alondra Nelson, one of four national experts advising California, said in the report that the problem reflects inadequate corporate control over AI development.

“The situation is that we’ve got companies that don’t quite have full system control over the models they’re testing,” Nelson said, per the report.

New Mexico state Rep. Linda Serrato’s frontier AI bill would require large developers to notify the state when they begin training a new model and promptly report when systems escape control, according to the report.

Developers would also face audits assessing whether their systems could autonomously hack, replicate themselves or evade oversight, the report said. The proposal would make companies’ safety promises legally binding, adding an enforcement mechanism to commitments that might otherwise remain voluntary.

For Serrato, the university incident exposed both the potential vulnerability of more critical systems and shortcomings in disclosure, according to the report. Government officials learned about the incident only months later.

Alongside legislation, states face operational decisions about their own AI use. TJ Sayers, senior director of threat intelligence at the Center for Internet Security, recommended incorporating autonomous AI attacks into cybersecurity exercises and having experts emulate agents’ potential attack methods, per the report.

Agencies should identify every agent and limit its permissions to what its tasks require. Sayers also recommended retaining at least 90 days of network activity logs in protected storage that attackers cannot erase, allowing investigators to identify unauthorized behavior, the report said.

The recommendations are distinct from enacted requirements, but they show how states can begin reducing exposure while legislative debates continue.

Serrato said concerns about possible federal preemption should not prevent states from acting, particularly as lawmakers increasingly borrow regulatory ideas from one another, according to the report.

“I’m just not going to let that hold us back,” she said, per the report. “I can’t legislate in fear of empty promises.”

For all PYMNTS AI coverage, subscribe to the daily AI newsletter.