Utah is expanding its artificial intelligence sandbox with three healthcare pilots. At the same time, regulators are developing answers to questions about patient protection, such as who bears liability when an automated system causes harm, what evidence establishes its safety and where state experimentation meets federal oversight.
The new pilots will test AI that refills prescriptions for chronic conditions, writes initial acne prescriptions and provides virtual physical therapy, Nextgov/FCW reported Friday (Oct. 16). They expand a program that allows companies to test technologies with regulatory relief and state supervision, creating a proving ground for medical applications whose oversight remains unsettled.
But questions remain as to who bears liability when an automated system causes harm, what evidence establishes its safety and where state experimentation meets federal oversight.
Utah has “evidentiary standards,” Zach Boyd, director of the state’s Office of Artificial Intelligence Policy, told the ADAPT conference in Santa Fe, New Mexico, the report said. Regulators are also developing governance guidance and a liability bill to begin establishing a statutory framework around the sandbox.
The initiative began in 2024 with a mental health support application, according to the report. It attracted broader attention this year when Utah authorized a pilot with health technology company Doctronic, allowing AI to refill certain prescriptions with clinician oversight.
The decision exposed a dispute over how medical expertise should inform regulatory experimentation, the report said. In April, the Utah Medical Licensing Board called for the pilot’s immediate suspension and criticized the state for failing to consult it.
“We hit a nerve,” Boyd said, per the report.
Following six months of discussions and changes to the pilot, the board established a working group that meets weekly with state regulators, according to the report. Utah also removed headache medication butalbital and antiarrhythmic drug flecainide from the prescription refill list following the board’s scrutiny. The episode illustrated how professional licensing oversight can shape the boundaries of sandbox approvals.
Liability remains unresolved, however. According to Boyd, regulators interpret the sandbox’s authority as insufficient to change liability rules, the report said. As an interim measure, the state requires Doctronic and future participating companies to carry special liability insurance and refrain from waiving liability.
Boyd said a more durable framework must address the responsibility of technology suppliers amid concern that physicians supervising automated systems could otherwise shoulder excessive exposure, according to the report.
We’d love to be your preferred source for news.
Please add us to your preferred sources list so our news, data and interviews show up in your feed. Thanks!
“In the long run, we think [insurance is] less sustainable, and we need a way to hold the vendor directly liable for the actions of their automated system,” he said, per the report.
Boyd said he hopes Utah will unveil legislation during its interim legislative session, beginning what he described as a multiyear effort to refine liability, according to the report.
Federal jurisdiction presents another uncertainty. The Food and Drug Administration generally requires software that treats or diagnoses patients to undergo premarket authorization, the report said. Lawyers said that requirement could encompass technology such as Doctronic’s prescription refill system. The FDA had not publicly addressed the Utah experiment.
Boyd declined to speak for the agency but expressed interest in a coordinated pathway across jurisdictions, according to the report. He described the appeal of a “vet once, deployed everywhere, aggregate oversight across jurisdictions” approach.
The state is also building an external evaluation system, tapping six organizations to assess companies’ technical claims. Future pilots will have to contract with an independent evaluator, the report said.
But evaluation standards remain a work in progress. Utah’s technical guidance addresses matters including uncertainty in AI inputs and clinical context. Boyd acknowledged its provisional nature, saying, “Half of [the document] is probably wrong…,” per the report
Vega Health Co-Founder Mark Sendak said assessments must reflect each pilot’s intended use, patient population and outcomes, according to the report. Because models change and reach new populations, evaluations would need to recur.
Agreements with Intermountain Health and University of Utah Health could bring additional proposals, the report said.
For participating businesses, the regulatory challenge extends beyond securing initial approval. Liability allocation, federal authorization and continuing evidence of safety remain central to whether these experiments can become routine care.
For all PYMNTS AI coverage, subscribe to the daily AI newsletter.