If technology can help collect information, can it also be trusted to act on it?

Terry Gerton ARPA-H has some fascinating projects and you’ve got one out that’s really different. I do have to give you props for acronym development. ADVOCATE is aiming to add AI to cardiac care. Tell us a little bit more about what you’re hoping to accomplish here.

Haider Warraich You know, a great acronym is one in which you can, if you break it down, you can explain what you’re trying to do, but just the name implies what the purpose of the program is. And I’m a clinician, I’m cardiologist. And one of the things that I get to do is that when friends or friends of friends or loved ones, they have some type of heart issue, they can call me. Even if I’m not their cardiologist, they could call me, they can text me, they’ll send me their reports, and Oftentimes they just want me to tell them that everything is okay, or does everything look good? And usually that’s always the case. And I think that it gives people a lot of comfort to have someone like me that they can call because heart disease is very, very scary. And the purpose of this program is how can we get that type of advocate for everyone with heart disease? Right? Because if you have a system in which people are relying on whether they happen to know a cardiologist and oncologist and endocrinologist to be able to get that type of personalized attention, I think that we will continue to live in a system where people really feel isolated when they’re living with chronic disease. And so the purpose of ADVOCATE is to use, is to essentially develop a patient-facing technology that can do most things a clinician like myself can do over the phone, which means that this is for patients with, this is sick patients, this is patients with heart failure, and so this is a technology that should be able to triage a patient, figure out can they stay at home, or do they need to go to the hospital? It should be to make a diagnosis of heart failure. If the patient is short of breath, it should be be able tell them this might be because your heart failure is getting worse. And lastly, it should able to close the loop and actually do the things that the patient needs for them to do, which is to optimize their medications or help them feel better, or order tests that they might need to be able to get all the information they and their clinical team needs. A few important things, this is not a replacement for clinicians. I’m a clinician, I practice cardiology in Boston, so I certainly don’t wanna replace myself, but I do want help. As a clinician, I wanna be able to make sure that I am not just attending to the patients that I’m seeing in clinic or those I see in the hospital, but that my patients are getting 24/7 care when they’re at home in between visits when they are not in front of me. The second thing we want to do is that we’re not really looking for a regulatory loophole. If anything, we’re saying that we wanna go straight to the regulator and we want to collect all the evidence needed to be able to give people the reassurance and trust that this is a technology that works very, very well. And so our goal is to go to the FDA with these technologies and seek market authorization within the next two years. No one’s really done that. There’s no real generative AI technology or agentic AI technology that has ever received FDA authorization. So that’s a huge milestone that we are focused towards. And the second thing we’re also doing is developing what we’re calling it by a different set of teams, an independent set of themes, what we are calling a supervisory agent or supervisory layer that can provide an additional redundant almost layer of safety to make sure that these technologies are acting on the patient’s behalf in the appropriate way as a part of the care team, not as an entity that exists outside of the clinical care team that the patient has. But really as an extension of that. So that’s the program in a nutshell and just really excited to be able to talk about that more.

Terry Gerton That is a great overview. I wanna come back to a couple of the specific pieces that you mentioned there. First you talked about a clinical facing AI agent and then the supervisory AI agent. Help us understand a little bit about how this AI is gonna get signals from a patient and what they’re gonna respond to and how you see them making decisions, what kind of decisions they’ll be ‘authorized,’ in my air quotes here, to make.

Haider Warraich So I think if you were an engineer, you would be, the way that you would ask this question is, okay, what are your inputs and outputs? But I think that you’ve explained it in a way that is much more, I think, that people can understand. So this is gonna be patient-facing technology. That immediately makes it high risk because now you’re not dealing with an expert user like myself. Right, as a clinician or a nurse or, you know, this is a, you know patients, some patients are obviously very sophisticated, but many of them are not expected to have this sort of clinical knowledge a physician has. And therefore there is the need to be safe is very, very important. What is the information that is going into this technology that is shaping its actions? So first of all, there’s a patient facing technology that will live in a patient’s phone. So the patient can either talk to it. So some companies are just building a voice agent. So the the patient just communicates with that in natural language and the technology responds to it in natural languages as well. This is incredibly important, especially for older adults. Many older adults are much more comfortable just talking rather than using an application or on a phone or computer. And then there are other teams that are developing either both capabilities or just developing an application or an app, in which you can also upload other information like pictures or other things. So that is one piece. The patient is engaging with the technology either through voice or text. The second way that input that is going into this technology is essentially all the data in the patient’s electronic health record. And this is a hard thing to do because the electronic health records and contains a lot of redundancy, contains a lots of missing information, there can be a lot mistakes. I mean, this is not a perfect data set and that makes this a bit challenging. Certainly clinicians are able to still use this data and provide lots of value for the patient. So we have, we are able to discern what is there, what is not, what to trust and what we cannot to be able to build what we call a ground truth for the patients that we can use to then shape our actions. Now, the. Additional thing that this technology has that even a clinician like myself often doesn’t have is actual physiologic data from the patient’s own body that is being collected through some type of variable technology. Maybe it’s a watch, maybe it’s the ring, maybe a smart scale, maybe some other type of device that can give a sense of how active is this patient or is this patients heart rate elevated. Is a patient’s EKG, for example, showing abnormalities? So that we can use this to actually build a surveillance layer. Any technology that waits for the patient to tell the technology or to tell the clinician that, hey, I’m having difficult breathing, most of the times it’s already too late. Most of the time, by the time a patient becomes symptomatic, there’s very little you can do other than just tell them to go to the emergency room. And the medication, so really using physiologic data to catch patients before they are symptomatic is I think another key advance that we want to do as part of this program. So that’s all the information that’s going into the technology. What comes out of this technology? So I would say that there are two types of things that this technology can do. The FDA regulates a certain set of functions called medical device functions. These are higher risk functions, so these can include diagnosis, these can include triage, this can include management. So this technology can do a whole lot of those types of functions. So for example, being able to triage a patient, deciding most patients use this technology at home. So the first thing that this technology has to do is that can I even have this conversation at home? Or is there a better place for this patient to go to? Urgent care, clinic, ER, et cetera., and that’s a hard thing. The figuring that out is hard, but we believe it’s doable. You cannot triage a patient until you at least make a diagnosis. You can’t treat them until you figure out, okay, what am I dealing with? And so, being able to diagnose heart failure is going, or heart failure exacerbation, as we call it. These are patients who already have heart failure, but heart failure is kind of a roller coaster, right? You have these lulls, and then sometimes things get worse. Those exacerbations mean that heart failure is actually the number one cause of hospital admissions amongst older Americans. It’s a very big and hard problem to solve. But this technology should be able to do that. And the third thing that this technology should be be able do is that for what is in scope for this technology, so for in scope of this technology would be treatment of heart failure. It should be to be able ultimately autonomously manage medications that can improve a patient’s heart failure, both their symptoms in the short term but also their long term symptoms. So these are some of the high device functions, quote unquote, that we are focused on, and I think that they’ll be really important to evaluate and study and standardize. And then there’s a lot of other things that are not quote device functions but are very important for a patient. For example, a patient wants to know what can they eat if they have heart failure? They’re in the grocery store and they’re trying to figure out, okay, can I have… You know, can I have this canned soup, for example, or what are choices that I can make, or what type of exercise I can do, or, you know other questions that may not meet, you know that device definition, but are actually very, very important and probably most of the questions that patients actually have living with chronic disease do not meet that bar. So this technology should be able to respond to most of those questions, you now, you know really autonomously back to the patient. And so that’s really what the sort of vision is. The vision is that you have these set of inputs, the patient’s prompt, text or voice, electronic health record data, variable data that goes into the technology and outcomes and actions that really span the spectrum of really everything a patient might need as they live a life with chronic disease.

Terry Gerton Dr. Haider Warraich is the program manager at ARPA-H. So Dr. Warraich, you mentioned earlier the regulatory process for this, that this is not just a device, it’s a process and you’re trying to figure out as you’re developing the solution what the regulatory pathway is going to be. How is ARPA-H working with FDA to kind of scope out that regulatory process?

Haider Warraich So Terry, one of the reasons I’m excited about this interview is because I have a long history with the federal government. So right now I work at RFAH. Before that, long before that, I was at the VA. I was a heart failure cardiologist at the VA in Boston. But after that, I went to the FDA where I was the senior advisor to the F.D.A commissioner. And one of sort of the key things that I focused on was AI. And in fact, I developed this program while I was there at the FDA, knowing that this is a real gap in the field, which is that we really need to understand how best to regulate, evaluate, monitor these technologies, not just before they touch a patient, but really long after they’ve been deployed and they’re in health systems and they are part of clinical practice, we need to still have a system of how do we continue to monitor them, that phase is called post-market monitoring. And so, This was really a program that I built with loss of input from colleagues at FDA when I was there. And since then, the FDA has been really deeply, deeply engaged with this program at really every step from how it was conceived to what types of metrics and milestones we’re looking at, some of our timelines. Just a couple of days ago, we had our in-person kickoff event in DC, and we had. Jared Seehofer, who is the Deputy Commissioner for Technology at FDA, and Dr. Rick Abramson, who is Director of the Digital Health Center of Excellence at the FDA as well. We’ve had lots of engagement, really, from the entire, I would say, stack of folks who are going to be involved in this in a deep way. This includes teams in the commissioner’s office. All the way to the centers and then in the science and research laboratories, in the review divisions. And they have been excellent partners in this journey with us. I think that they also feel a sense of urgency and the sense of emergency comes from the fact that millions of people are using generative AI every single day. Patients I see in clinic use generative AI for life and death decisions. Should they go to the emergency room if they’re having a stroke, right? I think it’s the performance of the technology. People are using this technology and getting results that is blowing their minds. So they’re really trusting this technology in ways that they really haven’t anything else in the past. But we really don’t have a clear sense of how well are these technologies actually doing for real patients. And I think that’s one of the reasons why there’s a sense that we really need to figure this out because clearly there’s the huge signal that’s coming from people that they want a technology like this that can help them at all times. The regulatory processes and the other partners in government, we really need to make sure that we create a pathway that is transparent, that is accessible, but also engenders trust through rigor. And so the FDA has been a fantastic partner for this program and will continue to be over its entire course.

Terry Gerton As you go through this regulatory process, you use the terms life and death decisions. I have got to imagine that you are dealing with the issue of what happens when this AI makes a life worth death decision and it has a bad outcome. How do you manage liability responsibility and response?

Haider Warraich I think the reality, Terry, is that this technology will make mistakes. If anyone tells you otherwise, they’re not being honest, they are not being straightforward. The real question is, what is the counterfactual? What is the alternative? And I will say a few things. First is that research that my team has led shows that almost 50% of counties in country do not have a single cardiologist. Cardiovascular disease is the number one cause of death in this country, but I practice in Boston. We have more cardiologists here than any other place on earth, and yet it takes five to six months to see a cardiologist. There are a lot of stuff I can have, and then once you see a cardiologist, you see them once every six months, maybe once a year, once every other year, and we have not been able to make more progress in cardiovascular disease. We have so many treatments. Uh… They’re not reaching patients even something as simple as high blood pressure we have so many treatments for high blood pressure half the people in this country don’t have their blood pressure under control if you look at heart failure there are four medications that are evidence-based that can extend life in patients with heart failure do you know what is the percentage of patients who are actually on all four of those medications whatever number you come up with is going to be lower than that because the number is two point five percent Right, so those are things that we’re not able to capture. Those are things that we’re not able to capture and that we’ve kind of accepted that this is just part of our health system. That all these people are going to have preventable death and disability and that’s the best we can do. So I do think that this technology is going to make mistakes. We have so many processes in place to be able to understand how it’s making mistakes. Its mistakes may be different from the types of mistakes people make. And we want to sort of go above and beyond to mitigate those risks. And in the end, what we want to do is we want actually compare this against cardiologists, against the type of care that you’d get in a very well, even in the best possible circumstances. That if you have the best, possible cardiovascular care, what is the additive benefit of a technology like this on top of that? And so, my sense about this is that this is not a perfect technology. I do think that we want to make sure that this is a technology that is not just as good as clinical, as the best clinical care, that it is significantly better than the best clinical care. And I’m not saying that that’s going to be an easy thing to show or easy thing to do. If anything, this is probably one of the hardest things that we’ve ever done. But given the teams that we have on board, given the support we have from across of government given the pace of how this technology is getting better. I am confident that we will get there.