What Makes Vaccine Safety Research Credible?

— Michael Osterholm says transparency and reproducibility are essential

Questions about vaccine safety can carry enormous consequences for public trust, but Michael Osterholm, PhD, MPH, argues that the credibility of any new research depends as much on how the studies are conducted as on what they ultimately find.

In a recent viewpointopens in a new tab or window, Osterholm, executive director of the Vaccine Integrity Project, emphasized the need for predetermined study designs, transparent methods, reproducible analyses, and independent scrutiny to ensure that vaccine safety research is scientifically sound.

In the installment of "Off the 'Page," MedPage Today's video series exploring thought-provoking perspectives in medicine, Osterholm discusses what credible vaccine-safety research should look like and why transparency is essential to maintaining trust.

The following is a transcript of this video:

The most important word that one can think about in talking about these kinds of studies is "transparency."

The most important thing is to understand, why are you doing these studies? What is the deficit in information that these studies will help address? And I think one of the challenges that we have right out of the chute is the fact that we're really talking about people who have preconceived ideas about vaccines and have made those ideas very clear.

If you're doing studies just to confirm those ideas, that's very different than the way you should approach a topic and say, "We never can have enough information on safety." We want more information all the time. But it has to be obtained in a way that is scientifically sound. Something that is predetermined before you ever start looking at the data itselfopens in a new tab or window ... is now going to make the study designed to work so that it supports that.

And I think that the real challenge we have here right now is that we don't really see any benefits, from the kinds of studies that are being proposed, that aren't already being addressed with the very rigorous safety overview we have of our current vaccines.

How's it set up? How do you define who is a vaccinee and who is not? How do you determine the outcome that patient has? How do you understand what in fact the patient experienced in terms of their illness if they had an adverse reaction? And so that's really important to have that decision-making done up ahead of time so it's not somehow later on, picking and choosing the numbers you want to have, therefore that very well may support your program.

We actually have a tool that we provide to the public, or other [healthcare] professionals, that you can go in and do the same analysis we did. All the data are there. Every ounce of data we use is there. The tool allows you to actually replicate, or if you find differences, to actually document that. And so that's also another thing that, really, I think is an important aspect to what goes on.

Now, once you do the analysis, and you find what you do, at that point again, it's having the experts review what is it that that finding actually brought you? And remember if you do enough analyses just by chance alone, you'll find something that might be associated, not cause-and-effect. What's your process for determining what the next step is? Is that something that deserves further attention that you then should study?

One of the things that's going to be really important with these studies that are being proposed is that we stay on top of them. We understand what's happening. And if we see challenges to study design, if we have questions about definitions of what they're doing or how they're doing it, we need to call it out at the time. Don't wait until there's a finding one day that's been launched on a public program basically saying that vaccines are unsafe or that vaccines are not effective. We need to stay on top of this all the time because one of the challenges that we have today is trust, and that trust is really difficult to respond to when you have a U.S. government, a senior health official, making claims about the danger of vaccinesopens in a new tab or window.