→ In a new evidence review published today in Health Affairs, the IFL team finds that when researchers were testing the impact of COVID-19 misinformation interventions, they used significantly different examples of misinformation, assessed 47 different outcome measures, and only 18% of COVID-19 misinformation interventions measured public health outcomes, such as intent to vaccinate.
→ To move the field forward, the research community should align on standardized outcome measures and develop a typology of misinformation to make evidence comparable and actionable. Also, public health researchers and practitioners should consistently be included in health misinformation intervention research.
PROVIDENCE, R.I., Nov. 15, 2023 — Health misinformation and its impact on people and communities remains an urgent issue across the world. To identify what works and what doesn’t when responding to misleading health information online and offline, a team of researchers at the Information Futures Lab (IFL) at the Brown University School of Public Health reviewed 50 papers that investigated the efficacy of COVID-19 misinformation interventions fielded during the pandemic.
Instead of actionable evidence, the researchers found that most of the studies were so different in their approaches and what was measured that the majority of the evidence was not meaningfully comparable. They also found that most studies measured outcomes such as likelihood to share misinformation or perceived accuracy of misinformation. Only 18% of studies measured any public health-related outcomes, such as intent to vaccinate or self-reported mask wearing. These findings underscore the urgent need to include more public health voices in health misinformation research.
The study, “A Systematic Review of COVID-19 Misinformation Interventions: Lessons Learned” will appear in the December 2023 issue of Health Affairs, and is available via open access online as of today. It reviewed 50 COVID-19 misinformation studies published in English between Jan. 1, 2020 and Feb. 24, 2023, covering 119 distinct interventions.
KEY FINDINGS
In their review, the researchers found a large variety of outcome measures (47) applied across the 50 studies, and a significant lack of consistency across studies. Other key findings include:
- Only 18% of interventions were tested using public health-related outcome measures, such as attitudes toward vaccines, intent to vaccinate or willingness to pay for unproven treatments.
- Only 6% of studies exposed participants to any video content (video only or video and text), despite the rising influence of video content.
- Only 9% of interventions were tested against conspiracy theories related to COVID-19 vaccines. There were even fewer instances of COVID-19 vaccine misinformation related to issues of liberty and freedom, despite the prevalence of such narratives in the U.S., Canada, and Europe.
- Only 14% of studies assessed outcomes longitudinally.
- Only 28% of the interventions used populations outside the U.S.
WHY IT MATTERS
“Public health practitioners, journalists, community organizations and other trusted messengers are tasked with responding to health misinformation every day,” says Stefanie Friedhoff, co-director of the Information Futures Lab, associate professor of the practice of health services, policy and practice at the Brown University School of Public Health, and a co-author of the study. “While this is a complex area of study, we have a responsibility toward those on the frontlines to generate evidence that is meaningful and as actionable as possible. Our review can move the needle by identifying what is missing and where the research community needs to go next.”
“Misinformation research is a young field, so diverse approaches are good and important,” says Claire Wardle, co-director of the Information Futures Lab, professor of the practice of health services, policy and practice at the Brown University School of Public Health, and a co-author of the study. “What’s also clear is that this field emerged after concerns about the role of misinformation in elections, so many of the key researchers come from political science. As we have seen misinformation impact a number of different topics and issues, it is time researchers from different disciplines investigating misinformation come together to connect the dots.”
In the new study, the authors systematically extracted, categorized, and explored the different types of COVID-19 misinformation examples used in intervention studies, such as “vaccines are not safe” or “garlic water can cure COVID-19.” The team also extracted the ways in which study participants were exposed to such content — whether through video, text, images, audio, or combinations of these.
“Examining misinformation and its impact with greater granularity allows us to more clearly discern if an intervention worked on a specific kind of misinformation, and in what context,” says Rory Smith, research and investigation manager at the Information Futures Lab and a co-author of the study. “That is important because not all misinformation is the same, and details such as the delivery mechanisms and messengers matter, as other studies have also shown. If an intervention protects people from believing more innocuous forms of health misinformation, such as taking a hot bath prevents COVID-19, but doesn’t protect people from more dangerous conspiracy theories about vaccines, this is vital information practitioners who are applying the interventions should have.”
A PATHWAY TO ACTION
Recognizing the need for more standardization and collaboration in misinformation research revealed by the new study, the IFL team has started to engage researchers (including some whose studies are covered by the review), journalists, and public health practitioners in qualitative interviews and conversations about what is needed to generate more generalizable, actionable evidence. “We can’t just stop at publishing our findings,” say IFL co-directors Wardle and Friedhoff. “We now have an opportunity to facilitate solutions.”
An early readout from this follow-up research reveals that many misinformation researchers are aware of the challenges but struggle to find ways to overcome barriers. Researchers point out that the current methodological challenges and roadblocks they are experiencing are common when new academic disciplines emerge, but also that standardization is urgently needed. They list barriers such as academic incentives, funding, a rapidly changing information landscape, lack of access to better data, and lack of opportunities to collaborate.
“We’re trying to make the case that our research is fundamentally more important than someone else’s, when the reality is we’re all holding puzzle pieces. And we can’t actually make a puzzle until we put all the pieces on the table,” said one of the interview participants, Brandi Collins-Dexter, a former associate director of research at the Tech and Social Change Project at Harvard University.
To make the findings from the literature review as well as the follow-up interviews broadly accessible, the authors are also publishing summaries, visualizations, additional data not covered in the Health Affairs paper, and other supporting materials on the Information Futures Lab website.
The translation of this research into accessible content and the qualitative follow-up interviews with researchers, journalists, and public health practitioners are supported by a grant from the Peter G. Peterson Foundation, and awarded and administered through Brown University’s 2022 Peter G. Peterson Foundation Pandemic Response Policy Research Fund.