Building trust in peer review practices for science reporting: Reflecting on Peer Review Week 2022 

How reliable is a peer review in filtering out poor-quality science?  

How can press offices and media responsibly report early-stage research?   

As research review culture shifts from embargoed press releases to open science, how can best practices adapt alongside it?  

F1000 and Taylor & Francis, in association with the Association of British Science Writers, brought together an expert panel of researchers, publishers, press officers, and journalists to explore the relationship between peer review and science reporting as part of our Open Thinking program’s contribution to Peer Review Week 2022. Taking cues from the Peer Review Week theme of research integrity, the panel debated whether the peer review process helps or hinders science reporting. 

“The only thing worse than slow science is wrong science.” 

The COVID pandemic illustrated to members of the public – many for the first time – how science evolves and changes, and sometimes at a rapid pace. The panel highlighted the frenzied pace of updates to clinical guidelines in the initial stages of the crisis. Updates to scientific advice could be about minute details, such as which type of mask was most effective. Or they could be drastic about-turns, such as in the case of the early-stage recommendation to put patients on ventilators changing to pharmacological treatment. Given the urgency of treating patients and finding solutions, speed and collaboration took preference, and some of this scientific data was not fully peer reviewed.  

In the context of pandemic research, it became readily apparent how peer review can both help and hinder science reporting. The problems with peer review are well-known; sluggish times, reviewer biases, and the rise of predatory journals, for example. Conversely – and crucially – peer review does act as a filter for bad science.  

The panel agreed that while peer review at its heart is a solid system to build trust in academic research, the process must continue to evolve to remain relevant. Given the nuances and complexities of different peer review methods, rather than focusing on one ‘right’ way to do peer review, the panel suggested that the research community should focus instead on creating more opportunities to establish trust in peer review processes.  

Open Data, Open Methods, Open Peer review 

The discussion opened with some of the vital conversations that arise around publishers themselves in the process of science reporting.  

Research is becoming increasingly interdisciplinary. Some commented that, given the complexity involved, one or two peer reviewers from a single field of research may not be able to fully address the validity or potential implications of a study. To counterbalance this, the panel emphasized the importance of transparency – of trust – within any form of the review process, so that journalists can be secure when evaluating studies, and deciding what to print or broadcast.  

The panel also considered the role that press offices also play in science reporting – particularly when promoting research that has not been fully peer reviewed, as was the case during the early stages of the COVID pandemic. They suggested that an obligation lies with press offices to apply additional layers of scrutiny in selecting research for publicization and to establish processes that help build confidence in the research. These include featuring caveats of the research within press releases and maintaining an understanding of journalistic criteria for news reporting. Press officers should also understand the environment journalists are operating in, their time pressure, and the resources they work with. Clear communication also should underline all interactions between press offices and the media, safeguards should be built in with publication, and limitations of the study should be explained and easy to find. Panelists further clarified the position of trust that press offices must work hard to build. Touching on the irreversibility of releasing bad science to the press and the impact it could have, the panel warned, “once a story is out there, it’s really hard to put back in the box.” 

Needing trust signals in journalism 

In some cases, members of the panel noted that peer review can act as a barrier to science reporting. When interesting scientific developments have been made, some of the panel thought the peer review process is out of sync with the speed of today’s news media cycle. To some reporters and some readers, it was argued that the practice of withholding articles under embargo until they are peer reviewed could even be considered a form of gatekeeping, or that it wrongly assumes readers are not interested in the initial stages of an idea. Journalists on the panel also highlighted that as the public funds a lot of research, they have the right to access it.  

It was suggested that journalists may prefer to source stories directly from scientists themselves or set up alerts to pre-print articles on a particular subject. There were cases where this approach has proved risky, however, as in the case of a now-infamous study that questioned the effects of mRNA vaccines. 

Members of the panel also questioned whether peer review itself should be considered as the ultimate deciding factor regarding which research stories are worthy of publication by a news outlet. What determines if a science story was ready to report on, they said, instead relies on how trustworthy the research is.  

 
Altogether, the panel brought insights and perspectives from every phase of research communication, from research paper to headline, emphasizing the importance of trust at every stage. The result was an insightful discussion on the challenges and opportunities that peer review and science reporting pose, and an important basis for further constructive discussion. 

 

Panelists:  

• Jodie Bell, Senior Press & Media Relations Manager, Taylor & Francis 
• Boyana Konforti, Director of Strategic Initiatives, F1000 
• Parth Patel, Senior Research Fellow, IPPR; media commentator; Wellcome Open Research author 
• Rachael Pells, a freelance journalist in science and research (Times, Nature, Wired, iPaper) 
• Tom Sheldon, Senior Press Manager, Science Media Centre 
• Richard Van Noorden, Features Editor, Nature 
• The Chair of the panel: Martin Wilson, Head of Content, Taylor & Francis; former Head of Product, BBC Digital Creativity 

For more discussions and debates on the latest in research and publishing, visit Open Thinking on F1000. 

You can also subscribe to our mailing list to receive our latest updates on upcoming webinars and new resources.