Issues affecting the public health relevance of Cochrane Reviews

Background: Cochrane Reviews are widely seen as trustworthy and informed by rigorous, evidence-based methods. Although these methods are expanding, they were historically developed with a focus on clinical treatments. Cochrane authors and review groups may experience significant challenges with public health and health systems reviews due to the scope of reviews, complexity of interventions, sources of evidence and types of relevant outcomes. Therefore, it is important to identify and document the issues faced whilst conducting these
reviews, to anticipate them and address them adequately.
Objectives: to identify key issues impacting the public health relevance of Cochrane Reviews evaluating prevention measures, harms, exposures, policies/programmes and/or their implementation.
Methods: we collated issues from various relevant sources:
1) reviews identified as high priority, potentially controversial or referred by an editor from the Public Health and Health Systems (PHHS) Network;
2) feedback/examples from editors within Cochrane and the PHHS Network;
3) external commentaries on PHHS reviews published in 2018;
4) screening reports by the Cochrane Editorial and Methods Department of PHHS Network reviews.
We categorized the identified issues into themes. We reviewed and discussed these iteratively with the editors within the PHHS Network until we reached agreement on categories.
Results: the common issues affecting the relevance of reviews were grouped into the following topics:
1) review aims/framing of questions;
2) choice of included study designs, especially when to include non-randomized studies (NRS);
3) population(s) and the corresponding perspectives/outcomes;
4) choice of outcomes, including non-standard outcomes;
5) use and interpretation of surrogate outcomes;
6) analysis issues, such as synthesis without meta-analysis;
7) interpretation of findings, such as effect sizes and applying GRADE in public health reviews.
There is often an underlying tension between taking a broader and more inclusive perspective in reviews, versus pragmatic considerations of the timely delivery of those reviews within the resources available. This often contributes to decisions which could eventually affect the public health relevance of reviews produced.
Conclusions: some of the issues faced in these reviews could be at least partially anticipated through careful consideration of the review’s context, perspective and the potential application of evidence when questions are framed to ensure that inclusion criteria, outcomes, analysis methods, and interpretation are closely aligned. Other issues are more 'methodological' in nature and would benefit from evidence-based guidance. Authors of public health and health systems reviews need specifically tailored guidance, as the issues faced and methods needed are different from clinical treatment reviews.