An algorithm to assign GRADE levels of evidence to comparisons within systematic reviews

Alex Pollock, Sybil E. Farmer, Marian C. Brady, Peter Langhorne, Gillian E. Mead, Jan Mehrholz, Frederike van Wijck, Philip J Wiffen

Research output: Contribution to journalArticlepeer-review

53 Downloads (Pure)


Objectives: One recommended use of the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach is supporting quality assessment of evidence of comparisons included within a Cochrane overview of reviews. Within our overview, reviewers found that current GRADE guidance was insufficient to make reliable and consistent judgments. To support our ratings, we developed an algorithm to grade quality of evidence using concrete rules.
Methods: Using a pragmatic, exploratory approach, we explored the challenges of applying GRADE levels of evidence and developed an algorithm to applying GRADE levels of evidence in a consistent and transparent approach. Our methods involved application of algorithms and formulas to samples of reviews, expert panel discussion, and iterative refinement and revision.
Results: The developed algorithm incorporated four key criteria: number of participants, risk of bias of trials, heterogeneity, and methodological
quality of the review. A formula for applying GRADE level of evidence from the number of downgrades assigned by the algorithm was agreed.
Conclusion: Our algorithm which assigns GRADE levels of evidence using a set of concrete rules was successfully applied within our Cochrane overview. We propose that this methodological approach has complications for assessment of quality of evidence within future evidence syntheses.
Original languageEnglish
Pages (from-to)106-110
Number of pages5
JournalJournal of Clinical Epidemiology
Publication statusPublished - 1 Sep 2015


  • review
  • overview
  • algorithm
  • methodology
  • quality of evidence


Dive into the research topics of 'An algorithm to assign GRADE levels of evidence to comparisons within systematic reviews'. Together they form a unique fingerprint.

Cite this