- Email: [email protected]
Rapid reviews for evidence synthesis
Med Clin (Barc). 2017;148(9):424–428
www.elsevier.es/medicinaclinica
Special article
Rapid reviews for evidence synthesis夽 Revisiones rápidas para la síntesis de la evidencia Ferrán Catalá-López a,b,∗ , Adrienne Stevens b,c , Chantelle Garritty b,c , Brian Hutton b,d a Departamento de Medicina, Universidad de Valencia/Instituto de Investigación Sanitaria INCLIVA, Centro de Investigación Biomédica en Red de Salud Mental (CIBERSAM), Valencia, Spain b Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada c Translational Research in Biomedicine (TRIBE) Program, School of Medicine, University of Split, Split, Croatia d School of Epidemiology, Public Health and Preventive Medicine, Faculty of Medicine, University of Ottawa, Ottawa, Ontario, Canada
a r t i c l e
i n f o
Article history: Received 30 November 2016 Accepted 24 December 2016 Available online 20 May 2017
Introduction Decision-making in clinical practice and health care, such as requesting a diagnostic test or treatment, should be supported by the best available evidence. For this purpose, it is generally recommended that the information of all available studies on a particular question be taken into account, rather than considering only the results of a single study.1–4 Evidence synthesis from systematic reviews and meta-analysis uses explicit and rigorous methods to identify, select, collect, critically evaluate and synthesize the results of all existing studies. This facilitates the correct understanding of the results of individual studies in the context of all available evidence. It also contributes to knowledge transfer by bridging the gap between research and clinical practice.1–4 However, planning and preparing a comprehensive and transparent systematic review reporting decision-making involves time and resources not always available. For example, it is estimated that a quality systematic review performed by an experienced team may involve over a thousand hours of work (interval 216–2518 h) distributed over at least 1–2 years.5–8 On many occasions, clinical and health decisions must be made quickly to respond immediately to the requirements of professionals, managers, planners and patients. With the proliferation of new interventions, services and health technologies in increasingly complex healthcare contexts, the need for independent, continuous and rapid assessments that provide information
夽 Please cite this article as: Catalá-López F, Stevens A, Garritty C, Hutton B. Revisiones rápidas para la síntesis de la evidencia. Med Clin (Barc). 2017;148:424–428. ∗ Corresponding author. E-mail address: ferran [email protected] (F. Catalá-López). ˜ S.L.U. All rights reserved. 2387-0206/© 2017 Elsevier Espana,
for objective, rational and transparent decision-making is a reality. In recent years evidence-based synthesis programs have been proposed in addition to traditional systematic reviews, some of which may be framed within rapid response assessment programs8–14 to report the decision-making. In this special article, the concept of “rapid review” has been introduced, showing its differences versus narrative reviews and systematic reviews, and its importance is discussed through some published examples. Rapid review A rapid review7–9 can be defined as a review of the scientific literature using simplified and rapid methods compared to a traditional systematic review. In the Anglo-Saxon context, where rapid reviews are routinely being used,15–20 rapid reviews have also been described as rapid evidence assessment, rapid synthesis, rapid health technology assessment, and rapid evidence summary, among others.7 Planning and preparing a rapid review may be conditioned by a number of decision-making factors, including a health or clinical emergency, an immediate request to implement or adopt a new technology, procedure or service, and the limited time and resources that prevents a more thorough assessment. Table 1 shows the differences between narrative, rapid and systematic reviews.8,14 Unlike the narrative reviews, in systematic reviews we use the scientific method to search for potentially relevant studies together with explicit and reproducible criteria, fixed in advance in the selection. In contrast to systematic reviews, rapid reviews often omit or simplify stages to draft a rapid synthesis (e.g., limiting information sources and searches, limiting language and date of publication, not assessing the quality of primary studies, not
F. Catalá-López et al. / Med Clin (Barc). 2017;148(9):424–428
425
Table 1 Differences between narrative, rapid and systematic reviews. Feature
Narrative review
Rapid review
Systematic review
Period of time (approximately) Question of interest
Indeterminate
3 weeks to 6 months
6 months to 2–3 years
Non-specific, non-structured Non-detailed, non-systematic
Specific (although it may be extensive), structured They may be limited, non-comprehensive, although explicit sources and strategies Based on explicit criteria, uniformly applied Structured, it can include critical assessment (from previous systematic reviews, sometimes from primary studies) Descriptive, qualitative (rarely quantitative with meta-analysis when appropriate and time/resources allow)
Specific (on a well delimited clinical question), structured Structured, comprehensive, explicit sources and strategies
Sources and searches
Selection Evaluation of quality and information bias
Non-detailed, non-reproducible Absent
Synthesis
Descriptive, qualitative
Inferences
Generally based on opinion (bias), limited, cautious interpretation
performing meta-analysis). According to the Cochrane manual,21 in a systematic review all stages, including screening of studies (titles and abstracts; full text), data extraction and quality assessment (biases) of information should be typically carried out by 2 evaluators independently. In this regard, in rapid reviews, the stages of screening and selection of studies, and the data extraction are often performed by a single evaluator or reviewer (but not always). Some of these situations may make it possible to simplify and accelerate the synthesis of knowledge on a particular subject. However, it must also be considered that errors and potential biases can be added into the evaluation (e.g., omission of studies, errors in data extraction, publication and language biases, etc.).7 The evidence generated by rapid reviews may be considered to be exploratory in nature, generally aimed at reporting a decision, rather than responding more specific responses such as it happens with more comprehensive and thorough systematic reviews.1,3 Therefore, it should be a priority that the potential user (or promoter) of the evaluation understand, value and discuss these issues with the research team, when defining the question of interest and the methods (and the completeness of the analyses) to be applied considering the period of time available. Some examples of rapid reviews The 2 examples of rapid reviews19,20 shown below were developed by the Knowledge Synthesis Group of the Ottawa Hospital Health Research Institute to report decision making. In the first review, the researchers tried to respond to the requirements of the center professionals and management team to ensure the effective and efficient use of the hospital emergency services through a short-stay unit.19 In the second review, information was used to respond to the highest level of decision-making (international agencies, ministries of health, etc.) regarding the recommendations on personal protective equipment in the standards of practice in cases of suspected infection by Ebola virus.20 In both cases, the researchers performed the rapid reviews in less than 2 months. In 2011, the management team of the Ottawa Hospital, a public university hospital with over 1100 beds located in the Ontario region (Canada), suggested the idea of introducing short-stay units as an alternative hospitalization strategy facing the organizational challenge of the overloaded hospital emergency services. In this context, a rapid review on the effectiveness and safety of
Sometimes evidence-based, limited, cautious interpretation
Based on explicit criteria, uniformly applied Structured, rigorous, critical appraisal
Descriptive, qualitative and quantitative (usually includes meta-analysis when appropriate) Generally evidence-based
short-stay units was requested to report decision-making19 (Table 2). To this purpose, the research team defined the question of interest, and provided the methods and documents that would be delivered within the established time frame (4 weeks). We searched electronic databases and sources of gray literature. The eligibility criteria used included only papers published in English available in electronic full-text format. Researchers limited their study selection to published systematic reviews (given the heterogeneity of primary studies and the limited time available to collect information and understand the complexity of the evidence). They excluded papers published prior to year 2000 prioritizing the most recent evidence. The selection of papers was carried out by 2 evaluators. However, the extraction of information, quality evaluation and narrative synthesis were carried out by a single evaluator. The quality of the included systematic reviews was evaluated using the Assessing the Methodological Quality of Systematic Reviews – AMSTAR-tool.22 Totally, 4 published systematic reviews were included. Evidence from a systematic review (12 studies) of moderate quality indicated that short-stay units can lead to better outcomes (patient satisfaction, reduced stay, etc.) and improve efficiency in Emergency services. However, this systematic review was published over a decade ago. Therefore, the recommendation was to carry out a new comprehensive, rigorous and updated systematic review on the subject. Similarly, the researchers found that most of the comparative evaluations of short-stay units used “before and after” type designs, which have limitations such as confusion with natural evolution or the effects of any other (hidden) intervention, and the lack of masked assessment. The generalization of results was limited by the specific approach of the units evaluated (e.g., Cardiology, Pneumology). For this reason, and given the paucity of quality clinical trials, the researchers recommended caution in interpreting the findings. Evidence from the rapid review was positively responded by the hospital management team and was used to report the discussion on the implementation of a short-stay unit. It also helped strengthen partnerships between researchers and the management team, which has led to further research on this issue. Specifically, this rapid review was used as the basis for requesting for funding for a systematic review project that was successfully obtained by the team of researchers and other collaborators.23 The authors of the systematic review concluded that there is insufficient evidence to draw conclusions about the effectiveness and safety of short-stay units compared to hospital care.23 Therefore, the results of the systematic review support the
426
F. Catalá-López et al. / Med Clin (Barc). 2017;148(9):424–428
Table 2 Example of rapid review on the effectiveness and safety of short-stay units. Component
Characteristics
Requirement (potential user) Question of interest Methods
Overloaded hospital emergency services (management team and professionals)
Results Dissemination
What is the effectiveness and safety of short-stay units? Define inclusion/exclusion criteria (for example, inclusion of systematic reviews published in English available in full-text electronic format; exclusion of papers published before year 2000) Define and conduct searches of the medical literature (by an expert documentalist) in Medline, the Cochrane Library, Embase and the Center for Reviews and Dissemination) and gray literature sources (websites of scientific societies and assessment of health technologies) Screen and select the previous systematic reviews (by two researchers, one expert in methodology and one clinical expert in hospital emergencies) Data extraction (by a single researcher) Evaluate the quality of previous systematic reviews (by a single researcher, using the AMSTAR tool) Synthesize the results (narrative synthesis by a single researcher) Prepare the report of the rapid review and disseminate between the management team and health professionals (available through the institutional website: http://www.ohri.ca/kta/docs/KTA-ED-SSU-Evidence-Summary.pdf) [consulted Oct 2016] Publication of scientific article in medical journals19
idea that the authors of the rapid review considered this as a cautious conclusion until the results of a consistent systematic review were available. A rapid review performed in 7 weeks was issued in October 2015 to report on the development of the Rapid Advice Guideline on Personal Protective Equipment in the context of the filovirus outbreak of Ebola in West Africa, with special attention to the professionals directly responsible for patients with Ebola or Marburg filovirus disease20 (Table 3). Based on a research protocol developed in July 2014, search was performed in electronic databases and gray literature sources. The eligibility criteria used initially included comparative studies on Ebola and Marburg disease reported in English or French, but subsequently the criteria were expanded for studies on further viral hemorrhagic fevers and non-comparative designs due to the low number of studies. In this review, screening of title and abstract as well as the full text of the relevant articles were evaluated twice (2 people to exclude studies). However, the information was drawn from the studies by a single evaluator, and a second evaluator checked the information in just over half (57%) of the studies. The authors of the studies were not contacted to obtain any additional information due to the lack of time. The quality of the evidence was evaluated through the Grading of Recommendations Assessment, Development and Evaluation – GRADE – system.24 We identified 30 noncomparative observational studies (8 related to Ebola virus disease) and 27 provided data on viral transmission. Due to the nature and heterogeneity of the included studies, no meta-analysis of the results was performed. Based on low-quality evidence, it was concluded that the presentation of the protective equipment, and infection prevention and control protocols was deficient, which prevented from drawing conclusions about the comparative effectiveness of the various types of personal protection equipment (“absence of evidence” should not be understood as “evidence of absence”). In the rapid review, it was also identified the urgent need for further research that would improve the available evidence on the comparative effectiveness of the protective equipment currently used. This example demonstrates how a rapid review can be incorporated into a guideline, identify the need to improve available evidence and be useful to report urgent public health decisionmaking.25 Final considerations An important advantage of rapid reviews is their time of development and, therefore, their ability to guide and assist timely
decision making. However, it should also be recognized that in some situations the rapidity of the review may be limited. Often, to complete a review quickly, simplifying the stages of a traditional systematic review is required, which may result in potential errors and/or biases.7,8,14 There are differences between the development of a standard systematic review and a rapid review. However, the basic principles of systematic reviews also apply to rapid reviews, which would include the application of transparent processes, the use of explicit methods and the reduction of biases. Systematic reviews and meta-analysis play an important role in the planning, design and development of biomedical research.26,27 In fact, it has been suggested that all researchers should begin their research training by developing at least one systematic review.28 Perhaps a first step in this direction would be for future undergraduate and graduate students to begin with the methodology of evidence synthesis through rapid reviews. The best qualification of future generations (authors, reviewers and editors) might help increase the value and reduce the potential waste occurring in preparing reviews with little methodological accuracy.3 Currently, multiple and varied methodologies have been applied for preparing rapid reviews.29,30 However, the level of detail in the description of the methods and results of the reviews should be improved.7,13,29,30 Rapidness should not compromise the validity (or transparency) of the methods and results reported in a rapid review. Rapid reviews that do not use explicit or structured methods run the risk of becoming narrative reviews or opinion articles. In situations where key aspects of the methods and results of the rapid review are omitted due to lack of report, any assessment of the validity of the review is seriously hampered. Published rapid reviews should adhere to the guidelines for developing and submitting reviews. Work is currently being performed on initiatives that would include extensions of the current recommendations to cover more specifically aspects and particularities of rapid reviews. Specifically, an extension of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) for rapid reviews (PRISMA-RR) is being conducted.31 In addition, a new Cochrane Rapid Review Methods Group32 has recently been established, aimed at having an international leadership role in guiding the development of rapid reviews. It is expected that these initiatives will contribute to improving the methods and reporting future rapid reviews, including also those evidence-based programs using analogous terminologies (e.g. rapid assessments of health technologies, etc.).
F. Catalá-López et al. / Med Clin (Barc). 2017;148(9):424–428
427
Table 3 Example of a rapid review on the effectiveness of protective equipment for the prevention of Ebola virus transmission. Component
Characteristics
Requirement (potential user)
Outbreak and epidemic due to filovirus disease in West Africa, with special attention on Ebola or Marburg virus infection. Situation of international health emergency (international agencies, ministries of health, health professionals) What is the effectiveness of protective equipment for professionals/caregivers to prevent from Ebola virus transmission? Define inclusion/exclusion criteria (for example, inclusion of comparative studies on protective equipment in Ebola and/or Marburg disease reported in English or French, although the criteria were later expanded for studies on other viral hemorrhagic fevers and non-comparative designs due to the lack of studies) Define and conduct medical literature searches (by an expert documentalist) in Medline, the Cochrane Library, Embase and African Index Medicus and gray literature sources (ProQuest Dissertation and Theses Databases, Google, ClinicalTrials.gov and the international registry of WHO clinical trials) Screen and select the identified studies (by a single researcher, although reviewed by a second researcher) Data extraction (by a single researcher, although 57% of the information reviewed by a second researcher) Evaluate the quality of evidence (by a single researcher using the GRADE tool, although it does not assess the risk of bias in single studies, as there are no validated tools for non-comparative studies) Synthesize the results (narrative synthesis not including meta-analysis) Prepare the rapid review report used as a guideline for the discussions on providing the recommendations by the WHO: Personal protective equipment in the context of filovirus disease outbreak response. WHO/EVD/Guidance/PPE/14.1. Geneva, Switzerland: World Health Organization; 2014 Publication of scientific article in medical journals20
Question of interest Methods
Results Dissemination
Funding FCL has been granted by the Generalitat Valenciana (PROMETEOII/2015/021). BH has received funding from the Canadian Institutes of Health Research (New Investigator Award, Canadian Institutes of Health Research/Drug Safety and Effectiveness Network).
Conflict of interests The authors report no conflict of interest.
References 1. Moher D, Liberati A, Tetzlaff J, Altman DG, Group PRISMA. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6:e1000097. 2. Hutton B, Catalá-López F, Moher D. The PRISMA statement extension for systematic reviews incorporating network meta-analysis: PRISMA-NMA. Med Clin (Barc). 2016;147:262–6. 3. Page MJ, Shamseer L, Altman DG, Tetzlaff J, Sampson M, Tricco AC, et al. Epidemiology and reporting characteristics of systematic reviews of biomedical research: a cross-sectional study. PLoS Med. 2016;13:e1002028. 4. Alonso-Coello P, Schünemann HJ, Moberg J, Brignardello-Petersen R, Akl EA, Davoli M, et al. GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 1: introduction. BMJ. 2016;353:i2016. 5. Petticrew M, Roberts H. Systematic reviews in the social sciences: a practical guide. Malden, MA: Blackwell Publishing Co.; 2006. 6. Sampson M, Shojania KG, Garritty C, Horsley T, Ocampo M, Moher D. Systematic reviews can be produced and published faster. J Clin Epidemiol. 2008;61: 531–6. 7. Tricco AC, Antony J, Zarin W, Strifler L, Ghassemi M, Ivory J, et al. A scoping review of rapid review methods. BMC Med. 2015;13:224. 8. Khangura S, Konnyu K, Cushman R, Grimshaw J, Moher D. Evidence summaries: the evolution of a rapid review approach. Syst Rev. 2012;1:10. 9. Harker J, Kleijnen J. What is a rapid review? A methodological exploration of rapid reviews in Health Technology Assessments. Int J Evid Based Healthc. 2012;10:397–410. 10. Hailey D, Corabian P, Harstall C, Schneider W. The use and impact of rapid health technology assessments. Int J Technol Assess Health Care. 2000;16: 651–6. 11. Thomas J, Newman M, Oliver S. Rapid evidence assessments of research to inform social policy: taking stock and moving forward. Evid Policy. 2013;9: 5–27. 12. Khangura S, Polisena J, Clifford TJ, Farrah K, Kamel C. Rapid review: an emerging approach to evidence synthesis in health technology assessment. Int J Technol Assess Health Care. 2014;30:20–7.
13. Polisena J, Garritty C, Kamel C, Stevens A, Abou-Setta AM. Rapid review programs to support health care and policy decision making: a descriptive analysis of processes and methods. Syst Rev. 2015;4:26. 14. Watt A, Cameron A, Sturm L, Lathlean T, Babidge W, Blamey S, et al. Rapid reviews versus full systematic reviews: an inventory of current methods and practice in health technology assessment. Int J Technol Assess Health Care. 2008;24: 133–9. ˜ 15. Branas P, Jordan R, Fry-Smith A, Burls A, Hyde C. Treatments for fatigue in multiple sclerosis: a rapid and systematic review. Health Technol Assess. 2000;4:1–61. 16. Bridle C, Palmer S, Bagnall AM, Darba J, Duffy S, Sculpher M, et al. A rapid and systematic review and economic evaluation of the clinical and cost-effectiveness of newer drugs for treatment of mania associated with bipolar affective disorder. Health Technol Assess. 2004;8:iii–v, 1-187. 17. Beall RF, Baskerville N, Golfam M, Saeed S, Little J. Modes of delivery in preventive intervention studies: a rapid review. Eur J Clin Invest. 2014;44:688–96. 18. Luckett T, Phillips J, Agar M, Virdun C, Green A, Davidson PM. Elements of effective palliative care models: a rapid review. BMC Health Serv Res. 2014;14: 136. 19. Konnyu KJ, Kwok E, Skidmore B, Moher D. The effectiveness and safety of emergency department short stay units: a rapid review. Open Med. 2012;6: e10–6. 20. Hersi M, Stevens A, Quach P, Hamel C, Thavorn K, Garritty C, et al. Effectiveness of personal protective equipment for healthcare workers caring for patients with filovirus disease: a rapid review. PLoS One. 2015;10:e0140290. 21. The Cochrane Collaboration. Manual Cochrane de revisiones sistemáticas de intervenciones. Versión 5.1.0 (traducida). Barcelona: Centro Cochrane Iberoamericano; 2011 [consulted 17 Oct 2016]. Available in: https://es. cochrane.org/sites/es.cochrane.org/files/uploads/Manual Cochrane 510 reduit.pdf 22. Shea BJ, Grimshaw JM, Wells GA, Boers M, Andersson N, Hamel C, et al. Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol. 2007;7:10. 23. Galipeau J, Pussegoda K, Stevens A, Brehaut JC, Curran J, Forster AJ, et al. Effectiveness and safety of short-stay units in the emergency department: a systematic review. Acad Emerg Med. 2015;22:893–907. 24. Alonso-Coello P, Rigau D, Solà I, Martínez García L. Formulating health care recommendations: the GRADE system. Med Clin (Barc). 2013;140: 366–73. 25. Garritty CM, Norris SL, Moher D. Developing WHO rapid advice guidelines in the setting of a public health emergency. J Clin Epidemiol. 2016, http://dx.doi.org/10.1016/j.jclinepi.2016.08.010, pii: S0895-4356(16)30386-9. 26. Catalá-López F, González-Bermejo D, de la Fuente Honrubia C, Macías SaintGerons D. Clinical equipoise and systematic reviews of randomized controlled trials. Med Clin (Barc). 2015;145:496–8. 27. Ioannidis JP. Why most clinical research is not useful. PLoS Med. 2016;13:e1002049. 28. Mahtani KR. All health researchers should begin their training by preparing at least one systematic review. J R Soc Med. 2016;109:264–8. 29. Tricco AC, Zarin W, Antony J, Hutton B, Moher D, Sherifali D, et al. An international survey and modified Delphi approach revealed numerous rapid review methods. J Clin Epidemiol. 2016;70:61–7. 30. Kelly SE, Moher D, Clifford TJ. Quality of conduct and reporting in rapid reviews: an exploration of compliance with PRISMA and AMSTAR guidelines. Syst Rev. 2016;5:79.
428
F. Catalá-López et al. / Med Clin (Barc). 2017;148(9):424–428
31. EQUATOR network. Reporting guidelines under development. PRISMA-RR 2017: an extension to PRISMA for rapid reviews. [consulted 17 Oct 2016]. Available in: http://www.equator-network.org/library/reporting-guidelines-underdevelopment/#51
32. Garritty C, Stevens A, Gartlehner G, King V, Kamel C, Cochrane Rapid Reviews Methods Group. Cochrane Rapid Reviews Methods Group to play a leading role in guiding the production of informed high-quality, timely research evidence syntheses. Syst Rev. 2016;5:184.
www.elsevier.es/medicinaclinica
Special article
Rapid reviews for evidence synthesis夽 Revisiones rápidas para la síntesis de la evidencia Ferrán Catalá-López a,b,∗ , Adrienne Stevens b,c , Chantelle Garritty b,c , Brian Hutton b,d a Departamento de Medicina, Universidad de Valencia/Instituto de Investigación Sanitaria INCLIVA, Centro de Investigación Biomédica en Red de Salud Mental (CIBERSAM), Valencia, Spain b Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada c Translational Research in Biomedicine (TRIBE) Program, School of Medicine, University of Split, Split, Croatia d School of Epidemiology, Public Health and Preventive Medicine, Faculty of Medicine, University of Ottawa, Ottawa, Ontario, Canada
a r t i c l e
i n f o
Article history: Received 30 November 2016 Accepted 24 December 2016 Available online 20 May 2017
Introduction Decision-making in clinical practice and health care, such as requesting a diagnostic test or treatment, should be supported by the best available evidence. For this purpose, it is generally recommended that the information of all available studies on a particular question be taken into account, rather than considering only the results of a single study.1–4 Evidence synthesis from systematic reviews and meta-analysis uses explicit and rigorous methods to identify, select, collect, critically evaluate and synthesize the results of all existing studies. This facilitates the correct understanding of the results of individual studies in the context of all available evidence. It also contributes to knowledge transfer by bridging the gap between research and clinical practice.1–4 However, planning and preparing a comprehensive and transparent systematic review reporting decision-making involves time and resources not always available. For example, it is estimated that a quality systematic review performed by an experienced team may involve over a thousand hours of work (interval 216–2518 h) distributed over at least 1–2 years.5–8 On many occasions, clinical and health decisions must be made quickly to respond immediately to the requirements of professionals, managers, planners and patients. With the proliferation of new interventions, services and health technologies in increasingly complex healthcare contexts, the need for independent, continuous and rapid assessments that provide information
夽 Please cite this article as: Catalá-López F, Stevens A, Garritty C, Hutton B. Revisiones rápidas para la síntesis de la evidencia. Med Clin (Barc). 2017;148:424–428. ∗ Corresponding author. E-mail address: ferran [email protected] (F. Catalá-López). ˜ S.L.U. All rights reserved. 2387-0206/© 2017 Elsevier Espana,
for objective, rational and transparent decision-making is a reality. In recent years evidence-based synthesis programs have been proposed in addition to traditional systematic reviews, some of which may be framed within rapid response assessment programs8–14 to report the decision-making. In this special article, the concept of “rapid review” has been introduced, showing its differences versus narrative reviews and systematic reviews, and its importance is discussed through some published examples. Rapid review A rapid review7–9 can be defined as a review of the scientific literature using simplified and rapid methods compared to a traditional systematic review. In the Anglo-Saxon context, where rapid reviews are routinely being used,15–20 rapid reviews have also been described as rapid evidence assessment, rapid synthesis, rapid health technology assessment, and rapid evidence summary, among others.7 Planning and preparing a rapid review may be conditioned by a number of decision-making factors, including a health or clinical emergency, an immediate request to implement or adopt a new technology, procedure or service, and the limited time and resources that prevents a more thorough assessment. Table 1 shows the differences between narrative, rapid and systematic reviews.8,14 Unlike the narrative reviews, in systematic reviews we use the scientific method to search for potentially relevant studies together with explicit and reproducible criteria, fixed in advance in the selection. In contrast to systematic reviews, rapid reviews often omit or simplify stages to draft a rapid synthesis (e.g., limiting information sources and searches, limiting language and date of publication, not assessing the quality of primary studies, not
F. Catalá-López et al. / Med Clin (Barc). 2017;148(9):424–428
425
Table 1 Differences between narrative, rapid and systematic reviews. Feature
Narrative review
Rapid review
Systematic review
Period of time (approximately) Question of interest
Indeterminate
3 weeks to 6 months
6 months to 2–3 years
Non-specific, non-structured Non-detailed, non-systematic
Specific (although it may be extensive), structured They may be limited, non-comprehensive, although explicit sources and strategies Based on explicit criteria, uniformly applied Structured, it can include critical assessment (from previous systematic reviews, sometimes from primary studies) Descriptive, qualitative (rarely quantitative with meta-analysis when appropriate and time/resources allow)
Specific (on a well delimited clinical question), structured Structured, comprehensive, explicit sources and strategies
Sources and searches
Selection Evaluation of quality and information bias
Non-detailed, non-reproducible Absent
Synthesis
Descriptive, qualitative
Inferences
Generally based on opinion (bias), limited, cautious interpretation
performing meta-analysis). According to the Cochrane manual,21 in a systematic review all stages, including screening of studies (titles and abstracts; full text), data extraction and quality assessment (biases) of information should be typically carried out by 2 evaluators independently. In this regard, in rapid reviews, the stages of screening and selection of studies, and the data extraction are often performed by a single evaluator or reviewer (but not always). Some of these situations may make it possible to simplify and accelerate the synthesis of knowledge on a particular subject. However, it must also be considered that errors and potential biases can be added into the evaluation (e.g., omission of studies, errors in data extraction, publication and language biases, etc.).7 The evidence generated by rapid reviews may be considered to be exploratory in nature, generally aimed at reporting a decision, rather than responding more specific responses such as it happens with more comprehensive and thorough systematic reviews.1,3 Therefore, it should be a priority that the potential user (or promoter) of the evaluation understand, value and discuss these issues with the research team, when defining the question of interest and the methods (and the completeness of the analyses) to be applied considering the period of time available. Some examples of rapid reviews The 2 examples of rapid reviews19,20 shown below were developed by the Knowledge Synthesis Group of the Ottawa Hospital Health Research Institute to report decision making. In the first review, the researchers tried to respond to the requirements of the center professionals and management team to ensure the effective and efficient use of the hospital emergency services through a short-stay unit.19 In the second review, information was used to respond to the highest level of decision-making (international agencies, ministries of health, etc.) regarding the recommendations on personal protective equipment in the standards of practice in cases of suspected infection by Ebola virus.20 In both cases, the researchers performed the rapid reviews in less than 2 months. In 2011, the management team of the Ottawa Hospital, a public university hospital with over 1100 beds located in the Ontario region (Canada), suggested the idea of introducing short-stay units as an alternative hospitalization strategy facing the organizational challenge of the overloaded hospital emergency services. In this context, a rapid review on the effectiveness and safety of
Sometimes evidence-based, limited, cautious interpretation
Based on explicit criteria, uniformly applied Structured, rigorous, critical appraisal
Descriptive, qualitative and quantitative (usually includes meta-analysis when appropriate) Generally evidence-based
short-stay units was requested to report decision-making19 (Table 2). To this purpose, the research team defined the question of interest, and provided the methods and documents that would be delivered within the established time frame (4 weeks). We searched electronic databases and sources of gray literature. The eligibility criteria used included only papers published in English available in electronic full-text format. Researchers limited their study selection to published systematic reviews (given the heterogeneity of primary studies and the limited time available to collect information and understand the complexity of the evidence). They excluded papers published prior to year 2000 prioritizing the most recent evidence. The selection of papers was carried out by 2 evaluators. However, the extraction of information, quality evaluation and narrative synthesis were carried out by a single evaluator. The quality of the included systematic reviews was evaluated using the Assessing the Methodological Quality of Systematic Reviews – AMSTAR-tool.22 Totally, 4 published systematic reviews were included. Evidence from a systematic review (12 studies) of moderate quality indicated that short-stay units can lead to better outcomes (patient satisfaction, reduced stay, etc.) and improve efficiency in Emergency services. However, this systematic review was published over a decade ago. Therefore, the recommendation was to carry out a new comprehensive, rigorous and updated systematic review on the subject. Similarly, the researchers found that most of the comparative evaluations of short-stay units used “before and after” type designs, which have limitations such as confusion with natural evolution or the effects of any other (hidden) intervention, and the lack of masked assessment. The generalization of results was limited by the specific approach of the units evaluated (e.g., Cardiology, Pneumology). For this reason, and given the paucity of quality clinical trials, the researchers recommended caution in interpreting the findings. Evidence from the rapid review was positively responded by the hospital management team and was used to report the discussion on the implementation of a short-stay unit. It also helped strengthen partnerships between researchers and the management team, which has led to further research on this issue. Specifically, this rapid review was used as the basis for requesting for funding for a systematic review project that was successfully obtained by the team of researchers and other collaborators.23 The authors of the systematic review concluded that there is insufficient evidence to draw conclusions about the effectiveness and safety of short-stay units compared to hospital care.23 Therefore, the results of the systematic review support the
426
F. Catalá-López et al. / Med Clin (Barc). 2017;148(9):424–428
Table 2 Example of rapid review on the effectiveness and safety of short-stay units. Component
Characteristics
Requirement (potential user) Question of interest Methods
Overloaded hospital emergency services (management team and professionals)
Results Dissemination
What is the effectiveness and safety of short-stay units? Define inclusion/exclusion criteria (for example, inclusion of systematic reviews published in English available in full-text electronic format; exclusion of papers published before year 2000) Define and conduct searches of the medical literature (by an expert documentalist) in Medline, the Cochrane Library, Embase and the Center for Reviews and Dissemination) and gray literature sources (websites of scientific societies and assessment of health technologies) Screen and select the previous systematic reviews (by two researchers, one expert in methodology and one clinical expert in hospital emergencies) Data extraction (by a single researcher) Evaluate the quality of previous systematic reviews (by a single researcher, using the AMSTAR tool) Synthesize the results (narrative synthesis by a single researcher) Prepare the report of the rapid review and disseminate between the management team and health professionals (available through the institutional website: http://www.ohri.ca/kta/docs/KTA-ED-SSU-Evidence-Summary.pdf) [consulted Oct 2016] Publication of scientific article in medical journals19
idea that the authors of the rapid review considered this as a cautious conclusion until the results of a consistent systematic review were available. A rapid review performed in 7 weeks was issued in October 2015 to report on the development of the Rapid Advice Guideline on Personal Protective Equipment in the context of the filovirus outbreak of Ebola in West Africa, with special attention to the professionals directly responsible for patients with Ebola or Marburg filovirus disease20 (Table 3). Based on a research protocol developed in July 2014, search was performed in electronic databases and gray literature sources. The eligibility criteria used initially included comparative studies on Ebola and Marburg disease reported in English or French, but subsequently the criteria were expanded for studies on further viral hemorrhagic fevers and non-comparative designs due to the low number of studies. In this review, screening of title and abstract as well as the full text of the relevant articles were evaluated twice (2 people to exclude studies). However, the information was drawn from the studies by a single evaluator, and a second evaluator checked the information in just over half (57%) of the studies. The authors of the studies were not contacted to obtain any additional information due to the lack of time. The quality of the evidence was evaluated through the Grading of Recommendations Assessment, Development and Evaluation – GRADE – system.24 We identified 30 noncomparative observational studies (8 related to Ebola virus disease) and 27 provided data on viral transmission. Due to the nature and heterogeneity of the included studies, no meta-analysis of the results was performed. Based on low-quality evidence, it was concluded that the presentation of the protective equipment, and infection prevention and control protocols was deficient, which prevented from drawing conclusions about the comparative effectiveness of the various types of personal protection equipment (“absence of evidence” should not be understood as “evidence of absence”). In the rapid review, it was also identified the urgent need for further research that would improve the available evidence on the comparative effectiveness of the protective equipment currently used. This example demonstrates how a rapid review can be incorporated into a guideline, identify the need to improve available evidence and be useful to report urgent public health decisionmaking.25 Final considerations An important advantage of rapid reviews is their time of development and, therefore, their ability to guide and assist timely
decision making. However, it should also be recognized that in some situations the rapidity of the review may be limited. Often, to complete a review quickly, simplifying the stages of a traditional systematic review is required, which may result in potential errors and/or biases.7,8,14 There are differences between the development of a standard systematic review and a rapid review. However, the basic principles of systematic reviews also apply to rapid reviews, which would include the application of transparent processes, the use of explicit methods and the reduction of biases. Systematic reviews and meta-analysis play an important role in the planning, design and development of biomedical research.26,27 In fact, it has been suggested that all researchers should begin their research training by developing at least one systematic review.28 Perhaps a first step in this direction would be for future undergraduate and graduate students to begin with the methodology of evidence synthesis through rapid reviews. The best qualification of future generations (authors, reviewers and editors) might help increase the value and reduce the potential waste occurring in preparing reviews with little methodological accuracy.3 Currently, multiple and varied methodologies have been applied for preparing rapid reviews.29,30 However, the level of detail in the description of the methods and results of the reviews should be improved.7,13,29,30 Rapidness should not compromise the validity (or transparency) of the methods and results reported in a rapid review. Rapid reviews that do not use explicit or structured methods run the risk of becoming narrative reviews or opinion articles. In situations where key aspects of the methods and results of the rapid review are omitted due to lack of report, any assessment of the validity of the review is seriously hampered. Published rapid reviews should adhere to the guidelines for developing and submitting reviews. Work is currently being performed on initiatives that would include extensions of the current recommendations to cover more specifically aspects and particularities of rapid reviews. Specifically, an extension of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) for rapid reviews (PRISMA-RR) is being conducted.31 In addition, a new Cochrane Rapid Review Methods Group32 has recently been established, aimed at having an international leadership role in guiding the development of rapid reviews. It is expected that these initiatives will contribute to improving the methods and reporting future rapid reviews, including also those evidence-based programs using analogous terminologies (e.g. rapid assessments of health technologies, etc.).
F. Catalá-López et al. / Med Clin (Barc). 2017;148(9):424–428
427
Table 3 Example of a rapid review on the effectiveness of protective equipment for the prevention of Ebola virus transmission. Component
Characteristics
Requirement (potential user)
Outbreak and epidemic due to filovirus disease in West Africa, with special attention on Ebola or Marburg virus infection. Situation of international health emergency (international agencies, ministries of health, health professionals) What is the effectiveness of protective equipment for professionals/caregivers to prevent from Ebola virus transmission? Define inclusion/exclusion criteria (for example, inclusion of comparative studies on protective equipment in Ebola and/or Marburg disease reported in English or French, although the criteria were later expanded for studies on other viral hemorrhagic fevers and non-comparative designs due to the lack of studies) Define and conduct medical literature searches (by an expert documentalist) in Medline, the Cochrane Library, Embase and African Index Medicus and gray literature sources (ProQuest Dissertation and Theses Databases, Google, ClinicalTrials.gov and the international registry of WHO clinical trials) Screen and select the identified studies (by a single researcher, although reviewed by a second researcher) Data extraction (by a single researcher, although 57% of the information reviewed by a second researcher) Evaluate the quality of evidence (by a single researcher using the GRADE tool, although it does not assess the risk of bias in single studies, as there are no validated tools for non-comparative studies) Synthesize the results (narrative synthesis not including meta-analysis) Prepare the rapid review report used as a guideline for the discussions on providing the recommendations by the WHO: Personal protective equipment in the context of filovirus disease outbreak response. WHO/EVD/Guidance/PPE/14.1. Geneva, Switzerland: World Health Organization; 2014 Publication of scientific article in medical journals20
Question of interest Methods
Results Dissemination
Funding FCL has been granted by the Generalitat Valenciana (PROMETEOII/2015/021). BH has received funding from the Canadian Institutes of Health Research (New Investigator Award, Canadian Institutes of Health Research/Drug Safety and Effectiveness Network).
Conflict of interests The authors report no conflict of interest.
References 1. Moher D, Liberati A, Tetzlaff J, Altman DG, Group PRISMA. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6:e1000097. 2. Hutton B, Catalá-López F, Moher D. The PRISMA statement extension for systematic reviews incorporating network meta-analysis: PRISMA-NMA. Med Clin (Barc). 2016;147:262–6. 3. Page MJ, Shamseer L, Altman DG, Tetzlaff J, Sampson M, Tricco AC, et al. Epidemiology and reporting characteristics of systematic reviews of biomedical research: a cross-sectional study. PLoS Med. 2016;13:e1002028. 4. Alonso-Coello P, Schünemann HJ, Moberg J, Brignardello-Petersen R, Akl EA, Davoli M, et al. GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 1: introduction. BMJ. 2016;353:i2016. 5. Petticrew M, Roberts H. Systematic reviews in the social sciences: a practical guide. Malden, MA: Blackwell Publishing Co.; 2006. 6. Sampson M, Shojania KG, Garritty C, Horsley T, Ocampo M, Moher D. Systematic reviews can be produced and published faster. J Clin Epidemiol. 2008;61: 531–6. 7. Tricco AC, Antony J, Zarin W, Strifler L, Ghassemi M, Ivory J, et al. A scoping review of rapid review methods. BMC Med. 2015;13:224. 8. Khangura S, Konnyu K, Cushman R, Grimshaw J, Moher D. Evidence summaries: the evolution of a rapid review approach. Syst Rev. 2012;1:10. 9. Harker J, Kleijnen J. What is a rapid review? A methodological exploration of rapid reviews in Health Technology Assessments. Int J Evid Based Healthc. 2012;10:397–410. 10. Hailey D, Corabian P, Harstall C, Schneider W. The use and impact of rapid health technology assessments. Int J Technol Assess Health Care. 2000;16: 651–6. 11. Thomas J, Newman M, Oliver S. Rapid evidence assessments of research to inform social policy: taking stock and moving forward. Evid Policy. 2013;9: 5–27. 12. Khangura S, Polisena J, Clifford TJ, Farrah K, Kamel C. Rapid review: an emerging approach to evidence synthesis in health technology assessment. Int J Technol Assess Health Care. 2014;30:20–7.
13. Polisena J, Garritty C, Kamel C, Stevens A, Abou-Setta AM. Rapid review programs to support health care and policy decision making: a descriptive analysis of processes and methods. Syst Rev. 2015;4:26. 14. Watt A, Cameron A, Sturm L, Lathlean T, Babidge W, Blamey S, et al. Rapid reviews versus full systematic reviews: an inventory of current methods and practice in health technology assessment. Int J Technol Assess Health Care. 2008;24: 133–9. ˜ 15. Branas P, Jordan R, Fry-Smith A, Burls A, Hyde C. Treatments for fatigue in multiple sclerosis: a rapid and systematic review. Health Technol Assess. 2000;4:1–61. 16. Bridle C, Palmer S, Bagnall AM, Darba J, Duffy S, Sculpher M, et al. A rapid and systematic review and economic evaluation of the clinical and cost-effectiveness of newer drugs for treatment of mania associated with bipolar affective disorder. Health Technol Assess. 2004;8:iii–v, 1-187. 17. Beall RF, Baskerville N, Golfam M, Saeed S, Little J. Modes of delivery in preventive intervention studies: a rapid review. Eur J Clin Invest. 2014;44:688–96. 18. Luckett T, Phillips J, Agar M, Virdun C, Green A, Davidson PM. Elements of effective palliative care models: a rapid review. BMC Health Serv Res. 2014;14: 136. 19. Konnyu KJ, Kwok E, Skidmore B, Moher D. The effectiveness and safety of emergency department short stay units: a rapid review. Open Med. 2012;6: e10–6. 20. Hersi M, Stevens A, Quach P, Hamel C, Thavorn K, Garritty C, et al. Effectiveness of personal protective equipment for healthcare workers caring for patients with filovirus disease: a rapid review. PLoS One. 2015;10:e0140290. 21. The Cochrane Collaboration. Manual Cochrane de revisiones sistemáticas de intervenciones. Versión 5.1.0 (traducida). Barcelona: Centro Cochrane Iberoamericano; 2011 [consulted 17 Oct 2016]. Available in: https://es. cochrane.org/sites/es.cochrane.org/files/uploads/Manual Cochrane 510 reduit.pdf 22. Shea BJ, Grimshaw JM, Wells GA, Boers M, Andersson N, Hamel C, et al. Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol. 2007;7:10. 23. Galipeau J, Pussegoda K, Stevens A, Brehaut JC, Curran J, Forster AJ, et al. Effectiveness and safety of short-stay units in the emergency department: a systematic review. Acad Emerg Med. 2015;22:893–907. 24. Alonso-Coello P, Rigau D, Solà I, Martínez García L. Formulating health care recommendations: the GRADE system. Med Clin (Barc). 2013;140: 366–73. 25. Garritty CM, Norris SL, Moher D. Developing WHO rapid advice guidelines in the setting of a public health emergency. J Clin Epidemiol. 2016, http://dx.doi.org/10.1016/j.jclinepi.2016.08.010, pii: S0895-4356(16)30386-9. 26. Catalá-López F, González-Bermejo D, de la Fuente Honrubia C, Macías SaintGerons D. Clinical equipoise and systematic reviews of randomized controlled trials. Med Clin (Barc). 2015;145:496–8. 27. Ioannidis JP. Why most clinical research is not useful. PLoS Med. 2016;13:e1002049. 28. Mahtani KR. All health researchers should begin their training by preparing at least one systematic review. J R Soc Med. 2016;109:264–8. 29. Tricco AC, Zarin W, Antony J, Hutton B, Moher D, Sherifali D, et al. An international survey and modified Delphi approach revealed numerous rapid review methods. J Clin Epidemiol. 2016;70:61–7. 30. Kelly SE, Moher D, Clifford TJ. Quality of conduct and reporting in rapid reviews: an exploration of compliance with PRISMA and AMSTAR guidelines. Syst Rev. 2016;5:79.
428
F. Catalá-López et al. / Med Clin (Barc). 2017;148(9):424–428
31. EQUATOR network. Reporting guidelines under development. PRISMA-RR 2017: an extension to PRISMA for rapid reviews. [consulted 17 Oct 2016]. Available in: http://www.equator-network.org/library/reporting-guidelines-underdevelopment/#51
32. Garritty C, Stevens A, Gartlehner G, King V, Kamel C, Cochrane Rapid Reviews Methods Group. Cochrane Rapid Reviews Methods Group to play a leading role in guiding the production of informed high-quality, timely research evidence syntheses. Syst Rev. 2016;5:184.