- Email: [email protected]
Evidence into practice: How do we get past the roadblocks?
Paediatric Respiratory Reviews 15 (2014) 45–46
Contents lists available at ScienceDirect
Paediatric Respiratory Reviews
Cochrane Corner
Evidence into practice: How do we get past the roadblocks? Alan R. Smyth Division of Child Health, Obstetrics and Gynaecology, School of Medicine, University of Nottingham
In this Cochrane Corner we take a helicopter view of the whole enterprise of bringing best evidence into clinical practice. On this long road there are many potential roadblocks. These blocks are summarised in the figure. The Cochrane Collaboration addresses the third block – the lack of systematic reviews of existing trials. The collaboration was established to prepare impartial and up to date systematic reviews, to ensure that health professionals, patients and policy makers have access to the best possible evidence to make decisions. Eighteen years elapsed between the first trial clearly showing the effectiveness of antenatal steroids in reducing the risk of respiratory distress syndrome in premature neonates [1] and the systematic review which changed practice [2]. Many infants may have died or suffered unnecessary harm because of that 18 year delay. The founders of the collaboration believed that an international network of volunteers, preparing systematic reviews of therapy, in every medical specialty, could avoid such long delays in applying evidence to practice. The outcome would be to improve human health and save lives (Figure 1). But the diagram makes it clear that there are other blocks in the evidence pathway, both upstream and downstream. Recruitment into clinical trials is a key challenge (1st block). Around 5% of a sample of clinical trials in children, identified through the clinicaltrials.gov database [3] were suspended or terminated half because of poor recruitment. Completed trials may experience poor recruitment and fail to achieve the sample size needed for sufficient statistical power to answer the research question. One study found that only half of negative or indeterminate phase III trials in rheumatology had an adequate sample size [4]. A number of underpowered trials showing no difference, may create a misconception that there is no difference between treatments, where in fact a true difference has been obscured by a type II error. The 2nd block is non publication. An important step to overcome this obstacle was made in the US, with the introduction of the FDA Amendment Act 2007, which required the publication of summary results from specified trials on clinicaltrials.gov (the major US trials registry). However, a study of over 700 trials which were subject to
1526-0542/$ – see front matter ß 2013 Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.prrv.2013.11.008
mandatory reporting showed that only 22% of RCTs registered on clinicaltrials.gov had results published on line within one year as mandated by US law [5]. Even when robust evidence exists to guide the choice of therapy and this has been evaluated in a systematic review, clinical practice may not change (4th block). Six years elapsed between a metaanalysis showing thrombolysis saves lives in acute myocardial infarction [6] and the routine recommendation of thrombolysis by experts [7]. The 5th and final block arises where guideline evidence is ignored. Even when there is a published guideline to guide therapy, incorporating the best available evidence, clinicians don’t always follow guidelines. Surveys of guideline adherence in respiratory disease have a had varying results. A study of adherence [8] to the US Cystic Fibrosis Foundation Pulmonary Guidelines [9] found 78% of respondents prescribed 4 medications which were either level A (‘‘strongly recommended’’ - nebulised tobramycin and dornase alfa) or level B (‘‘recommended’’ - Azithromycin and nebulised hypertonic saline). In contrast, a recent US survey [10] showed that adherence to American Thoracic Society antimicrobial guidelines for treating pulmonary infection with non-tuberculous mycobacteria [11] was only 13%. What can be done to get past the roadblocks? The Cochrane Collaboration has made huge strides in the last 20 years towards overcoming the 3rd block, though it faces challenges in the next decade, including finding the best way to use evidence from nonrandomised studies, keeping a growing body of systematic reviews up to date and compiling reviews of diagnostic test accuracy. Recruitment difficulties (1st block) may be addressed with greater patient participation in trial design – patients may wish to participate if the trial design reflects their priorities. The scandal of unpublished trials data has been the subject of the ‘‘AllTrials’’ campaign. http://www.alltrials.net In the US, new legislation has been proposed - the Trial and Experimental Studies Transparency (TEST) Act, which will require publication of trial data, regardless of phase, design or approval status [12]. However, perhaps the greatest challenge will be changing the behaviour of the guideline
A.R. Smyth / Paediatric Respiratory Reviews 15 (2014) 45–46
46
Figure 1. The pathway to better healthcare, through applying the findings of randomised controlled trials in clinical practice - examples of the major blocks.
writers and health professionals to address the 4th and 5th blocks to implementing best evidence. Anyone for a new collaboration? In collaboration with the Cochrane CF and Genetic Disorders Group’ http://cfgd.cochrane.org/. References [1] Liggins GC, Howie RN. A controlled trial of antepartum glucocorticoid treatment for prevention of the respiratory distress syndrome in premature infants. Pediatr 1972;50(4):515–25. [2] Crowley P, Chalmers I, Keirse MJ. The effects of corticosteroid administration before preterm delivery: an overview of the evidence from controlled trials. Br J Obstet Gynaecol 1990;97(1):11–25. [3] Shamliyan T, Kane RL. Clinical research involving children: registration, completeness, and publication. Pediatr 2012;129(5):e1291–300. [4] Keen HI, Pile K, Hill CL. The prevalence of underpowered randomized clinical trials in rheumatology. J Rheumatol 2005;32(11):2083–8. [5] Prayle AP, Hurley MN, Smyth AR. Compliance with mandatory reporting of clinical trial results on ClinicalTrials.gov: cross sectional study. BMJ 2012;344: d7373.
[6] Stampfer MJ, Goldhaber SZ, Yusuf S, Peto R, Hennekens CH. Effect of intravenous streptokinase on acute myocardial infarction: pooled results from randomized trials. N Engl J Med 1982;307(19):1180–2. [7] Antman EM, Lau J, Kupelnick B, Mosteller F, Chalmers TC. A comparison of results of meta-analyses of randomized control trials and recommendations of clinical experts. Treatments for myocardial infarction. JAMA 1992;268(2): 240–8. [8] Glauser TA, Nevins PH, Williamson JC, et al. Adherence to the 2007 cystic fibrosis pulmonary guidelines: A national survey of CF care centers. Pediatr Pulmonol 2012;47(5):434–40. [9] Flume PA, O’Sullivan BP, Robinson KA, et al. Cystic fibrosis pulmonary guidelines: chronic medications for maintenance of lung health. Am J Respir Crit Care Med 2007;176(10):957–69. [10] Adjemian J, Prevots DR, Gallagher J, Heap K, Gupta R, Griffith D. Lack of Adherence to Evidence-based Treatment Guidelines for Nontuberculous Mycobacterial Lung Disease. Ann Am Thorac Soc 2013. [11] Griffith DE, Aksamit T, Brown-Elliott BA, et al. An official ATS/IDSA statement: diagnosis, treatment, and prevention of nontuberculous mycobacterial diseases. Am J Respir Crit Care Med 2007;175(4):367–416. [12] Drazen JM. Transparency for Clinical Trials: The TEST Act. N Engl J Med 2012;367(9):863–4.
In collaboration with the Cochrane CF and Genetic Disorders Group’ http://cfgd.cochrane.org/.
Contents lists available at ScienceDirect
Paediatric Respiratory Reviews
Cochrane Corner
Evidence into practice: How do we get past the roadblocks? Alan R. Smyth Division of Child Health, Obstetrics and Gynaecology, School of Medicine, University of Nottingham
In this Cochrane Corner we take a helicopter view of the whole enterprise of bringing best evidence into clinical practice. On this long road there are many potential roadblocks. These blocks are summarised in the figure. The Cochrane Collaboration addresses the third block – the lack of systematic reviews of existing trials. The collaboration was established to prepare impartial and up to date systematic reviews, to ensure that health professionals, patients and policy makers have access to the best possible evidence to make decisions. Eighteen years elapsed between the first trial clearly showing the effectiveness of antenatal steroids in reducing the risk of respiratory distress syndrome in premature neonates [1] and the systematic review which changed practice [2]. Many infants may have died or suffered unnecessary harm because of that 18 year delay. The founders of the collaboration believed that an international network of volunteers, preparing systematic reviews of therapy, in every medical specialty, could avoid such long delays in applying evidence to practice. The outcome would be to improve human health and save lives (Figure 1). But the diagram makes it clear that there are other blocks in the evidence pathway, both upstream and downstream. Recruitment into clinical trials is a key challenge (1st block). Around 5% of a sample of clinical trials in children, identified through the clinicaltrials.gov database [3] were suspended or terminated half because of poor recruitment. Completed trials may experience poor recruitment and fail to achieve the sample size needed for sufficient statistical power to answer the research question. One study found that only half of negative or indeterminate phase III trials in rheumatology had an adequate sample size [4]. A number of underpowered trials showing no difference, may create a misconception that there is no difference between treatments, where in fact a true difference has been obscured by a type II error. The 2nd block is non publication. An important step to overcome this obstacle was made in the US, with the introduction of the FDA Amendment Act 2007, which required the publication of summary results from specified trials on clinicaltrials.gov (the major US trials registry). However, a study of over 700 trials which were subject to
1526-0542/$ – see front matter ß 2013 Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.prrv.2013.11.008
mandatory reporting showed that only 22% of RCTs registered on clinicaltrials.gov had results published on line within one year as mandated by US law [5]. Even when robust evidence exists to guide the choice of therapy and this has been evaluated in a systematic review, clinical practice may not change (4th block). Six years elapsed between a metaanalysis showing thrombolysis saves lives in acute myocardial infarction [6] and the routine recommendation of thrombolysis by experts [7]. The 5th and final block arises where guideline evidence is ignored. Even when there is a published guideline to guide therapy, incorporating the best available evidence, clinicians don’t always follow guidelines. Surveys of guideline adherence in respiratory disease have a had varying results. A study of adherence [8] to the US Cystic Fibrosis Foundation Pulmonary Guidelines [9] found 78% of respondents prescribed 4 medications which were either level A (‘‘strongly recommended’’ - nebulised tobramycin and dornase alfa) or level B (‘‘recommended’’ - Azithromycin and nebulised hypertonic saline). In contrast, a recent US survey [10] showed that adherence to American Thoracic Society antimicrobial guidelines for treating pulmonary infection with non-tuberculous mycobacteria [11] was only 13%. What can be done to get past the roadblocks? The Cochrane Collaboration has made huge strides in the last 20 years towards overcoming the 3rd block, though it faces challenges in the next decade, including finding the best way to use evidence from nonrandomised studies, keeping a growing body of systematic reviews up to date and compiling reviews of diagnostic test accuracy. Recruitment difficulties (1st block) may be addressed with greater patient participation in trial design – patients may wish to participate if the trial design reflects their priorities. The scandal of unpublished trials data has been the subject of the ‘‘AllTrials’’ campaign. http://www.alltrials.net In the US, new legislation has been proposed - the Trial and Experimental Studies Transparency (TEST) Act, which will require publication of trial data, regardless of phase, design or approval status [12]. However, perhaps the greatest challenge will be changing the behaviour of the guideline
A.R. Smyth / Paediatric Respiratory Reviews 15 (2014) 45–46
46
Figure 1. The pathway to better healthcare, through applying the findings of randomised controlled trials in clinical practice - examples of the major blocks.
writers and health professionals to address the 4th and 5th blocks to implementing best evidence. Anyone for a new collaboration? In collaboration with the Cochrane CF and Genetic Disorders Group’ http://cfgd.cochrane.org/. References [1] Liggins GC, Howie RN. A controlled trial of antepartum glucocorticoid treatment for prevention of the respiratory distress syndrome in premature infants. Pediatr 1972;50(4):515–25. [2] Crowley P, Chalmers I, Keirse MJ. The effects of corticosteroid administration before preterm delivery: an overview of the evidence from controlled trials. Br J Obstet Gynaecol 1990;97(1):11–25. [3] Shamliyan T, Kane RL. Clinical research involving children: registration, completeness, and publication. Pediatr 2012;129(5):e1291–300. [4] Keen HI, Pile K, Hill CL. The prevalence of underpowered randomized clinical trials in rheumatology. J Rheumatol 2005;32(11):2083–8. [5] Prayle AP, Hurley MN, Smyth AR. Compliance with mandatory reporting of clinical trial results on ClinicalTrials.gov: cross sectional study. BMJ 2012;344: d7373.
[6] Stampfer MJ, Goldhaber SZ, Yusuf S, Peto R, Hennekens CH. Effect of intravenous streptokinase on acute myocardial infarction: pooled results from randomized trials. N Engl J Med 1982;307(19):1180–2. [7] Antman EM, Lau J, Kupelnick B, Mosteller F, Chalmers TC. A comparison of results of meta-analyses of randomized control trials and recommendations of clinical experts. Treatments for myocardial infarction. JAMA 1992;268(2): 240–8. [8] Glauser TA, Nevins PH, Williamson JC, et al. Adherence to the 2007 cystic fibrosis pulmonary guidelines: A national survey of CF care centers. Pediatr Pulmonol 2012;47(5):434–40. [9] Flume PA, O’Sullivan BP, Robinson KA, et al. Cystic fibrosis pulmonary guidelines: chronic medications for maintenance of lung health. Am J Respir Crit Care Med 2007;176(10):957–69. [10] Adjemian J, Prevots DR, Gallagher J, Heap K, Gupta R, Griffith D. Lack of Adherence to Evidence-based Treatment Guidelines for Nontuberculous Mycobacterial Lung Disease. Ann Am Thorac Soc 2013. [11] Griffith DE, Aksamit T, Brown-Elliott BA, et al. An official ATS/IDSA statement: diagnosis, treatment, and prevention of nontuberculous mycobacterial diseases. Am J Respir Crit Care Med 2007;175(4):367–416. [12] Drazen JM. Transparency for Clinical Trials: The TEST Act. N Engl J Med 2012;367(9):863–4.
In collaboration with the Cochrane CF and Genetic Disorders Group’ http://cfgd.cochrane.org/.