- Email: [email protected]
Different methods of allocation to groups in randomized trials are associated with different levels of bias. A meta-epidemiological study
Journal of Clinical Epidemiology 64 (2011) 1070e1075
Different methods of allocation to groups in randomized trials are associated with different levels of bias. A meta-epidemiological study Peter Herbisona,*, Jean Hay-Smithb, William J. Gillespiec b
a Department of Preventive and Social Medicine, Dunedin School of Medicine, University of Otago, PO Box 913, Dunedin 9054, New Zealand Rehabilitation Teaching and Research Unit, University of Otago, Wellington, c/- Women’s and Children’s Health, PO Box 913, Dunedin 9054, New Zealand c Hull York Medical School, Cottingham Road, Hull HU6 7RX, UK
Accepted 10 December 2010
Abstract Objective: Insecure hiding of the treatment allocation in randomized trials is associated with bias. It is less certain how much bias is associated with different methods of treatment allocation. Study Design and Setting: Meta-epidemiological study of 389 randomized trials from 19 systematic reviews and 65 meta-analyses with differing methods of treatment allocation. Pooled ratios of odds ratios (RORs) and 95% confidence intervals (95% CI) were calculated from trials with different methods of treatment allocation. An ROR less than one shows exaggeration of treatment effect. Results: There is no evidence that the use of sealed envelopes with enhancement was different from central randomization (ROR 1.02, 95% CI: 0.85e1.23). Sealed envelopes without enhancement were associated with an exaggeration of the estimate of effect (ROR 0.87, 95% CI: 0.76e1.00). Where allocation concealment for double-blind trials was unclear, the ROR is 0.86 (95% CI: 0.78e0.96) and if not hidden, the ROR is 0.89 (95% CI: 0.70e1.15). Conclusion: Sealed envelopes with some form of enhancement (opaque, sequentially numbered, and so forth) may give adequate concealment. Description of a study as "double blind" does not imply a lack of bias when concealment of allocation is unclear. Ó 2011 Elsevier Inc. All rights reserved. Keywords: Allocation concealment; Bias; Meta-epidemiology; Blinding; Randomized trials; Meta-analysis
1. Introduction Although the best evidence of the effect of medical interventions comes from systematic reviews and metaanalyses of randomized trials, these are potentially subject to bias. Often these biases are associated with how trials are carried out. A number of methodological studies have examined the method of allocation to groups and have found empirical evidence of bias [1e6]. Meta-analysis of these methodological studies [7e9] indicates that this bias results in an overestimate of treatment effect of around 20% for studies with inadequate allocation concealment compared with those that have more secure methods of concealing the allocation. In another study, which reported data from 499 trials included in 70 metaanalyses [10], two-thirds of conclusions in favor of one of the interventions were no longer supported if only trials * Corresponding author. Department of Preventive and Social Medicine, Dunedin School of Medicine, PO Box 913, Dunedin 9054, New Zealand. Tel.: þ64-3-479-7217; fax: þ64-3-479-7298. E-mail address: [email protected] (P. Herbison). 0895-4356/$ - see front matter Ó 2011 Elsevier Inc. All rights reserved. doi: 10.1016/j.jclinepi.2010.12.018
with adequate allocation concealment were included in the meta-analysis. The amount of bias associated with allocation concealment may also depend on the outcome, with objective outcomes being associated with less bias than subjective outcomes [11]. Previous studies examining bias associated with allocation concealment mostly coded the method of group allocation as hidden and unclear [2e6], or in one case, hidden, unclear, and not hidden [1]. However, there are many ways of allocating participants to groups, ranging from those that are more securely hidden, such as a third party Web site or telephone access or allocation of numbered prepacked medications or identical placebos by a pharmacy (central randomization), to those that are clearly not hidden, such as whether the participants date of birth is odd or even. Between central randomization and the clearly not hidden lies other methods, which may be somewhat insecure. Reporting of allocation concealment is often insufficient to make a clear judgment. This study explores, at a finer level of detail than has been done previously, the methods of group allocation in individual trials and the associated biases.
P. Herbison et al. / Journal of Clinical Epidemiology 64 (2011) 1070e1075
What is new? Key findings: Inadequate concealment of allocation can lead to exaggeration of treatment effects in randomized controlled trials. Many methods of allocation to groups are used, and it is not clear which provide adequate concealment. This study provides estimates of the degree of exaggeration associated with different allocation methods. What this adds: Sealed envelopes with extra security features appear to provide adequate concealment. Saying only that a study is double blind is not sufficient to ensure lack of exaggeration of effect. Implications: If it is not possible to use a truly secure method of allocation, then sealed, opaque, sequentially numbered envelopes may provide adequate allocation concealment.
2. Methods Full details of the methods of data collection were reported previously [12]. Briefly, systematic reviews in issue 1, 2001 of the Cochrane Library were searched for binary outcomes that had at least 10 included trials, at least one of which had more than 500 people randomized to each arm. For these outcomes, the report of the contributing trials was found, and the data were extracted in duplicate by two trained research assistants. The results for the binary outcomes were taken from the Cochrane Review as the total number and number positive for each arm in the study. Each included trial was coded for the method of allocation into one of six categories. Category 1 comprised trials that used some form of central randomization that clearly should hide the allocation, such as a remote telephone service or randomization by a pharmacy. Category 2 comprised trials that used sealed envelopes with some form of security enhancement, such as ensuring that envelopes were opaque and numbered. Category 3 comprised trials that used sealed envelopes without any further details. Category 4 comprised trials that were reported as randomized without details, and also as "double blind." Category 5 comprised trials that simply said they were randomized with no further details. Category 6 comprised trials where the allocation was clearly not hidden, for example, being based on an open list, odd or even days of the week, participant’s birth date, or the team on duty at enrollment. Categorization into one of these six groups was conducted
1071
independently by two people, with discussion with a third person to resolve any disagreements. To facilitate comparison with previous meta-analyses, these six categories were combined to create two groups: an "adequate concealment" group combining groups 1 and 2 and the remainder to form an "inadequate or unclear concealment" group. Analysis was done in Stata V10 (StataCorp, College Station, TX) using the meta-epidemiology approach described by Sterne et al. [13]. Odds ratios (ORs) and their standard errors were calculated from the appropriate two by two tables using the metan routine in Stata [14]. This was used as metan adds variables for the natural log of the OR and its standard error to the file. This process was not used to produce pooled results. Then random-effects meta-regression was used on each meta-analysis using the method of allocation as the covariate [15]. The central randomization group was used as the reference group for the six-group comparison, and "adequate concealment" was used for the two-group comparison. The coefficients in the meta-regression give ratios of ORs (RORs), which were themselves combined using metan with random effects to give pooled RORs with 95% confidence intervals (95% CIs). The data from the meta-analyses were not independent as more than one meta-analysis from a systematic review was included. These often contained the same studies. To allow for this, the meta-analysis of the RORs was bootstrapped to get bootstrap CIs with one metaanalysis being chosen at random from each systematic review, and the results were analyzed. This was repeated 1,000 times. Because of the results of this analysis, a post hoc comparison between security-enhanced envelopes and envelopes without enhancement was carried out. Calculations were always done so that ORs less than one indicated beneficial effects of the treatment. This implies that an exaggeration of the treatment effect would lead to an ROR less than one. The pooled ROR for the two-category result (adequate/not adequate) was added to the meta-analysis of similar studies reported by J€uni et al. [7] and then Egger and Ebrahim [8] and since updated by Gluud [9] but with the correction from Kjaergard et al. [16]. Random-effects meta-analysis was used for this calculation.
3. Results The search identified 67 meta-analyses from 19 systematic reviews. Two of these, from the same systematic review, had to be discarded as too many of the studies had zero events in both arms. The remaining 65 meta-analyses from 18 systematic reviews included 389 studies. Details of the systematic reviews and outcomes are given in Table 1. The number of studies in individual meta-analyses ranged from 10 to 63. Sixty-seven studies used central randomization (category 1), 18 used enhanced envelopes (category 2), 48 envelopes with no further details (category 3), 121 reported that they were double blind but had unclear allocation
1072
P. Herbison et al. / Journal of Clinical Epidemiology 64 (2011) 1070e1075
Table 1 Meta-analyses and outcomes included 1. Henry DA, Moxey AJ, Carless PA, O’Connell D, McClelland B, Henderson KM, Sly K, Laupacis A, Fergusson D. Anti-fibrinolytic use for minimising perioperative allogeneic blood transfusion. In: The Cochrane Library, Issue 1, 2001. Oxford: Update Software. Outcomes 1. No. exposed to allogeneic blood 2. No. exposed to allogeneic blood by operation type (cardiac) 3. No. exposed to allogeneic blood by Transfusion Protocol 2. Knight M, Duley L, Henderson-Smart DJ, King JF. Antiplatelet agents for preventing and treating pre-eclampsia. The Cochrane Library, Issue 1, 2001. Oxford: Update Software. Outcomes 1. Pregnancy-induced hypertension (moderate risk) 2. Proteinuric pre-eclampsia (moderate risk) 3. C-section (moderate risk) 4. Preterm delivery !37 wk (moderate risk) 5. Fetal, neonatal, or infant death (moderate risk) 6. Fetal or neonatal deaths subgrouped by time of death (stillbirth) 7. Small for gestational age (moderate risk) 8. Pregnancy-induced hypertension (>20 wk) 9. Proteinuric pre-eclampsia (!20 wk) 10. Proteinuric pre-eclampsia (>20 wk) 11. Fetal neonatal or infant death (>20 wk) 12. Small for gestational age (>20 wk) 13. Pregnancy-induced hypertension 14. Proteinuric pre-eclampsia 15. Preterm delivery 3. Williams Jr. JW, Aguilar C, Makela M, Cornell J, Hollman DR, Chiquette E, Simel DL. Antibiotics for acute maxillary sinusitis. In: The Cochrane Library, Issue 1, 2001. Oxford: Update Software. Outcomes 1. Clinical cure or improvement 2. Dropout because of adverse effects 4. Hodnett ED. Caregiver support for women during childbirth. In: The Cochrane Library, Issue 1, 2001. Oxford: Update Software Outcomes 1. Oxytocin augmentation 2. Any analgesia/anesthesia during labor 3. Operative vaginal delivery (all women) 4. Caesarean delivery 5. Horn J, Limburg M. Calcium antagonists for acute ischemic stroke. In: The Cochrane Library, Issue 1, 2001. Oxford: Update Software. Outcomes 1. Poor outcome at end of follow-up (nimodipine vs. control) 2. Death at end of follow-up (nimodipine vs. control) 3. Death at end of treatment period (nimodipine vs. control) 4. Adverse events (ALL) during treatment period (nimodipine vs. control) 5. Poor outcome by time of start of treatment (treatment started O12 hr) 6. Methodology: randomization, concealment (poor outcome in good-quality trials) 7. Methodology: publication status (poor outcome in published trials) 6. Neilson JP, Alfirevic Z. Doppler ultrasound for foetal assessment in high-risk pregnancies. In: The Cochrane Library, Issue 1, 2001. Oxford: Update Software. Outcomes 1. Perinatal deaths 2. Stillbirths, normally formed 3. Neonatal deaths, normally formed 7. Asplund K, Israelsson K, Schampi I. Haemodilution for acute ischaemic stroke. In: The Cochrane Library, Issue 1, 2001. Oxford: Update Software. Outcomes 1. Case fatality at early follow-up (!28 days) 2. Case fatality at late follow-up (3e6 mo) 8. K Rees, M Beranek-Stanley, M Burke, S Ebrahim. Hypothermia to reduce neurological damage following coronary artery bypass surgery. In: The Cochrane Library, Issue 1, 2001. Oxford: Update Software. Outcomes 1. Nonfatal stroke 2. Perioperative deaths (not strokes) 3. Nonfatal myocardial infarction 4. Low output syndrome 5. Intra-aortic balloon pump use 6. Pooled "bad" outcomes 9. Alejandria MM, Lansang MA, Dans LF, Mantaring JBV. Intravenous immunoglobulin for treating sepsis and septic shock. In: The Cochrane Library, Issue 1, 2001. Oxford: Update Software. Outcomes 1. All-cause mortality (adults) 2. All-cause mortality (all ages) (Continued )
P. Herbison et al. / Journal of Clinical Epidemiology 64 (2011) 1070e1075
1073
Table 1 Continued 10. Ohlsson A, Lacy JB. Intravenous immunoglobulin for preventing infection in preterm and/or low-birth-weight infants. In: The Cochrane Library, Issue 1, 2001. Oxford: Update Software. Outcomes 1. Any serious infection 2. Mortality (total) 11. Schmitt B, Bennett C, Seidenfeld J, Samson D, Wilt T. Maximal androgen blockade for advanced prostate cancer. In: The Cochrane Library, Issue 1, 2001. Oxford: Update Software. Outcomes 1. Overall survival 1 yr 2. Overall survival 2 yr 12. Silagy C, Mant D, Fowler G, Lancaster T. Nicotine replacement therapy for smoking cessation. In: The Cochrane Library, Issue 1, 2001. Oxford: Update Software. (Gum). Outcomes 1. Smoking cessation at max. follow-up 6e12 mo (total) 2. Smoking cessation at max. follow-up 6e12 mo (low intensity) 3. Smoking cessation at max. follow-up 6e12 mo (high intensity) 13. Silagy C, Mant D, Fowler G, Lancaster T. Nicotine replacement therapy for smoking cessation. In: The Cochrane Library, Issue 1, 2001. Oxford: Update Software. (Patch). Outcomes 1. Smoking cessation at max. follow-up 6e12 mo (total) 2. Smoking cessation at max. follow-up 6e12 mo (low intensity) 3. Smoking cessation at max. follow-up 6e12 mo (high intensity) 14. Tan BP, Hannah ME. Oxytocin for prelabour rupture of membranes at or near term. In: The Cochrane Library, Issue 1, 2001. Oxford: Update Software. Outcomes 1. Chorioamnionitis 2. Caesarean section 3. Neonatal infection 15. Tan BP, Hannah ME. Prostaglandins versus oxytocin for prelabour rupture of membranes at or near term. In: The Cochrane Library, Issue 1, 2001. Oxford: Update Software. Outcomes 1. Caesarean section 2. Operative delivery 16. Crowley P. Prophylactic corticosteroids for preterm birth. In: The Cochrane Library, Issue 1, 2001. Oxford: Update Software. Outcomes 1. Neonatal death (all babies) 2. Stillbirth (all babies) 17. Tan BP, Hannah ME. Prostaglandins for prelabour rupture of membranes at or near term. In: The Cochrane Library, Issue 1, 2001. Oxford: Update Software. Outcomes 1. Caesarean section 2. Neonatal infection 18. Gupta JK, Nikodem VC. Woman’s position during second stage of labour. In: The Cochrane Library, Issue 1, 2001. Oxford: Update Software. Outcomes 1. Mode of delivery (assisted delivery) 2. Episiotomy
(category 4), 104 had unclear allocation (category 5), and in 31 the allocation was clearly not hidden (category 6). Nine meta-analyses were omitted from the six-group comparison as they had no studies in the reference group, and two were omitted from the two-group comparison. The pooled RORs for the different allocation methods compared with central randomization are given in Table 2. Overall, studies with inadequate allocation concealment (groups 3e6) show an exaggeration of the effect size by 9% (95% CI: 1e17%) when compared with studies with adequate concealment. The comparisons of groups 2e6, individually, with group 1 show that sealed opaque envelopes with enhanced security (group 2) were not significantly different (ROR 1.02, 95% CI: 0.85e1.22) from central randomization although the CI is wide. Envelopes without enhanced security (group 3) show increased exaggeration although this was only just statistically significant. Studies that gave no details of the randomization (group 4) but said they
were double blind exaggerate the effect size (ROR 0.86, 95% CI: 0.78e0.96). The estimated ROR from the post hoc comparison of security-enhanced envelopes with no enhancement is 0.87 (95% CI: 0.73e1.05). Incorporation of our overall result into the most recent meta-analysis of studies of allocation concealment changes the overall ROR from 0.83 (95% CI: 0.71e0.96) to 0.84 (95% CI: 0.75e0.95) (Fig. 1).
4. Discussion This study confirmed the finding of previous studies that apparently less secure methods of allocating participants to groups in randomized controlled trials are indeed associated with bias. Our results, overall, are consistent with those studies [1e6,16], which reported RORs ranging from 0.60 to 1.02. This study found an ROR of 0.91 for the same
1074
P. Herbison et al. / Journal of Clinical Epidemiology 64 (2011) 1070e1075
Table 2 RORs associated with different methods of allocation to groups in randomized controlled trials Method of allocation
RORs (bootstrap 95% CI)
Number of meta-analyses contributing to estimate
t2
Six-group comparison Central randomization Enhanced envelopes Envelopes with no further elaboration Unclear but stated that trial is double blind Unclear Clearly not hidden
Reference 1.02 (0.85e1.22) 0.87 (0.76e1.00) 0.86 (0.78e0.96) 0.76 (0.66e0.87) 0.89 (0.70e1.15)
16 39 44 50 22
0 0 0.047 0.130 0.123
Two-group comparison Adequate concealment Inadequate or unclear concealment
Reference 0.91 (0.83e0.99)
63
0.036
Abbreviations: ROR, ratio of odds ratio; CI, confidence interval.
comparison. The exaggeration of effect size in the clearly not hidden group, compared with central randomization, is of the same order as found in other studies. We found that, in the data set, which we examined, use of sealed envelopes with extra precautions appeared in practice to be a secure way of hiding allocation. This is despite the well-recognized theoretical danger that researchers recruiting and randomizing participants could have circumvented the randomization process [17]. Perhaps few have done so. We also found that saying a trial is double blind does not imply hidden allocation. The term "double blind" has been used to refer to blinding of participants, health care providers, investigators, data collectors, judicial assessors, or data analyzers [9]. Another study found that 19% of 200 ‘‘double-blind’’ trials had not blinded participants, health care providers, or data collectors [18]. Our study supports others that find that description of a trial as "double blind" does not imply hidden allocation. Neither does absence of double blinding preclude it. These study attributes are different. Insecure allocation may occur in the presence of blinding of participants and health care providers. On the other hand, if the allocation sequence is securely created, recorded, and implemented (our category 1) as, for example, in a trial comparing two surgical interventions, it can be considered concealed despite the difficulties of blinding of participants and caregivers. Our post hoc analysis found that the use of sealed envelopes with extra security precautions is not statistically significantly different from using envelopes without any report of extra precautions. Misclassification is more likely in the group with the lower security as the lack of extra precautions may be a result of poor reporting rather than of poor practice. This would tend to bring the two groups closer together. We used more than one meta-analysis from each systematic review. This means that studies often contributed more than once to the results. We attempted to adjust for this by bootstrapping the analysis to get bootstrap CIs where the bootstrap sample included only one meta-analysis from each systematic review. In addition, the systematic reviews were drawn from many different medical fields. Including studies that contribute more than once is likely to lead to
CIs being smaller than they would be if that were not the case although the bootstrap came up with intervals that were only slightly wider. The variety of subject areas could possibly lead to wider CIs than restricting the study to one area. It is difficult to decide how these issues would affect the results. The data for this study came from the first issue of the Cochrane Library in 2001. Thus, many of the studies contributing to the meta-analyses included in this study were published before the first consolidated standards of reporting trials (CONSORT) statement [19]. This would mean that the standard of reporting of those trials may not be as good as that expected these days. This may have led to some misclassification of the different methods of allocation to groups, with perhaps the largest effect on those with unclear allocation. This misclassification would be expected to increase the CIs and move the estimate of the RORs closer to one. Appraisal and categorization of security of allocation concealment in any trial involve a judgment based on the report; some misclassification is inevitable. For example, a study with central randomization may have the allocation concealment
Fig. 1. Forest plot of a random-effects meta-analysis of methodological studies calculating the ratios of odds ratios between groups of trials with and without adequate allocation concealment.
P. Herbison et al. / Journal of Clinical Epidemiology 64 (2011) 1070e1075
compromised if it has an allocation sequence with a block size of two and is testing a drug with obvious side effects compared with the comparison treatment. These problems would, on the whole, lessen the differences between the methods of allocation and increase their variability. Addition of the data from this study to the previous metaanalysis [9] changes the pooled point estimate from 0.83 to 0.84. Because it is possible that this study includes some of the meta-analyses and studies also included in the previous studies, this combined result should be treated with some caution, but the change in estimate is small. In addition, all previous studies used a fixed-effect logistic regression approach to estimate the average effect of inadequate/unclear allocation concealment compared with adequate concealment, and thus standard errors may well be too small. This study clarifies what constitutes hidden allocation in randomized controlled trials. We compared more securely concealed allocation, such as a remote telephone service or pharmacy allocation, with other methods. We found little evidence that results from trials using opaque, sequentially opened envelopes were systematically biased, compared with those from trials using central randomization, even though such methods are potentially insecure. However, central randomization, combined with adequate sequence generation, should be used whenever possible and, as recommended in the CONSORT statement 2010 [20,21], details of both sequence generation and concealment of allocation should always be clearly reported.
Acknowledgments The authors thank Annette Baker and Marianne de Graaf for extracting the data and Rebecca George for independently categorizing the allocation concealment. Funding: The New Zealand Lottery Grants Board provided funding for this study and had no role other than providing funding.
Appendix Supplementary material Supplementary material can be found, in the online version, at 10.1016/j.jclinepi.2010.12.018. References [1] Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 1995;273:408e12. [2] Moher D, Pham B, Jones A, Olkin I, Rennie D, Stroup DF. Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses? Lancet 1998;352:609e13.
1075
[3] J€uni P, Tallon D, Egger M. ‘‘Garbage inegarbage out?’’ Assessment of the quality of controlled trials in meta-analyses published in leading journals. Proceedings of the 3rd symposium on systematic reviews: beyond the basics, St. Catherine’s College, Oxford. Oxford, UK: Centre for Statistics in Medicine; 2000. p. 19. [4] Kjaergard LL, Villumsen J, Gluud C. Reported methodological quality and discrepancies between large and small randomized trials in meta-analyses. Ann Intern Med 2001;135:982e9. [5] Balk EM, Bonis PA, Moskowitz H, Schmid CH, Ioannidis JP, Wang C, et al. Correlation of quality measures with estimates of treatment effect in meta-analyses of randomized controlled trials. JAMA 2002;287:2973e82. [6] Als-Nielsen B, Chen W, Gluud LL, Siersma V, Hilden J, Gluud C. Are trials size and reported methodological quality associated with treatment effects? Observational study of 523 randomised trials. Cochrane Colloquium Abstract 2004. Available at http:// www.cochrane.org/colloquia/abstracts/ottawa/P-003.htm. Accessed 2 February 2009. [7] J€uni P, Altman DG, Egger M. Assessing the quality of controlled clinical trials. BMJ 2001;323:42e6. [8] Egger M, Ebrahim S, Davey Smith G. Where now for meta-analysis? Int J Epidemiol 2002;31:1e5. [9] Gluud LL. Bias in clinical intervention research. Am J Epidemiol 2006;163:493e501. [10] Pildal J, Hrobjartsson A, Jørgensen KJ, Hilden J, Altman DG, Gøtzsche PC. Impact of allocation concealment on conclusions drawn from meta-analyses of randomised trials. Int J Epidemiol 2007;36:847e57. [11] Wood L, Egger M, Gluud LL, Schulz KF, J€uni P, Altman DG, et al. Empirical evidence of bias in treatment effect estimates in controlled trials with different intervention and outcomes: meta-epidemiological study. BMJ 2008;336:601e5. [12] Herbison P, Hay-Smith J, Gillespie WJ. Adjustment of meta-analyses on the basis of quality scores should be abandoned. J Clin Epidemiol 2006;59:1249e56. [13] Sterne JAC, J€uni P, Schulz KF, Altman DG, Bartlett C, Egger M. Statistical methods for assessing the influence of study characteristics on treatment effects in ‘meta-epidemiological’ research. Stat Med 2002;21:1513e24. [14] Harris RJ, Bradburn MJ, Deeks JJ, Harbord RM, Altman DG, Sterne JAC. Metan: fixed- and random-effects meta-analysis. Stata J 2008;8:3e28. [15] Harbord RM, Higgins JPT. Meta-regression in Stata. Stata J 2008;8: 493e519. [16] Kjaergard LL, Villumesen J, Gluud C. Correction: reported methodologic quality and discrepancies between large and small randomized trials in meta-analyses. Ann Intern Med 2008;149:219. [17] Berger VW. Quantifying the magnitude of baseline covariate imbalances resulting from selection bias in randomized clinical trials. Biometrical J 2005;47:119e27. [18] Haahr MT, Hrobjartsson A. Who is blinded in randomized clinical trials? A study of 200 trials and a survey of authors. Clin Trials 2006;3:360e5. [19] Begg C, Cho M, Eastwood S, Horton R, Moher D, Oikin I, et al. Improving the quality of reporting of randomized controlled trials: the CONSORT statement. JAMA 1996;276:637e9. [20] Schulz KF, Altman DG, Moher D; for the CONSORT Group. CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials. J Clin Epidemiol 2010;63:843e40. [21] Moher D, Hopewell S, Schulz KF, Montori V, Gøtzsche PC, Devereaux PJ, et al; for the CONSORT Group. CONSORT 2010 Explanation and Elaboration: updated guidelines for reporting parallel group randomised trial. BMJ 2010;340:c869.
Different methods of allocation to groups in randomized trials are associated with different levels of bias. A meta-epidemiological study Peter Herbisona,*, Jean Hay-Smithb, William J. Gillespiec b
a Department of Preventive and Social Medicine, Dunedin School of Medicine, University of Otago, PO Box 913, Dunedin 9054, New Zealand Rehabilitation Teaching and Research Unit, University of Otago, Wellington, c/- Women’s and Children’s Health, PO Box 913, Dunedin 9054, New Zealand c Hull York Medical School, Cottingham Road, Hull HU6 7RX, UK
Accepted 10 December 2010
Abstract Objective: Insecure hiding of the treatment allocation in randomized trials is associated with bias. It is less certain how much bias is associated with different methods of treatment allocation. Study Design and Setting: Meta-epidemiological study of 389 randomized trials from 19 systematic reviews and 65 meta-analyses with differing methods of treatment allocation. Pooled ratios of odds ratios (RORs) and 95% confidence intervals (95% CI) were calculated from trials with different methods of treatment allocation. An ROR less than one shows exaggeration of treatment effect. Results: There is no evidence that the use of sealed envelopes with enhancement was different from central randomization (ROR 1.02, 95% CI: 0.85e1.23). Sealed envelopes without enhancement were associated with an exaggeration of the estimate of effect (ROR 0.87, 95% CI: 0.76e1.00). Where allocation concealment for double-blind trials was unclear, the ROR is 0.86 (95% CI: 0.78e0.96) and if not hidden, the ROR is 0.89 (95% CI: 0.70e1.15). Conclusion: Sealed envelopes with some form of enhancement (opaque, sequentially numbered, and so forth) may give adequate concealment. Description of a study as "double blind" does not imply a lack of bias when concealment of allocation is unclear. Ó 2011 Elsevier Inc. All rights reserved. Keywords: Allocation concealment; Bias; Meta-epidemiology; Blinding; Randomized trials; Meta-analysis
1. Introduction Although the best evidence of the effect of medical interventions comes from systematic reviews and metaanalyses of randomized trials, these are potentially subject to bias. Often these biases are associated with how trials are carried out. A number of methodological studies have examined the method of allocation to groups and have found empirical evidence of bias [1e6]. Meta-analysis of these methodological studies [7e9] indicates that this bias results in an overestimate of treatment effect of around 20% for studies with inadequate allocation concealment compared with those that have more secure methods of concealing the allocation. In another study, which reported data from 499 trials included in 70 metaanalyses [10], two-thirds of conclusions in favor of one of the interventions were no longer supported if only trials * Corresponding author. Department of Preventive and Social Medicine, Dunedin School of Medicine, PO Box 913, Dunedin 9054, New Zealand. Tel.: þ64-3-479-7217; fax: þ64-3-479-7298. E-mail address: [email protected] (P. Herbison). 0895-4356/$ - see front matter Ó 2011 Elsevier Inc. All rights reserved. doi: 10.1016/j.jclinepi.2010.12.018
with adequate allocation concealment were included in the meta-analysis. The amount of bias associated with allocation concealment may also depend on the outcome, with objective outcomes being associated with less bias than subjective outcomes [11]. Previous studies examining bias associated with allocation concealment mostly coded the method of group allocation as hidden and unclear [2e6], or in one case, hidden, unclear, and not hidden [1]. However, there are many ways of allocating participants to groups, ranging from those that are more securely hidden, such as a third party Web site or telephone access or allocation of numbered prepacked medications or identical placebos by a pharmacy (central randomization), to those that are clearly not hidden, such as whether the participants date of birth is odd or even. Between central randomization and the clearly not hidden lies other methods, which may be somewhat insecure. Reporting of allocation concealment is often insufficient to make a clear judgment. This study explores, at a finer level of detail than has been done previously, the methods of group allocation in individual trials and the associated biases.
P. Herbison et al. / Journal of Clinical Epidemiology 64 (2011) 1070e1075
What is new? Key findings: Inadequate concealment of allocation can lead to exaggeration of treatment effects in randomized controlled trials. Many methods of allocation to groups are used, and it is not clear which provide adequate concealment. This study provides estimates of the degree of exaggeration associated with different allocation methods. What this adds: Sealed envelopes with extra security features appear to provide adequate concealment. Saying only that a study is double blind is not sufficient to ensure lack of exaggeration of effect. Implications: If it is not possible to use a truly secure method of allocation, then sealed, opaque, sequentially numbered envelopes may provide adequate allocation concealment.
2. Methods Full details of the methods of data collection were reported previously [12]. Briefly, systematic reviews in issue 1, 2001 of the Cochrane Library were searched for binary outcomes that had at least 10 included trials, at least one of which had more than 500 people randomized to each arm. For these outcomes, the report of the contributing trials was found, and the data were extracted in duplicate by two trained research assistants. The results for the binary outcomes were taken from the Cochrane Review as the total number and number positive for each arm in the study. Each included trial was coded for the method of allocation into one of six categories. Category 1 comprised trials that used some form of central randomization that clearly should hide the allocation, such as a remote telephone service or randomization by a pharmacy. Category 2 comprised trials that used sealed envelopes with some form of security enhancement, such as ensuring that envelopes were opaque and numbered. Category 3 comprised trials that used sealed envelopes without any further details. Category 4 comprised trials that were reported as randomized without details, and also as "double blind." Category 5 comprised trials that simply said they were randomized with no further details. Category 6 comprised trials where the allocation was clearly not hidden, for example, being based on an open list, odd or even days of the week, participant’s birth date, or the team on duty at enrollment. Categorization into one of these six groups was conducted
1071
independently by two people, with discussion with a third person to resolve any disagreements. To facilitate comparison with previous meta-analyses, these six categories were combined to create two groups: an "adequate concealment" group combining groups 1 and 2 and the remainder to form an "inadequate or unclear concealment" group. Analysis was done in Stata V10 (StataCorp, College Station, TX) using the meta-epidemiology approach described by Sterne et al. [13]. Odds ratios (ORs) and their standard errors were calculated from the appropriate two by two tables using the metan routine in Stata [14]. This was used as metan adds variables for the natural log of the OR and its standard error to the file. This process was not used to produce pooled results. Then random-effects meta-regression was used on each meta-analysis using the method of allocation as the covariate [15]. The central randomization group was used as the reference group for the six-group comparison, and "adequate concealment" was used for the two-group comparison. The coefficients in the meta-regression give ratios of ORs (RORs), which were themselves combined using metan with random effects to give pooled RORs with 95% confidence intervals (95% CIs). The data from the meta-analyses were not independent as more than one meta-analysis from a systematic review was included. These often contained the same studies. To allow for this, the meta-analysis of the RORs was bootstrapped to get bootstrap CIs with one metaanalysis being chosen at random from each systematic review, and the results were analyzed. This was repeated 1,000 times. Because of the results of this analysis, a post hoc comparison between security-enhanced envelopes and envelopes without enhancement was carried out. Calculations were always done so that ORs less than one indicated beneficial effects of the treatment. This implies that an exaggeration of the treatment effect would lead to an ROR less than one. The pooled ROR for the two-category result (adequate/not adequate) was added to the meta-analysis of similar studies reported by J€uni et al. [7] and then Egger and Ebrahim [8] and since updated by Gluud [9] but with the correction from Kjaergard et al. [16]. Random-effects meta-analysis was used for this calculation.
3. Results The search identified 67 meta-analyses from 19 systematic reviews. Two of these, from the same systematic review, had to be discarded as too many of the studies had zero events in both arms. The remaining 65 meta-analyses from 18 systematic reviews included 389 studies. Details of the systematic reviews and outcomes are given in Table 1. The number of studies in individual meta-analyses ranged from 10 to 63. Sixty-seven studies used central randomization (category 1), 18 used enhanced envelopes (category 2), 48 envelopes with no further details (category 3), 121 reported that they were double blind but had unclear allocation
1072
P. Herbison et al. / Journal of Clinical Epidemiology 64 (2011) 1070e1075
Table 1 Meta-analyses and outcomes included 1. Henry DA, Moxey AJ, Carless PA, O’Connell D, McClelland B, Henderson KM, Sly K, Laupacis A, Fergusson D. Anti-fibrinolytic use for minimising perioperative allogeneic blood transfusion. In: The Cochrane Library, Issue 1, 2001. Oxford: Update Software. Outcomes 1. No. exposed to allogeneic blood 2. No. exposed to allogeneic blood by operation type (cardiac) 3. No. exposed to allogeneic blood by Transfusion Protocol 2. Knight M, Duley L, Henderson-Smart DJ, King JF. Antiplatelet agents for preventing and treating pre-eclampsia. The Cochrane Library, Issue 1, 2001. Oxford: Update Software. Outcomes 1. Pregnancy-induced hypertension (moderate risk) 2. Proteinuric pre-eclampsia (moderate risk) 3. C-section (moderate risk) 4. Preterm delivery !37 wk (moderate risk) 5. Fetal, neonatal, or infant death (moderate risk) 6. Fetal or neonatal deaths subgrouped by time of death (stillbirth) 7. Small for gestational age (moderate risk) 8. Pregnancy-induced hypertension (>20 wk) 9. Proteinuric pre-eclampsia (!20 wk) 10. Proteinuric pre-eclampsia (>20 wk) 11. Fetal neonatal or infant death (>20 wk) 12. Small for gestational age (>20 wk) 13. Pregnancy-induced hypertension 14. Proteinuric pre-eclampsia 15. Preterm delivery 3. Williams Jr. JW, Aguilar C, Makela M, Cornell J, Hollman DR, Chiquette E, Simel DL. Antibiotics for acute maxillary sinusitis. In: The Cochrane Library, Issue 1, 2001. Oxford: Update Software. Outcomes 1. Clinical cure or improvement 2. Dropout because of adverse effects 4. Hodnett ED. Caregiver support for women during childbirth. In: The Cochrane Library, Issue 1, 2001. Oxford: Update Software Outcomes 1. Oxytocin augmentation 2. Any analgesia/anesthesia during labor 3. Operative vaginal delivery (all women) 4. Caesarean delivery 5. Horn J, Limburg M. Calcium antagonists for acute ischemic stroke. In: The Cochrane Library, Issue 1, 2001. Oxford: Update Software. Outcomes 1. Poor outcome at end of follow-up (nimodipine vs. control) 2. Death at end of follow-up (nimodipine vs. control) 3. Death at end of treatment period (nimodipine vs. control) 4. Adverse events (ALL) during treatment period (nimodipine vs. control) 5. Poor outcome by time of start of treatment (treatment started O12 hr) 6. Methodology: randomization, concealment (poor outcome in good-quality trials) 7. Methodology: publication status (poor outcome in published trials) 6. Neilson JP, Alfirevic Z. Doppler ultrasound for foetal assessment in high-risk pregnancies. In: The Cochrane Library, Issue 1, 2001. Oxford: Update Software. Outcomes 1. Perinatal deaths 2. Stillbirths, normally formed 3. Neonatal deaths, normally formed 7. Asplund K, Israelsson K, Schampi I. Haemodilution for acute ischaemic stroke. In: The Cochrane Library, Issue 1, 2001. Oxford: Update Software. Outcomes 1. Case fatality at early follow-up (!28 days) 2. Case fatality at late follow-up (3e6 mo) 8. K Rees, M Beranek-Stanley, M Burke, S Ebrahim. Hypothermia to reduce neurological damage following coronary artery bypass surgery. In: The Cochrane Library, Issue 1, 2001. Oxford: Update Software. Outcomes 1. Nonfatal stroke 2. Perioperative deaths (not strokes) 3. Nonfatal myocardial infarction 4. Low output syndrome 5. Intra-aortic balloon pump use 6. Pooled "bad" outcomes 9. Alejandria MM, Lansang MA, Dans LF, Mantaring JBV. Intravenous immunoglobulin for treating sepsis and septic shock. In: The Cochrane Library, Issue 1, 2001. Oxford: Update Software. Outcomes 1. All-cause mortality (adults) 2. All-cause mortality (all ages) (Continued )
P. Herbison et al. / Journal of Clinical Epidemiology 64 (2011) 1070e1075
1073
Table 1 Continued 10. Ohlsson A, Lacy JB. Intravenous immunoglobulin for preventing infection in preterm and/or low-birth-weight infants. In: The Cochrane Library, Issue 1, 2001. Oxford: Update Software. Outcomes 1. Any serious infection 2. Mortality (total) 11. Schmitt B, Bennett C, Seidenfeld J, Samson D, Wilt T. Maximal androgen blockade for advanced prostate cancer. In: The Cochrane Library, Issue 1, 2001. Oxford: Update Software. Outcomes 1. Overall survival 1 yr 2. Overall survival 2 yr 12. Silagy C, Mant D, Fowler G, Lancaster T. Nicotine replacement therapy for smoking cessation. In: The Cochrane Library, Issue 1, 2001. Oxford: Update Software. (Gum). Outcomes 1. Smoking cessation at max. follow-up 6e12 mo (total) 2. Smoking cessation at max. follow-up 6e12 mo (low intensity) 3. Smoking cessation at max. follow-up 6e12 mo (high intensity) 13. Silagy C, Mant D, Fowler G, Lancaster T. Nicotine replacement therapy for smoking cessation. In: The Cochrane Library, Issue 1, 2001. Oxford: Update Software. (Patch). Outcomes 1. Smoking cessation at max. follow-up 6e12 mo (total) 2. Smoking cessation at max. follow-up 6e12 mo (low intensity) 3. Smoking cessation at max. follow-up 6e12 mo (high intensity) 14. Tan BP, Hannah ME. Oxytocin for prelabour rupture of membranes at or near term. In: The Cochrane Library, Issue 1, 2001. Oxford: Update Software. Outcomes 1. Chorioamnionitis 2. Caesarean section 3. Neonatal infection 15. Tan BP, Hannah ME. Prostaglandins versus oxytocin for prelabour rupture of membranes at or near term. In: The Cochrane Library, Issue 1, 2001. Oxford: Update Software. Outcomes 1. Caesarean section 2. Operative delivery 16. Crowley P. Prophylactic corticosteroids for preterm birth. In: The Cochrane Library, Issue 1, 2001. Oxford: Update Software. Outcomes 1. Neonatal death (all babies) 2. Stillbirth (all babies) 17. Tan BP, Hannah ME. Prostaglandins for prelabour rupture of membranes at or near term. In: The Cochrane Library, Issue 1, 2001. Oxford: Update Software. Outcomes 1. Caesarean section 2. Neonatal infection 18. Gupta JK, Nikodem VC. Woman’s position during second stage of labour. In: The Cochrane Library, Issue 1, 2001. Oxford: Update Software. Outcomes 1. Mode of delivery (assisted delivery) 2. Episiotomy
(category 4), 104 had unclear allocation (category 5), and in 31 the allocation was clearly not hidden (category 6). Nine meta-analyses were omitted from the six-group comparison as they had no studies in the reference group, and two were omitted from the two-group comparison. The pooled RORs for the different allocation methods compared with central randomization are given in Table 2. Overall, studies with inadequate allocation concealment (groups 3e6) show an exaggeration of the effect size by 9% (95% CI: 1e17%) when compared with studies with adequate concealment. The comparisons of groups 2e6, individually, with group 1 show that sealed opaque envelopes with enhanced security (group 2) were not significantly different (ROR 1.02, 95% CI: 0.85e1.22) from central randomization although the CI is wide. Envelopes without enhanced security (group 3) show increased exaggeration although this was only just statistically significant. Studies that gave no details of the randomization (group 4) but said they
were double blind exaggerate the effect size (ROR 0.86, 95% CI: 0.78e0.96). The estimated ROR from the post hoc comparison of security-enhanced envelopes with no enhancement is 0.87 (95% CI: 0.73e1.05). Incorporation of our overall result into the most recent meta-analysis of studies of allocation concealment changes the overall ROR from 0.83 (95% CI: 0.71e0.96) to 0.84 (95% CI: 0.75e0.95) (Fig. 1).
4. Discussion This study confirmed the finding of previous studies that apparently less secure methods of allocating participants to groups in randomized controlled trials are indeed associated with bias. Our results, overall, are consistent with those studies [1e6,16], which reported RORs ranging from 0.60 to 1.02. This study found an ROR of 0.91 for the same
1074
P. Herbison et al. / Journal of Clinical Epidemiology 64 (2011) 1070e1075
Table 2 RORs associated with different methods of allocation to groups in randomized controlled trials Method of allocation
RORs (bootstrap 95% CI)
Number of meta-analyses contributing to estimate
t2
Six-group comparison Central randomization Enhanced envelopes Envelopes with no further elaboration Unclear but stated that trial is double blind Unclear Clearly not hidden
Reference 1.02 (0.85e1.22) 0.87 (0.76e1.00) 0.86 (0.78e0.96) 0.76 (0.66e0.87) 0.89 (0.70e1.15)
16 39 44 50 22
0 0 0.047 0.130 0.123
Two-group comparison Adequate concealment Inadequate or unclear concealment
Reference 0.91 (0.83e0.99)
63
0.036
Abbreviations: ROR, ratio of odds ratio; CI, confidence interval.
comparison. The exaggeration of effect size in the clearly not hidden group, compared with central randomization, is of the same order as found in other studies. We found that, in the data set, which we examined, use of sealed envelopes with extra precautions appeared in practice to be a secure way of hiding allocation. This is despite the well-recognized theoretical danger that researchers recruiting and randomizing participants could have circumvented the randomization process [17]. Perhaps few have done so. We also found that saying a trial is double blind does not imply hidden allocation. The term "double blind" has been used to refer to blinding of participants, health care providers, investigators, data collectors, judicial assessors, or data analyzers [9]. Another study found that 19% of 200 ‘‘double-blind’’ trials had not blinded participants, health care providers, or data collectors [18]. Our study supports others that find that description of a trial as "double blind" does not imply hidden allocation. Neither does absence of double blinding preclude it. These study attributes are different. Insecure allocation may occur in the presence of blinding of participants and health care providers. On the other hand, if the allocation sequence is securely created, recorded, and implemented (our category 1) as, for example, in a trial comparing two surgical interventions, it can be considered concealed despite the difficulties of blinding of participants and caregivers. Our post hoc analysis found that the use of sealed envelopes with extra security precautions is not statistically significantly different from using envelopes without any report of extra precautions. Misclassification is more likely in the group with the lower security as the lack of extra precautions may be a result of poor reporting rather than of poor practice. This would tend to bring the two groups closer together. We used more than one meta-analysis from each systematic review. This means that studies often contributed more than once to the results. We attempted to adjust for this by bootstrapping the analysis to get bootstrap CIs where the bootstrap sample included only one meta-analysis from each systematic review. In addition, the systematic reviews were drawn from many different medical fields. Including studies that contribute more than once is likely to lead to
CIs being smaller than they would be if that were not the case although the bootstrap came up with intervals that were only slightly wider. The variety of subject areas could possibly lead to wider CIs than restricting the study to one area. It is difficult to decide how these issues would affect the results. The data for this study came from the first issue of the Cochrane Library in 2001. Thus, many of the studies contributing to the meta-analyses included in this study were published before the first consolidated standards of reporting trials (CONSORT) statement [19]. This would mean that the standard of reporting of those trials may not be as good as that expected these days. This may have led to some misclassification of the different methods of allocation to groups, with perhaps the largest effect on those with unclear allocation. This misclassification would be expected to increase the CIs and move the estimate of the RORs closer to one. Appraisal and categorization of security of allocation concealment in any trial involve a judgment based on the report; some misclassification is inevitable. For example, a study with central randomization may have the allocation concealment
Fig. 1. Forest plot of a random-effects meta-analysis of methodological studies calculating the ratios of odds ratios between groups of trials with and without adequate allocation concealment.
P. Herbison et al. / Journal of Clinical Epidemiology 64 (2011) 1070e1075
compromised if it has an allocation sequence with a block size of two and is testing a drug with obvious side effects compared with the comparison treatment. These problems would, on the whole, lessen the differences between the methods of allocation and increase their variability. Addition of the data from this study to the previous metaanalysis [9] changes the pooled point estimate from 0.83 to 0.84. Because it is possible that this study includes some of the meta-analyses and studies also included in the previous studies, this combined result should be treated with some caution, but the change in estimate is small. In addition, all previous studies used a fixed-effect logistic regression approach to estimate the average effect of inadequate/unclear allocation concealment compared with adequate concealment, and thus standard errors may well be too small. This study clarifies what constitutes hidden allocation in randomized controlled trials. We compared more securely concealed allocation, such as a remote telephone service or pharmacy allocation, with other methods. We found little evidence that results from trials using opaque, sequentially opened envelopes were systematically biased, compared with those from trials using central randomization, even though such methods are potentially insecure. However, central randomization, combined with adequate sequence generation, should be used whenever possible and, as recommended in the CONSORT statement 2010 [20,21], details of both sequence generation and concealment of allocation should always be clearly reported.
Acknowledgments The authors thank Annette Baker and Marianne de Graaf for extracting the data and Rebecca George for independently categorizing the allocation concealment. Funding: The New Zealand Lottery Grants Board provided funding for this study and had no role other than providing funding.
Appendix Supplementary material Supplementary material can be found, in the online version, at 10.1016/j.jclinepi.2010.12.018. References [1] Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 1995;273:408e12. [2] Moher D, Pham B, Jones A, Olkin I, Rennie D, Stroup DF. Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses? Lancet 1998;352:609e13.
1075
[3] J€uni P, Tallon D, Egger M. ‘‘Garbage inegarbage out?’’ Assessment of the quality of controlled trials in meta-analyses published in leading journals. Proceedings of the 3rd symposium on systematic reviews: beyond the basics, St. Catherine’s College, Oxford. Oxford, UK: Centre for Statistics in Medicine; 2000. p. 19. [4] Kjaergard LL, Villumsen J, Gluud C. Reported methodological quality and discrepancies between large and small randomized trials in meta-analyses. Ann Intern Med 2001;135:982e9. [5] Balk EM, Bonis PA, Moskowitz H, Schmid CH, Ioannidis JP, Wang C, et al. Correlation of quality measures with estimates of treatment effect in meta-analyses of randomized controlled trials. JAMA 2002;287:2973e82. [6] Als-Nielsen B, Chen W, Gluud LL, Siersma V, Hilden J, Gluud C. Are trials size and reported methodological quality associated with treatment effects? Observational study of 523 randomised trials. Cochrane Colloquium Abstract 2004. Available at http:// www.cochrane.org/colloquia/abstracts/ottawa/P-003.htm. Accessed 2 February 2009. [7] J€uni P, Altman DG, Egger M. Assessing the quality of controlled clinical trials. BMJ 2001;323:42e6. [8] Egger M, Ebrahim S, Davey Smith G. Where now for meta-analysis? Int J Epidemiol 2002;31:1e5. [9] Gluud LL. Bias in clinical intervention research. Am J Epidemiol 2006;163:493e501. [10] Pildal J, Hrobjartsson A, Jørgensen KJ, Hilden J, Altman DG, Gøtzsche PC. Impact of allocation concealment on conclusions drawn from meta-analyses of randomised trials. Int J Epidemiol 2007;36:847e57. [11] Wood L, Egger M, Gluud LL, Schulz KF, J€uni P, Altman DG, et al. Empirical evidence of bias in treatment effect estimates in controlled trials with different intervention and outcomes: meta-epidemiological study. BMJ 2008;336:601e5. [12] Herbison P, Hay-Smith J, Gillespie WJ. Adjustment of meta-analyses on the basis of quality scores should be abandoned. J Clin Epidemiol 2006;59:1249e56. [13] Sterne JAC, J€uni P, Schulz KF, Altman DG, Bartlett C, Egger M. Statistical methods for assessing the influence of study characteristics on treatment effects in ‘meta-epidemiological’ research. Stat Med 2002;21:1513e24. [14] Harris RJ, Bradburn MJ, Deeks JJ, Harbord RM, Altman DG, Sterne JAC. Metan: fixed- and random-effects meta-analysis. Stata J 2008;8:3e28. [15] Harbord RM, Higgins JPT. Meta-regression in Stata. Stata J 2008;8: 493e519. [16] Kjaergard LL, Villumesen J, Gluud C. Correction: reported methodologic quality and discrepancies between large and small randomized trials in meta-analyses. Ann Intern Med 2008;149:219. [17] Berger VW. Quantifying the magnitude of baseline covariate imbalances resulting from selection bias in randomized clinical trials. Biometrical J 2005;47:119e27. [18] Haahr MT, Hrobjartsson A. Who is blinded in randomized clinical trials? A study of 200 trials and a survey of authors. Clin Trials 2006;3:360e5. [19] Begg C, Cho M, Eastwood S, Horton R, Moher D, Oikin I, et al. Improving the quality of reporting of randomized controlled trials: the CONSORT statement. JAMA 1996;276:637e9. [20] Schulz KF, Altman DG, Moher D; for the CONSORT Group. CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials. J Clin Epidemiol 2010;63:843e40. [21] Moher D, Hopewell S, Schulz KF, Montori V, Gøtzsche PC, Devereaux PJ, et al; for the CONSORT Group. CONSORT 2010 Explanation and Elaboration: updated guidelines for reporting parallel group randomised trial. BMJ 2010;340:c869.