- Email: [email protected]
Pneumococcal polysaccharide vaccine effectiveness: study quality must not be ignored
Reflection and Reaction
Pneumococcal polysaccharide vaccine effectiveness: study quality must not be ignored
CNRI/Science Photo Library
Angel Vila-Corcoles1 challenged the conclusion from a recently updated Cochrane Review2 (and, puzzlingly, those presented in a previous letter he coauthored in 20043) that there is insufficient evidence to support the routine use of the 23-valent pneumococcal polysaccharide vaccine (PPV23) to prevent pneumonia. One of the arguments presented is that evidence from observational studies should be included in metaanalyses, and the results from a cohort study in Spain4 (for which Vila-Corcoles is first author) are discussed in some detail. The findings of any systematic review depend crucially on which studies are included and decisions regarding the inclusion and exclusion of studies must be clear and transparent. If observational studies are to be included then they too must be identified systematically and not selectively. However, there are good reasons for excluding observational studies, which generally are at higher risk of bias than randomised controlled trials. Indeed, the situation with PPV is reminiscent of the arguments surrounding the use of influenza vaccines in elderly people, where cohort studies, which are prone to selection bias, have shown (implausibly) large benefit, by contrast with the results from randomised controlled trials.5–7 We stress that in meta-analyses of randomised controlled trials it is also essential to assess study quality and the risk of bias. The Cochrane Review used the Jadad scoring scale to assess study quality.1 This is surprising, since the Cochrane guidelines explicitly discourage the use of this scale because it emphasises reporting rather than conduct, and does not account for concealment of allocation, an important measure to prevent selection bias.8 Lack of blinding in outcome assessment can also be problematic when the outcome in question is open to diagnostic opinion. We recently undertook a systematic review and meta-analysis of randomised trials to evaluate the efficacy of PPV in the prevention of pneumonia in adults and to explore reasons for
664
heterogeneous results of clinical trials.9 Unlike the 17 previous meta-analyses that we identified on the same subject, we formally considered study quality as a means to explore causes of heterogeneity of results. We found that trial quality, in particular double blinding, explained a substantial proportion of heterogeneity and there was little evidence that PPV offered protection against pneumonia in trials of higher methodological quality. Prevention of pneumonia in elderly people is an important public-health challenge and although pneumococcal conjugate vaccines are likely to be more expensive, they are also likely to be more effective than the currently available PPV23. We await with interest results from high quality randomised controlled trials of pneumococcal conjugate vaccines in elderly people. Caroline Trotter, Pippa Scott, Anke Huss, *Matthias Egger Department of Social Medicine, University of Bristol, Bristol, UK (CT, ME); and Institute of Social and Preventive Medicine (ISPM), University of Bern, Bern, Switzerland (PS, AH, ME) [email protected] We declare that we have no conflicts of interest. 1 2
3
4
5
6 7
8
9
Vila-Corcoles A. Is the pneumococcal polysaccharide vaccine effective in preventing pneumonia? Lancet Infect Dis 2008; 8: 405–06. Moberley SA, Holden J, Tatham DP, Andrews RM. Vaccines for preventing pneumococcal infection in adults. Cochrane Database Syst Rev 2008; 1: CD000422. Vila Corcoles A, Ochoa Gondar O, Llor C. Methodology of observational studies of pneumococcal polysaccharide vaccine. Lancet Infect Dis 2004; 4: 71. Vila-Corcoles A, Ochoa-Gondar O, Hospital I, et al. Protective effects of the 23-valent pneumococcal polysaccharide vaccine in the elderly population: the EVAN-65 study. Clin Infect Dis 2006; 43: 860–68. Jefferson T, Rivetti D, Rivetti A, Rudin M, Di Pietrantonj C, Demicheli V. Efficacy and effectiveness of influenza vaccines in elderly people: a systematic review. Lancet 2005; 366: 1165–74 Jefferson T. Inactivated influenza vaccines in the elderly—are you sure? Lancet Infect Dis 2007; 7: 1199–200. Simonsen L, Taylor RJ, Viboud C, Miller MA, Jackson LA. Mortality benefits of influenza vaccination in elderly people: an ongoing controversy. Lancet Infect Dis 2007; 7: 658–66. Higgins JP, Green S, eds. Cochrane handbook for systematic reviews of interventions version 5.0.0 [updated February 2008]. The Cochrane Collaboration, 2008. http://www.cochrane-handbook.org (accessed Sept 29, 2008). Huss A, Scott P, Stuck AE, Trotter C, Egger M. Pneumococcal polysaccharide vaccine in adults: systematic review and meta-analysis of clinical trials of higher and lower methodological quality. CMAJ (in press).
www.thelancet.com/infection Vol 8 November 2008
Pneumococcal polysaccharide vaccine effectiveness: study quality must not be ignored
CNRI/Science Photo Library
Angel Vila-Corcoles1 challenged the conclusion from a recently updated Cochrane Review2 (and, puzzlingly, those presented in a previous letter he coauthored in 20043) that there is insufficient evidence to support the routine use of the 23-valent pneumococcal polysaccharide vaccine (PPV23) to prevent pneumonia. One of the arguments presented is that evidence from observational studies should be included in metaanalyses, and the results from a cohort study in Spain4 (for which Vila-Corcoles is first author) are discussed in some detail. The findings of any systematic review depend crucially on which studies are included and decisions regarding the inclusion and exclusion of studies must be clear and transparent. If observational studies are to be included then they too must be identified systematically and not selectively. However, there are good reasons for excluding observational studies, which generally are at higher risk of bias than randomised controlled trials. Indeed, the situation with PPV is reminiscent of the arguments surrounding the use of influenza vaccines in elderly people, where cohort studies, which are prone to selection bias, have shown (implausibly) large benefit, by contrast with the results from randomised controlled trials.5–7 We stress that in meta-analyses of randomised controlled trials it is also essential to assess study quality and the risk of bias. The Cochrane Review used the Jadad scoring scale to assess study quality.1 This is surprising, since the Cochrane guidelines explicitly discourage the use of this scale because it emphasises reporting rather than conduct, and does not account for concealment of allocation, an important measure to prevent selection bias.8 Lack of blinding in outcome assessment can also be problematic when the outcome in question is open to diagnostic opinion. We recently undertook a systematic review and meta-analysis of randomised trials to evaluate the efficacy of PPV in the prevention of pneumonia in adults and to explore reasons for
664
heterogeneous results of clinical trials.9 Unlike the 17 previous meta-analyses that we identified on the same subject, we formally considered study quality as a means to explore causes of heterogeneity of results. We found that trial quality, in particular double blinding, explained a substantial proportion of heterogeneity and there was little evidence that PPV offered protection against pneumonia in trials of higher methodological quality. Prevention of pneumonia in elderly people is an important public-health challenge and although pneumococcal conjugate vaccines are likely to be more expensive, they are also likely to be more effective than the currently available PPV23. We await with interest results from high quality randomised controlled trials of pneumococcal conjugate vaccines in elderly people. Caroline Trotter, Pippa Scott, Anke Huss, *Matthias Egger Department of Social Medicine, University of Bristol, Bristol, UK (CT, ME); and Institute of Social and Preventive Medicine (ISPM), University of Bern, Bern, Switzerland (PS, AH, ME) [email protected] We declare that we have no conflicts of interest. 1 2
3
4
5
6 7
8
9
Vila-Corcoles A. Is the pneumococcal polysaccharide vaccine effective in preventing pneumonia? Lancet Infect Dis 2008; 8: 405–06. Moberley SA, Holden J, Tatham DP, Andrews RM. Vaccines for preventing pneumococcal infection in adults. Cochrane Database Syst Rev 2008; 1: CD000422. Vila Corcoles A, Ochoa Gondar O, Llor C. Methodology of observational studies of pneumococcal polysaccharide vaccine. Lancet Infect Dis 2004; 4: 71. Vila-Corcoles A, Ochoa-Gondar O, Hospital I, et al. Protective effects of the 23-valent pneumococcal polysaccharide vaccine in the elderly population: the EVAN-65 study. Clin Infect Dis 2006; 43: 860–68. Jefferson T, Rivetti D, Rivetti A, Rudin M, Di Pietrantonj C, Demicheli V. Efficacy and effectiveness of influenza vaccines in elderly people: a systematic review. Lancet 2005; 366: 1165–74 Jefferson T. Inactivated influenza vaccines in the elderly—are you sure? Lancet Infect Dis 2007; 7: 1199–200. Simonsen L, Taylor RJ, Viboud C, Miller MA, Jackson LA. Mortality benefits of influenza vaccination in elderly people: an ongoing controversy. Lancet Infect Dis 2007; 7: 658–66. Higgins JP, Green S, eds. Cochrane handbook for systematic reviews of interventions version 5.0.0 [updated February 2008]. The Cochrane Collaboration, 2008. http://www.cochrane-handbook.org (accessed Sept 29, 2008). Huss A, Scott P, Stuck AE, Trotter C, Egger M. Pneumococcal polysaccharide vaccine in adults: systematic review and meta-analysis of clinical trials of higher and lower methodological quality. CMAJ (in press).
www.thelancet.com/infection Vol 8 November 2008