- Email: [email protected]
Efficacy of polysaccharide pneumococcal vaccine in adults in more developed countries: another view of the evidence
Reflection & Reaction Efficacy of polysaccharide pneumococcal vaccine in adults in more developed countries: another view of the evidence Punam Mangtani and colleagues review1 of the evidence on efficacy of pneumococcal vaccine for adults who live in more developed countries focuses on ten prospective clinical trials and five published metaanalyses. These studies show no protective effect against pneumococcal pneumonia and a small but statistically non-significant effect in preventing pneumococcal bacteraemia. The authors give less attention to the results of many observational studies that show significant protection against bacteraemic disease. Instead they recommend that protection should be confirmed “against hard endpoints such as pneumonia and mortality . . . with new trials or observational studies using new diagnostic tools”. Their cautious views raise doubts about the value of pneumococcal vaccination. Yet the use of pneumococcal vaccine has grown remarkably in many countries in recent years.2 What then are we to make of their conclusions? Pneumococcal vaccination of elderly and high-risk adults can be considered worthwhile if (1) the incidence of disease is high enough to be of concern to health officials and physicians, (2) vaccination is associated with a level of protection considered to be clinically important, and (3) protection can be achieved at a cost society finds acceptable. Most observers recognise that pneumococcal disease occurs with sufficient frequency to be worth preventing.2 Differences arise, however, over whether the vaccine is efficacious and vaccination effective, and whether widespread vaccination would be worthwhile. Four case-control studies, three indirect cohort studies, and one retrospective cohort study have shown that the aggregate effectiveness of pneumococcal vaccination is about 50–80% in preventing pneumococcal bacteraemia or invasive pneumococcal disease.2 Protection in all of these studies is statistically significant. Mangtani et al list five of these
272
observational studies, but focus their discussion on the one observational study that failed to show protection. This indirect cohort (not case-control) study by Forrester et al3 analysed 89 patients with pneumococcal bacteraemia. The vaccination status of patients was ascertained only by examining the hospital records. All other observational studies required documentation of vaccination status from all sites where care was given; one case-control study of 90 patients with community-acquired invasive pneumococcal disease checked 210 sources of medical care and found that 22 vaccinated patients had received their vaccine in 20 different sites.4 The patients studied by Forrester et al also were at unusually high risk: 22% were immunocompromised, 18% were admitted from nursing homes, and 30% had nosocomial infection. In an analysis that did not include risk adjustment, they found vaccination to be –21% effective (95% CI –221% to 55%). By contrast, three indirect cohort studies evaluated 450,5 861,6 and 28377 cases of invasive pneumococcal disease. At least two of the analyses included risk adjustment.6,7 In immunocompetent older adults, vaccination effectiveness was statistically significant at 89%, 62%, and 75%, respectively. These findings are far more persuasive than those of Forrester et al regarding the effectiveness of pneumococcal vaccination. Two prospective clinical trials in older adults suggested that pneumococcal vaccine is efficacious in preventing bacteraemic disease,8,9 although the results lacked statistical significance. A larger number of clinical trials and their meta-analyses have failed to show protection against all cases of pneumococcal pneumonia (non-bacteraemic and bacteraemic cases). To account for the apparent discrepancy in the trial results, Mangtani et al offer a biological explanation: pneumococcal vaccine prevents bacteraemia from developing in patients with pneumococcal pneumonia, but fails to prevent
pneumonia itself. However, they provide no biological evidence to support their conjecture. It is well known that pneumococcal vaccine offers younger adults excellent protection against pneumococcal pneumonia (sputum culture-positive) and radiographically diagnosed pneumonia.2 It is also known that antibody responses to vaccination in older and younger adults are similar.2 Why then should pnemococcal vaccine not protect at least some older adults against pneumococcal pneumonia? Could it be that problems with the clinical trials themselves and not the vaccine have made it impossible to reach a firm conclusion about its protective efficacy in older adults? Among the ten clinical trials reviewed by Mangtani et al, three were small immunogenicity studies in highrisk patients and cannot be regarded as serious efforts to assess clinical protection. In the trial conducted in long-term care institutions in France, randomisation generated study groups of strikingly unequal size (749 and 937), the 80 investigators differed greatly in the extent to which they obtained diagnostic tests, and 33% of the study patients either died or were lost to follow-up.10 In the Veterans Administration Cooperative Study, patients in the vaccine group had more episodes of pneumococcal pneumonia before the trial and significantly higher all-cause mortality during the trial. In the Swedish and Finnish trials, most of the diagnoses of pneumococcal pneumonia were made with serological tests that are known to be unreliable. But looming over these and other methodological problems is the question of sample size.11 The sample size question can be addressed by using data gathered from the control group during the conduct of a clinical trial to calculate retrospectively the sample size (number of person-years of observation) that would have been required to rule out a false-negative result regarding vaccine efficacy (VE).12 Charles Liss and I have shown that for preventing
THE LANCET Infectious Diseases Vol 3 May 2003
http://infection.thelancet.com
For personal use. Only reproduce with permission from The Lancet Publishing Group.
Reflection & Reaction bacteraemia (VE=50%), each of the clinical trials was ten to 50 times too small, and for preventing all-cause pneumonia (VE=15%), they were five to ten times too small (VE against vaccine-type [VT] pneumococcal pneumonia=50%, and 30% of all pneumonias are VT-pneumococcal infections).2 Each of the meta-analyses also included too few person-years of observation to rule out false-negative results regarding VE in preventing pneumococcal bacteraemia and allcause pneumonia.2 Thus the metaanalyses of the clinical trials of pneumococcal vaccine were destined from the outset to be inconclusive. If clinical trials and their metaanalyses provide no firm evidence regarding the efficacy of pneumococcal vaccine in older adults, what should be done? The Potsdam report on methods for the meta-analysis of clinical trials cautioned that, “when data are too sparse, of low quality, or too heterogeneous to proceed with their statistical aggregation, . . . (investigators should) . . . perform a narrative, qualitative summary and avoid meta-analysis”.13 The report also advised investigators to interpret their results “in the context of other available evidence (human, experimental or observational)”, and noted that clinical recommendations should be “practical and explicit”. Others have pointed out that there is no theoretical or empirical basis for rejecting the findings of observational studies or for regarding them as inferior to those of randomised controlled trials.14 Although clinical trials have failed to show protection
again pneumonia, a retrospective cohort study has shown that pneumococcal vaccination significantly reduces hospital admissions for allcause pneumonia in patients with chronic obstructive pulmonary disease.15 These findings challenge directly Mangtani and colleagues’ conclusions. When all of the evidence is considered, it is clear that the debate over the efficacy of pneumococcal vaccine and the effectiveness of vaccination does not need to centre on preventing pneumococcal pneumonia. Pneumococcal vaccination is effective in preventing bacteraemic disease, and this is “a hard endpoint”. Studies covering ten western European countries show that vaccination to prevent bacteraemic disease alone is very cost-effective16 (Silvia Evers, Andre Ament, University of Maastricht, Netherlands, personal communication). These findings fully justify “a major public health programme”1 to vaccinate older adults. No doubt it would be interesting to know whether pneumococcal vaccine reduces hospitalisation for pneumococcal pneumonia or all-cause pneumonia; case-control studies could answer this question. Nonetheless, for an increasing number of health officials who make recommendations and physicians who use pneumococcal vaccine, such evidence simply is not needed.
References 1
2
3 4
5
6
7
8
9
10
11
12
13
14
Mangtani P, Cutts F, Hall AJ. Efficacy of polysaccharide pneumococcal vaccine in adults in more developed countries: the state of the evidence. Lancet Infect Dis 2003; 3: 71–78. Fedson DS, Musher DM. Pneumococcal polysaccharide vaccine. In: Plotkin SA, Orenstein WA, eds. Vaccines, 4th edn. Philadelphia: WB Saunders (in press). Forrester HL, Jahnigen DW, LaForce FM. Inefficacy of pneumococcal vaccine in a high-risk population. Am J Med 1987; 83: 425–30. Shapiro ED, Clemens JD. A controlled evaluation of the protective efficacy of pneumococcal vaccine for patients at high risk of serious pneumococcal infections. Ann Intern Med 1984; 101: 325–30. Green K, Landry L, Goldenberg E, et al. Effectiveness of a pneumococcal vaccination program in preventing invasive pneumococcal disease. 39th Interscience Conference on Antimicrobial Agents and Chemotherapy; San Francisco; September 26–29, 1999. p 673. Shapiro ED, Berg AT, Austrian R, et al. The protective efficacy of polyvalent pneumococcal polysaccharide vaccine. N Engl J Med 1991; 325: 1453–60. Butler JC, Breiman RF, Campbell JF, et al. Pneumococcal polysaccharide vaccine efficacy. An evaluation of current recommendations. JAMA 1993; 270: 1826–31. Ortqvist A, Hedlund J, Burman LA, et al. Randomized trial of 23-valent pneumococcal capsular polysaccharide vaccine in the prevention of pneumonia in middle-aged and elderly people. Lancet 1998; 351: 399–403. Honkanen PO, Keistinen T, Miettinen L, et al. Incremental effectiveness of pneumococcal vaccine on simultaneously administered influenza vaccine in preventing pneumonia and pneumococcal pneumonia among persons aged 65 years or older. Vaccine 1999; 17: 2493–2500. Gaillat J, Zmirou D, Mallaret MR, et al. Clinical trial of pneumococcal vaccine among institutionalized elderly [in French]. Rev Epidemiol Sante Publ 1985; 33: 437–44. Clemens JD, Shapiro ED. Resolving the pneumococcal vaccine controversy: are there alternatives to randomized clinical trials? Rev Infect Dis 1984; 6: 589–600. Detsky AS, Sackett DL. When was a ‘negative’ clinical trial big enough? How many patients you needed depends on what you found. Arch Intern Med 1985; 145: 709–12. Cook DJ, Sackett DL, Spitzer WO. Methodologic guidelines for systematic reviews of randomized controlled trials in health care from the Potsdam consultation on meta-analysis. J Clin Epidemiol 1995; 48: 167–71. Abel U, Koch A. The role of randomization in clinical studies: myths and beliefs. J Clin Epidemiol 1999; 52: 487–97. Nichol KL, Baken L, Wuorenma J, Nelson A. The health and economic benefits associated with pneumococcal vaccination of elderly persons with chronic lung disease. Arch Intern Med 1999; 159: 2437–42. Ament A, Baltussen R, Duru G, et al. Costeffectiveness of pneumococcal vaccination of older people: a study in 5 western European countries. Clin Infect Dis 2000; 31: 444–50.
David S Fedson
15
DS is a former employee of Aventis Pasteur MSD Correspondence: Dr David S Fedson, 57 Chemin du Lavoir, 01630 Sergy Haut, France. Tel +33 4 50 99 13 06; email [email protected]
16
We consider that the lack of robust data on efficacy against outcomes of public-health importance subsumes any discussion about whether one observational study that did not show a protective effect1 is more flawed than others that do show protection. We agree that observational studies support the indication of a reduced risk of bacteraemia from trial data. We
highlighted the one observational study (by Forrester et al1) that did not show a protective effect against bacteraemia only because such heterogeneity precludes a summary estimate across the studies. Although decisions may have to be made based on observational studies, there is far greater room for confounding and bias as rightly noted by Fedson. In case-control studies
Authors’ reply David Fedson questions one of our conclusions that, although many observational studies of the polysaccharide pneumococcal vaccine have shown a significant protective effect against bacteraemia, there is no evidence that this reduces illness or mortality from bacteraemia. Our review, we hope, will prompt evaluation of such outcomes as a research priority.
THE LANCET Infectious Diseases Vol 3 May 2003
http://infection.thelancet.com
273
For personal use. Only reproduce with permission from The Lancet Publishing Group.
272
observational studies, but focus their discussion on the one observational study that failed to show protection. This indirect cohort (not case-control) study by Forrester et al3 analysed 89 patients with pneumococcal bacteraemia. The vaccination status of patients was ascertained only by examining the hospital records. All other observational studies required documentation of vaccination status from all sites where care was given; one case-control study of 90 patients with community-acquired invasive pneumococcal disease checked 210 sources of medical care and found that 22 vaccinated patients had received their vaccine in 20 different sites.4 The patients studied by Forrester et al also were at unusually high risk: 22% were immunocompromised, 18% were admitted from nursing homes, and 30% had nosocomial infection. In an analysis that did not include risk adjustment, they found vaccination to be –21% effective (95% CI –221% to 55%). By contrast, three indirect cohort studies evaluated 450,5 861,6 and 28377 cases of invasive pneumococcal disease. At least two of the analyses included risk adjustment.6,7 In immunocompetent older adults, vaccination effectiveness was statistically significant at 89%, 62%, and 75%, respectively. These findings are far more persuasive than those of Forrester et al regarding the effectiveness of pneumococcal vaccination. Two prospective clinical trials in older adults suggested that pneumococcal vaccine is efficacious in preventing bacteraemic disease,8,9 although the results lacked statistical significance. A larger number of clinical trials and their meta-analyses have failed to show protection against all cases of pneumococcal pneumonia (non-bacteraemic and bacteraemic cases). To account for the apparent discrepancy in the trial results, Mangtani et al offer a biological explanation: pneumococcal vaccine prevents bacteraemia from developing in patients with pneumococcal pneumonia, but fails to prevent
pneumonia itself. However, they provide no biological evidence to support their conjecture. It is well known that pneumococcal vaccine offers younger adults excellent protection against pneumococcal pneumonia (sputum culture-positive) and radiographically diagnosed pneumonia.2 It is also known that antibody responses to vaccination in older and younger adults are similar.2 Why then should pnemococcal vaccine not protect at least some older adults against pneumococcal pneumonia? Could it be that problems with the clinical trials themselves and not the vaccine have made it impossible to reach a firm conclusion about its protective efficacy in older adults? Among the ten clinical trials reviewed by Mangtani et al, three were small immunogenicity studies in highrisk patients and cannot be regarded as serious efforts to assess clinical protection. In the trial conducted in long-term care institutions in France, randomisation generated study groups of strikingly unequal size (749 and 937), the 80 investigators differed greatly in the extent to which they obtained diagnostic tests, and 33% of the study patients either died or were lost to follow-up.10 In the Veterans Administration Cooperative Study, patients in the vaccine group had more episodes of pneumococcal pneumonia before the trial and significantly higher all-cause mortality during the trial. In the Swedish and Finnish trials, most of the diagnoses of pneumococcal pneumonia were made with serological tests that are known to be unreliable. But looming over these and other methodological problems is the question of sample size.11 The sample size question can be addressed by using data gathered from the control group during the conduct of a clinical trial to calculate retrospectively the sample size (number of person-years of observation) that would have been required to rule out a false-negative result regarding vaccine efficacy (VE).12 Charles Liss and I have shown that for preventing
THE LANCET Infectious Diseases Vol 3 May 2003
http://infection.thelancet.com
For personal use. Only reproduce with permission from The Lancet Publishing Group.
Reflection & Reaction bacteraemia (VE=50%), each of the clinical trials was ten to 50 times too small, and for preventing all-cause pneumonia (VE=15%), they were five to ten times too small (VE against vaccine-type [VT] pneumococcal pneumonia=50%, and 30% of all pneumonias are VT-pneumococcal infections).2 Each of the meta-analyses also included too few person-years of observation to rule out false-negative results regarding VE in preventing pneumococcal bacteraemia and allcause pneumonia.2 Thus the metaanalyses of the clinical trials of pneumococcal vaccine were destined from the outset to be inconclusive. If clinical trials and their metaanalyses provide no firm evidence regarding the efficacy of pneumococcal vaccine in older adults, what should be done? The Potsdam report on methods for the meta-analysis of clinical trials cautioned that, “when data are too sparse, of low quality, or too heterogeneous to proceed with their statistical aggregation, . . . (investigators should) . . . perform a narrative, qualitative summary and avoid meta-analysis”.13 The report also advised investigators to interpret their results “in the context of other available evidence (human, experimental or observational)”, and noted that clinical recommendations should be “practical and explicit”. Others have pointed out that there is no theoretical or empirical basis for rejecting the findings of observational studies or for regarding them as inferior to those of randomised controlled trials.14 Although clinical trials have failed to show protection
again pneumonia, a retrospective cohort study has shown that pneumococcal vaccination significantly reduces hospital admissions for allcause pneumonia in patients with chronic obstructive pulmonary disease.15 These findings challenge directly Mangtani and colleagues’ conclusions. When all of the evidence is considered, it is clear that the debate over the efficacy of pneumococcal vaccine and the effectiveness of vaccination does not need to centre on preventing pneumococcal pneumonia. Pneumococcal vaccination is effective in preventing bacteraemic disease, and this is “a hard endpoint”. Studies covering ten western European countries show that vaccination to prevent bacteraemic disease alone is very cost-effective16 (Silvia Evers, Andre Ament, University of Maastricht, Netherlands, personal communication). These findings fully justify “a major public health programme”1 to vaccinate older adults. No doubt it would be interesting to know whether pneumococcal vaccine reduces hospitalisation for pneumococcal pneumonia or all-cause pneumonia; case-control studies could answer this question. Nonetheless, for an increasing number of health officials who make recommendations and physicians who use pneumococcal vaccine, such evidence simply is not needed.
References 1
2
3 4
5
6
7
8
9
10
11
12
13
14
Mangtani P, Cutts F, Hall AJ. Efficacy of polysaccharide pneumococcal vaccine in adults in more developed countries: the state of the evidence. Lancet Infect Dis 2003; 3: 71–78. Fedson DS, Musher DM. Pneumococcal polysaccharide vaccine. In: Plotkin SA, Orenstein WA, eds. Vaccines, 4th edn. Philadelphia: WB Saunders (in press). Forrester HL, Jahnigen DW, LaForce FM. Inefficacy of pneumococcal vaccine in a high-risk population. Am J Med 1987; 83: 425–30. Shapiro ED, Clemens JD. A controlled evaluation of the protective efficacy of pneumococcal vaccine for patients at high risk of serious pneumococcal infections. Ann Intern Med 1984; 101: 325–30. Green K, Landry L, Goldenberg E, et al. Effectiveness of a pneumococcal vaccination program in preventing invasive pneumococcal disease. 39th Interscience Conference on Antimicrobial Agents and Chemotherapy; San Francisco; September 26–29, 1999. p 673. Shapiro ED, Berg AT, Austrian R, et al. The protective efficacy of polyvalent pneumococcal polysaccharide vaccine. N Engl J Med 1991; 325: 1453–60. Butler JC, Breiman RF, Campbell JF, et al. Pneumococcal polysaccharide vaccine efficacy. An evaluation of current recommendations. JAMA 1993; 270: 1826–31. Ortqvist A, Hedlund J, Burman LA, et al. Randomized trial of 23-valent pneumococcal capsular polysaccharide vaccine in the prevention of pneumonia in middle-aged and elderly people. Lancet 1998; 351: 399–403. Honkanen PO, Keistinen T, Miettinen L, et al. Incremental effectiveness of pneumococcal vaccine on simultaneously administered influenza vaccine in preventing pneumonia and pneumococcal pneumonia among persons aged 65 years or older. Vaccine 1999; 17: 2493–2500. Gaillat J, Zmirou D, Mallaret MR, et al. Clinical trial of pneumococcal vaccine among institutionalized elderly [in French]. Rev Epidemiol Sante Publ 1985; 33: 437–44. Clemens JD, Shapiro ED. Resolving the pneumococcal vaccine controversy: are there alternatives to randomized clinical trials? Rev Infect Dis 1984; 6: 589–600. Detsky AS, Sackett DL. When was a ‘negative’ clinical trial big enough? How many patients you needed depends on what you found. Arch Intern Med 1985; 145: 709–12. Cook DJ, Sackett DL, Spitzer WO. Methodologic guidelines for systematic reviews of randomized controlled trials in health care from the Potsdam consultation on meta-analysis. J Clin Epidemiol 1995; 48: 167–71. Abel U, Koch A. The role of randomization in clinical studies: myths and beliefs. J Clin Epidemiol 1999; 52: 487–97. Nichol KL, Baken L, Wuorenma J, Nelson A. The health and economic benefits associated with pneumococcal vaccination of elderly persons with chronic lung disease. Arch Intern Med 1999; 159: 2437–42. Ament A, Baltussen R, Duru G, et al. Costeffectiveness of pneumococcal vaccination of older people: a study in 5 western European countries. Clin Infect Dis 2000; 31: 444–50.
David S Fedson
15
DS is a former employee of Aventis Pasteur MSD Correspondence: Dr David S Fedson, 57 Chemin du Lavoir, 01630 Sergy Haut, France. Tel +33 4 50 99 13 06; email [email protected]
16
We consider that the lack of robust data on efficacy against outcomes of public-health importance subsumes any discussion about whether one observational study that did not show a protective effect1 is more flawed than others that do show protection. We agree that observational studies support the indication of a reduced risk of bacteraemia from trial data. We
highlighted the one observational study (by Forrester et al1) that did not show a protective effect against bacteraemia only because such heterogeneity precludes a summary estimate across the studies. Although decisions may have to be made based on observational studies, there is far greater room for confounding and bias as rightly noted by Fedson. In case-control studies
Authors’ reply David Fedson questions one of our conclusions that, although many observational studies of the polysaccharide pneumococcal vaccine have shown a significant protective effect against bacteraemia, there is no evidence that this reduces illness or mortality from bacteraemia. Our review, we hope, will prompt evaluation of such outcomes as a research priority.
THE LANCET Infectious Diseases Vol 3 May 2003
http://infection.thelancet.com
273
For personal use. Only reproduce with permission from The Lancet Publishing Group.