- Email: [email protected]
Adverse events and vaccination-the lack of power and predictability of infrequent events in pre-licensure study
Vaccine 19 (2001) 2428– 2433 www.elsevier.com/locate/vaccine
Adverse events and vaccination-the lack of power and predictability of infrequent events in pre-licensure study Robert M. Jacobson a,*, Adedunni Adegbenro b, V. Shane Pankratz c, Gregory A. Poland d a
Department of Pediatric and Adolescent Medicine, Vaccine Research Group, Mayo Clinic Baldwin 3B, Rochester, MN 55905 -0001, USA b Department of Pediatric and Adolescent Medicine, Vaccine Research Group, Mayo Clinic, Rochester, MN, USA c Department of Health Sciences Research, Vaccine Research Group, Mayo Clinic, Rochester, MN, USA d Department of Medicine, Vaccine Research Group, Mayo Clinic, Rochester, MN, USA
Abstract The recent setback in the development of a safe and effective rotavirus vaccine illustrates an important problem regarding prelicensure testing and its ability to identify rare vaccine-related adverse effects. It is our contention that the possibility of a rare but serious vaccine adverse effect is difficult to detect in prelicensure testing. In this paper, we review the history regarding the testing and eventual studies that led to the permanent withdrawal of that vaccine. The post-licensure discovery of a serious adverse event associated with the rotavirus vaccine is not unique among vaccines, but represents a recurrent phenomenon that in fact is mathematically predictable. Prelicensure studies examine thousands of subjects and not hundreds of thousands. A sample size of 10,000 subjects may provide excellent estimates of efficacy, but cannot provide an adequate denominator to rule out rare adverse events. It lacks the power. Just as with the rotavirus vaccine, only after hundreds of thousands of doses of vaccines are distributed, will such rare events appear often enough to permit detection. For that reason, we must depend upon the modern post-licensure surveillance programs that we already have in place. © 2001 Elsevier Science Ltd. All rights reserved. Keywords: Detection; Prelicensure testing; Rotavirus vaccine; Sample size; Serious vaccine adverse effects
1. Introduction The recent setback in the development of a safe and effective rotavirus vaccine illustrates an important problem concerning the identification of rare vaccinerelated adverse effects [1 – 3]. The rotavirus vaccine had been tested in thousands of subjects prior to licensure in the US [3,4]. Although intussusception was seriously considered as a possible adverse effect, pre-licensure studies failed to reveal a statistically significant increase in the incidence of intussusception among vaccine recipients. Approximately 1 year after the vaccine was adopted for routine use in infants across the country, public health officials became aware of a possible association between that particular vaccine and intussusception and called for a moratorium of the vaccine’s use * Corresponding author. Tel.: +1-507-2664408; fax: + 1-5072849744. E-mail address: [email protected] (R.M. Jacobson).
[5]. Further study confirmed that association and led to a permanent cessation of the vaccine’s manufacture, distribution, and use [1]. The impact that this discovery will have upon the eventual development and promulgation of a successful rotavirus vaccine remains unknown. It is unclear even at this time, how this may affect the public’s acceptance of new vaccines in general. These events concern experts with regard to the future of vaccine licensure and in particular with candidate rotavirus vaccines, [2,3,6] The Wall Street Journal reported in October 1999, that the US Food and Drug Administration (FDA) was reevaluating its pre-licensure procedures [7]. The FDA was considering not only extensive testing for future rotavirus vaccine candidates to rule out associations with intussusception but also larger, pre-licensure trials for all the new candidate vaccines. It is not evident at all that mistakes were made in the pre-licensure process. Indeed, it is our contention that
0264-410X/01/$ - see front matter © 2001 Elsevier Science Ltd. All rights reserved. PII: S 0 2 6 4 - 4 1 0 X ( 0 0 ) 0 0 4 6 7 - 9
R.M. Jacobson et al. / Vaccine 19 (2001) 2428–2433
the possibility of a rare but serious vaccine adverse effect is difficult to detect in pre-licensure testing. We do not believe that expanding the size of the pre-licensure trials will have practical significance in reducing the possibility of rare but serious vaccine adverse effects. In this paper, we will review the pre-licensure testing of the rotavirus vaccine and the post-licensure discovery of the intussusception-association. We will then generalize our discussion to include pre-licensure testing of vaccines in general. From this, we will develop our argument through a demonstration of the problem of pre-licensure sample-size and power. We will conclude with a review of various solutions to the underlying problem of the identification of rare but serious adverse effects resulting from vaccination.
2. Pre-licensure testing of rotavirus vaccine In the pre-licensure studies of the Wyeth Lederle’s RotaShield®, a tetravalent rhesus-based recombinant rotavirus vaccine, intussusception occurred in only five of the 10 054 recipients of the vaccine [5]. Three of the cases occurred during the first week after vaccination. One case was reported among the 4633 controls. The difference between the two rates of intussusception of the two groups was not statistically significant [3,8]. After careful review of these cases in anticipation of licensure, the Rotavirus working group of the Advisory Committee on Immunization Practices (ACIP) of the US Centers for Disease Control and Prevention (CDC) concluded the association between the cases of intussusception was temporal but not causal [3]. Upon licensure, the US Food and Drug Agency chose to include intussusception as a potential adverse reaction in the package insert, and the ACIP recommended post-licensure surveillance for this adverse event [3,9]. This was indeed carried out and led to the eventual discovery of the association [1,5]. On 31 August, 1998, the US FDA licensed RotaShield® for routine use in infants [9]. The ACIP recommended its universal use in routine infant immunization later that year as did the American Academy of Pediatrics and the American Academy of Family Physicians [5,9,10]. Upon licensure, vaccine providers worked diligently to vaccinate infants in anticipation of the coming rotavirus season.
3. Post-licensure reports of intussusception In May 1999, the Centers for Disease Control became aware of a sharp increase in instances of intussusception associated with RotaShield® [7]. By 7 July,
2429
the Centers for Disease Control had learned of 15 cases. On 15 July, the Centers for Disease Control recommended that clinicians halt the use of the vaccine. Wyeth Lederle Vaccines suspended the sales of the vaccine simultaneously [5]. Further study revealed additional cases totalling 102 cases reported to the Vaccine Adverse Events Reporting System [1,11]. About 57 infants developed intussusception within 7 days of vaccination. Out of those 29 underwent surgery, and seven involved bowel resection. Among the 102 cases, one infant died. A case series analysis conducted by the National Immunization Program and presented at the October, 1999 ACIP meeting indicated that the risk of intussusception was increased by 60% after rotavirus vaccination among cases ever vaccinated as compared with cases, who were never vaccinated (PB 0.0003) [11]. For the first dose, the risk of intussusception was increased 19-fold in the first 3–7 days after vaccination and almost 4-fold in the 8–14 days after vaccination (PB0.0002). The National Immunization Program’s case-control data presented at the same meeting supported these findings [11]. The risk of intussusception was increased by 80% after rotavirus vaccination for those who received any dose ever as compared with those who never received a dose. For the first dose, the risk increased 25-fold in the first 3–7 days after vaccination. The risk increased approximately 7-fold in the following 8–14 days. For the second dose, the risk of intussusception increased approximately 13-fold in the first 3–7 days after vaccination. A population-based cohort study’s preliminary findings also indicate an increased risk with intussusception [11]. Again, in this analysis, the highest incidence of intussusception occurred in the first week following vaccination. This study drew upon the resources of the Vaccine Safety Datlink project described below. The three analyses supported a strong association between the vaccine and intussusception with the intussusception occurring in the first weeks following vaccination. The data of course only became available after licensure and specifically after the manufacturer distributed 1.8 million doses (as of 1 June 1999) [5]. The CDC estimated 1.5 million doses (83%) had been actually administered. On 22 October 1999, the ACIP, after reviewing these analyses, made its final decision [1]. It concluded that intussusception occurs with significantly increased frequency in the first 1–2 weeks after vaccination with RotaSchield®, and withdrew its recommendation for the routine vaccination of infants in the US. At the same time, Wyeth Lederle withdrew the vaccine from the market and requested an immediate return of all vaccines.
2430
R.M. Jacobson et al. / Vaccine 19 (2001) 2428–2433
4. Other post-licensure associations The post-licensure discovery of a serious adverse event associated with the rotavirus vaccine is not unique among vaccines. Other examples include the post-licensure discoveries of atypical measles in recipients of the killed measles vaccine, [12] of hypotonic hyporesponsive episodes following the whole cell pertussis vaccine, [13] and of thrombocytopenic purpura associated with the live attenuated measles vaccine [14]. We must consider what vaccines currently in use or under consideration for future licensing will be similarly linked to earlier undiscovered adverse events. It is our contention that such rare adverse events typically will not be discovered in pre-licensure testing and that we must instead rely upon post-licensure vigilance to detect rare but real adverse events.
5. The phases of pre-licensure testing To consider this, let us review the process by which vaccines are currently approved in the US Vaccines, like other biologics and pharmaceuticals, undergo extensive testing in animals and in human subjects prior to licensure. The Food and Drug Act Regulations refer to various phases of drug studies in humans [Title 21 of the Code of Federal Regulations (CFR), Part 312 (21 CFR 312)]. Clinical testing prior to licensure involves three phases of testing that occasionally overlap. Prephase I studies refer to animal testing and include tests for safety, tolerability, and efficacy. The entire pre-licensure process from Phase I through Phase III and then licensure takes an average 8.5 years [15]. Of note, for every five drugs for which phase I studies are initiated, only one is licensed. Phase I studies are the first human subjects studies [16 – 18]. These are focused primarily on safety and tolerability and specifically designed to identify a dose to use in further studies. These studies are typically open-label and nonrandomized. They involve 20–80 subjects and last several months. Subjects may be normal subjects or patients or both. Phase II studies involve a larger number of subjects of approximately 100 – 300 in number [19 – 22]. These studies seek to determine evidence for efficacy and only secondarily data on safety and tolerability. They are frequently conducted as open-label, single-arm studies. These data are gathered over several months to one or two years. Often, immunogenicity is studied rather than efficacy. Should the vaccine appear immunogenic, these studies provide the information necessary to design successful Phase III studies of adequate power to demonstrate efficacy. Phase III studies involve larger numbers. For vaccine studies, they involve several hundred to several thou-
sand subjects [23,24]. The goals of these studies typically include efficacy as well as immunogenicity, safety, tolerability, and dose-schedule. These randomized, double-blinded, controlled trials are the pivotal studies upon which the US FDA bases its decision to approve or license the vaccine. Phase IV studies refer to post-licensure studies and take many different forms for many different objectives [25,26]. Recent examples include studies that explore alternate routes of dosing and focus on new populations. Phase IV testing may involve much larger groups of subjects than studied in the first three phases and may have as the primary study goal safety issues [27]. These phase IV studies may be conducted and reported years or even decades after licensure [25,26]. Thus, at the time of licensure, at the end of phase III testing, no more than 10 000 individuals may have actually received the study vaccine. Furthermore, this process involves only a few years observation of the recipients post-vaccination.
6. Sample size, rare events, and the rule of three A sample size of 10 000 subjects may provide excellent estimates of efficacy but cannot provide an adequate denominator to rule out rare adverse events. To demonstrate this, we will first introduce the so-called Rule of Three [28]. The rule refers to an approximate estimate of an upper limit to a 95% confidence interval, when no occurrences of a particular event have yet occurred despite numerous observations. The Rule of Three draws upon the binomial distribution of the occurrence or non-occurrence of events. The rule states that with 30 or more observations, when no events have occurred, the upper limit of the 95% confidence interval for the rate of events is approximately 3/N, when N is the number of observations. For example one may wish to estimate the possibility of a particular complication following a new surgical procedure despite the fact that such a complication has not occurred despite 30 cases. One could use the Rule of Three to estimate that the true rate of that complication may be much higher than zero. The Rule of Three indicates that the 95% confidence interval includes as its upper limit approximately 3/30 or 10%. Of course, with more observations, the denominator expands and the upper limit falls. With 100 observations, the upper limit is approximately 3/100 or 3%. With 1000, the upper limit is approximately 3/1000 or 0.3%. In terms of pre-licensure testing, given 10 000 subjects studied, despite the lack of any observed events, for any particular concern, the upper limit of its occurrence would be approximately 3/10 000 or 0.03%.
R.M. Jacobson et al. / Vaccine 19 (2001) 2428–2433
7. Power, rare events, and background rates In the case of rotavirus vaccine, pre-licensure testing, however, did reveal some occurrences of intussusception. Investigators identified a total of six cases of intussusception among the 14 687 study participants in the pre-licensure phases of testing. Five of the six cases of intussusception occurred among the 10 054 recipients of the vaccine or 0.050%, with an exact 95% confidence interval ranging from 0.02 to 0.12%. One case occurred among the 4633 controls for a rate of 0.022%, with an exact 95% confidence interval ranging from 0.005 to 0.12%. A background rate of intussusception does occur among infants, and, in most cases of infantile intussusception, an etiology is not identified, unlike in adults. The rates found in these studies were similar to rates reported by others as the typical background incidence rate of intussusception among infants [8]. The confidence intervals for the two rates overlap, indicating that the difference between the two rates is not statistically significant. Performing Fisher’s exact test results in a two-tailed P value of 0.67, which is nowhere near the 0.05 threshold used to establish statistical significance. (Bonferroni corrections of the P value given the multiple independent safety outcomes examined would create a far more stringent threshold). One might argue that a two-tailed test is too severe, as we were looking for an adverse event (using the two-tailed test implies that the vaccine might have had either a deleterious effect or a beneficial one. One might support the use of two-tailed statistics arguing a priori that the vaccine might protect against intussusception by preventing rotavirus infection. In fact, experts did argue that. Nonetheless, a one-tailed test still does not provide an adequate solution. A one-tailed test only gives a P value of 0.09. What would it take to drive the P value below 0.05 given the control or background rate of one out of 4633? A doubling of cases would not do it. The twotailed Fischer exact would give a P value of 0.19. Tripling the rate of intussusception with 15 out of 10 054, and performing a one-tailed test succeeds with a 2 P value less than 0.03. The magic number for the one-tailed P value to cross 0.05 appears to be 13 cases out of 10 054. There were not 13 cases. There were not 10. There were five. What if more subjects had been studied prior to licensure? What if the number studied doubled from 10 054 recipients and 4633 controls to 20 108 recipients and 9266 controls? With 10 cases among recipients and two among controls, the two-tailed P value is 0.36. What if the number studied was tripled to 30 162 recipients and 13 899 controls? This would give us a 2 P-value of still only 0.17. What the number studied had been quadrupled to 40 216 recipients and 18 532 con-
2431
trols? This still would not have crossed the threshold of statistical significance. It gives a P value of only 0.12. One would not obtain a P value less than 0.05 until one had studied not 6 but almost seven times the number of subjects or approximately 100 000 subjects altogether! And even this sample size would only provide approximately 50% power to detect the observed difference. The Table 1 illustrates the power to detect differences in intussusception rates of the magnitude observed in pre-licensure. Not until there were 251 234 subjects would there be 90% power to statistically distinguish between rates of 0.050 and 0.022%. Indeed it was not just the number of cases of intussusception post-licensure that caught the attention of the epidemiologists but the timing of the events and specifically the number of cases of intussusception occurring in the first week following vaccination [3]. We must also concern ourselves with the problem of delayed adverse events such as atypical measles, which develop years after the receipt of the killed measles vaccine. The average length of time between vaccination and event in the initial reports were 5–6 years [12]. This time span greatly exceeds the typical length of Phase III studies.
8. Possible solutions The numbers of subjects studied in the pre-licensure testing for the rotavirus vaccine are very similar to the numbers of other recently licensed vaccines. Merck’s Hepatitis A Vaccine (VAQTA®) included 9181 recipients in its phase II/III studies.(Package insert) Lederle’s Pneumococcal Conjugate Vaccine (Prevnar™) included 18 906 recipients in its phase II/III studies (Package Table 1 Power to detect a difference as small as the pre-licensure rates of intussusception achieved with increasing sample sizes by multiples of the original sample size (10 054 and 4633) (h =0.05) Multiple of pre-licensure sample size 1 2 3 4 5 6 7 8 9 10 11 12 12.8 13 17.1
Power (%) 12 20 27 35 42 49 55 60 65 70 74 78 80 81 90
2432
R.M. Jacobson et al. / Vaccine 19 (2001) 2428–2433
insert). We must accept the fact that rare events are not likely to be discovered in pre-licensure testing. Therefore, we must rely upon post-licensure vigilance to detect real but rare adverse events. Solutions lay not in increasing pre-licensure numbers but in sustaining and building post-licensure surveillance systems and protocols. Specific examples of post-licensure surveillances include the US Vaccine Adverse Event Reporting System, commonly referred to its acronym VAERS [29]. VAERS is a passive surveillance system operated by the FDA and the CDC. Vaccine manufacturers are required to report to VAERS any adverse event reported to them, and health-care providers are encouraged to report any adverse event possibly attributable to vaccine. Vaccine recipients and their families also can report adverse events to VAERS. VAERS played a pivotal role in identifying the problem of intussusception with RotaShield®. Its great number of reports and national coverage makes VAERS useful for post-licensure surveillance of rare adverse events that are unlikely to be identified through clinical trials or phase IV studies carried out by manufacturers after licensure [27]. A second example is the Vaccine Safety Datalink Project [30 –32]. This is a partnership of the US CDC with four US health maintenance organizations. This project involves the creation of organization-wide, computerized vaccination databases that permit studies of large numbers of vaccine recipients and avoid the limitations of passive surveillance that VAERS suffers. Testing known rare adverse events in the studied populations has validated this method [30,31]. Farrington and colleagues in the UK have developed similar methods for active post-marketing surveillance [33]. They linked vaccination records with computerized hospital admission records and have validated their methodology through the examination of the earlier established links between DTP and febrile convulsion and MMR and thrombocytopenic purpura. Re-analysis of original vaccine trial data permits the creation of retrospective exposure cohorts that can be linked to historical controls for a given hypothesis [34]. This was the method used to study and dismiss the claim that Haemophilus influenzae type b vaccines were causally linked with diabetes mellitus [35]. An additional source of re-analysis resides in the powerful tool of meta-analysis and in particular in the Cochrane Library, which provides a systematic review of the known effects of vaccines [34,36]. We conclude with two cautionary notes. First, no vaccine is perfectly safe, and as we move from pre-licensure testing of thousands of subjects to post-licensure routine use in millions of children, we will necessarily see the appearance and increased occurrence of a number of true vaccine reactions and adverse
events. Chen et al. report that the VAERS in the US now receives approximately 10 000 reports each year, a number, which exceeds the reported incidence of most vaccine-preventable childhood diseases combined [29]. Second, valiant efforts to achieve universal routine vaccination of children have incited a vigorous antivaccine movement [2]. This movement has repeatedly seized upon spurious links between various vaccines and adverse outcomes to bolster its arguments against routine childhood vaccination. These spurious links have included, among others, sudden infant death syndrome and DTP, multiple sclerosis and hepatitis B, autism and measles vaccine, and diabetes mellitus and early infant vaccination. These claimed associations have resulted in the public’s loss of confidence and refusal of vaccination, and, while these claimed associations were eventually repudiated from a lack of confirmation through further study, they continue to create doubt and suspicion. Due to the very real possibility of rare but true adverse effects, vaccine researchers cannot routinely dismiss claims of adverse events but must work diligently to explore the possibility of association and etiology, identify true adverse effects, even when rare or latent, and expose false claims for what they are. References [1] CDC. Withdrawal of Rotavirus Vaccine Recommendation. MMWR, 1999; 48:1007. [2] Poland GA, Jacobson RM. Vaccine safety: injecting a dose of common sense [editorial]. Mayo Clin Proc 2000;75:135–9. [3] Rennels MB. The rotavirus vaccine story: a clinical investigator’s view. Pediatrics 2000;106:123– 5. [4] Jacobson RM. The current status of the rotavirus vaccine. Vaccine 1999;17:1690– 9. [5] CDC. Intussusception Among Recipients of Rotavirus Vaccine — United States, 1998– 1999. MMWR 1999; 48:577– 581. [6] Anonymous. ACIP reverses recommendation for rotavirus vaccine use: Wyeth Lederle voluntarily pulled RotaShield from the market because of reports of intussusception http:// www.slackinc.com/child/idc/199911/rota.asp. Slack Incorporated, 1999. [7] Harris G. After vaccine’s recall, regulators look for holes in the safety net. Wall Street Journal. New York, 1999: B1, B4. [8] Rennels M, Parashar U, Holman R, Le C, Chang H, Glass R. Lack of an apparent association between intussusception and wild or vaccine rotavirus infection. Pediatr Infect Dis J 1998;17:924– 5. [9] CDC. Rotavirus vaccine for the prevention of rotavirus gastroenteritis among children. Recommendations of the Advisory Committee on Immunization Practices (ACIP). MMWR. 1999; 48:1– 20. [10] Committee on Infectious Diseases. American Academy of Pediatrics. Prevention of rotavirus disease: guidelines for use of rotavirus vaccine. Pediatrics 1998; 102:1483– 1491. [11] Anonymous. ACIP reverses recommendation for rotavirus vaccine: Wyeth Lederle voluntarily pulled RotaShield from the market because of reports of intussusception. http:// www.slackinc.com/child/idc/199912/rota.asp. SLACK Incorporated, 1999.
R.M. Jacobson et al. / Vaccine 19 (2001) 2428–2433 [12] Fulginiti V, Eller J, Downie A, Kempe C. Altered reactivity to measles virus: a typical measles in children previously immunized with inactivated measles virus vaccines. J Am Med Assoc 1967;202:1075– 80. [13] Blumberg D, Morgan C, Lewis K, et al. An ongoing surveillance study of persistent crying and hypotonic-hyporesponsive episodes following routine DTP immunization: a preliminary report. Tokai J Exp Clin Med 1988;13(Suppl.):133–6. [14] Stratton K, Howe C, Johnston RJ. Adverse events associated with childhood vaccines other than pertussis and rubella. Summary of a report from the Institute of Medicine. J Am Med Assoc 1994;271:1602–5. [15] Spilker BA. The Drug Development and Approval Process. http://www.hcnor.com/rx – help/approval.html, 2000. [16] Gluck U, Gebbers JO, Gluck R. Phase 1 evaluation of intranasal virosomal influenza vaccine with and without Escherichia coli heat-labile toxin in adult volunteers. J Virol 1999;73:7780– 6. [17] Doherty JF, Pinder M, Tornieporth N, et al. A phase I safety and immunogenicity trial with the candidate malaria vaccine RTS,S/SBAS2 in semi-immune adults in the Gambia. Am J Trop Med Hyg 1999;61:865–8. [18] McCormack S, Tilzey A, Carmichael A, et al. A phase I trial in HIV negative healthy volunteers evaluating the effect of potent adjuvants on immunogenicity of a recombinant gp120(W61D) derived from dual tropic R5X4 HIV-1(ACH320). Vaccine 2000;18:1166– 77. [19] Uehling DT, Hopkins WJ, Balish E, Xing YN, Heisey DM. Vaginal mucosal immunization for recurrent urinary tract infection — Phase II clinical trial. J Urol 1997;157:2049–52. [20] Schedel I, Sutor GC, Hunsmann G, Jurkiewicz E. Phase II study of anti-CD4 idiotype vaccination in HIV positive volunteers. Vaccine 1999;17:1837–45. [21] Konadu EY, Lin FYC, Ho VA, et al. Phase 1 and phase 2 studies of Salmonella enterica serovar Paratyphi A O-specific polysaccharide-tetanus toroid conjugates in adults, teenagers, and 2 – 4 year old children in Vietnam, Infec. Immun. 1529; 68:1529– 1534. [22] Tacket, CO, Sztein, MB, Wasserman, SS, et al. Phase 2 clinical trial of attenuated Salmonella enterica serovar Typhi oral live vector vaccine CVD 908-htrA in US volunteers, Infec. Immun. 1196; 68:1196– 1201. [23] Francis DP, Gregory T, McElrath MJ, et al. Advancing aidsvax(Tm) to phase 3-safety, immunogenicity, and plans for phase 3. AIDS Res Hum Retrovir 1998;14:325S–31S.
.
2433
[24] Beck HP, Felger I, Huber W, et al. Analysis of multiple Plasmodium falciparum infections in Tanzanian children during the phase III trial of the malaria vaccine Spf66. J Infect Dis 1997;175:921– 6. [25] Lagos R, Levine OS, Avendano A, Horwitz I, Levine MM. The introduction of routine Haemophilus influenzae type b conjugate vaccine in Chile: a framework for evaluating new vaccines in newly industrializing countries. Pediatr Infect Dis J 1998;17:139S– 48S. [26] Ragni MV, Lusher JM, Koerper MA, Manco-Johnson M, Krause DS. Safety and immunogenicity of subcutaneous hepatitis. A vaccine in children with haemophilia. Haemophilia 2000;6:98– 103. [27] Braun MM, Ellenberg SS. Descriptive epidemiology of adverse events after immunization: reports to the Vaccine Adverse Event Reporting System (VAERS), 1991– 1994. J Pediatr 1997;131:529– 35. [28] Eyspasch E, Lefering R, Kum C, Troidl H. Probability of adverse events that have not yet occurred: a statistical reminder. Br Med J 1995;311:619– 20. [29] Chen RT, Rastogi SC, Mullen JR, et al. The Vaccine Adverse Event Reporting System (VAERS). Vaccine 1994;12:542–50. [30] Mullooly J, Drew L, DeStefano F, et al. Quality of HMO vaccination databases used to monitor childhood vaccine safety. Vaccine Safety DataLink Team. Am J Epidemiol 1999;149:186– 94. [31] Chen RT, Glasser JW, Rhodes PH, et al. Vaccine Safety Datalink project: a new tool for improving vaccine safety monitoring in the United States. The Vaccine Safety Datalink Team. Pediatrics 1997;99:765– 73. [32] Wassilak SG, Glasser JW, Chen RT, Hadler SC. Utility of large-linked databases in vaccine safety, particularly in distinguishing independent and synergistic effects. The Vaccine Safety Datalink Investigators. Ann New York Acad Sci 1995;754:377– 82. [33] Farrington P, Pugh S, Colville A, et al. A new method for active surveillance of adverse events from diphtheria/tetanus/pertussis and measles/mumps/rubella vaccines. Lancet 1995;345:567–9. [34] Jefferson T. Vaccination and its adverse effects: real or perceived. Society should think about means of linking exposure to potential long term effect [editorial]. Br Med J 1998;317:159–60. [35] Karvonen M, Cepaitis Z, Tuomilehto J. Association between type 1 diabetes and Haemophilus influenzae type b vaccination: birth cohort study. Br Med J 1999;318:1169– 72. [36] Jacobson RM. Promises and pitfalls of meta-analysis in vaccine research. Vaccine 1999;17:1628– 34.
Adverse events and vaccination-the lack of power and predictability of infrequent events in pre-licensure study Robert M. Jacobson a,*, Adedunni Adegbenro b, V. Shane Pankratz c, Gregory A. Poland d a
Department of Pediatric and Adolescent Medicine, Vaccine Research Group, Mayo Clinic Baldwin 3B, Rochester, MN 55905 -0001, USA b Department of Pediatric and Adolescent Medicine, Vaccine Research Group, Mayo Clinic, Rochester, MN, USA c Department of Health Sciences Research, Vaccine Research Group, Mayo Clinic, Rochester, MN, USA d Department of Medicine, Vaccine Research Group, Mayo Clinic, Rochester, MN, USA
Abstract The recent setback in the development of a safe and effective rotavirus vaccine illustrates an important problem regarding prelicensure testing and its ability to identify rare vaccine-related adverse effects. It is our contention that the possibility of a rare but serious vaccine adverse effect is difficult to detect in prelicensure testing. In this paper, we review the history regarding the testing and eventual studies that led to the permanent withdrawal of that vaccine. The post-licensure discovery of a serious adverse event associated with the rotavirus vaccine is not unique among vaccines, but represents a recurrent phenomenon that in fact is mathematically predictable. Prelicensure studies examine thousands of subjects and not hundreds of thousands. A sample size of 10,000 subjects may provide excellent estimates of efficacy, but cannot provide an adequate denominator to rule out rare adverse events. It lacks the power. Just as with the rotavirus vaccine, only after hundreds of thousands of doses of vaccines are distributed, will such rare events appear often enough to permit detection. For that reason, we must depend upon the modern post-licensure surveillance programs that we already have in place. © 2001 Elsevier Science Ltd. All rights reserved. Keywords: Detection; Prelicensure testing; Rotavirus vaccine; Sample size; Serious vaccine adverse effects
1. Introduction The recent setback in the development of a safe and effective rotavirus vaccine illustrates an important problem concerning the identification of rare vaccinerelated adverse effects [1 – 3]. The rotavirus vaccine had been tested in thousands of subjects prior to licensure in the US [3,4]. Although intussusception was seriously considered as a possible adverse effect, pre-licensure studies failed to reveal a statistically significant increase in the incidence of intussusception among vaccine recipients. Approximately 1 year after the vaccine was adopted for routine use in infants across the country, public health officials became aware of a possible association between that particular vaccine and intussusception and called for a moratorium of the vaccine’s use * Corresponding author. Tel.: +1-507-2664408; fax: + 1-5072849744. E-mail address: [email protected] (R.M. Jacobson).
[5]. Further study confirmed that association and led to a permanent cessation of the vaccine’s manufacture, distribution, and use [1]. The impact that this discovery will have upon the eventual development and promulgation of a successful rotavirus vaccine remains unknown. It is unclear even at this time, how this may affect the public’s acceptance of new vaccines in general. These events concern experts with regard to the future of vaccine licensure and in particular with candidate rotavirus vaccines, [2,3,6] The Wall Street Journal reported in October 1999, that the US Food and Drug Administration (FDA) was reevaluating its pre-licensure procedures [7]. The FDA was considering not only extensive testing for future rotavirus vaccine candidates to rule out associations with intussusception but also larger, pre-licensure trials for all the new candidate vaccines. It is not evident at all that mistakes were made in the pre-licensure process. Indeed, it is our contention that
0264-410X/01/$ - see front matter © 2001 Elsevier Science Ltd. All rights reserved. PII: S 0 2 6 4 - 4 1 0 X ( 0 0 ) 0 0 4 6 7 - 9
R.M. Jacobson et al. / Vaccine 19 (2001) 2428–2433
the possibility of a rare but serious vaccine adverse effect is difficult to detect in pre-licensure testing. We do not believe that expanding the size of the pre-licensure trials will have practical significance in reducing the possibility of rare but serious vaccine adverse effects. In this paper, we will review the pre-licensure testing of the rotavirus vaccine and the post-licensure discovery of the intussusception-association. We will then generalize our discussion to include pre-licensure testing of vaccines in general. From this, we will develop our argument through a demonstration of the problem of pre-licensure sample-size and power. We will conclude with a review of various solutions to the underlying problem of the identification of rare but serious adverse effects resulting from vaccination.
2. Pre-licensure testing of rotavirus vaccine In the pre-licensure studies of the Wyeth Lederle’s RotaShield®, a tetravalent rhesus-based recombinant rotavirus vaccine, intussusception occurred in only five of the 10 054 recipients of the vaccine [5]. Three of the cases occurred during the first week after vaccination. One case was reported among the 4633 controls. The difference between the two rates of intussusception of the two groups was not statistically significant [3,8]. After careful review of these cases in anticipation of licensure, the Rotavirus working group of the Advisory Committee on Immunization Practices (ACIP) of the US Centers for Disease Control and Prevention (CDC) concluded the association between the cases of intussusception was temporal but not causal [3]. Upon licensure, the US Food and Drug Agency chose to include intussusception as a potential adverse reaction in the package insert, and the ACIP recommended post-licensure surveillance for this adverse event [3,9]. This was indeed carried out and led to the eventual discovery of the association [1,5]. On 31 August, 1998, the US FDA licensed RotaShield® for routine use in infants [9]. The ACIP recommended its universal use in routine infant immunization later that year as did the American Academy of Pediatrics and the American Academy of Family Physicians [5,9,10]. Upon licensure, vaccine providers worked diligently to vaccinate infants in anticipation of the coming rotavirus season.
3. Post-licensure reports of intussusception In May 1999, the Centers for Disease Control became aware of a sharp increase in instances of intussusception associated with RotaShield® [7]. By 7 July,
2429
the Centers for Disease Control had learned of 15 cases. On 15 July, the Centers for Disease Control recommended that clinicians halt the use of the vaccine. Wyeth Lederle Vaccines suspended the sales of the vaccine simultaneously [5]. Further study revealed additional cases totalling 102 cases reported to the Vaccine Adverse Events Reporting System [1,11]. About 57 infants developed intussusception within 7 days of vaccination. Out of those 29 underwent surgery, and seven involved bowel resection. Among the 102 cases, one infant died. A case series analysis conducted by the National Immunization Program and presented at the October, 1999 ACIP meeting indicated that the risk of intussusception was increased by 60% after rotavirus vaccination among cases ever vaccinated as compared with cases, who were never vaccinated (PB 0.0003) [11]. For the first dose, the risk of intussusception was increased 19-fold in the first 3–7 days after vaccination and almost 4-fold in the 8–14 days after vaccination (PB0.0002). The National Immunization Program’s case-control data presented at the same meeting supported these findings [11]. The risk of intussusception was increased by 80% after rotavirus vaccination for those who received any dose ever as compared with those who never received a dose. For the first dose, the risk increased 25-fold in the first 3–7 days after vaccination. The risk increased approximately 7-fold in the following 8–14 days. For the second dose, the risk of intussusception increased approximately 13-fold in the first 3–7 days after vaccination. A population-based cohort study’s preliminary findings also indicate an increased risk with intussusception [11]. Again, in this analysis, the highest incidence of intussusception occurred in the first week following vaccination. This study drew upon the resources of the Vaccine Safety Datlink project described below. The three analyses supported a strong association between the vaccine and intussusception with the intussusception occurring in the first weeks following vaccination. The data of course only became available after licensure and specifically after the manufacturer distributed 1.8 million doses (as of 1 June 1999) [5]. The CDC estimated 1.5 million doses (83%) had been actually administered. On 22 October 1999, the ACIP, after reviewing these analyses, made its final decision [1]. It concluded that intussusception occurs with significantly increased frequency in the first 1–2 weeks after vaccination with RotaSchield®, and withdrew its recommendation for the routine vaccination of infants in the US. At the same time, Wyeth Lederle withdrew the vaccine from the market and requested an immediate return of all vaccines.
2430
R.M. Jacobson et al. / Vaccine 19 (2001) 2428–2433
4. Other post-licensure associations The post-licensure discovery of a serious adverse event associated with the rotavirus vaccine is not unique among vaccines. Other examples include the post-licensure discoveries of atypical measles in recipients of the killed measles vaccine, [12] of hypotonic hyporesponsive episodes following the whole cell pertussis vaccine, [13] and of thrombocytopenic purpura associated with the live attenuated measles vaccine [14]. We must consider what vaccines currently in use or under consideration for future licensing will be similarly linked to earlier undiscovered adverse events. It is our contention that such rare adverse events typically will not be discovered in pre-licensure testing and that we must instead rely upon post-licensure vigilance to detect rare but real adverse events.
5. The phases of pre-licensure testing To consider this, let us review the process by which vaccines are currently approved in the US Vaccines, like other biologics and pharmaceuticals, undergo extensive testing in animals and in human subjects prior to licensure. The Food and Drug Act Regulations refer to various phases of drug studies in humans [Title 21 of the Code of Federal Regulations (CFR), Part 312 (21 CFR 312)]. Clinical testing prior to licensure involves three phases of testing that occasionally overlap. Prephase I studies refer to animal testing and include tests for safety, tolerability, and efficacy. The entire pre-licensure process from Phase I through Phase III and then licensure takes an average 8.5 years [15]. Of note, for every five drugs for which phase I studies are initiated, only one is licensed. Phase I studies are the first human subjects studies [16 – 18]. These are focused primarily on safety and tolerability and specifically designed to identify a dose to use in further studies. These studies are typically open-label and nonrandomized. They involve 20–80 subjects and last several months. Subjects may be normal subjects or patients or both. Phase II studies involve a larger number of subjects of approximately 100 – 300 in number [19 – 22]. These studies seek to determine evidence for efficacy and only secondarily data on safety and tolerability. They are frequently conducted as open-label, single-arm studies. These data are gathered over several months to one or two years. Often, immunogenicity is studied rather than efficacy. Should the vaccine appear immunogenic, these studies provide the information necessary to design successful Phase III studies of adequate power to demonstrate efficacy. Phase III studies involve larger numbers. For vaccine studies, they involve several hundred to several thou-
sand subjects [23,24]. The goals of these studies typically include efficacy as well as immunogenicity, safety, tolerability, and dose-schedule. These randomized, double-blinded, controlled trials are the pivotal studies upon which the US FDA bases its decision to approve or license the vaccine. Phase IV studies refer to post-licensure studies and take many different forms for many different objectives [25,26]. Recent examples include studies that explore alternate routes of dosing and focus on new populations. Phase IV testing may involve much larger groups of subjects than studied in the first three phases and may have as the primary study goal safety issues [27]. These phase IV studies may be conducted and reported years or even decades after licensure [25,26]. Thus, at the time of licensure, at the end of phase III testing, no more than 10 000 individuals may have actually received the study vaccine. Furthermore, this process involves only a few years observation of the recipients post-vaccination.
6. Sample size, rare events, and the rule of three A sample size of 10 000 subjects may provide excellent estimates of efficacy but cannot provide an adequate denominator to rule out rare adverse events. To demonstrate this, we will first introduce the so-called Rule of Three [28]. The rule refers to an approximate estimate of an upper limit to a 95% confidence interval, when no occurrences of a particular event have yet occurred despite numerous observations. The Rule of Three draws upon the binomial distribution of the occurrence or non-occurrence of events. The rule states that with 30 or more observations, when no events have occurred, the upper limit of the 95% confidence interval for the rate of events is approximately 3/N, when N is the number of observations. For example one may wish to estimate the possibility of a particular complication following a new surgical procedure despite the fact that such a complication has not occurred despite 30 cases. One could use the Rule of Three to estimate that the true rate of that complication may be much higher than zero. The Rule of Three indicates that the 95% confidence interval includes as its upper limit approximately 3/30 or 10%. Of course, with more observations, the denominator expands and the upper limit falls. With 100 observations, the upper limit is approximately 3/100 or 3%. With 1000, the upper limit is approximately 3/1000 or 0.3%. In terms of pre-licensure testing, given 10 000 subjects studied, despite the lack of any observed events, for any particular concern, the upper limit of its occurrence would be approximately 3/10 000 or 0.03%.
R.M. Jacobson et al. / Vaccine 19 (2001) 2428–2433
7. Power, rare events, and background rates In the case of rotavirus vaccine, pre-licensure testing, however, did reveal some occurrences of intussusception. Investigators identified a total of six cases of intussusception among the 14 687 study participants in the pre-licensure phases of testing. Five of the six cases of intussusception occurred among the 10 054 recipients of the vaccine or 0.050%, with an exact 95% confidence interval ranging from 0.02 to 0.12%. One case occurred among the 4633 controls for a rate of 0.022%, with an exact 95% confidence interval ranging from 0.005 to 0.12%. A background rate of intussusception does occur among infants, and, in most cases of infantile intussusception, an etiology is not identified, unlike in adults. The rates found in these studies were similar to rates reported by others as the typical background incidence rate of intussusception among infants [8]. The confidence intervals for the two rates overlap, indicating that the difference between the two rates is not statistically significant. Performing Fisher’s exact test results in a two-tailed P value of 0.67, which is nowhere near the 0.05 threshold used to establish statistical significance. (Bonferroni corrections of the P value given the multiple independent safety outcomes examined would create a far more stringent threshold). One might argue that a two-tailed test is too severe, as we were looking for an adverse event (using the two-tailed test implies that the vaccine might have had either a deleterious effect or a beneficial one. One might support the use of two-tailed statistics arguing a priori that the vaccine might protect against intussusception by preventing rotavirus infection. In fact, experts did argue that. Nonetheless, a one-tailed test still does not provide an adequate solution. A one-tailed test only gives a P value of 0.09. What would it take to drive the P value below 0.05 given the control or background rate of one out of 4633? A doubling of cases would not do it. The twotailed Fischer exact would give a P value of 0.19. Tripling the rate of intussusception with 15 out of 10 054, and performing a one-tailed test succeeds with a 2 P value less than 0.03. The magic number for the one-tailed P value to cross 0.05 appears to be 13 cases out of 10 054. There were not 13 cases. There were not 10. There were five. What if more subjects had been studied prior to licensure? What if the number studied doubled from 10 054 recipients and 4633 controls to 20 108 recipients and 9266 controls? With 10 cases among recipients and two among controls, the two-tailed P value is 0.36. What if the number studied was tripled to 30 162 recipients and 13 899 controls? This would give us a 2 P-value of still only 0.17. What the number studied had been quadrupled to 40 216 recipients and 18 532 con-
2431
trols? This still would not have crossed the threshold of statistical significance. It gives a P value of only 0.12. One would not obtain a P value less than 0.05 until one had studied not 6 but almost seven times the number of subjects or approximately 100 000 subjects altogether! And even this sample size would only provide approximately 50% power to detect the observed difference. The Table 1 illustrates the power to detect differences in intussusception rates of the magnitude observed in pre-licensure. Not until there were 251 234 subjects would there be 90% power to statistically distinguish between rates of 0.050 and 0.022%. Indeed it was not just the number of cases of intussusception post-licensure that caught the attention of the epidemiologists but the timing of the events and specifically the number of cases of intussusception occurring in the first week following vaccination [3]. We must also concern ourselves with the problem of delayed adverse events such as atypical measles, which develop years after the receipt of the killed measles vaccine. The average length of time between vaccination and event in the initial reports were 5–6 years [12]. This time span greatly exceeds the typical length of Phase III studies.
8. Possible solutions The numbers of subjects studied in the pre-licensure testing for the rotavirus vaccine are very similar to the numbers of other recently licensed vaccines. Merck’s Hepatitis A Vaccine (VAQTA®) included 9181 recipients in its phase II/III studies.(Package insert) Lederle’s Pneumococcal Conjugate Vaccine (Prevnar™) included 18 906 recipients in its phase II/III studies (Package Table 1 Power to detect a difference as small as the pre-licensure rates of intussusception achieved with increasing sample sizes by multiples of the original sample size (10 054 and 4633) (h =0.05) Multiple of pre-licensure sample size 1 2 3 4 5 6 7 8 9 10 11 12 12.8 13 17.1
Power (%) 12 20 27 35 42 49 55 60 65 70 74 78 80 81 90
2432
R.M. Jacobson et al. / Vaccine 19 (2001) 2428–2433
insert). We must accept the fact that rare events are not likely to be discovered in pre-licensure testing. Therefore, we must rely upon post-licensure vigilance to detect real but rare adverse events. Solutions lay not in increasing pre-licensure numbers but in sustaining and building post-licensure surveillance systems and protocols. Specific examples of post-licensure surveillances include the US Vaccine Adverse Event Reporting System, commonly referred to its acronym VAERS [29]. VAERS is a passive surveillance system operated by the FDA and the CDC. Vaccine manufacturers are required to report to VAERS any adverse event reported to them, and health-care providers are encouraged to report any adverse event possibly attributable to vaccine. Vaccine recipients and their families also can report adverse events to VAERS. VAERS played a pivotal role in identifying the problem of intussusception with RotaShield®. Its great number of reports and national coverage makes VAERS useful for post-licensure surveillance of rare adverse events that are unlikely to be identified through clinical trials or phase IV studies carried out by manufacturers after licensure [27]. A second example is the Vaccine Safety Datalink Project [30 –32]. This is a partnership of the US CDC with four US health maintenance organizations. This project involves the creation of organization-wide, computerized vaccination databases that permit studies of large numbers of vaccine recipients and avoid the limitations of passive surveillance that VAERS suffers. Testing known rare adverse events in the studied populations has validated this method [30,31]. Farrington and colleagues in the UK have developed similar methods for active post-marketing surveillance [33]. They linked vaccination records with computerized hospital admission records and have validated their methodology through the examination of the earlier established links between DTP and febrile convulsion and MMR and thrombocytopenic purpura. Re-analysis of original vaccine trial data permits the creation of retrospective exposure cohorts that can be linked to historical controls for a given hypothesis [34]. This was the method used to study and dismiss the claim that Haemophilus influenzae type b vaccines were causally linked with diabetes mellitus [35]. An additional source of re-analysis resides in the powerful tool of meta-analysis and in particular in the Cochrane Library, which provides a systematic review of the known effects of vaccines [34,36]. We conclude with two cautionary notes. First, no vaccine is perfectly safe, and as we move from pre-licensure testing of thousands of subjects to post-licensure routine use in millions of children, we will necessarily see the appearance and increased occurrence of a number of true vaccine reactions and adverse
events. Chen et al. report that the VAERS in the US now receives approximately 10 000 reports each year, a number, which exceeds the reported incidence of most vaccine-preventable childhood diseases combined [29]. Second, valiant efforts to achieve universal routine vaccination of children have incited a vigorous antivaccine movement [2]. This movement has repeatedly seized upon spurious links between various vaccines and adverse outcomes to bolster its arguments against routine childhood vaccination. These spurious links have included, among others, sudden infant death syndrome and DTP, multiple sclerosis and hepatitis B, autism and measles vaccine, and diabetes mellitus and early infant vaccination. These claimed associations have resulted in the public’s loss of confidence and refusal of vaccination, and, while these claimed associations were eventually repudiated from a lack of confirmation through further study, they continue to create doubt and suspicion. Due to the very real possibility of rare but true adverse effects, vaccine researchers cannot routinely dismiss claims of adverse events but must work diligently to explore the possibility of association and etiology, identify true adverse effects, even when rare or latent, and expose false claims for what they are. References [1] CDC. Withdrawal of Rotavirus Vaccine Recommendation. MMWR, 1999; 48:1007. [2] Poland GA, Jacobson RM. Vaccine safety: injecting a dose of common sense [editorial]. Mayo Clin Proc 2000;75:135–9. [3] Rennels MB. The rotavirus vaccine story: a clinical investigator’s view. Pediatrics 2000;106:123– 5. [4] Jacobson RM. The current status of the rotavirus vaccine. Vaccine 1999;17:1690– 9. [5] CDC. Intussusception Among Recipients of Rotavirus Vaccine — United States, 1998– 1999. MMWR 1999; 48:577– 581. [6] Anonymous. ACIP reverses recommendation for rotavirus vaccine use: Wyeth Lederle voluntarily pulled RotaShield from the market because of reports of intussusception http:// www.slackinc.com/child/idc/199911/rota.asp. Slack Incorporated, 1999. [7] Harris G. After vaccine’s recall, regulators look for holes in the safety net. Wall Street Journal. New York, 1999: B1, B4. [8] Rennels M, Parashar U, Holman R, Le C, Chang H, Glass R. Lack of an apparent association between intussusception and wild or vaccine rotavirus infection. Pediatr Infect Dis J 1998;17:924– 5. [9] CDC. Rotavirus vaccine for the prevention of rotavirus gastroenteritis among children. Recommendations of the Advisory Committee on Immunization Practices (ACIP). MMWR. 1999; 48:1– 20. [10] Committee on Infectious Diseases. American Academy of Pediatrics. Prevention of rotavirus disease: guidelines for use of rotavirus vaccine. Pediatrics 1998; 102:1483– 1491. [11] Anonymous. ACIP reverses recommendation for rotavirus vaccine: Wyeth Lederle voluntarily pulled RotaShield from the market because of reports of intussusception. http:// www.slackinc.com/child/idc/199912/rota.asp. SLACK Incorporated, 1999.
R.M. Jacobson et al. / Vaccine 19 (2001) 2428–2433 [12] Fulginiti V, Eller J, Downie A, Kempe C. Altered reactivity to measles virus: a typical measles in children previously immunized with inactivated measles virus vaccines. J Am Med Assoc 1967;202:1075– 80. [13] Blumberg D, Morgan C, Lewis K, et al. An ongoing surveillance study of persistent crying and hypotonic-hyporesponsive episodes following routine DTP immunization: a preliminary report. Tokai J Exp Clin Med 1988;13(Suppl.):133–6. [14] Stratton K, Howe C, Johnston RJ. Adverse events associated with childhood vaccines other than pertussis and rubella. Summary of a report from the Institute of Medicine. J Am Med Assoc 1994;271:1602–5. [15] Spilker BA. The Drug Development and Approval Process. http://www.hcnor.com/rx – help/approval.html, 2000. [16] Gluck U, Gebbers JO, Gluck R. Phase 1 evaluation of intranasal virosomal influenza vaccine with and without Escherichia coli heat-labile toxin in adult volunteers. J Virol 1999;73:7780– 6. [17] Doherty JF, Pinder M, Tornieporth N, et al. A phase I safety and immunogenicity trial with the candidate malaria vaccine RTS,S/SBAS2 in semi-immune adults in the Gambia. Am J Trop Med Hyg 1999;61:865–8. [18] McCormack S, Tilzey A, Carmichael A, et al. A phase I trial in HIV negative healthy volunteers evaluating the effect of potent adjuvants on immunogenicity of a recombinant gp120(W61D) derived from dual tropic R5X4 HIV-1(ACH320). Vaccine 2000;18:1166– 77. [19] Uehling DT, Hopkins WJ, Balish E, Xing YN, Heisey DM. Vaginal mucosal immunization for recurrent urinary tract infection — Phase II clinical trial. J Urol 1997;157:2049–52. [20] Schedel I, Sutor GC, Hunsmann G, Jurkiewicz E. Phase II study of anti-CD4 idiotype vaccination in HIV positive volunteers. Vaccine 1999;17:1837–45. [21] Konadu EY, Lin FYC, Ho VA, et al. Phase 1 and phase 2 studies of Salmonella enterica serovar Paratyphi A O-specific polysaccharide-tetanus toroid conjugates in adults, teenagers, and 2 – 4 year old children in Vietnam, Infec. Immun. 1529; 68:1529– 1534. [22] Tacket, CO, Sztein, MB, Wasserman, SS, et al. Phase 2 clinical trial of attenuated Salmonella enterica serovar Typhi oral live vector vaccine CVD 908-htrA in US volunteers, Infec. Immun. 1196; 68:1196– 1201. [23] Francis DP, Gregory T, McElrath MJ, et al. Advancing aidsvax(Tm) to phase 3-safety, immunogenicity, and plans for phase 3. AIDS Res Hum Retrovir 1998;14:325S–31S.
.
2433
[24] Beck HP, Felger I, Huber W, et al. Analysis of multiple Plasmodium falciparum infections in Tanzanian children during the phase III trial of the malaria vaccine Spf66. J Infect Dis 1997;175:921– 6. [25] Lagos R, Levine OS, Avendano A, Horwitz I, Levine MM. The introduction of routine Haemophilus influenzae type b conjugate vaccine in Chile: a framework for evaluating new vaccines in newly industrializing countries. Pediatr Infect Dis J 1998;17:139S– 48S. [26] Ragni MV, Lusher JM, Koerper MA, Manco-Johnson M, Krause DS. Safety and immunogenicity of subcutaneous hepatitis. A vaccine in children with haemophilia. Haemophilia 2000;6:98– 103. [27] Braun MM, Ellenberg SS. Descriptive epidemiology of adverse events after immunization: reports to the Vaccine Adverse Event Reporting System (VAERS), 1991– 1994. J Pediatr 1997;131:529– 35. [28] Eyspasch E, Lefering R, Kum C, Troidl H. Probability of adverse events that have not yet occurred: a statistical reminder. Br Med J 1995;311:619– 20. [29] Chen RT, Rastogi SC, Mullen JR, et al. The Vaccine Adverse Event Reporting System (VAERS). Vaccine 1994;12:542–50. [30] Mullooly J, Drew L, DeStefano F, et al. Quality of HMO vaccination databases used to monitor childhood vaccine safety. Vaccine Safety DataLink Team. Am J Epidemiol 1999;149:186– 94. [31] Chen RT, Glasser JW, Rhodes PH, et al. Vaccine Safety Datalink project: a new tool for improving vaccine safety monitoring in the United States. The Vaccine Safety Datalink Team. Pediatrics 1997;99:765– 73. [32] Wassilak SG, Glasser JW, Chen RT, Hadler SC. Utility of large-linked databases in vaccine safety, particularly in distinguishing independent and synergistic effects. The Vaccine Safety Datalink Investigators. Ann New York Acad Sci 1995;754:377– 82. [33] Farrington P, Pugh S, Colville A, et al. A new method for active surveillance of adverse events from diphtheria/tetanus/pertussis and measles/mumps/rubella vaccines. Lancet 1995;345:567–9. [34] Jefferson T. Vaccination and its adverse effects: real or perceived. Society should think about means of linking exposure to potential long term effect [editorial]. Br Med J 1998;317:159–60. [35] Karvonen M, Cepaitis Z, Tuomilehto J. Association between type 1 diabetes and Haemophilus influenzae type b vaccination: birth cohort study. Br Med J 1999;318:1169– 72. [36] Jacobson RM. Promises and pitfalls of meta-analysis in vaccine research. Vaccine 1999;17:1628– 34.