ARTICLE IN PRESS
Controlled Clinical Trials 24 (2003) 99–104
Unique roles of a data and safety monitoring board in vaccine safety trials with compressed timelines and urgent implications David Scheifele, M.D.a,*, Bernard Duval, M.D.b, Gaston De Serres, M.D.b, Danuta M. Skowronski, M.D.a a
Canadian Association for Immunization Research and Evaluation (CAIRE), Vancouver, British Columbia, Canada b Canadian Association for Immunization Research and Evaluation (CAIRE), Quebec City, Quebec, Canada Manuscript received June 27, 2002; manuscript accepted September 12, 2002
Abstract Studies with urgent implications for public policy and programs represent a special case for monitoring boards, whose responsibilities must include ensuring applicability to policy making and timely communication of results as well as data and safety monitoring. We recently conducted two preseason evaluations of an influenza vaccine, results of which would influence the supply of vaccine for the Canadian annual program. Only 12 weeks were available for planning the studies and 6 weeks for fieldwork and data analysis. The enlisted board played key roles in determining appropriate study design, setting and enforcing rules for study initiation and early termination, monitoring safety, and encouraging timely dissemination of results. An extended role in data monitoring would have been desirable. Providing a safety valve for inevitable tensions between investigators and sponsor was an understated but significant role. Lessons from our experience will be particularly pertinent to future studies of pandemic influenza vaccines. © 2003 Elsevier Science Inc. All rights reserved. Keywords: Influenza vaccine; Randomized controlled trials; Research personnel; Immunization
Introduction Compared with other clinical trials those with a rushed timeline and high-stakes results require different types of help from a data and safety monitoring board (DSMB), particularly
* Corresponding author: Dr. David Scheifele, L427, BC’s Children’s Hospital, 4500 Oak Street, Vancouver, British Columbia, Canada, V6H 3N1. Tel.: 1-604-875-2422; Fax: 1-604-875-2635. E-mail address:
[email protected] 0197-2456/03/$—see front matter © 2003 Elsevier Science Inc. All rights reserved. PII: S0 1 9 7 - 2 4 5 6 ( 0 2 ) 0 0 2 6 9 -6
ARTICLE IN PRESS 100
D. Scheifele et al./Controlled Clinical Trials 24 (2003) 99–104
for calm objectivity. We were recently in such a situation, involving the preseason evaluation of an influenza vaccine, when only a few months were available for planning and fieldwork and with Canada’s vaccine supply influenced by the results. Our DSMB was of great assistance at several points during the study, as they served both conventional and unconventional supporting roles. Background Each autumn millions of Canadians receive inactivated influenza vaccine to reduce their risk of disease complications and death. The vaccine is reformulated annually to match anticipated changes in circulating strains. In North America the new formulations are not usually tested in humans before being approved for sale, on the assumption that safety and immunogenicity are consistent from year to year. In the autumn of 2000 a trivalent, inactivated, split formulation influenza vaccine made by a Canadian company was associated with unusual ocular and respiratory adverse effects. Retrospective surveys indicated that 2–3% of recipients developed red eyes, respiratory symptoms, and/or facial swelling during the 24 hours after vaccination [1]. Over 950 reports consistent with this new “oculorespiratory syndrome” (ORS) were received by Health Canada [2]. Many patients complained of throat tightness and difficulty swallowing and some had significant respiratory distress. About 25% of reported cases sought medical attention. Investigations revealed that the vaccine contained many aggregated unsplit virus particles [2]. No similar particles were evident in previous formulations or in current products of another company that were not associated with ORS, suggesting that viral aggregates triggered ORS [3]. For the following year the virus splitting process was improved. Before the reformulated vaccine was approved for sale the Canadian vaccines regulatory agency required controlled data regarding the ORS occurrence risk in a general adult population and the recurrence risk for persons who had ORS the previous year. The company contracted with the Canadian Association for Immunization Research and Evaluation (CAIRE), an academic research organization, to plan, conduct, analyze, and report the two studies within 4 months. The logistics were challenging because a preseason evaluation of an influenza vaccine had not been required in Canada or the United States for decades and has a window of only 4–6 weeks between first vaccine availability and the normal start of delivery programs. The situation for the sponsor was intense because the lengthy preparation time for this vaccine meant that full production would have to be completed before the results of the trials (and marketability of the new product) were known. Study design and conduct Both studies were designed as multicenter, double-blind, randomized crossover trials in which subjects received influenza vaccine and saline placebo about 7 days apart, in random order. Safety assessments were carried out by structured telephone interviews conducted 1 and 6 days after each injection. The principal outcome measure was the vaccine-attributable risk of ORS. The general study was designed to detect a minimum 2% absolute difference in the risk of ORS following vaccine versus placebo with 80% power and an alpha type error of 5%, requiring enrollment of 600 subjects. The particular age group targeted for enrollment
ARTICLE IN PRESS D. Scheifele et al./Controlled Clinical Trials 24 (2003) 99–104
101
had an estimated 15% risk of ORS with the previous year’s vaccine. The recurrence risk study was similarly powered to detect an absolute ORS risk difference of 5% after vaccine versus placebo, requiring enrollment of 150 subjects affected by ORS the previous year. In the context of a very short period of follow-up of a closed population, the ORS risk difference (vaccine risk minus placebo risk) was equated with the attributable risk of the vaccine [4]. Four sites participated enrolling subjects into both studies. To minimize risks to participants, the recurrence study design included a delayed start until preliminary data from at least 200 participants in the general study indicated a low risk of ORS associated with the new vaccine. In addition, a stopping rule was created to protect participants from a vaccineattributable ORS recurrence risk in excess of 10%. Enrollment, immunization, and follow-up were completed for both trials during September 2001. Preliminary analysis of ORS risks was completed in about 10 days and a report to the regulator submitted on October 9. The vaccine-attributable ORS risk in the general study was 2.8% (95% confidence interval [CI] 0.5, 5.1) [5] and in the recurrence study it was 24.8% (95% CI 7.0, 42.5) [6]. Approval for sale of the product was granted the next day, with a caution regarding the recurrence risk [2]. Had approval not been obtained by mid-October, large sales contracts would have been canceled and much of the company’s annual production of millions of doses of vaccine gone unsold. From a public health perspective, longer delays or nonapproval would have resulted in nationwide vaccine shortages, placing those who depend upon vaccine at significant risk. Roles of the DSMB Board composition The investigators selected the six board members with emphasis on ensuring that the study results would be useful for policy makers and the study safe for participants. To meet the unusual requirement to inform subsequent policy decisions, the board included three senior public health officials with responsibility for influenza immunization programs in provinces that previously experienced ORS cases. The board also included a vaccine safety specialist from the U.S. Centers for Disease Control and Prevention, an American expert on influenza vaccine, and an ethicist (a Canadian physician who experienced ORS in 2000). The American members were chosen to provide impartial expertise since the vaccine under study had not been used in the United States. Because of the highly compressed timeline for data analysis and reporting to the regulator, a role for the board in reviewing the fully assembled data and draft reports was considered impractical. Consequently a statistician was not included in the board membership. Role in trial planning The key initial contribution of the board to study planning was determination of the clinically acceptable risks of ORS occurrence or recurrence in public immunization programs, which in turn determined trial enrollment numbers. Members’ impressions of the risks of occurrence or recurrence acceptable to the general public and the anticipated impact of safety concerns on vaccination coverage levels were highly instructive in setting measurable outcome levels and decision rules.
ARTICLE IN PRESS 102
D. Scheifele et al./Controlled Clinical Trials 24 (2003) 99–104
Roles in trial conduct The next decision aided by the board involved determining when sufficient data from the general study had accumulated to judge that it was safe to start the recurrence risk study, given that such preliminary data posed a risk of type I error. Interim analysis was performed by the CAIRE study statistician 12 days after the start of the trial when about one third of the total number of injections and early assessments had been completed. At that point, data were available for 344 subjects, 11 of whom had ORS (seven after vaccine, four after placebo). Only the numerator data were unblinded. Assuming that half of the subjects were in each treatment group, the estimated vaccine-attributable ORS risk was 1.7% (95% CI 2, 5.5). The protocol had specified (with board approval) a rate below 2% as permissible, without further decision rules. Given that the observed risk was not statistically significant and that cases were mild, the investigators and sponsor agreed to start the second study and apprised the board of the data and decision, which members endorsed. The most critical decision was to interpret the stopping rule when the ORS recurrence risk exceeded 10% early in the second study. Sites advised the CAIRE study statistician daily of the number of subjects immunized and of individuals reporting ORS, whose treatment group he confidentially determined. He daily calculated the vaccine-attributable ORS risk using estimated denominators and advised the team when this approached 10%. When work in progress was assembled for review with the DSMB less than 10 days after the start of the study, 61 subjects had received dose one and 11 had ORS, ten (of 32) after vaccine and one (of 29) after placebo, for an estimated vaccine-attributable ORS recurrence risk of 27.8% (95% CI 10.4, 45.2). Although the decision criteria had not been fully defined, the board had little difficulty determining that the 10% stopping rule had been exceeded despite the limited number of vaccinations given. Since the decision to stop was a weighty one for the sponsor, the unanimous vote from the board to do so was helpful and reassuring to the investigators who were ill at ease to continue the study because ORS symptoms were of moderate severity in some subjects. At termination 65 subjects had been immunized so final results were slightly different. Role in conflict resolution We anticipated that the DSMB would also be a useful safety valve in the event of a disagreement between investigators and sponsor during the pressure-filled few weeks of fieldwork. This proved to be the case following early termination of the recurrence study. The sponsor demanded complete confidentiality according to contractual agreements until the results were reported to the regulator, whereas the investigators also wanted to apprise in confidence the National Advisory Committee on Immunization (NACI) so that a cautionary statement could be readied if the product was approved for sale. Since the public health members of the DSMB forcefully reflected the needs and expectations of provincial vaccine purchasers and policy makers, the sponsor accepted a compromise (see below). Deliberations, dilemmas, and influence of the board The three board members with responsibility for provincial vaccine programs had a difficult dilemma upon learning of the ORS recurrence risk. None of the board members had
ARTICLE IN PRESS D. Scheifele et al./Controlled Clinical Trials 24 (2003) 99–104
103
signed a confidentiality agreement but the expectation of confidentiality was clear to all. Faced with their own influenza vaccination programs starting up, they wanted planners to make rapid policy adjustments for persons previously affected by ORS. The sponsor’s reluctance to divulge the results prior to informing the regulator and an uncertain communication plan thereafter troubled the investigators, some of whom also had responsibilities to their provincial influenza vaccination programs. Recognizing the expectations of board members for rapid information sharing, the sponsor developed a communication plan that satisfied all of the parties. In quick succession the regulator received the study reports, NACI was given a summary of the studies to assist with preparation of a policy statement [2], and provincial program directors across the country were sent a fax with a synopsis of both studies, enabling them to begin to respond. Clearly the potential for conflicts of interest has to be carefully considered in selecting board members and in their agreeing to participate. The remedy reached here of resolving the conflict with a communication plan was superior to avoiding conflict by choosing persons unaffected by the situation as such advisors would have been less passionate about the need for rapid information sharing and perhaps less influential with the sponsor as a result. In retrospect we underestimated the importance of the board for ensuring appropriate dissemination of results and would be explicit about this role in future studies of this type. Dissolution of the board The DSMB was formally dissolved when the product was approved for sale. In retrospect this was premature as the study team still needed to prepare a final report to the regulator and manuscripts. During these tasks the sponsor unexpectedly limited access of coinvestigators to the databases, requiring them to work through the principal investigator and study statistician for any questions related to the data. Had the board remained active with a role in data oversight this issue might have been resolved more favorably or even avoided. Conclusion In summary, in carrying out this rushed pair of trials a DSMB was invaluable in providing practical advice during study planning, guarding the integrity of starting and stopping rules, and ensuring rapid communication of important findings to policy makers. In retrospect, a more formalized and extended role of the board in data oversight would have been useful. Similar circumstances may arise again in other trials with compressed timelines and urgent implications — in particular, this can be anticipated when vaccines against a pandemic strain of influenza must be evaluated. Those involved in long-range planning of responses to a pandemic should carefully consider the roles to be played by a DSMB in support of prelicensure vaccine trials, including sensitivity to informing urgent policy decisions. References [1] Boulianne N, De Serres G, Duval B, Shadmani R, Rochette L. Clinical manifestations and incidence of oculorespiratory syndrome following influenza vaccination — Quebec, 2000. Can Commun Dis Rep 2001;27:85–90. [2] National Advisory Committee on Immunization. Supplementary statement for the 2001–2002 season: influenza vaccination
ARTICLE IN PRESS 104
[3]
[4] [5]
[6]
D. Scheifele et al./Controlled Clinical Trials 24 (2003) 99–104
of persons who experienced oculorespiratory syndrome following previous influenza vaccination. Can Commun Dis Rep 2001;27:1–7 [Advisory Committee Statement-7]. National Advisory Committee on Immunization. Supplementary statement for the 2002–2003 influenza season: update on oculorespiratory syndrome in association with influenza vaccination. Can Commun Dis Rep 2002;28:1–8 [Advisory Committee Statement-6]. Greenland S, Robbins JM. Conceptual problems in the definition and interpretation of attributable fraction. Am J Epidemiol 1988;128:1185–1197. Scheifele DW, Duval B, Russell M, et al. Oculorespiratory syndrome (ORS) among healthy adults given a reformulated influenza vaccine previously linked with ORS (abstract). Fifth Annual Conference on Vaccine Research, Baltimore, Maryland. Abstract S43, 2002. Skowronski DM, De Serres G, Warrington R, et al. Oculo-respiratory syndrome following influenza vaccine: recurrence risk with re-formulated vaccine for 2001-2002 (abstract). Fifth Annual Conference on Vaccine Research, Baltimore, Maryland. Abstract S42, 2002.