- Email: [email protected]
Data and Safety Monitoring in the Delta Trial
Data and Safety Monitoring in the Delta Trial Peter Armitage, PhD, on behalf of the Delta Data and Safety Monitoring Committee St. Peter’s College, Oxford, United Kingdom.
ABSTRACT: Delta was a randomized trial designed to test whether combinations of zidovudine (AZT) with didanosine (ddI) or zalcitabine (ddC) were more effective than AZT alone for HIV-infected persons. A Data and Safety Monitoring Committee (DSMC) monitored accumulating data. This paper describes the deliberations and recommendations of the DSMC, as clear-cut differences in efficacy gradually emerged during the later stages of the trial. Control Clin Trials 1999;20:229–241 Elsevier Science Inc., 1999 KEY WORDS: AIDS trials, HIV infection, data and safety monitoring, interim analyses, survival data
INTRODUCTION Delta was a double-blind randomized trial designed to compare the efficacy, in delaying death or the progression of disease, of three regimens: (1) zidovudine (AZT) alone, (2) AZT in combination with didanosine (ddI), or (3) AZT in combination with zalcitabine (ddC), for HIV-infected persons. Between March 1992 and May 1994, 3207 participants were allocated to AZT alone (1055), AZT 1 ddI (1080) or AZT 1 ddC (1072). The blinded phase of the trial ended in September 1995. Participants, who came from France, the UK, Ireland, Australia, New Zealand, Italy, the Netherlands, Germany, and Switzerland, had either symptoms of HIV disease or CD4 counts less than 350 3 106/L. The protocol distinguished two “subtrials”: Delta 1 for those who had not previously received AZT; and Delta 2 for those who had taken it for at least 3 months. The two subtrials were to be analysed both separately and together. Delta summarized the results as follows: “Initiation of treatments with combinations of AZT plus ddI or ddC prolongs life and delays disease progression compared with AZT alone. The addition of ddI to participants already treated with AZT also improves survival, although the benefit appears less” [1]. An independent Data and Safety Monitoring Committee (DSMC) monitored the accumulating data from Delta at frequent intervals. This article describes the procedures followed by the DSMC and summarizes the data available at interim stages as well as the decisions made by the DSMC. Members of the Delta Data and Safety Monitoring Committee are listed in the Appendix. Address reprint requests to: Professor P. Armitage, MRC Clinical Trials Unit, UCLMS, The Mortimer Market Centre, Cappes Street, London WC1E 6AU, UK. Received July 22, 1998; accepted October 16, 1998. Controlled Clinical Trials 20:229–241 (1999) Elsevier Science Inc. 1999 655 Avenue of the Americas, New York, NY 10010
0197-2456/99/$–see front matter PII S0197-2456(99)00003-3
230
P. Armitage, et al.
THE STRUCTURE AND ROLE OF THE DSMC Delta proceeded under the aegis of an international Coordinating Committee, and in collaboration with Bristol-Myers Squibb, Roche, and Glaxo-Wellcome. (See Reference [1] for sources of funding.) Randomization was undertaken independently by national trials centers. Administration and data management occurred at national trials centers in the UK, France, Australia, the Netherlands, Italy, and Switzerland, while the UK center combined and analysed the data. The UK and French teams had already collaborated on the planning and execution of the earlier trials Concorde [2] and Alpha [3], the latter having also had Dutch, Australian, and Swiss participation, and together they formed the basis of Delta’s organization. Concorde and Alpha had had DSMCs with identical membership, which were now modified and enlarged to provide representation from other participant countries. The terms of reference of the DSMC given in the Delta protocol were essentially the same as in Concorde and Alpha, and I have reproduced them fully elsewhere [4]. The DSMC was to monitor the accumulating data on efficacy and safety, meeting at approximately 4- to 6-month intervals, and to consider relevant findings from other studies. A full interim analysis for Delta 1 was to be undertaken after 1500 person-years of observation, and for Delta 2 after 1000 person-years. The DSMC would make appropriate recommendations to the Coordinating Committee about possible termination of the trial or amendments to the protocol. In particular, the DSMC would advise the Coordinating Committee if, in their view, the randomized comparisons in the trial have provided both: (a) ‘proof beyond reasonable doubt’ that for all, or for some, types of patient one particular treatment is clearly indicated or clearly contra-indicated in terms of a net difference in mortality or clear evidence that any delay in progression from ARC to AIDS or from AIDS to severe AIDS outweighs any serious adverse effects of treatment or vice versa, and (b) evidence that might reasonably be expected to influence the patient management of many clinicians who are already aware of the results of any other studies.
The DSMC was thus, as in Concorde and Alpha, allowed to exercise judgment flexibly rather than following a rigid rule for possible termination. A footnote to the protocol suggested a version of what is often termed the “Haybittle–Peto” approach [5]: Appropriate criteria of proof beyond reasonable doubt cannot be specified precisely, but, for example, a difference of at least three standard deviations in an interim analysis of a major endpoint may be needed to justify halting, or modifying, such a study prematurely. If this criterion were to be adopted, it would have the practical advantage that the exact number of interim analyses would be of little importance, and so no fixed schedule is proposed.
The fact that Delta involved four treatment arms (if the two placebo groups were considered separately) and two subtrials raised issues of multiplicity that the DSMC needed to take into account, and was an additional reason for caution in drawing inferences about differences in outcome between different groups.
Data and Safety Monitoring in Delta
231
The DSMC noted two issues in particular. First, the two subtrials needed separate consideration because the treatments might have different relative effects for the two types of participants. On the other hand, if the distinction between Delta 1 and Delta 2 turned out to be unimportant, useful information about the relative merits of the treatments might be gained if data from the two subtrials were pooled. Secondly, the ddI and ddC placebo groups (receiving AZT alone) had to be separated because the different formulations used for the two groups might conceivably affect the outcome. Again, if this proved not to be the case, the placebo groups could be pooled, thus constituting a control group of the same size as each of the active drug groups. The DSMC meetings normally began with short open sessions attended by one or more Principal Investigator and/or the Chairman of the Coordinating Committee. The DSMC did not discuss data on safety or efficacy during these sessions. The closed sessions were attended by two statisticians, from the UK and France, who provided summaries and analyses of the data, and by a secretary. As in Concorde and Alpha, the DSMC maintained strict confidentiality. The documents prepared by the trial statisticians were returned at the end of each meeting. The statisticians and secretary also attended some meetings of the Coordinating Committee, but they maintained complete confidentiality. At the termination of each meeting the DSMC drafted a brief report to the Coordinating Committee. It always included a clear statement as to whether or not the DSMC recommended termination or any change to the protocol, and occasionally referred to other aspects of the trial organization.
THE MONITORING PROCESS The DSMC met eight times. Table 1 shows the dates and locations of these meetings and summarizes the data on patient accrual presented at each. The table and text use the following abbreviations for the four treatment arms: ddIP (AZT 1 ddI placebo); ddCP (AZT 1 ddC placebo); ddI (AZT 1 ddI); and ddC (AZT 1 ddC).
Meeting 1 (June 30, 1992) At Meeting 1, held a few months after the trial had started, no data were presented. The main purpose of the meeting was to enable the DSMC members to meet representatives of the Coordinating Committee and discuss with them the general aims of the trial. We knew that a similar trial, ACTG 175, was being conducted in the USA by the AIDS Clinical Trials Group (ACT 6), with participants somewhat less advanced and with more previous AZT use than in Delta. We agreed to explore the possible exchange of information, especially unblinded data on toxicity, between the Delta DSMC and the Data and Safety Monitoring Board (DSMB) of ACTG 175. Subsequently, we agreed that exchanges should normally share numerical information on efficacy only when issues requiring decisions arose on either side.
June 30, 1992 February 11, 1993 June 28, 1993 November 26, 1993 July 13, 1994 December 20, 1994 May 10, 1995 August 21, 1995
Date P P L P L P L L
Place — January 15, 1993 May 15, 1993 October 31, 1993 April 11, 1994 September 30, 1994 January 31, 1995 May 31, 1995 September 25, 1995
Data freeze
Delta 2
Delta 1
Delta 2
Person-years (to death or last date alive)
256 285 328 335 336 336
—
— 368b 281 314 362 367 367 367 700b
— 364 557 608 711 721 719 720 718
— 350 546 604 694 708 709 708 706 156 176 186 187 187 187
—
— 207b 141 156 166 167 167 168 355b
— 217 305 334 355 360 361 362 362
— 205 297 345 360 363 364 366 366 124 228 355 474 565 667
—
— — — — 123b 125 127 144 269 272 80 252 482 488 138 400 765 760 211 536 1034 1030 276 636 1235 1227 320 756 1474 1465 380
— 75b 75 123 192 251 290 340
— 75 161 267 415 531 616 749
b
— 72 156 264 416 544 627 756
ddIP ddCP ddI ddC ddIP ddCP ddI ddC ddIP ddCP ddI ddC ddIP ddCP ddI ddC
Delta 1
Persons starting treatment
Data on Accrual Available at Meetings of the Delta DSMC
Interim analysis. Combined placebo groups. L, London; P, Paris. ddIP: zidovudine (AZT) 1 didanosine (ddI) placebo. ddCP: AZT 1 zalcitabine (ddC) placebo. ddI: AZT 1 ddI. ddC: AZT 1 ddC.
a
1 2 3 4 5a 6 7 8 Ref[1]
Meeting
Table 1
232 P. Armitage, et al.
Data and Safety Monitoring in Delta
233
Meeting 2 (February 11, 1993) At the preliminary meeting with Coordinating Committee representatives, the DSMC learned that recruitment was satisfactory and that targets were likely to be reached within a few months. There were differences between countries in the acceptability of the placebo groups (AZT only) in Delta 2, which was lower in countries where ddI was already licensed. There had been some discussion in the Coordinating Committee as to whether the protocol should be modified to introduce an additional clinical endpoint based on a substantial fall in CD4 counts. The DSMC members expressed their opposition to this proposal, recalling the disparity shown in Concorde between treatment effects on CD4 counts and on progressions and deaths. In the closed meeting, the DSMC received information on progressions, deaths, and serious adverse events, as summarized in Table 2. The numbers of these events were still too few to indicate any clear differences between treatment groups. The data indicated, however, that withdrawals from assigned treatment were highest in the ddI group (Delta 1: ddI, 29; ddC, 6; combined placebo groups, 17. Delta 2: ddI, 18; ddC, 11; placebo, 14), and, in at least some countries, higher in the ddIP placebo group than in ddIC, suggesting unpalatability of the ddI buffer. The DSMC drew the Coordinating Committee’s attention to the points about the unsuitability of the CD4 count as an endpoint and the potentially poor acceptability of the ddI buffer. The latter point suggested that noncompliance might be an increasingly serious problem as the trial progressed. The protocol referred to the use of serum levels of ddI, ddC, and AZT for measurement of compliance, and the DSMC asked the Coordinating Committee for further information about the plans for implementing this provision. Otherwise it found no ground for proposing a change in either the protocol or the conduct of the trial. Meeting 3 (June 28, 1993) The DSMC again had the benefit of preliminary discussions with Coordinating Committee representatives. In response to the concern expressed at Meeting 2 about the unpalatability of ddI and/or its buffer, the Coordinating Committee intended to introduce a new formulation (which was in fact implemented in August 1994). Delta would shortly start assessing compliance from serum samples. The Coordinating Committee representatives asked the DSMC to reconsider the sample size calculations. The protocol targets of 1500 participants in Delta 1 and 1000 in Delta 2 had recently been increased to 1740 and 1020, respectively, to allow for the possibility that the two placebo groups might have to be kept separate in the analyses. The calculations had not, however, allowed for noncompliance, which we now saw as potentially important. Moreover, the requirement for high power against a 50% decrease in the progression rate now appeared unrealistic, in view of recently announced results from ACTG 155, a trial of AZT alone and in combination with ddC, which was later published [6]. The DSMC thought it prudent to assume 80% compliance (i.e., adherence to randomized treatment before an efficacy endpoint), and to require
Delta 2
Delta 1
Deaths Delta 2
Delta 1
Delta 2
Persons with serious adverse effects
11 25 50 78 87 116
—
188
270c 231
— — 5 6 26 19 48 38 74 77 106 113 121 145 116 197
— 14c 12 26 49 71 92 129 17 25 38 58 72 80
—
178c
— 15c 16 22 47 60 69 78 165
175
— — 14 14 34 34 53 59 77 96 107 124 127 137 146 162 4 10 16 31 43 60
—
149c
— 3c 2 8 12 23 42 56 93
— 2 8 13 23 34 51 69 107
— 2 6 12 30 40 56 82 7 13 19 28 41 47
—
126c
— 3c 3 6 16 30 39 47 103
— 1 11 18 37 56 71 84 121
— 2 7 15 37 59 76 96
— 26c 27 34 62 80 105 112 136
176
Delta 1 & 2:
21 28 51 64 71 74
—
289
— 37 55 71 121 142 169 179
368
— 23 56 70 155 183 209 222
13 24 38 40 45 47
—
— 21c 23 30 39 51 52 53
— 14 30 50 73 84 87 91
— 21 38 64 87 105 117 124
ddIP ddCP ddI ddC ddIP ddCP ddI ddC ddIP ddCP ddI ddC ddIP ddCP ddI ddC ddIP ddCP ddI ddC ddIP ddCP ddI ddC
Delta 1
Progressionsa
Clinical Endpoints and Serious Adverse Effects
b
Meetings 2–4, 6, 7: Progressions to severe ARC, AIDS, severe AIDS, or death. Meetings 5, 8 and Ref[1]: Progressions to new AIDS-defining event or death. Interim analysis. c Combined placebo groups.
a
Ref[1]
1 2 3 4 5b 6 7 8
Meeting
Table 2
234 P. Armitage, et al.
235
Data and Safety Monitoring in Delta
Table 3 Withdrawal by Patient Request from Delta by Group Group
ddIP
ddCP
ddI
ddC
Delta 1 Delta 2
23 25
12 14
48 32
29 15
high power against a 33% decrease in progression rates. These changes would lead to an increase of about 1000 in the total number of participants. (The Coordinating Committee subequently agreed to increase the intake to 2250 in Delta 1 and 1200 in Delta 2, a total increase of 690.) The Coordinating Committee representatives also reported some unease among certain participating physicians about the use of AZT alone in the two control groups, in view of the preliminary publication of results from Concorde. The DSMC took the view that continued use of AZT alone for this group of trial participants was not unethical. In its closed session, the DSMC reviewed the data on efficacy and safety summarized in Table 2, which displays the composite numbers of progressions to severe ARC, AIDS, severe AIDS, or death. In fact, the protocol defined the primary endpoints as (1) death in all patients, and (2) progression to AIDS or death in patients without AIDS at entry, while secondary endpoints included progression to severe AIDS or death in patients with AIDS at entry. The different categories of progression were shown separately in the tabulations presented to the DSMC, but, for simplicity, I do not distinguish them here. No striking differences appeared at this stage between treatment groups in either subtrial. The numbers of withdrawals from assigned treatment in the ddI and ddIP groups, after a mean follow-up of only about 6 months, again caused concern. In particular, the numbers withdrawn from assigned treatment at the patient’s request are given in Table 3. The DSMC report to the Coordinating Committee noted again the problem of noncompliance in the ddI and ddIP groups, but otherwise recommended no change on grounds of efficacy or safety. Meeting 4 (November 26, 1993) At the preliminary meeting, the Coordinating Committee representatives and the DSMC explored further some of the issues raised earlier about sample size and withdrawals from assigned treatment. In particular, the DSMC learned that the investigators would study more fully the reasons for withdrawal for what appeared to be minor side effects. The Coordinating Committee representatives suggested that the previously arranged link with the DSMB of ACTG 175 should be extended to cover another trial monitored by the same DSMB: the NUCOMBO study (CPCRA 007), conducted by the Community Programs for Clinical Research on AIDS. This trial was similar to Delta, comparing AZT with either ddI or ddC in patients with low CD4 counts or AIDS-defining opportunistic infections. In the closed session, the DSMC agreed to liaise with the DSMB for CPCRA 007, and, exceptionally, to release information about progressions and deaths in Delta. Because there was an early indication in CPCRA 007 of increased rates of progression and death in ddIP, the DSMC felt uneasy about the ddI
236
P. Armitage, et al.
buffer. The data currently available for Delta (see Table 2) showed very little evidence of differences in progression or death rates, the main outlier, however, being a somewhat higher rate of progressions and deaths on ddIP in each subtrial. The DSMC recommended continuation of the trial, but agreed to prepare a document based on results from both Delta and CPCRA 007. The DSMC prepared a report on January 21, 1994 and communicated it to the CPCRA 007 DSMB. The report concluded that: 1. For neither Delta 1 nor Delta 2, nor for the two subtrials combined, was there conventionally significant heterogeneity in mortality. 2. There was no conventionally significant heterogeneity in progression for all three trials (Delta 1, Delta 2, and CPCRA 007) combined. 3. There was a moderately significant (p 5 0.02) difference in mortality between the four treatment arms in all three trials combined: 56 deaths on ddI, 49 on ddC, 41 on ddIP, and 20 on ddCP (the two latter groups being half the size of the others). However, these differences could well be attributed to chance in an interim analysis with multiple comparisons, especially because the main contrast was that between the two placebo arms. In fact, an analysis based on a later data-freeze for Delta (December 31, 1993) reduced the contrast, with 63 deaths on ddI, 68 on ddC, 41 on ddIP, and 22 on ddCP, yielding p 5 0.1.
Meeting 5 (July 13, 1994) At this meeting, the DSMC was scheduled to receive the full interim analysis required by the protocol, the specified number of person-years having been observed. In the preliminary discussion, the Coordinating Committee representatives emphasized that Delta had reached an important phase. Several of the aspects of the trial report caused concern within the Coordinating Committee. Patients were continuing to withdraw from assigned treatment before reaching endpoints, and some were expressing impatience that the new formulation of ddI had not yet appeared. Many participants had now experienced the minimum follow-up period of 2 years, and although the protocol allowed for continuation beyond that point, they tended to ask for changes of medication even though progressions had not occurred. More generally, some clinicians felt that the trial had evidently not detected any important treatment effects, and that it would now be better to move to another protocol. In the closed session, the DSMC reviewed the results of the interim analysis. This included tabulations similar to those examined at previous meetings, but with many additional analyses. The DSMC compared progression and survival rates by time-to-event analyses, separately for different subgroups of participants. Delta 1 showed no convincing differences in either progression or survival, for either symptomatic or asymptomatic patients. In Delta 2, patients with AIDS at entry had higher rates of progression to severe AIDS or death in the ddIP and ddC groups. The analyses presented at earlier meetings had used the definitions of endpoints given in the protocol. Unfortunately, the definitions of severe ARC and severe AIDS endpoints included a component based on the occurrence of low
Data and Safety Monitoring in Delta
237
CD4 counts, in addition to symptomatic changes. The CD4 criterion would be removed in view of the recent findings in Concorde that treatments might affect CD4 counts but not clinical endpoints. In the meantime, the present findings in Delta 2 might be caused, at least partially, by differential treatment effects on CD4 counts. For the interim analysis (and in the final publication), a progression would be defined as the occurrence of a new AIDS-defining event or death. The interim analysis permitted a more detailed examination of adverse effects than at previous meetings. In each subtrial, the overall rates of serious adverse events were similar in the four treatment arms, although differences were apparent for specific categories. In Delta 1, the rate of gastroenterologic signs and symptoms (mainly nausea and vomiting) was higher in the ddI group. In both subtrials, the incidence of peripheral neuropathy events was higher in the active ddC group than in the other groups (Delta 1: ddIP, 5; ddCP, 3; ddI, 5; ddC, 17. Delta 2: ddIP, 3; ddCP, 5; ddI, 5; ddC, 15). There were no notable differences between treatments in the numbers of patients with laboratory abnormalities. Tables of change in CD4 counts at 8-week intervals showed higher counts in the active than in the placebo groups, with no marked difference between the placebo groups. The DSMC supported a suggestion from the trial statisticians that future presentations should include time-to-event analyses, for all participants, for the first new AIDS-defining event or death. The DSMC again recommended to the Coordinating Committee that the trial should continue. The withdrawal rate continued to be a concern, but the DSMC remarked that most patients who withdrew at their own wish had nevertheless already taken the assigned treatment for a high proportion of the 2-year period. Withdrawals did not appear to be related to serious toxicity. In the light of the earlier discussion with Coordinating Committee representatives, the DSMC pointed out that Delta was the largest trial in the world evaluating combination therapy for HIV infection, particularly in previously untreated patients, and expressed its wish that all participants should be strongly encouraged to continue their vital contribution. Meeting 6 (December 20, 1994) In the preliminary meeting, the Coordinating Committee representatives reported continuing resistance among some patients to the extension of protocol treatment beyond 2 years. Recent research reports suggested that the combination of AZT with lamivudine (3TC) might be beneficial, at least in its effect on surrogate markers, and regulatory changes in France enabled some patients there to use that drug on a compassionate basis. The data presented in the closed session included a wide range of time-toevent analyses. These now revealed for the first time convincing evidence of differences between treatments in the incidence of progressions, but with little suggestion of an effect on survival. An analysis of time to a new AIDS-defining event or death for Delta 1 showed significant differences (p 5 0.003) in a logrank test between the three groups (ddI, ddC, and combined placebo). Patients on ddI had a Kaplan-Meier estimated 2-year progression-free survival of 0.790
238
P. Armitage, et al.
(standard error [SE] 0.019), those on ddC 0.763 (SE 0.021), and those on placebo 0.675 (SE 0.023). The main contrast was between the two combination groups, ddI and ddC (with a combined progression rate 11.4 per 100 person-years) and the combined placebo group (15.3 per 100 person-years). However, the primary endpoint—progression to AIDS or death in patients without AIDS at entry— showed differences between the three groups of only marginal significance (p 5 0.03), with estimated 2-year progression-free survival 0.879 (SE 0.015) for ddI, 0.854 (SE 0.016) for ddC, and 0.834 (SE 0.017) for the combined placebo group. Some subgroup analyses of progressions for Delta 2 showed modest significance, with lower rates for ddI, but the evidence was less convincing than for Delta 1. The DSMC held the view that the evidence was too fragmentary to justify a recommendation to the Coordinating Committee in favor of termination, and emphasized in its report the crucial importance of encouraging patients to remain on assigned treatment for at least 2 years. The DSMC welcomed an impending protocol amendment that would allow a change of randomized assignment after 2 years with new treatments to include AZT plus 3TC. This modification would provide an incentive for patients to remain on the originally assigned treatment for the full 2-year period. The decision to recommend continuation carried an implication that a statistically significant difference in one of a number of possible comparisons, without the emergence of a coherent picture across the whole trial, was insufficient to justify termination. The DSMC discussed this approach to decision making at some length at the meeting and in subsequent correspondence, and endorsed it at the next meeting. The approach accorded with item (b) of the terms of reference quoted earlier, stating that advice in favor of termination would require evidence likely to influence clinical practice. Although recruitment had now ended, further follow-up of patients in the trial might well clarify the extent and nature of the apparent benefit of combination therapy. Meeting 7 (May 10, 1995) This meeting followed a similar course to that of Meeting 6. In the preliminary meeting, the Coordinating Committee representatives reported continued resistance among some clinicians and patients to extended treatment periods, particularly in France, and premature withdrawal from assigned treatment continued to occur. The Coordinating Committee invited the DSMC to comment on the plan it had outlined for the analysis to be adopted in the final publication. The analyses presented in the closed session confirmed the trends shown at Meeting 6. A time-to-event analysis of survival in Delta 1 showed a significant advantage to the two combination groups by log-rank test (p 5 0.003), but the effect was apparent only after a 2-year follow-up, the Kaplan-Meier estimates of 2-year survival proportions being 0.927 (SE 0.011) on ddI, 0.915 (SE 0.012) on ddC, and 0.881 (SE 0.015) for the combined placebo group. Various analyses of progressions in Delta 1 confirmed the advantage, especially to ddI, apparent after about 1-year follow-up. The differences between groups in the primary endpoint of progressions to AIDS or death in patients without AIDS at entry had become more significant than at the last meeting: the numbers of progressions were ddI 94, ddC 118, combined placebo 139 (log-rank test between the
Data and Safety Monitoring in Delta
239
three groups, p 5 0.005). These differences had now become apparent after about 18 months follow-up, the estimates of 18-month survival proportions being 0.878 (SE 0.014) on ddI, 0.853 (SE 0.015) on ddC, and 0.823 (SE 0.017) for the combined placebo group. Significant effects in Delta 2 were again inconsistent across subgroups. The proportions of participants withdrawn from assigned treatment before an endpoint had been reached were 48% in Delta 1 and 56% in Delta 2. These figures were discouraging. On the other hand, participants had remained on scheduled treatment for 75% (Delta 1) and 64% (Delta 2) of the follow-up period. The incidence of serious adverse effects was not unduly high, the main feature again being the excess of peripheral neuropathy in the active ddC group. The main changes from the data presented at Meeting 6 were, for Delta 1, the emergence of a difference in survival and a confirmation of a difference in progression rate. However, termination on this account would be unlikely to benefit trial participants, as the effect appeared only after a prolonged period of treatment. The DSMC therefore renewed its recommendation for continuation of the trial. Meeting 8 (August 21, 1995 and September 14, 1995) The timing of this meeting was complicated by the fact that the results of ACTG 175 would become known within a few weeks of the scheduled DSMC meeting. The meeting was consequently adjourned, after preliminary discussions, and restarted on September 14, 1995. During the discussion on August 21, it was clear that previously noted trends had continued and strengthened. In Delta 1, the ddI and ddC arms each showed substantial improvement over AZT monotherapy in both survival and progression. The DSMC took the view that Delta 1 should now be discontinued. Delta 2 did not provide clear evidence of a benefit in favor of combination therapy, but there was an indication of a benefit in the ddI group. On practical grounds it appeared unlikely that Delta 2 could continue once Delta 1 had stopped, and the DSMC accordingly recommended termination of both subtrials. The DSMC formulated these views at the August 21 session, but deferred making a recommendation until it had the opportunity, at the September 14 session, to see the results from ACTG 175. These were largely consistent with those from Delta and confirmed the original position taken by the DSMC, which was now communicated to the Coordinating Committee. The DSMC also recommended maintenance of long-term follow-up of patients. DISCUSSION The data monitoring procedure for Delta differed in various respects from those for Concorde and Alpha [4], two other trials for treatments of HIV infection that overlapped with Delta in time and DSMC membership. The structure of Delta, with two subtrials and four treatment arms, was more complex than that of the other trials, and the clinical profile of its participants wider. It was consequently more difficult to specify clearly in advance the criteria for possible termination. When evidence of possible differences in efficacy started to appear during the second half of the monitoring, the DSMC
240
P. Armitage, et al.
used its discretion in waiting for clearer and more coherent evidence to emerge before recommending termination. When such a recommendation became inevitable, the trial investigators were in any case on the point of stopping. The outcomes of the three trials were very different. In Concorde, an early suggestion of an effect needed to be treated with caution and was eventually shown to be illusory. In Alpha, no difference in efficacy between the two dose levels of ddI was established. In Delta, clear evidence emerged of an advantage in favor of combination therapy, but only toward the end of the trial, and perhaps only after a prolonged period of drug administration. In none of the three trials was safety a major issue. Alpha and Delta demonstrated clearly some adverse effects, but they had been expected on the basis of previous experience and were easily managed. The schedule of meetings followed a similar pattern throughout all three trials. The regular preliminary meetings with Coordinating Committee representatives provided an extremely valuable resource for the DSMC. Their value was particularly apparent during Delta, when participants and clinicians became aware of potentially important alternative treatments, and their anxiety to explore new regimens affected compliance with the trial protocol. In such a rapidly developing field of therapy, this shifting background places a premium on short-term assessments of efficacy, causing problems in the measurement of long-term effects. In any trial with rapid accrual of patients, such as Alpha and Delta, interim presentations of accumulated data will inevitably include some that are erroneous due to inaccuracies in patient records that are corrected at a later stage. Table 1, for example, shows minor irregularities in the numbers of persons starting treatment, as reported at Meetings 6–8 and in the final report, after recruitment had stopped. In some circumstances, the comparisons between treatments may be biased because errors occur more frequently with some treatments than with others. We have no reason to believe that these lacunae were at all frequent in Delta, or that biases occurred. Nevertheless, the possibility of inaccuracies is an additional reason for hesitation in drawing conclusions from early and rapidly assembled data.
I am grateful to my colleagues on the DSMC, the Chairmen of the Coordinating Committee, the Principal Investigators, and Dr. A.G. Babiker, for substantial help in the drafting of this paper. I thank also the editor and a referee for constructive comments on the first draft. The DSMC members express their appreciation to the participants in the trial and to their physicians. We thank also the Coordinating Committee Chairmen and Principal Investigators for their invaluable briefing during the open sessions. We are particularly indebted to the trial statisticians, Drs. A.G. Babiker and G. Cheˆne, who presented tabulations and analyses of the data, often under pressure of time. We received valuable administrative and secretarial help from Roberta Withnall.
REFERENCES 1. Delta Coordinating Committee. Delta: a randomised double-blind controlled trial comparing combinations of zidovudine plus didanosine or zalcitabine with zidovudine alone in HIV-infected individuals. Lancet 1996;348:283–291. 2. Concorde Coordinating Committee. Concorde: MRC/ANRS randomised doubleblind controlled trial of immediate and deferred zidovudine in symptom-free HIV infection. Lancet 1994;343:871–881.
Data and Safety Monitoring in Delta
241
3. Alpha International Coordinating Committee. The Alpha trial: European/Australian randomised double-blind trial of two doses of didanosine in zidovudine-intolerant patients with symptomatic HIV disease. AIDS 1996;10:867–880. 4. Armitage, P. Data and Safety Monitoring in the Concorde and Alpha Trials. Control Clin Trials 1999;20:207–228. 5. Armitage P. Interim analysis in clinical trials. Stat Med 1991;10:925–937. 6. Fischl MA, Stanley K, Collier AC, et al. Combination and monotherapy with zidovudine and zalcitabine in patients with advanced HIV disease. Ann Intern Med 1995;122:24–32.
APPENDIX Data and Safety Monitoring Committee B. Bailey, A. Fagot-Largeault (Joint Chairpersons), P. Armitage, V. Beral, H. Lambert, J. Lellouch, J. Tijssen, B. Varet.
ABSTRACT: Delta was a randomized trial designed to test whether combinations of zidovudine (AZT) with didanosine (ddI) or zalcitabine (ddC) were more effective than AZT alone for HIV-infected persons. A Data and Safety Monitoring Committee (DSMC) monitored accumulating data. This paper describes the deliberations and recommendations of the DSMC, as clear-cut differences in efficacy gradually emerged during the later stages of the trial. Control Clin Trials 1999;20:229–241 Elsevier Science Inc., 1999 KEY WORDS: AIDS trials, HIV infection, data and safety monitoring, interim analyses, survival data
INTRODUCTION Delta was a double-blind randomized trial designed to compare the efficacy, in delaying death or the progression of disease, of three regimens: (1) zidovudine (AZT) alone, (2) AZT in combination with didanosine (ddI), or (3) AZT in combination with zalcitabine (ddC), for HIV-infected persons. Between March 1992 and May 1994, 3207 participants were allocated to AZT alone (1055), AZT 1 ddI (1080) or AZT 1 ddC (1072). The blinded phase of the trial ended in September 1995. Participants, who came from France, the UK, Ireland, Australia, New Zealand, Italy, the Netherlands, Germany, and Switzerland, had either symptoms of HIV disease or CD4 counts less than 350 3 106/L. The protocol distinguished two “subtrials”: Delta 1 for those who had not previously received AZT; and Delta 2 for those who had taken it for at least 3 months. The two subtrials were to be analysed both separately and together. Delta summarized the results as follows: “Initiation of treatments with combinations of AZT plus ddI or ddC prolongs life and delays disease progression compared with AZT alone. The addition of ddI to participants already treated with AZT also improves survival, although the benefit appears less” [1]. An independent Data and Safety Monitoring Committee (DSMC) monitored the accumulating data from Delta at frequent intervals. This article describes the procedures followed by the DSMC and summarizes the data available at interim stages as well as the decisions made by the DSMC. Members of the Delta Data and Safety Monitoring Committee are listed in the Appendix. Address reprint requests to: Professor P. Armitage, MRC Clinical Trials Unit, UCLMS, The Mortimer Market Centre, Cappes Street, London WC1E 6AU, UK. Received July 22, 1998; accepted October 16, 1998. Controlled Clinical Trials 20:229–241 (1999) Elsevier Science Inc. 1999 655 Avenue of the Americas, New York, NY 10010
0197-2456/99/$–see front matter PII S0197-2456(99)00003-3
230
P. Armitage, et al.
THE STRUCTURE AND ROLE OF THE DSMC Delta proceeded under the aegis of an international Coordinating Committee, and in collaboration with Bristol-Myers Squibb, Roche, and Glaxo-Wellcome. (See Reference [1] for sources of funding.) Randomization was undertaken independently by national trials centers. Administration and data management occurred at national trials centers in the UK, France, Australia, the Netherlands, Italy, and Switzerland, while the UK center combined and analysed the data. The UK and French teams had already collaborated on the planning and execution of the earlier trials Concorde [2] and Alpha [3], the latter having also had Dutch, Australian, and Swiss participation, and together they formed the basis of Delta’s organization. Concorde and Alpha had had DSMCs with identical membership, which were now modified and enlarged to provide representation from other participant countries. The terms of reference of the DSMC given in the Delta protocol were essentially the same as in Concorde and Alpha, and I have reproduced them fully elsewhere [4]. The DSMC was to monitor the accumulating data on efficacy and safety, meeting at approximately 4- to 6-month intervals, and to consider relevant findings from other studies. A full interim analysis for Delta 1 was to be undertaken after 1500 person-years of observation, and for Delta 2 after 1000 person-years. The DSMC would make appropriate recommendations to the Coordinating Committee about possible termination of the trial or amendments to the protocol. In particular, the DSMC would advise the Coordinating Committee if, in their view, the randomized comparisons in the trial have provided both: (a) ‘proof beyond reasonable doubt’ that for all, or for some, types of patient one particular treatment is clearly indicated or clearly contra-indicated in terms of a net difference in mortality or clear evidence that any delay in progression from ARC to AIDS or from AIDS to severe AIDS outweighs any serious adverse effects of treatment or vice versa, and (b) evidence that might reasonably be expected to influence the patient management of many clinicians who are already aware of the results of any other studies.
The DSMC was thus, as in Concorde and Alpha, allowed to exercise judgment flexibly rather than following a rigid rule for possible termination. A footnote to the protocol suggested a version of what is often termed the “Haybittle–Peto” approach [5]: Appropriate criteria of proof beyond reasonable doubt cannot be specified precisely, but, for example, a difference of at least three standard deviations in an interim analysis of a major endpoint may be needed to justify halting, or modifying, such a study prematurely. If this criterion were to be adopted, it would have the practical advantage that the exact number of interim analyses would be of little importance, and so no fixed schedule is proposed.
The fact that Delta involved four treatment arms (if the two placebo groups were considered separately) and two subtrials raised issues of multiplicity that the DSMC needed to take into account, and was an additional reason for caution in drawing inferences about differences in outcome between different groups.
Data and Safety Monitoring in Delta
231
The DSMC noted two issues in particular. First, the two subtrials needed separate consideration because the treatments might have different relative effects for the two types of participants. On the other hand, if the distinction between Delta 1 and Delta 2 turned out to be unimportant, useful information about the relative merits of the treatments might be gained if data from the two subtrials were pooled. Secondly, the ddI and ddC placebo groups (receiving AZT alone) had to be separated because the different formulations used for the two groups might conceivably affect the outcome. Again, if this proved not to be the case, the placebo groups could be pooled, thus constituting a control group of the same size as each of the active drug groups. The DSMC meetings normally began with short open sessions attended by one or more Principal Investigator and/or the Chairman of the Coordinating Committee. The DSMC did not discuss data on safety or efficacy during these sessions. The closed sessions were attended by two statisticians, from the UK and France, who provided summaries and analyses of the data, and by a secretary. As in Concorde and Alpha, the DSMC maintained strict confidentiality. The documents prepared by the trial statisticians were returned at the end of each meeting. The statisticians and secretary also attended some meetings of the Coordinating Committee, but they maintained complete confidentiality. At the termination of each meeting the DSMC drafted a brief report to the Coordinating Committee. It always included a clear statement as to whether or not the DSMC recommended termination or any change to the protocol, and occasionally referred to other aspects of the trial organization.
THE MONITORING PROCESS The DSMC met eight times. Table 1 shows the dates and locations of these meetings and summarizes the data on patient accrual presented at each. The table and text use the following abbreviations for the four treatment arms: ddIP (AZT 1 ddI placebo); ddCP (AZT 1 ddC placebo); ddI (AZT 1 ddI); and ddC (AZT 1 ddC).
Meeting 1 (June 30, 1992) At Meeting 1, held a few months after the trial had started, no data were presented. The main purpose of the meeting was to enable the DSMC members to meet representatives of the Coordinating Committee and discuss with them the general aims of the trial. We knew that a similar trial, ACTG 175, was being conducted in the USA by the AIDS Clinical Trials Group (ACT 6), with participants somewhat less advanced and with more previous AZT use than in Delta. We agreed to explore the possible exchange of information, especially unblinded data on toxicity, between the Delta DSMC and the Data and Safety Monitoring Board (DSMB) of ACTG 175. Subsequently, we agreed that exchanges should normally share numerical information on efficacy only when issues requiring decisions arose on either side.
June 30, 1992 February 11, 1993 June 28, 1993 November 26, 1993 July 13, 1994 December 20, 1994 May 10, 1995 August 21, 1995
Date P P L P L P L L
Place — January 15, 1993 May 15, 1993 October 31, 1993 April 11, 1994 September 30, 1994 January 31, 1995 May 31, 1995 September 25, 1995
Data freeze
Delta 2
Delta 1
Delta 2
Person-years (to death or last date alive)
256 285 328 335 336 336
—
— 368b 281 314 362 367 367 367 700b
— 364 557 608 711 721 719 720 718
— 350 546 604 694 708 709 708 706 156 176 186 187 187 187
—
— 207b 141 156 166 167 167 168 355b
— 217 305 334 355 360 361 362 362
— 205 297 345 360 363 364 366 366 124 228 355 474 565 667
—
— — — — 123b 125 127 144 269 272 80 252 482 488 138 400 765 760 211 536 1034 1030 276 636 1235 1227 320 756 1474 1465 380
— 75b 75 123 192 251 290 340
— 75 161 267 415 531 616 749
b
— 72 156 264 416 544 627 756
ddIP ddCP ddI ddC ddIP ddCP ddI ddC ddIP ddCP ddI ddC ddIP ddCP ddI ddC
Delta 1
Persons starting treatment
Data on Accrual Available at Meetings of the Delta DSMC
Interim analysis. Combined placebo groups. L, London; P, Paris. ddIP: zidovudine (AZT) 1 didanosine (ddI) placebo. ddCP: AZT 1 zalcitabine (ddC) placebo. ddI: AZT 1 ddI. ddC: AZT 1 ddC.
a
1 2 3 4 5a 6 7 8 Ref[1]
Meeting
Table 1
232 P. Armitage, et al.
Data and Safety Monitoring in Delta
233
Meeting 2 (February 11, 1993) At the preliminary meeting with Coordinating Committee representatives, the DSMC learned that recruitment was satisfactory and that targets were likely to be reached within a few months. There were differences between countries in the acceptability of the placebo groups (AZT only) in Delta 2, which was lower in countries where ddI was already licensed. There had been some discussion in the Coordinating Committee as to whether the protocol should be modified to introduce an additional clinical endpoint based on a substantial fall in CD4 counts. The DSMC members expressed their opposition to this proposal, recalling the disparity shown in Concorde between treatment effects on CD4 counts and on progressions and deaths. In the closed meeting, the DSMC received information on progressions, deaths, and serious adverse events, as summarized in Table 2. The numbers of these events were still too few to indicate any clear differences between treatment groups. The data indicated, however, that withdrawals from assigned treatment were highest in the ddI group (Delta 1: ddI, 29; ddC, 6; combined placebo groups, 17. Delta 2: ddI, 18; ddC, 11; placebo, 14), and, in at least some countries, higher in the ddIP placebo group than in ddIC, suggesting unpalatability of the ddI buffer. The DSMC drew the Coordinating Committee’s attention to the points about the unsuitability of the CD4 count as an endpoint and the potentially poor acceptability of the ddI buffer. The latter point suggested that noncompliance might be an increasingly serious problem as the trial progressed. The protocol referred to the use of serum levels of ddI, ddC, and AZT for measurement of compliance, and the DSMC asked the Coordinating Committee for further information about the plans for implementing this provision. Otherwise it found no ground for proposing a change in either the protocol or the conduct of the trial. Meeting 3 (June 28, 1993) The DSMC again had the benefit of preliminary discussions with Coordinating Committee representatives. In response to the concern expressed at Meeting 2 about the unpalatability of ddI and/or its buffer, the Coordinating Committee intended to introduce a new formulation (which was in fact implemented in August 1994). Delta would shortly start assessing compliance from serum samples. The Coordinating Committee representatives asked the DSMC to reconsider the sample size calculations. The protocol targets of 1500 participants in Delta 1 and 1000 in Delta 2 had recently been increased to 1740 and 1020, respectively, to allow for the possibility that the two placebo groups might have to be kept separate in the analyses. The calculations had not, however, allowed for noncompliance, which we now saw as potentially important. Moreover, the requirement for high power against a 50% decrease in the progression rate now appeared unrealistic, in view of recently announced results from ACTG 155, a trial of AZT alone and in combination with ddC, which was later published [6]. The DSMC thought it prudent to assume 80% compliance (i.e., adherence to randomized treatment before an efficacy endpoint), and to require
Delta 2
Delta 1
Deaths Delta 2
Delta 1
Delta 2
Persons with serious adverse effects
11 25 50 78 87 116
—
188
270c 231
— — 5 6 26 19 48 38 74 77 106 113 121 145 116 197
— 14c 12 26 49 71 92 129 17 25 38 58 72 80
—
178c
— 15c 16 22 47 60 69 78 165
175
— — 14 14 34 34 53 59 77 96 107 124 127 137 146 162 4 10 16 31 43 60
—
149c
— 3c 2 8 12 23 42 56 93
— 2 8 13 23 34 51 69 107
— 2 6 12 30 40 56 82 7 13 19 28 41 47
—
126c
— 3c 3 6 16 30 39 47 103
— 1 11 18 37 56 71 84 121
— 2 7 15 37 59 76 96
— 26c 27 34 62 80 105 112 136
176
Delta 1 & 2:
21 28 51 64 71 74
—
289
— 37 55 71 121 142 169 179
368
— 23 56 70 155 183 209 222
13 24 38 40 45 47
—
— 21c 23 30 39 51 52 53
— 14 30 50 73 84 87 91
— 21 38 64 87 105 117 124
ddIP ddCP ddI ddC ddIP ddCP ddI ddC ddIP ddCP ddI ddC ddIP ddCP ddI ddC ddIP ddCP ddI ddC ddIP ddCP ddI ddC
Delta 1
Progressionsa
Clinical Endpoints and Serious Adverse Effects
b
Meetings 2–4, 6, 7: Progressions to severe ARC, AIDS, severe AIDS, or death. Meetings 5, 8 and Ref[1]: Progressions to new AIDS-defining event or death. Interim analysis. c Combined placebo groups.
a
Ref[1]
1 2 3 4 5b 6 7 8
Meeting
Table 2
234 P. Armitage, et al.
235
Data and Safety Monitoring in Delta
Table 3 Withdrawal by Patient Request from Delta by Group Group
ddIP
ddCP
ddI
ddC
Delta 1 Delta 2
23 25
12 14
48 32
29 15
high power against a 33% decrease in progression rates. These changes would lead to an increase of about 1000 in the total number of participants. (The Coordinating Committee subequently agreed to increase the intake to 2250 in Delta 1 and 1200 in Delta 2, a total increase of 690.) The Coordinating Committee representatives also reported some unease among certain participating physicians about the use of AZT alone in the two control groups, in view of the preliminary publication of results from Concorde. The DSMC took the view that continued use of AZT alone for this group of trial participants was not unethical. In its closed session, the DSMC reviewed the data on efficacy and safety summarized in Table 2, which displays the composite numbers of progressions to severe ARC, AIDS, severe AIDS, or death. In fact, the protocol defined the primary endpoints as (1) death in all patients, and (2) progression to AIDS or death in patients without AIDS at entry, while secondary endpoints included progression to severe AIDS or death in patients with AIDS at entry. The different categories of progression were shown separately in the tabulations presented to the DSMC, but, for simplicity, I do not distinguish them here. No striking differences appeared at this stage between treatment groups in either subtrial. The numbers of withdrawals from assigned treatment in the ddI and ddIP groups, after a mean follow-up of only about 6 months, again caused concern. In particular, the numbers withdrawn from assigned treatment at the patient’s request are given in Table 3. The DSMC report to the Coordinating Committee noted again the problem of noncompliance in the ddI and ddIP groups, but otherwise recommended no change on grounds of efficacy or safety. Meeting 4 (November 26, 1993) At the preliminary meeting, the Coordinating Committee representatives and the DSMC explored further some of the issues raised earlier about sample size and withdrawals from assigned treatment. In particular, the DSMC learned that the investigators would study more fully the reasons for withdrawal for what appeared to be minor side effects. The Coordinating Committee representatives suggested that the previously arranged link with the DSMB of ACTG 175 should be extended to cover another trial monitored by the same DSMB: the NUCOMBO study (CPCRA 007), conducted by the Community Programs for Clinical Research on AIDS. This trial was similar to Delta, comparing AZT with either ddI or ddC in patients with low CD4 counts or AIDS-defining opportunistic infections. In the closed session, the DSMC agreed to liaise with the DSMB for CPCRA 007, and, exceptionally, to release information about progressions and deaths in Delta. Because there was an early indication in CPCRA 007 of increased rates of progression and death in ddIP, the DSMC felt uneasy about the ddI
236
P. Armitage, et al.
buffer. The data currently available for Delta (see Table 2) showed very little evidence of differences in progression or death rates, the main outlier, however, being a somewhat higher rate of progressions and deaths on ddIP in each subtrial. The DSMC recommended continuation of the trial, but agreed to prepare a document based on results from both Delta and CPCRA 007. The DSMC prepared a report on January 21, 1994 and communicated it to the CPCRA 007 DSMB. The report concluded that: 1. For neither Delta 1 nor Delta 2, nor for the two subtrials combined, was there conventionally significant heterogeneity in mortality. 2. There was no conventionally significant heterogeneity in progression for all three trials (Delta 1, Delta 2, and CPCRA 007) combined. 3. There was a moderately significant (p 5 0.02) difference in mortality between the four treatment arms in all three trials combined: 56 deaths on ddI, 49 on ddC, 41 on ddIP, and 20 on ddCP (the two latter groups being half the size of the others). However, these differences could well be attributed to chance in an interim analysis with multiple comparisons, especially because the main contrast was that between the two placebo arms. In fact, an analysis based on a later data-freeze for Delta (December 31, 1993) reduced the contrast, with 63 deaths on ddI, 68 on ddC, 41 on ddIP, and 22 on ddCP, yielding p 5 0.1.
Meeting 5 (July 13, 1994) At this meeting, the DSMC was scheduled to receive the full interim analysis required by the protocol, the specified number of person-years having been observed. In the preliminary discussion, the Coordinating Committee representatives emphasized that Delta had reached an important phase. Several of the aspects of the trial report caused concern within the Coordinating Committee. Patients were continuing to withdraw from assigned treatment before reaching endpoints, and some were expressing impatience that the new formulation of ddI had not yet appeared. Many participants had now experienced the minimum follow-up period of 2 years, and although the protocol allowed for continuation beyond that point, they tended to ask for changes of medication even though progressions had not occurred. More generally, some clinicians felt that the trial had evidently not detected any important treatment effects, and that it would now be better to move to another protocol. In the closed session, the DSMC reviewed the results of the interim analysis. This included tabulations similar to those examined at previous meetings, but with many additional analyses. The DSMC compared progression and survival rates by time-to-event analyses, separately for different subgroups of participants. Delta 1 showed no convincing differences in either progression or survival, for either symptomatic or asymptomatic patients. In Delta 2, patients with AIDS at entry had higher rates of progression to severe AIDS or death in the ddIP and ddC groups. The analyses presented at earlier meetings had used the definitions of endpoints given in the protocol. Unfortunately, the definitions of severe ARC and severe AIDS endpoints included a component based on the occurrence of low
Data and Safety Monitoring in Delta
237
CD4 counts, in addition to symptomatic changes. The CD4 criterion would be removed in view of the recent findings in Concorde that treatments might affect CD4 counts but not clinical endpoints. In the meantime, the present findings in Delta 2 might be caused, at least partially, by differential treatment effects on CD4 counts. For the interim analysis (and in the final publication), a progression would be defined as the occurrence of a new AIDS-defining event or death. The interim analysis permitted a more detailed examination of adverse effects than at previous meetings. In each subtrial, the overall rates of serious adverse events were similar in the four treatment arms, although differences were apparent for specific categories. In Delta 1, the rate of gastroenterologic signs and symptoms (mainly nausea and vomiting) was higher in the ddI group. In both subtrials, the incidence of peripheral neuropathy events was higher in the active ddC group than in the other groups (Delta 1: ddIP, 5; ddCP, 3; ddI, 5; ddC, 17. Delta 2: ddIP, 3; ddCP, 5; ddI, 5; ddC, 15). There were no notable differences between treatments in the numbers of patients with laboratory abnormalities. Tables of change in CD4 counts at 8-week intervals showed higher counts in the active than in the placebo groups, with no marked difference between the placebo groups. The DSMC supported a suggestion from the trial statisticians that future presentations should include time-to-event analyses, for all participants, for the first new AIDS-defining event or death. The DSMC again recommended to the Coordinating Committee that the trial should continue. The withdrawal rate continued to be a concern, but the DSMC remarked that most patients who withdrew at their own wish had nevertheless already taken the assigned treatment for a high proportion of the 2-year period. Withdrawals did not appear to be related to serious toxicity. In the light of the earlier discussion with Coordinating Committee representatives, the DSMC pointed out that Delta was the largest trial in the world evaluating combination therapy for HIV infection, particularly in previously untreated patients, and expressed its wish that all participants should be strongly encouraged to continue their vital contribution. Meeting 6 (December 20, 1994) In the preliminary meeting, the Coordinating Committee representatives reported continuing resistance among some patients to the extension of protocol treatment beyond 2 years. Recent research reports suggested that the combination of AZT with lamivudine (3TC) might be beneficial, at least in its effect on surrogate markers, and regulatory changes in France enabled some patients there to use that drug on a compassionate basis. The data presented in the closed session included a wide range of time-toevent analyses. These now revealed for the first time convincing evidence of differences between treatments in the incidence of progressions, but with little suggestion of an effect on survival. An analysis of time to a new AIDS-defining event or death for Delta 1 showed significant differences (p 5 0.003) in a logrank test between the three groups (ddI, ddC, and combined placebo). Patients on ddI had a Kaplan-Meier estimated 2-year progression-free survival of 0.790
238
P. Armitage, et al.
(standard error [SE] 0.019), those on ddC 0.763 (SE 0.021), and those on placebo 0.675 (SE 0.023). The main contrast was between the two combination groups, ddI and ddC (with a combined progression rate 11.4 per 100 person-years) and the combined placebo group (15.3 per 100 person-years). However, the primary endpoint—progression to AIDS or death in patients without AIDS at entry— showed differences between the three groups of only marginal significance (p 5 0.03), with estimated 2-year progression-free survival 0.879 (SE 0.015) for ddI, 0.854 (SE 0.016) for ddC, and 0.834 (SE 0.017) for the combined placebo group. Some subgroup analyses of progressions for Delta 2 showed modest significance, with lower rates for ddI, but the evidence was less convincing than for Delta 1. The DSMC held the view that the evidence was too fragmentary to justify a recommendation to the Coordinating Committee in favor of termination, and emphasized in its report the crucial importance of encouraging patients to remain on assigned treatment for at least 2 years. The DSMC welcomed an impending protocol amendment that would allow a change of randomized assignment after 2 years with new treatments to include AZT plus 3TC. This modification would provide an incentive for patients to remain on the originally assigned treatment for the full 2-year period. The decision to recommend continuation carried an implication that a statistically significant difference in one of a number of possible comparisons, without the emergence of a coherent picture across the whole trial, was insufficient to justify termination. The DSMC discussed this approach to decision making at some length at the meeting and in subsequent correspondence, and endorsed it at the next meeting. The approach accorded with item (b) of the terms of reference quoted earlier, stating that advice in favor of termination would require evidence likely to influence clinical practice. Although recruitment had now ended, further follow-up of patients in the trial might well clarify the extent and nature of the apparent benefit of combination therapy. Meeting 7 (May 10, 1995) This meeting followed a similar course to that of Meeting 6. In the preliminary meeting, the Coordinating Committee representatives reported continued resistance among some clinicians and patients to extended treatment periods, particularly in France, and premature withdrawal from assigned treatment continued to occur. The Coordinating Committee invited the DSMC to comment on the plan it had outlined for the analysis to be adopted in the final publication. The analyses presented in the closed session confirmed the trends shown at Meeting 6. A time-to-event analysis of survival in Delta 1 showed a significant advantage to the two combination groups by log-rank test (p 5 0.003), but the effect was apparent only after a 2-year follow-up, the Kaplan-Meier estimates of 2-year survival proportions being 0.927 (SE 0.011) on ddI, 0.915 (SE 0.012) on ddC, and 0.881 (SE 0.015) for the combined placebo group. Various analyses of progressions in Delta 1 confirmed the advantage, especially to ddI, apparent after about 1-year follow-up. The differences between groups in the primary endpoint of progressions to AIDS or death in patients without AIDS at entry had become more significant than at the last meeting: the numbers of progressions were ddI 94, ddC 118, combined placebo 139 (log-rank test between the
Data and Safety Monitoring in Delta
239
three groups, p 5 0.005). These differences had now become apparent after about 18 months follow-up, the estimates of 18-month survival proportions being 0.878 (SE 0.014) on ddI, 0.853 (SE 0.015) on ddC, and 0.823 (SE 0.017) for the combined placebo group. Significant effects in Delta 2 were again inconsistent across subgroups. The proportions of participants withdrawn from assigned treatment before an endpoint had been reached were 48% in Delta 1 and 56% in Delta 2. These figures were discouraging. On the other hand, participants had remained on scheduled treatment for 75% (Delta 1) and 64% (Delta 2) of the follow-up period. The incidence of serious adverse effects was not unduly high, the main feature again being the excess of peripheral neuropathy in the active ddC group. The main changes from the data presented at Meeting 6 were, for Delta 1, the emergence of a difference in survival and a confirmation of a difference in progression rate. However, termination on this account would be unlikely to benefit trial participants, as the effect appeared only after a prolonged period of treatment. The DSMC therefore renewed its recommendation for continuation of the trial. Meeting 8 (August 21, 1995 and September 14, 1995) The timing of this meeting was complicated by the fact that the results of ACTG 175 would become known within a few weeks of the scheduled DSMC meeting. The meeting was consequently adjourned, after preliminary discussions, and restarted on September 14, 1995. During the discussion on August 21, it was clear that previously noted trends had continued and strengthened. In Delta 1, the ddI and ddC arms each showed substantial improvement over AZT monotherapy in both survival and progression. The DSMC took the view that Delta 1 should now be discontinued. Delta 2 did not provide clear evidence of a benefit in favor of combination therapy, but there was an indication of a benefit in the ddI group. On practical grounds it appeared unlikely that Delta 2 could continue once Delta 1 had stopped, and the DSMC accordingly recommended termination of both subtrials. The DSMC formulated these views at the August 21 session, but deferred making a recommendation until it had the opportunity, at the September 14 session, to see the results from ACTG 175. These were largely consistent with those from Delta and confirmed the original position taken by the DSMC, which was now communicated to the Coordinating Committee. The DSMC also recommended maintenance of long-term follow-up of patients. DISCUSSION The data monitoring procedure for Delta differed in various respects from those for Concorde and Alpha [4], two other trials for treatments of HIV infection that overlapped with Delta in time and DSMC membership. The structure of Delta, with two subtrials and four treatment arms, was more complex than that of the other trials, and the clinical profile of its participants wider. It was consequently more difficult to specify clearly in advance the criteria for possible termination. When evidence of possible differences in efficacy started to appear during the second half of the monitoring, the DSMC
240
P. Armitage, et al.
used its discretion in waiting for clearer and more coherent evidence to emerge before recommending termination. When such a recommendation became inevitable, the trial investigators were in any case on the point of stopping. The outcomes of the three trials were very different. In Concorde, an early suggestion of an effect needed to be treated with caution and was eventually shown to be illusory. In Alpha, no difference in efficacy between the two dose levels of ddI was established. In Delta, clear evidence emerged of an advantage in favor of combination therapy, but only toward the end of the trial, and perhaps only after a prolonged period of drug administration. In none of the three trials was safety a major issue. Alpha and Delta demonstrated clearly some adverse effects, but they had been expected on the basis of previous experience and were easily managed. The schedule of meetings followed a similar pattern throughout all three trials. The regular preliminary meetings with Coordinating Committee representatives provided an extremely valuable resource for the DSMC. Their value was particularly apparent during Delta, when participants and clinicians became aware of potentially important alternative treatments, and their anxiety to explore new regimens affected compliance with the trial protocol. In such a rapidly developing field of therapy, this shifting background places a premium on short-term assessments of efficacy, causing problems in the measurement of long-term effects. In any trial with rapid accrual of patients, such as Alpha and Delta, interim presentations of accumulated data will inevitably include some that are erroneous due to inaccuracies in patient records that are corrected at a later stage. Table 1, for example, shows minor irregularities in the numbers of persons starting treatment, as reported at Meetings 6–8 and in the final report, after recruitment had stopped. In some circumstances, the comparisons between treatments may be biased because errors occur more frequently with some treatments than with others. We have no reason to believe that these lacunae were at all frequent in Delta, or that biases occurred. Nevertheless, the possibility of inaccuracies is an additional reason for hesitation in drawing conclusions from early and rapidly assembled data.
I am grateful to my colleagues on the DSMC, the Chairmen of the Coordinating Committee, the Principal Investigators, and Dr. A.G. Babiker, for substantial help in the drafting of this paper. I thank also the editor and a referee for constructive comments on the first draft. The DSMC members express their appreciation to the participants in the trial and to their physicians. We thank also the Coordinating Committee Chairmen and Principal Investigators for their invaluable briefing during the open sessions. We are particularly indebted to the trial statisticians, Drs. A.G. Babiker and G. Cheˆne, who presented tabulations and analyses of the data, often under pressure of time. We received valuable administrative and secretarial help from Roberta Withnall.
REFERENCES 1. Delta Coordinating Committee. Delta: a randomised double-blind controlled trial comparing combinations of zidovudine plus didanosine or zalcitabine with zidovudine alone in HIV-infected individuals. Lancet 1996;348:283–291. 2. Concorde Coordinating Committee. Concorde: MRC/ANRS randomised doubleblind controlled trial of immediate and deferred zidovudine in symptom-free HIV infection. Lancet 1994;343:871–881.
Data and Safety Monitoring in Delta
241
3. Alpha International Coordinating Committee. The Alpha trial: European/Australian randomised double-blind trial of two doses of didanosine in zidovudine-intolerant patients with symptomatic HIV disease. AIDS 1996;10:867–880. 4. Armitage, P. Data and Safety Monitoring in the Concorde and Alpha Trials. Control Clin Trials 1999;20:207–228. 5. Armitage P. Interim analysis in clinical trials. Stat Med 1991;10:925–937. 6. Fischl MA, Stanley K, Collier AC, et al. Combination and monotherapy with zidovudine and zalcitabine in patients with advanced HIV disease. Ann Intern Med 1995;122:24–32.
APPENDIX Data and Safety Monitoring Committee B. Bailey, A. Fagot-Largeault (Joint Chairpersons), P. Armitage, V. Beral, H. Lambert, J. Lellouch, J. Tijssen, B. Varet.