- Email: [email protected]
Operational aspects of the early termination of the multicenter trial of cryotherapy for retinopathy of prematurity
320
Abstracts isation of the treatment effect is considered in several ways. For each parameterisation the significance level and power achieved by the triangler test are presented from simulation results.
Choosing Covariates in the Analysis of Clinical Trials Michael Beach
University of Chicago, Chicago, Illinois (S-12) It is typical in the conduct of clinical trials to measure a large number of covariates--more than can be induced in the analysis---so that some method is needed to choose among them if we are to adjust at all. One common method is to pick covariates showing the greatest disparity between the treatment and control groups. Another is to choose these strongly related to outcome, i.e., those with high influence. Canner 0981) showed that for the 2 × 2 × 2 contingency table, one can express the difference between the unadjusted test statistic, Zu, and the adjusted test statistic, ZA, as the ~ of the z-value for influence, ZI, and the z-value for disparity, ZD: Zu - ZA = Z~ × ZD/VN. Canner then suggests that choice of covariates which are both influential and disparate or those having a large product may be a useful alternative selection procedure. Different selection procedures were explored in six major clinical studies. Although the influence of the covariates was in each case small and no improvement in precision results from such adjustment, there were substantial shifts in the estimated treatment effect, leading to ambiguity in the interpretation ot the findings. Canner's work is extended to the multiple linear regression model, to generalized linear models, and to the Cox Proportional Hazards model. An extensive Monte Carlo experiment was carried out to study the performance of these three selection procedures. In many common settings covariate adjustment provides little if any advantage. Where adjustment is advantageous, that based on the product of influence and disparity can outperform selection based on either alone. Statistical Considerations in the Early Termination of the Multicenter Trial o f Cryotherapy for Retinopathy of Prematurity Robert J. H a r d y , Barry R. Davis, Betty T u n g , Earl A. P a l m e r University of Texas, Houston, Texas (13) The Multicenter Trial of Cryotherapy for Retinopathy of Prematurity (CRYO-ROP) was a randomized trial designed to test whether cryotherapy applied to one randomly selected eye when there was Stage 3 + ROP significantly decreased the incidence of an unfavorable outcome. An unfavorable outcome was defined as either retinal detachment or a retinal fold and was determined at three months post-treatment by the use of a masked photographic system. The trial was terminated on recommendation of the Data and Safety Monitoring Committee nine months before the scheduled closing date. At the time of the decision, an unfavorable outcome was significantly less frequent in the cryotherapy eyes (21.8%) as compared to the untreated eyes (43%), (chi-square = 20.5). The basic statistical method used in evaluating the significance of the p r i m a ~ outcome results was conditional power (or stochastic curtailment). This method was applied for the first time to a trial with a matched pair design. Several other issues were considered in deciding to terminate the trial including the examination of results across subgroups and clinical centers, the completeness of the follow-up, clinical assessment of outcome, and longterm consequences of the treatment.
Operational Aspects of the Early Termination of the Multicenter Trial o f Cryotherapy for Retinopathy of Prematurity Earl A. Palmer, Robert J. H a r d y , J o h n T. F l y n n , Dale L. P h e l p s , David B. Schaffer, Barry R. Davis, Richard M o w e r y , Betty T u n g , C y n t h i a Phillips Oregon Health Sciences University, Portland, Oregon (14) The Multicenter Trial of Cryotherapy for Retinopathy of Prematurity (CRYO-ROP) was a randomized trial designed to test v. hether cryotherapy applied to one randomly selected eye when there was Stage 3 + ROP significantly decreased the incidence of an unfavorable outcome. An unfavorable outcome was defined as either retinal detachment or a retinal fold and was determined at 3 month post-treatment by the use of a masked photographic system. The trial was
Abstracts
321
terminated on recommendation of the Data and Safety Monitoring Committee 9 months before the scheduled closing data. Operational procedures had to be set in motion to arrange for the orderly termination of randomization and dissemination of results. The purpose of this paper is to discuss close-out activities that are pertinent to the premature termination of a trial. These activities include preparation of trial participants and staff, release of trial information, continuing patient care, data collection and coding, and publication of trial results.
Stochastic Curtailment Methods and Confidence Intervals Barry R. Davis, R o b e r t J. H a r d y
The University of Texas School of Public Health, Houston, Texas (15) Stochastic curtailment methods (SCM) have been used to follow interim results in clinical trials. Using SCM, one can calculate the probability of being in the rejection region at the end of a trial given the current data and either the null or an alternative hypothesis. SCM have been used for normally distributed data (differences of mean, differences of proportions) and survival time data (hazard ratios). We describe methods for the construction of (l) confidence intervals for these estimates at repeated times in the course of a trial, and (2) prediction intervals for predicted estimates at the end of a trial. Specific examples are presented. To Stop or Not To Stop: A Case Study P a u l a K. R o b e r s o n , Jerry L. S h e n e p
St. Jude Children's Research Hospital, Memphis, Tennessee (16) Interim stopping rules are a desirable design feature in many clinical protocols. Often, however, the results apparent on early analysis raise ethical questions unforeseen in the original study plan. Recently, a double-blind, randomized clinical trial with one planned interim analysis was conducted at St. Jude Hospital to compare the efficacy and toxicity of vancomycin, ticarcillin and amikacin versus ticarcillin-clavulanate and amikacin as empirical therapy for febrile neutropenic cancer patients. Decisions regarding early stopping were complicated by the occurrence of unanticipated breakthrough bacteremia in ten percent of the patients, including one fatality. These cases were unevenly distributed between the treatment arms. The audience will be led to reenact the decision process of the study team, including a vote on the appropriate action. The session will conclude with a report and discussion of the action taken by investigators on the St. Jude study team.
Permutation Logrank Test in Controlled Clinical Trials Yudianto Pawitan, Alfred Hallstrom, Robin Reynoids-Haertle University of Washington, Seattle, Washington (17) In permutation or randomization testing one evaluates the statistical significance (or p-value) of a statistic by creating a reference set through repeated treatment assignments to the actual data. In highly stratified clinical trial data, which is quite common, the permutation logrank test (LRT) is shown to have comparable properties to that of the modified LRT of Schoenfeld & Tsiatis (Biometrika, 1987). For example, it is more efficient than the standard LRT or the stratified LRT if there is a strong strata effect. We also consider an application of the permutation LRT in the group sequential testing, the main advantage being that at k'th analysis one can compute easily the joint distribution of ($1. . . . ,Sk), the LRT statistics at k times, without having to estimate the "information time". We will illustrate the procedure by a real example.
Use of the Triangular Test in Phase II Cancer Clinical T r i a l s J a c q u e s B e n i c h o u , Eric Bellissant, C l a u d e C h a s t a n g
Department de Biostastique et Informatique Medicale, Hopital Saint-Louis, Paris, France (18) In phase II cancer clinical trials, the aim is to determine whether a new treatment is effective enough to warrant further study (i.e., a phase III study). The primary outcome variable is the response rate "rr and the study is designed to determine whether -~ is greater than a certain value
Abstracts isation of the treatment effect is considered in several ways. For each parameterisation the significance level and power achieved by the triangler test are presented from simulation results.
Choosing Covariates in the Analysis of Clinical Trials Michael Beach
University of Chicago, Chicago, Illinois (S-12) It is typical in the conduct of clinical trials to measure a large number of covariates--more than can be induced in the analysis---so that some method is needed to choose among them if we are to adjust at all. One common method is to pick covariates showing the greatest disparity between the treatment and control groups. Another is to choose these strongly related to outcome, i.e., those with high influence. Canner 0981) showed that for the 2 × 2 × 2 contingency table, one can express the difference between the unadjusted test statistic, Zu, and the adjusted test statistic, ZA, as the ~ of the z-value for influence, ZI, and the z-value for disparity, ZD: Zu - ZA = Z~ × ZD/VN. Canner then suggests that choice of covariates which are both influential and disparate or those having a large product may be a useful alternative selection procedure. Different selection procedures were explored in six major clinical studies. Although the influence of the covariates was in each case small and no improvement in precision results from such adjustment, there were substantial shifts in the estimated treatment effect, leading to ambiguity in the interpretation ot the findings. Canner's work is extended to the multiple linear regression model, to generalized linear models, and to the Cox Proportional Hazards model. An extensive Monte Carlo experiment was carried out to study the performance of these three selection procedures. In many common settings covariate adjustment provides little if any advantage. Where adjustment is advantageous, that based on the product of influence and disparity can outperform selection based on either alone. Statistical Considerations in the Early Termination of the Multicenter Trial o f Cryotherapy for Retinopathy of Prematurity Robert J. H a r d y , Barry R. Davis, Betty T u n g , Earl A. P a l m e r University of Texas, Houston, Texas (13) The Multicenter Trial of Cryotherapy for Retinopathy of Prematurity (CRYO-ROP) was a randomized trial designed to test whether cryotherapy applied to one randomly selected eye when there was Stage 3 + ROP significantly decreased the incidence of an unfavorable outcome. An unfavorable outcome was defined as either retinal detachment or a retinal fold and was determined at three months post-treatment by the use of a masked photographic system. The trial was terminated on recommendation of the Data and Safety Monitoring Committee nine months before the scheduled closing date. At the time of the decision, an unfavorable outcome was significantly less frequent in the cryotherapy eyes (21.8%) as compared to the untreated eyes (43%), (chi-square = 20.5). The basic statistical method used in evaluating the significance of the p r i m a ~ outcome results was conditional power (or stochastic curtailment). This method was applied for the first time to a trial with a matched pair design. Several other issues were considered in deciding to terminate the trial including the examination of results across subgroups and clinical centers, the completeness of the follow-up, clinical assessment of outcome, and longterm consequences of the treatment.
Operational Aspects of the Early Termination of the Multicenter Trial o f Cryotherapy for Retinopathy of Prematurity Earl A. Palmer, Robert J. H a r d y , J o h n T. F l y n n , Dale L. P h e l p s , David B. Schaffer, Barry R. Davis, Richard M o w e r y , Betty T u n g , C y n t h i a Phillips Oregon Health Sciences University, Portland, Oregon (14) The Multicenter Trial of Cryotherapy for Retinopathy of Prematurity (CRYO-ROP) was a randomized trial designed to test v. hether cryotherapy applied to one randomly selected eye when there was Stage 3 + ROP significantly decreased the incidence of an unfavorable outcome. An unfavorable outcome was defined as either retinal detachment or a retinal fold and was determined at 3 month post-treatment by the use of a masked photographic system. The trial was
Abstracts
321
terminated on recommendation of the Data and Safety Monitoring Committee 9 months before the scheduled closing data. Operational procedures had to be set in motion to arrange for the orderly termination of randomization and dissemination of results. The purpose of this paper is to discuss close-out activities that are pertinent to the premature termination of a trial. These activities include preparation of trial participants and staff, release of trial information, continuing patient care, data collection and coding, and publication of trial results.
Stochastic Curtailment Methods and Confidence Intervals Barry R. Davis, R o b e r t J. H a r d y
The University of Texas School of Public Health, Houston, Texas (15) Stochastic curtailment methods (SCM) have been used to follow interim results in clinical trials. Using SCM, one can calculate the probability of being in the rejection region at the end of a trial given the current data and either the null or an alternative hypothesis. SCM have been used for normally distributed data (differences of mean, differences of proportions) and survival time data (hazard ratios). We describe methods for the construction of (l) confidence intervals for these estimates at repeated times in the course of a trial, and (2) prediction intervals for predicted estimates at the end of a trial. Specific examples are presented. To Stop or Not To Stop: A Case Study P a u l a K. R o b e r s o n , Jerry L. S h e n e p
St. Jude Children's Research Hospital, Memphis, Tennessee (16) Interim stopping rules are a desirable design feature in many clinical protocols. Often, however, the results apparent on early analysis raise ethical questions unforeseen in the original study plan. Recently, a double-blind, randomized clinical trial with one planned interim analysis was conducted at St. Jude Hospital to compare the efficacy and toxicity of vancomycin, ticarcillin and amikacin versus ticarcillin-clavulanate and amikacin as empirical therapy for febrile neutropenic cancer patients. Decisions regarding early stopping were complicated by the occurrence of unanticipated breakthrough bacteremia in ten percent of the patients, including one fatality. These cases were unevenly distributed between the treatment arms. The audience will be led to reenact the decision process of the study team, including a vote on the appropriate action. The session will conclude with a report and discussion of the action taken by investigators on the St. Jude study team.
Permutation Logrank Test in Controlled Clinical Trials Yudianto Pawitan, Alfred Hallstrom, Robin Reynoids-Haertle University of Washington, Seattle, Washington (17) In permutation or randomization testing one evaluates the statistical significance (or p-value) of a statistic by creating a reference set through repeated treatment assignments to the actual data. In highly stratified clinical trial data, which is quite common, the permutation logrank test (LRT) is shown to have comparable properties to that of the modified LRT of Schoenfeld & Tsiatis (Biometrika, 1987). For example, it is more efficient than the standard LRT or the stratified LRT if there is a strong strata effect. We also consider an application of the permutation LRT in the group sequential testing, the main advantage being that at k'th analysis one can compute easily the joint distribution of ($1. . . . ,Sk), the LRT statistics at k times, without having to estimate the "information time". We will illustrate the procedure by a real example.
Use of the Triangular Test in Phase II Cancer Clinical T r i a l s J a c q u e s B e n i c h o u , Eric Bellissant, C l a u d e C h a s t a n g
Department de Biostastique et Informatique Medicale, Hopital Saint-Louis, Paris, France (18) In phase II cancer clinical trials, the aim is to determine whether a new treatment is effective enough to warrant further study (i.e., a phase III study). The primary outcome variable is the response rate "rr and the study is designed to determine whether -~ is greater than a certain value