- Email: [email protected]
Statistics in clinical trials
614
CLINICAL AND IMMUNOLOGICAL DATA
*Systemic features, tHistory of vascular events
rates, together with the absence of antibodies in the five without arteritis, does argue for a pathogenic role, and suggests a rationale for the use of immunosuppressive therapy. ACA should be sought in patients with ischaemic optic
erythrocyte sedimentation
neuropathy.
Royal Hallamshire Hospital, Sheffield S10 2JF, UK
M. T. WATTS M. GREAVES L. G. CLEARKIN R. G. MALIA S. M. COOPER
1. Anderson
R, Hansson GK, Soderstrom T, Jonsson R, Bengtsson BA, Nordberg E. HLA-DR expression in the vascular lesion and circulating T-lymphocytes of patients with giant cell arteritis. Clin Exp Immunol 1988; 73: 82-87. 2. Harris EN, Asherson RA, Hughes GRV. Antiphospholipid antibodies_ autoantibodies with a difference. Annu Rev Med 1988; 39: 261-71. 3. Harris EN, Gharavi AE, Patel SP, Hughes GRV. Evaluation of the anticardiolipin antibody test: report of an international workshop held on 4th April 1986. Clin Exp Immunol 1987; 68: 215-19.
Statistics in clinical trials SIR,-Mr Altman and Ms Dore (Jan 20, p 149) note the failure of others to explain sample size, yet they do not explain the sample size in their own study. Was their choice of 80 trials based on statistical power, the time period of the journals scanned, or the work load involved in detailed analysis of the papers? Before starting their study, Altman and Dore might have calculated that with a sign test, 80 trials, and a two-tailed p of 0-05, the statistical power is 80% (which is acceptable) for a medium-sized difference between groups, but only 16% for a small difference.The hypothesis testing that they did is on subgroups, the biggest n being 36. Here, for the sign test, power is 49% and 11% for a medium and a small difference, respectively. In case your contributors were "selling themselves short" by under-reporting procedures rather than failing to do them, I did consider telephoning’ them, but decided instead to keep tongue firmly in cheek. After all, too much detail can clutter an article and Lancet readers appreciate brevity in authors, including Altman and
Dore.
intention to treat was done in less than 20% of trials. To avoid bias in overviews it is necessary to include wellrandomised trials and intention-to-treat analysis.2 A checklist may improve the quality of reports but more needs to be done to improve the quality of the trials themselves. Doctors and students should be taught more about RCTs. Department of Medical Statistics, Institut Gustave Roussy, 94805 Villejuif, France
MORRIS BERNADT
J. Statistical power analysis for the behavioural sciences. New York and London: Academic Press, 1969: 155-56. 2. Liberati A, Himel HN, Chalmers TC. A quality assessment of randomised controlled trials of primary treatment of breast cancer. J Clin Oncol 1986; 4: 942-51. 1. Cohen
SIR,-Mr Altman and Ms Dore’s review of randomisation and baseline data in randomised clinical trials (RCT), because it was limited to four leading general medical journals, gives us an optimistic view of methodological quality. In an overview of studies on healing in duodenal ulcer we looked at 293 RCTs published between 1976 and 1987 as full papers or
J. P. PIGNON
Gastroenterology Service, Hôpital Antoine Béclère, Clamart
T. POYNARD
1.
Poynard T, Pignon JP. Acute treatment of duodenal ulcer: analysis of 293 randomized clinical trials. Montrouge: John Libbey Eurotext, 1989. 2. Collins R, Gray R, Godwin J, Peto R. Avoidance of large biases and large random errors in the assessment of moderate treatment effects: the need for systematic overview. Stat Med 1987; 6: 245-50.
SiR,—Overall, randomisation to groups in a controlled trial is balanced; for a given allocation it is not. Whether we take comfort from the first fact or take fright at the second, a significance test on Thus far I agree with Mr Altman and Ms Dore’s discussion of baseline comparability. If, however, it were appropriate to look at baseline comparability to decide what statistical analysis to do, the decision would depend on the correlation between baseline and outcome variable and on the standardised between-group difference2,3-not, as Altman and Dore claim, on absolute differences in means or proportions. However, examination for this purpose is not logical. For normally distributed outcomes the logical strategy is to select a set of prognostic variables before randomisation and to use these in an analysis of covariance whatever the imbalance.3Corresponding methods are available for other types of data. Using change from baseline is not, as Altman and Dore claim, an adequate method for dealing with baseline imbalance; if the correlation coefficient between outcome and baseline is less than 05 it is worse than just using the raw outcome measures.,,4 Looking at prognostic covariates, whether formally in terms of significance tests or by "subjective substantial difference in means or proportions" is no substitute for using them.
prognostic factors is irrelevant.’
Medical Department,
Ciba-Geigy Ltd, 4002 Basle, Switzerland
Department of Psychological Medicine, King’s College Hospital, London SE5 9RS, UK
proceedings in English, French, Italian, Portuguese, or Spanish.’ Methodological quality was assessed independently by each of us on a 14-item score ranging from 0 to 28. Disagreements were resolved by discussion. The trials were divided by year of publication into three groups: 1976-80 (n = 87), 1981-84 (n =116), and 1985-87 (n = 87). The method of generating random numbers and the mechanism used to allocate treatment were reported in only 3% of the trials; 25% of trials described one or other method. Details of randomisation did not improve with time despite a significant improvement in mean quality score from 14 to 16. Data on sample size calculation (0%, 3%, and 17% in the three consecutive periods) and adequate covariable analysis (30%, 47%, and 74%) improved significantly with time. 87% of RCTs reported the proportion of patients withdrawn but even in the latest period analysis by
STEPHEN SENN
1. Senn SJ. The use of baselines m clinical trials of bronchodilators. Stat Med 1989; 8: 1339-50. 2. Lewis J. Clinical trials: statistical developments of practical benefit to the pharmaceutical industry. J R Statist Soc A 1983; 146: 362-93. 3. Senn SJ. Covariate imbalance and random allocation in clinical trials. Stat Med 1989; 8: 467-75. 4. Hills M, Armitage P. The two-period cross-over clinical trial. Br J Clin Pharmacol 1979; 8: 7-20.
CORRECTION Long-term cardiorespiratory effects of amelioration of renal anaemia by erythropoietin.-In this article by Dr 1. Macdougall and others (March 3, p 489) the 4th sentence of the summary should read: "Carbon monoxide transfer rose from 15-5 (2-9) to 18-6 (3-7) ml. min-’. mm Hg-l."
CLINICAL AND IMMUNOLOGICAL DATA
*Systemic features, tHistory of vascular events
rates, together with the absence of antibodies in the five without arteritis, does argue for a pathogenic role, and suggests a rationale for the use of immunosuppressive therapy. ACA should be sought in patients with ischaemic optic
erythrocyte sedimentation
neuropathy.
Royal Hallamshire Hospital, Sheffield S10 2JF, UK
M. T. WATTS M. GREAVES L. G. CLEARKIN R. G. MALIA S. M. COOPER
1. Anderson
R, Hansson GK, Soderstrom T, Jonsson R, Bengtsson BA, Nordberg E. HLA-DR expression in the vascular lesion and circulating T-lymphocytes of patients with giant cell arteritis. Clin Exp Immunol 1988; 73: 82-87. 2. Harris EN, Asherson RA, Hughes GRV. Antiphospholipid antibodies_ autoantibodies with a difference. Annu Rev Med 1988; 39: 261-71. 3. Harris EN, Gharavi AE, Patel SP, Hughes GRV. Evaluation of the anticardiolipin antibody test: report of an international workshop held on 4th April 1986. Clin Exp Immunol 1987; 68: 215-19.
Statistics in clinical trials SIR,-Mr Altman and Ms Dore (Jan 20, p 149) note the failure of others to explain sample size, yet they do not explain the sample size in their own study. Was their choice of 80 trials based on statistical power, the time period of the journals scanned, or the work load involved in detailed analysis of the papers? Before starting their study, Altman and Dore might have calculated that with a sign test, 80 trials, and a two-tailed p of 0-05, the statistical power is 80% (which is acceptable) for a medium-sized difference between groups, but only 16% for a small difference.The hypothesis testing that they did is on subgroups, the biggest n being 36. Here, for the sign test, power is 49% and 11% for a medium and a small difference, respectively. In case your contributors were "selling themselves short" by under-reporting procedures rather than failing to do them, I did consider telephoning’ them, but decided instead to keep tongue firmly in cheek. After all, too much detail can clutter an article and Lancet readers appreciate brevity in authors, including Altman and
Dore.
intention to treat was done in less than 20% of trials. To avoid bias in overviews it is necessary to include wellrandomised trials and intention-to-treat analysis.2 A checklist may improve the quality of reports but more needs to be done to improve the quality of the trials themselves. Doctors and students should be taught more about RCTs. Department of Medical Statistics, Institut Gustave Roussy, 94805 Villejuif, France
MORRIS BERNADT
J. Statistical power analysis for the behavioural sciences. New York and London: Academic Press, 1969: 155-56. 2. Liberati A, Himel HN, Chalmers TC. A quality assessment of randomised controlled trials of primary treatment of breast cancer. J Clin Oncol 1986; 4: 942-51. 1. Cohen
SIR,-Mr Altman and Ms Dore’s review of randomisation and baseline data in randomised clinical trials (RCT), because it was limited to four leading general medical journals, gives us an optimistic view of methodological quality. In an overview of studies on healing in duodenal ulcer we looked at 293 RCTs published between 1976 and 1987 as full papers or
J. P. PIGNON
Gastroenterology Service, Hôpital Antoine Béclère, Clamart
T. POYNARD
1.
Poynard T, Pignon JP. Acute treatment of duodenal ulcer: analysis of 293 randomized clinical trials. Montrouge: John Libbey Eurotext, 1989. 2. Collins R, Gray R, Godwin J, Peto R. Avoidance of large biases and large random errors in the assessment of moderate treatment effects: the need for systematic overview. Stat Med 1987; 6: 245-50.
SiR,—Overall, randomisation to groups in a controlled trial is balanced; for a given allocation it is not. Whether we take comfort from the first fact or take fright at the second, a significance test on Thus far I agree with Mr Altman and Ms Dore’s discussion of baseline comparability. If, however, it were appropriate to look at baseline comparability to decide what statistical analysis to do, the decision would depend on the correlation between baseline and outcome variable and on the standardised between-group difference2,3-not, as Altman and Dore claim, on absolute differences in means or proportions. However, examination for this purpose is not logical. For normally distributed outcomes the logical strategy is to select a set of prognostic variables before randomisation and to use these in an analysis of covariance whatever the imbalance.3Corresponding methods are available for other types of data. Using change from baseline is not, as Altman and Dore claim, an adequate method for dealing with baseline imbalance; if the correlation coefficient between outcome and baseline is less than 05 it is worse than just using the raw outcome measures.,,4 Looking at prognostic covariates, whether formally in terms of significance tests or by "subjective substantial difference in means or proportions" is no substitute for using them.
prognostic factors is irrelevant.’
Medical Department,
Ciba-Geigy Ltd, 4002 Basle, Switzerland
Department of Psychological Medicine, King’s College Hospital, London SE5 9RS, UK
proceedings in English, French, Italian, Portuguese, or Spanish.’ Methodological quality was assessed independently by each of us on a 14-item score ranging from 0 to 28. Disagreements were resolved by discussion. The trials were divided by year of publication into three groups: 1976-80 (n = 87), 1981-84 (n =116), and 1985-87 (n = 87). The method of generating random numbers and the mechanism used to allocate treatment were reported in only 3% of the trials; 25% of trials described one or other method. Details of randomisation did not improve with time despite a significant improvement in mean quality score from 14 to 16. Data on sample size calculation (0%, 3%, and 17% in the three consecutive periods) and adequate covariable analysis (30%, 47%, and 74%) improved significantly with time. 87% of RCTs reported the proportion of patients withdrawn but even in the latest period analysis by
STEPHEN SENN
1. Senn SJ. The use of baselines m clinical trials of bronchodilators. Stat Med 1989; 8: 1339-50. 2. Lewis J. Clinical trials: statistical developments of practical benefit to the pharmaceutical industry. J R Statist Soc A 1983; 146: 362-93. 3. Senn SJ. Covariate imbalance and random allocation in clinical trials. Stat Med 1989; 8: 467-75. 4. Hills M, Armitage P. The two-period cross-over clinical trial. Br J Clin Pharmacol 1979; 8: 7-20.
CORRECTION Long-term cardiorespiratory effects of amelioration of renal anaemia by erythropoietin.-In this article by Dr 1. Macdougall and others (March 3, p 489) the 4th sentence of the summary should read: "Carbon monoxide transfer rose from 15-5 (2-9) to 18-6 (3-7) ml. min-’. mm Hg-l."