- Email: [email protected]
Two very different types of clinical importance
Contemporary Clinical Trials 46 (2016) 11
Contents lists available at ScienceDirect
Contemporary Clinical Trials journal homepage: www.elsevier.com/locate/conclintrial
Letter to the Editor Two very different types of clinical importance Keywords: Clinical importance Antidepressant efficacy
The recent article by Moncrieff and Kirsch [1] conflates the determination of the magnitude of improvement within patients (as assessed by patients or clinicians) that can be considered clinically important with the determination of the magnitude of group differences (typically between an active treatment and placebo) in clinical trials that can be considered clinically important. The distinction between these two different concepts of clinical importance has been recognized for many years across a variety of therapeutic areas. It has been concluded that the clinical importance of group differences should not be based on the magnitude of the within-patient improvements that patients (or clinicians) consider important but should rather be based on “the broader context of the disease being treated, the currently available treatment, and the overall risk–benefit ratio of the treatment” [2], ideally as evaluated by patients, clinicians, researchers, statisticians, and other stakeholders. Readers of the article by Moncrieff and Kirsch [1] should not accept its conclusions without careful consideration of the following quotations and the articles from which they are drawn: “Clinicians and investigators tend to assume that if the mean difference between a treatment and a control is appreciably less than the smallest change that is important, then the treatment has a trivial effect. This may not be so. Let us assume that a randomized clinical trial shows a mean difference of 0.25 in a questionnaire in which the minimal important difference is 0.5. It might be concluded that the difference is unimportant and that the result does not support giving the treatment. This interpretation assumes that every patient treated scored 0.25 better than they would have done had they received the control and ignores the possibility that treatment might have a heterogeneous effect” [3]. “Research has shown that the interpretation of change may vary depending on whether we are thinking at a group level (where smaller changes may be interpreted as important) or at an individual level, where larger changes are required before they are confidently accepted as indicating a meaningful change” [4]. “The clinical significance of a treatment is based on external standards provided by clinicians, patients, and/or researchers… It is not possible, however, to present more than a tentative recommendation for which effect size to use, or to provide any fixed standards for any such effect size that a clinician could universally use to conclude that an effect
http://dx.doi.org/10.1016/j.cct.2015.11.007 1551-7144/© 2015 Elsevier Inc. All rights reserved.
size was clinically significant. It makes a difference whether the treatment is for a deadly disease like polio, or the common cold, and whether the treatment is risky and costly or perfectly safe and free. The context in which an effect size is used matters in interpreting the size of the effect; the choice of effect size is only to facilitate consideration of the effect in the context of its use” [5]. “A lot of studies proposed values at the individual level for the CID (clinically important difference) with anchor-based approaches. At the group level, such important CID has not been proposed, because it can only be established on the broader context of the disease being treated, the currently available treatment, and the overall risk–benefit ratio of the treatment” [2]. Acknowledgments The author has received in the past 12 months research grants and contracts from U.S. Food and Drug Administration (U01 FD004187-05; HHSF22301400191C) and US National Institutes of Health (1R01 AR059102-01A1), and compensation for activities involving clinical trial research methods from Abide, Acetylon, Adynxx, Astellas, Axsome, Biogen, Coronado, Daiichi Sankyo, Eli Lilly, GlaxoSmithKline, Glenmark, Hope, Hydra, Immune, Johnson & Johnson, Lpath, Medavante, Novartis, NsGene, Olatec, Periphagen, Phosphagenics, Relmada, Spinifex, Teva, and Thar. References [1] J. Moncrieff, I. Kirsch, Empirically derived criteria cast doubt on the clinical significance of antidepressant-placebo differences, Contemp. Clin. Trials 43 (2015) 60–62. [2] A. Ruyssen-Witrand, F. Tubach, P. Ravaud, Systematic review reveals heterogeneity in definition of a clinically relevant difference in pain, J. Clin. Epidemiol. 64 (2011) 463–470. [3] G.H. Guyatt, E.F. Juniper, S.D. Walter, L.E. Griffith, R.S. Goldstein, Interpreting treatment effects in randomised trials, BMJ 316 (1998) 690–693. [4] D. Beaton, C. Bombardier, J.N. Katz, J.G. Wright, G. Wells, M. Boers, et al., Looking for important change/differences in studies of responsiveness, J. Rheumatol. 28 (2001) 400–405. [5] H.C. Kraemer, G.A. Morgan, N.L. Leech, J.A. Gliner, J.J. Vaske, R.J. Harmon, Measures of clinical significance, J. Am. Acad. Child Adolesc. Psychiatry 42 (2003) 1524–1529.
Robert H. Dworkin Department of Anesthesiology, University of Rochester School of Medicine and Dentistry, 601 Elmwood Avenue, Box 604, Rochester, NY 14642, United States 6 November 2015 Available online 10 November 2015
Contents lists available at ScienceDirect
Contemporary Clinical Trials journal homepage: www.elsevier.com/locate/conclintrial
Letter to the Editor Two very different types of clinical importance Keywords: Clinical importance Antidepressant efficacy
The recent article by Moncrieff and Kirsch [1] conflates the determination of the magnitude of improvement within patients (as assessed by patients or clinicians) that can be considered clinically important with the determination of the magnitude of group differences (typically between an active treatment and placebo) in clinical trials that can be considered clinically important. The distinction between these two different concepts of clinical importance has been recognized for many years across a variety of therapeutic areas. It has been concluded that the clinical importance of group differences should not be based on the magnitude of the within-patient improvements that patients (or clinicians) consider important but should rather be based on “the broader context of the disease being treated, the currently available treatment, and the overall risk–benefit ratio of the treatment” [2], ideally as evaluated by patients, clinicians, researchers, statisticians, and other stakeholders. Readers of the article by Moncrieff and Kirsch [1] should not accept its conclusions without careful consideration of the following quotations and the articles from which they are drawn: “Clinicians and investigators tend to assume that if the mean difference between a treatment and a control is appreciably less than the smallest change that is important, then the treatment has a trivial effect. This may not be so. Let us assume that a randomized clinical trial shows a mean difference of 0.25 in a questionnaire in which the minimal important difference is 0.5. It might be concluded that the difference is unimportant and that the result does not support giving the treatment. This interpretation assumes that every patient treated scored 0.25 better than they would have done had they received the control and ignores the possibility that treatment might have a heterogeneous effect” [3]. “Research has shown that the interpretation of change may vary depending on whether we are thinking at a group level (where smaller changes may be interpreted as important) or at an individual level, where larger changes are required before they are confidently accepted as indicating a meaningful change” [4]. “The clinical significance of a treatment is based on external standards provided by clinicians, patients, and/or researchers… It is not possible, however, to present more than a tentative recommendation for which effect size to use, or to provide any fixed standards for any such effect size that a clinician could universally use to conclude that an effect
http://dx.doi.org/10.1016/j.cct.2015.11.007 1551-7144/© 2015 Elsevier Inc. All rights reserved.
size was clinically significant. It makes a difference whether the treatment is for a deadly disease like polio, or the common cold, and whether the treatment is risky and costly or perfectly safe and free. The context in which an effect size is used matters in interpreting the size of the effect; the choice of effect size is only to facilitate consideration of the effect in the context of its use” [5]. “A lot of studies proposed values at the individual level for the CID (clinically important difference) with anchor-based approaches. At the group level, such important CID has not been proposed, because it can only be established on the broader context of the disease being treated, the currently available treatment, and the overall risk–benefit ratio of the treatment” [2]. Acknowledgments The author has received in the past 12 months research grants and contracts from U.S. Food and Drug Administration (U01 FD004187-05; HHSF22301400191C) and US National Institutes of Health (1R01 AR059102-01A1), and compensation for activities involving clinical trial research methods from Abide, Acetylon, Adynxx, Astellas, Axsome, Biogen, Coronado, Daiichi Sankyo, Eli Lilly, GlaxoSmithKline, Glenmark, Hope, Hydra, Immune, Johnson & Johnson, Lpath, Medavante, Novartis, NsGene, Olatec, Periphagen, Phosphagenics, Relmada, Spinifex, Teva, and Thar. References [1] J. Moncrieff, I. Kirsch, Empirically derived criteria cast doubt on the clinical significance of antidepressant-placebo differences, Contemp. Clin. Trials 43 (2015) 60–62. [2] A. Ruyssen-Witrand, F. Tubach, P. Ravaud, Systematic review reveals heterogeneity in definition of a clinically relevant difference in pain, J. Clin. Epidemiol. 64 (2011) 463–470. [3] G.H. Guyatt, E.F. Juniper, S.D. Walter, L.E. Griffith, R.S. Goldstein, Interpreting treatment effects in randomised trials, BMJ 316 (1998) 690–693. [4] D. Beaton, C. Bombardier, J.N. Katz, J.G. Wright, G. Wells, M. Boers, et al., Looking for important change/differences in studies of responsiveness, J. Rheumatol. 28 (2001) 400–405. [5] H.C. Kraemer, G.A. Morgan, N.L. Leech, J.A. Gliner, J.J. Vaske, R.J. Harmon, Measures of clinical significance, J. Am. Acad. Child Adolesc. Psychiatry 42 (2003) 1524–1529.
Robert H. Dworkin Department of Anesthesiology, University of Rochester School of Medicine and Dentistry, 601 Elmwood Avenue, Box 604, Rochester, NY 14642, United States 6 November 2015 Available online 10 November 2015