- Email: [email protected]
How should treatment results from clinical practice be reported?
Pain 138 (2008) 475–476 www.elsevier.com/locate/pain
Editorial
How should treatment results from clinical practice be reported? Articles reporting the results of randomized, placebocontrolled trials of new and old pharmacotherapies for pain are a feature of essentially every issue of PAIN. Much less frequent are independent reports of the results of these same therapies when used on a large scale in clinical practice. In this issue of PAIN, Toth and Au [13] report the results of treatment of 182 patients with neuropathic pain due to peripheral neuropathy. As part of a clinical practice initiative, patients meeting entry criteria after a comprehensive and standardized evaluation in a specialty clinic were treated and followed up at 3 and 6 months. Outcome data was obtained in almost all subjects. Reports such as this one differ in important ways from registry studies and longer term open label follow-on studies of treatments in the late stages of the new drug development process. Registry studies are relatively uncommon in the non-malignant pain literature. One of the few published in recent years was industry sponsored and all subjects had previously participated in one of several randomized controlled trials of sustained-release oxycodone [9]. Although such studies are valuable, the restricted access imposed by participation in a randomized controlled trial produce results of only limited relevance to the larger population of patients suffering from chronic neuropathic pain. Clinical trials require subjects to be relatively healthy, have one clearly defined type of neuropathic pain (most often post-herpetic neuralgia or painful diabetic neuropathy), place restrictions on prior and concomitant therapy, require moderate-severe pain at baseline, and perhaps most importantly, select for patients motivated and able to adhere to the requirements of a trial, including the possibility of being randomized to treatment with a placebo for a long period of time. For drugs not yet approved, long term follow-on open label results are included in the FDA review process prior to approval of a new drug. The report by Toth and Au [13] is to be strongly commended for the large sample size, careful followup, and use of a clinical practice initiative to gather prospective
data. However, the interpretation and reporting of the results raise important issues. The most important of these is the calculation of a new measure that Toth and Au call aNNT (approximated Number Needed to Treat) and aNNH (approximated Number Needed to Harm). NNT and NNH calculations have been widely used in constructing treatment algorithms [1,4,6,7,11,12] and reporting of large, randomized, placebo-controlled trials [10] for more than a decade. Not all evidence-based reviews have relied on NNTs and NNHs, however [3]. The cornerstone of the NNT concept is the use of the results of multiple randomized, placebo-controlled trials to estimate the true effect of a test therapy. The calculated NNT value for each trial takes into account the response to placebo within that particular trial. Placebo response rates can vary considerably from one trial to another, a finding that has proven especially true in trials of antidepressants for major depression [2,14]. When constructing an algorithm, the ideal evidence for the drug choices would be based on direct comparisons of one drug with another for both efficacy and side effects. As there are very few direct comparison studies available, NNT and NNH values are compared instead. Using NNT and NNH values in constructing comparisons is far from perfect. Unless a 95% confidence interval (or similar measure of variability within each trial) is calculated for the trial NNT, there is no way of knowing differences in the size of the trials from looking at the NNTs alone. NNTs may be calculated using different outcome measures of efficacy; in fact the various NNTs will show subtle but potentially important differences when calculated using each of the different outcome measures in a trial. Unless the primary efficacy outcome measure is prespecified and the NNT based on that measure, the opportunity exists to calculate the NNT for each available outcome measure and then report only the most favorable one. For NNH, the difficulty of comparing across trials may be even greater. Studies vary substantially in how adverse effect information is collected. NNH values are most readily comparable for
0304-3959/$34.00 Ó 2008 Published by Elsevier B.V. on behalf of International Association for the Study of Pain. doi:10.1016/j.pain.2008.08.003
476
Editorial / Pain 138 (2008) 475–476
easily defined severe adverse events (hospitalization, death, etc.). Using a standardized checklist of side effects will yield different results than relying on patient reports of side effects. Toth and Au attached the ‘approximated’ prefix onto the NNT and NNH terms in recognition of the lack of a placebo group in their study; all treatment was openlabel and the choice of treatment represented the results of a discussion between patient and physician. Although the authors state that they tried to avoid emphasizing any one particular medication, treatment assignment was not randomized or guided by a clearly defined and pre-specified decision tree. The lack of a placebo control group makes the choice of an appropriate control group for calculating aNNT and aNNH particularly important. In the Toth and Au paper, the control group was the 17 patients (of 182) electing not to receive pharmacotherapy. This is a very small number of patients to use as a comparison group, and makes statistical comparisons problematic. The fact that these comparison patients elected not to receive pharmacotherapy is presumably related to the minimal change in pain levels recorded after 3 months. The net result is exceptionally low NNT figures for 30% pain relief with the drugs used (2.2 for gabapentin and 2.3 for amitriptyline). Why the 108 patients already taking other medications for pain were excluded from the analysis is unclear; they would constitute a ‘no change in treatment’ comparison group not unlike the patients electing to receive no therapy. Little detail is provided on how adverse event data was collected. The aNNH numbers provided represent vaguely defined rates of ‘intolerable side effects’. To prevent confusion, it would have been better to avoid appending the approximated label onto the NNT and NNH concepts and choose a different term altogether for reporting outcomes. It is difficult to overstate the importance of random assignment to treatment; this is how the playing field is made level for comparing treatments. However, random assignment to treatment may not have been an option available to the authors in their clinical practice initiative. Consenting to treatment randomization means a loss of the ability to choose that many patients (and many physicians) resist. Certainly, randomization to blinded administration of a placebo is unlikely to ever be a feature of clinical practice. A clinical practice alternative is randomization to a period of no treatment or a waiting list. As Greenhalgh reminds us [5], cohort studies, case-control studies, and even cross-sectional surveys represent appropriate alternatives to collecting and reporting clinical practice results when randomized controlled trials are not feasible. Studies like the one conducted by Toth and Au are a positive step
in the direction of reporting clinical practice results in a format similar to the widely accepted CONSORT statement guidelines for parallel-group randomized trials [8]. Analogous guidelines are needed to facilitate standardized reporting of prospectively gathered practice results when randomization and blinding cannot be implemented.
References [1] Cook RJ, Sackett DL. The number needed to treat: a clinically useful measure of treatment effect. British Med J 1995;310:452–4. [2] Dworkin RH, Katz J, Gitlin MJ. Placebo response in clinical trials of depression and its implications for research on chronic neuropathic pain. Neurology 2005;65:S7–S19. [3] Dworkin RH, O’Connor AB, Backonja M, Farrar JT, Finnerup NB, Jensen TS, et al. Pharmacologic management of neuropathic pain: evidence-based recommendations. Pain 2007;132:237–51. [4] Finnerup NB, Otto M, McQuay HJ, Jensen TS, Sindrup SH. Algorithm for neuropathic pain treatment: an evidence-based proposal. Pain 2005;118:289–305. [5] Greenhalgh T. How to read a paper : getting your bearings (deciding what the paper is about). BMJ 1997;315:243–6. [6] McQuay HJ, Carroll D, Jadad AR, Wiffen P, Moore A. Anticonvulsant drugs for management of pain: a systematic review. BMJ 1995;311:1047–52. [7] McQuay HJ, Tramer M, Nye BA, Carroll D, Wiffen PJ, Moore RA. A systematic review of antidepressants in neuropathic pain. Pain 1996;68:217–27. [8] Moher D, Schultz KF, Altman D, Lepage L. The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomized trials. JAMA 2001;285: 1987–91. [9] Portenoy RK, Farrar JT, Backonja M, Cleeland CS, Yang K, Friedman M, et al. Longterm use of controlled-release oxycodone for noncancer pain: results of a three-year registry study. Clin J Pain 2007;23:300–2. [10] Rowbotham M, Harden N, Stacey B, Bernstein P, Magnus-Miller L. Gabapentin for the treatment of postherpetic neuralgia: a randomized controlled trial. JAMA 1998;280:1837–42. [11] Sindrup SH, Jensen TS. Efficacy of pharmacological treatments of neuropathic pain: an update and effect related to mechanism of drug action. Pain 1999;83:389–400. [12] Sindrup SH, Jensen TS. Pharmacological treatment of pain in polyneuropathy. Neurology 2000;55:915–20. [13] Toth C, Au S. A prospective identification of neuropathic pain in specific chronic polyneuropathy syndromes and response to pharmacological therapy. Pain 2008;138:657–66. [14] Walsh BT, Seidman SN, Sysko R, Gould M. Placebo response in studies of major depression: variable, substantial, and growing. JAMA 2002;287:1840–7.
Michael C. Rowbotham * UCSF Pain Clinical Research Center, School of Medicine, Department of Neurology, University of California, 1701 Divisadero Street, Suite 480, San Francisco, CA 94115, USA E-mail address: [email protected]
*
Tel.: +1 415 885 7899; fax: +1 415 885 7855.
Editorial
How should treatment results from clinical practice be reported? Articles reporting the results of randomized, placebocontrolled trials of new and old pharmacotherapies for pain are a feature of essentially every issue of PAIN. Much less frequent are independent reports of the results of these same therapies when used on a large scale in clinical practice. In this issue of PAIN, Toth and Au [13] report the results of treatment of 182 patients with neuropathic pain due to peripheral neuropathy. As part of a clinical practice initiative, patients meeting entry criteria after a comprehensive and standardized evaluation in a specialty clinic were treated and followed up at 3 and 6 months. Outcome data was obtained in almost all subjects. Reports such as this one differ in important ways from registry studies and longer term open label follow-on studies of treatments in the late stages of the new drug development process. Registry studies are relatively uncommon in the non-malignant pain literature. One of the few published in recent years was industry sponsored and all subjects had previously participated in one of several randomized controlled trials of sustained-release oxycodone [9]. Although such studies are valuable, the restricted access imposed by participation in a randomized controlled trial produce results of only limited relevance to the larger population of patients suffering from chronic neuropathic pain. Clinical trials require subjects to be relatively healthy, have one clearly defined type of neuropathic pain (most often post-herpetic neuralgia or painful diabetic neuropathy), place restrictions on prior and concomitant therapy, require moderate-severe pain at baseline, and perhaps most importantly, select for patients motivated and able to adhere to the requirements of a trial, including the possibility of being randomized to treatment with a placebo for a long period of time. For drugs not yet approved, long term follow-on open label results are included in the FDA review process prior to approval of a new drug. The report by Toth and Au [13] is to be strongly commended for the large sample size, careful followup, and use of a clinical practice initiative to gather prospective
data. However, the interpretation and reporting of the results raise important issues. The most important of these is the calculation of a new measure that Toth and Au call aNNT (approximated Number Needed to Treat) and aNNH (approximated Number Needed to Harm). NNT and NNH calculations have been widely used in constructing treatment algorithms [1,4,6,7,11,12] and reporting of large, randomized, placebo-controlled trials [10] for more than a decade. Not all evidence-based reviews have relied on NNTs and NNHs, however [3]. The cornerstone of the NNT concept is the use of the results of multiple randomized, placebo-controlled trials to estimate the true effect of a test therapy. The calculated NNT value for each trial takes into account the response to placebo within that particular trial. Placebo response rates can vary considerably from one trial to another, a finding that has proven especially true in trials of antidepressants for major depression [2,14]. When constructing an algorithm, the ideal evidence for the drug choices would be based on direct comparisons of one drug with another for both efficacy and side effects. As there are very few direct comparison studies available, NNT and NNH values are compared instead. Using NNT and NNH values in constructing comparisons is far from perfect. Unless a 95% confidence interval (or similar measure of variability within each trial) is calculated for the trial NNT, there is no way of knowing differences in the size of the trials from looking at the NNTs alone. NNTs may be calculated using different outcome measures of efficacy; in fact the various NNTs will show subtle but potentially important differences when calculated using each of the different outcome measures in a trial. Unless the primary efficacy outcome measure is prespecified and the NNT based on that measure, the opportunity exists to calculate the NNT for each available outcome measure and then report only the most favorable one. For NNH, the difficulty of comparing across trials may be even greater. Studies vary substantially in how adverse effect information is collected. NNH values are most readily comparable for
0304-3959/$34.00 Ó 2008 Published by Elsevier B.V. on behalf of International Association for the Study of Pain. doi:10.1016/j.pain.2008.08.003
476
Editorial / Pain 138 (2008) 475–476
easily defined severe adverse events (hospitalization, death, etc.). Using a standardized checklist of side effects will yield different results than relying on patient reports of side effects. Toth and Au attached the ‘approximated’ prefix onto the NNT and NNH terms in recognition of the lack of a placebo group in their study; all treatment was openlabel and the choice of treatment represented the results of a discussion between patient and physician. Although the authors state that they tried to avoid emphasizing any one particular medication, treatment assignment was not randomized or guided by a clearly defined and pre-specified decision tree. The lack of a placebo control group makes the choice of an appropriate control group for calculating aNNT and aNNH particularly important. In the Toth and Au paper, the control group was the 17 patients (of 182) electing not to receive pharmacotherapy. This is a very small number of patients to use as a comparison group, and makes statistical comparisons problematic. The fact that these comparison patients elected not to receive pharmacotherapy is presumably related to the minimal change in pain levels recorded after 3 months. The net result is exceptionally low NNT figures for 30% pain relief with the drugs used (2.2 for gabapentin and 2.3 for amitriptyline). Why the 108 patients already taking other medications for pain were excluded from the analysis is unclear; they would constitute a ‘no change in treatment’ comparison group not unlike the patients electing to receive no therapy. Little detail is provided on how adverse event data was collected. The aNNH numbers provided represent vaguely defined rates of ‘intolerable side effects’. To prevent confusion, it would have been better to avoid appending the approximated label onto the NNT and NNH concepts and choose a different term altogether for reporting outcomes. It is difficult to overstate the importance of random assignment to treatment; this is how the playing field is made level for comparing treatments. However, random assignment to treatment may not have been an option available to the authors in their clinical practice initiative. Consenting to treatment randomization means a loss of the ability to choose that many patients (and many physicians) resist. Certainly, randomization to blinded administration of a placebo is unlikely to ever be a feature of clinical practice. A clinical practice alternative is randomization to a period of no treatment or a waiting list. As Greenhalgh reminds us [5], cohort studies, case-control studies, and even cross-sectional surveys represent appropriate alternatives to collecting and reporting clinical practice results when randomized controlled trials are not feasible. Studies like the one conducted by Toth and Au are a positive step
in the direction of reporting clinical practice results in a format similar to the widely accepted CONSORT statement guidelines for parallel-group randomized trials [8]. Analogous guidelines are needed to facilitate standardized reporting of prospectively gathered practice results when randomization and blinding cannot be implemented.
References [1] Cook RJ, Sackett DL. The number needed to treat: a clinically useful measure of treatment effect. British Med J 1995;310:452–4. [2] Dworkin RH, Katz J, Gitlin MJ. Placebo response in clinical trials of depression and its implications for research on chronic neuropathic pain. Neurology 2005;65:S7–S19. [3] Dworkin RH, O’Connor AB, Backonja M, Farrar JT, Finnerup NB, Jensen TS, et al. Pharmacologic management of neuropathic pain: evidence-based recommendations. Pain 2007;132:237–51. [4] Finnerup NB, Otto M, McQuay HJ, Jensen TS, Sindrup SH. Algorithm for neuropathic pain treatment: an evidence-based proposal. Pain 2005;118:289–305. [5] Greenhalgh T. How to read a paper : getting your bearings (deciding what the paper is about). BMJ 1997;315:243–6. [6] McQuay HJ, Carroll D, Jadad AR, Wiffen P, Moore A. Anticonvulsant drugs for management of pain: a systematic review. BMJ 1995;311:1047–52. [7] McQuay HJ, Tramer M, Nye BA, Carroll D, Wiffen PJ, Moore RA. A systematic review of antidepressants in neuropathic pain. Pain 1996;68:217–27. [8] Moher D, Schultz KF, Altman D, Lepage L. The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomized trials. JAMA 2001;285: 1987–91. [9] Portenoy RK, Farrar JT, Backonja M, Cleeland CS, Yang K, Friedman M, et al. Longterm use of controlled-release oxycodone for noncancer pain: results of a three-year registry study. Clin J Pain 2007;23:300–2. [10] Rowbotham M, Harden N, Stacey B, Bernstein P, Magnus-Miller L. Gabapentin for the treatment of postherpetic neuralgia: a randomized controlled trial. JAMA 1998;280:1837–42. [11] Sindrup SH, Jensen TS. Efficacy of pharmacological treatments of neuropathic pain: an update and effect related to mechanism of drug action. Pain 1999;83:389–400. [12] Sindrup SH, Jensen TS. Pharmacological treatment of pain in polyneuropathy. Neurology 2000;55:915–20. [13] Toth C, Au S. A prospective identification of neuropathic pain in specific chronic polyneuropathy syndromes and response to pharmacological therapy. Pain 2008;138:657–66. [14] Walsh BT, Seidman SN, Sysko R, Gould M. Placebo response in studies of major depression: variable, substantial, and growing. JAMA 2002;287:1840–7.
Michael C. Rowbotham * UCSF Pain Clinical Research Center, School of Medicine, Department of Neurology, University of California, 1701 Divisadero Street, Suite 480, San Francisco, CA 94115, USA E-mail address: [email protected]
*
Tel.: +1 415 885 7899; fax: +1 415 885 7855.