- Email: [email protected]
Modeling Predictors, Moderators and Mediators of Treatment Outcome and Resistance in Depression
COMMENTARY
Modeling Predictors, Moderators and Mediators of Treatment Outcome and Resistance in Depression Madhukar H. Trivedi “The art of prophecy is very difficult, especially about the future.” —Mark Twain 1886
T
he article by Perlis (1) addresses a question of major importance for treatment matching of patients with major depressive disorder (MDD): Can we identify patients seeking treatment for depression that are unlikely to respond to currently available treatments? The ability to prospectively identify patient subgroups based on their predicted outcomes especially early in the course of treatment is likely to reduce the inordinate delays in identifying the correct treatment for patients, reduce morbidity and mortality, shorten suffering, reduce rates of treatment dropouts, and likely reduce the cost burden for the patient and society (2,3). The author also suggests that early identification of treatment resistant depression (TRD) could direct patients to more complicated treatments more rapidly. The author used data available from the STAR*D sample and developed a model that is designed to address these issues. The STAR*D sample was divided into two groups for the analysis: remitters from step 1 combined with remitters from step 2 and compared with nonremitters from step 2. This sample was then divided into three cohorts: a development cohort (n = 1571), a testing cohort (n = 523), and validation cohort (n = 461). The model was developed using a combination of expert clinical judgment and an automated process using the development cohort. The final model includes a series of sociodemographic, clinical illness, and severity characteristics. To understand the usefulness of the model, it is worthwhile to examine not only the reported accuracy (.71) but also the sensitivity and specificity. In other words, how many of those with TRD are correctly identified as having TRD (sensitivity = 26%), and how many of those without TRD are correctly identified as not having TRD (specificity = 91%). The model described by Perlis is thus better at identifying those without TRD than those with TRD.
Strengths Creation of Training, Testing, and Validation Samples from the STAR*D Sample In general it is ideal to assess how well a classifier performs as a predictor in a separate data set. However, using a subset of the STAR*D sample as was done by Perlis is reasonable to test validation of the predictor model. Selection of The Initial Set of Predictors Based on the Author’s Knowledge/Experience Rather than Using the Data to Select Predictors This approach has the advantage of simplicity, selecting predictors that are readily available in practice, and selecting From the Department of Psychiatry, University of Texas Southwestern Medical Center, Dallas, Texas. Address correspondence to Madhukar H. Trivedi, M.D., Department of Psychiatry, University of Texas Southwestern Medical Center, 5323 Harry Hines Boulevard, Dallas, TX 75390-9119; E-mail: madhukar. [email protected]. Received and accepted May 14, 2013.
0006-3223/$36.00 http://dx.doi.org/10.1016/j.biopsych.2013.05.009
predictors that make sense clinically but has the disadvantage that important predictors could be overlooked. Focusing on the Preemptive Identification of Patients With MDD who are Likely to be Treatment Resistant Results from this and future efforts could lead to clinically matching patients with more complicated treatments (e.g., combination pharmacotherapy, combination pharmacotherapy, and depression focused psychotherapy or electroconvulsive therapy) rapidly to achieve the desired outcomes.
Limitations Limitations of Predictive Ability Prediction of both response and nonresponse would be best if it can be associated with specific treatments (which was not done in this case). If this is not possible, the extent to which they are related to the underlying biology of depression might be the next best alternative (which was also not possible in this data set). Need for Subsequent Testing in a New Sample The model will need to be tested in a subsequent new sample to control for characteristics that were unique to the STAR*D study. The STAR*D sample has unique disadvantages because it was designed as a large pragmatic clinical trial and was thus sparse in terms of detailed clinical phenotyping of the population under study. In addition, the clinical utility is negatively affected by the lack of information about the approximately 37% that were excluded from the analysis. The Elements That Went Into the Prediction Model (With the Exception of Quick Inventory of Depressive Symptomatology Subscale Scores and Some Illness Characteristics) are not Easily Subject to Intervention In contrast, prediction models for risk factors for cardiovascular disease, diabetes, and other chronic disorders identify factors that can either be targeted for treatment or chosen for the development of new treatments. Furthermore, if biological based markers can be associated with treatment resistance, they may provide better guidance for the development of more specific novel treatments for TRD. Modest Clinical Utility in its Current Form The calculator provided gives the probability of TRD. The presentation is useful clinically when a histogram shows the subject’s place within the sample and is also available for several samples (male subjects, female subjects, etc.). However, the results showing a 56.7% probability of TRD requires careful interpretation by the clinician. Even with this model from a practical clinical perspective the strength of the prediction is relatively weak. Also, there would need to be a consideration of the potential for harm or risk to a patient identified as TRD who was not TRD and to a patient identified as not TRD who was TRD. Models with relatively weak prediction may still be useful in situations that result in BIOL PSYCHIATRY 2013;74:2–4 & 2013 Society of Biological Psychiatry
Commentary recommendations for closer monitoring for the development of the disorder (e.g., the risk for developing breast cancer). In contrast, in cases in which classification is likely to lead to recommendations for a potentially more complicated treatment with either greater risk or burden, the need for accuracy is greater.
What Is the Next Step? The timely selection of the best treatment for patients with TRD is critical to the goal of improving remission rates. To accomplish this goal, we need to identify differential predictors of positive response as well as treatment resistance. Because of the biological heterogeneity and variable symptom presentation of depression, it is unlikely that only clinical markers can guide treatment selection even accounting for all the domains of clinical assessments. Rather, a biosignature developed from a careful exploration of a group of several clinical and biological markers is more likely to be successful. Furthermore, predictors are unable to assist with selection of specific treatments but are purely general predictors of clinical status that are unlikely to be clinically useful. Kraemer (4) has suggested that we need to look for moderators and mediators of treatment response as opposed to simple
BIOL PSYCHIATRY 2013;74:2–4 3 predictors. As shown in Figure 1, presence of a moderator would indicate that the patient should be treated with treatment B, whereas the presence of a predictor only indicates that the patient is less likely to respond to treatment, but it does not help in treatment selection. As demonstrated by Perlis, to date we have been most successful at identifying negative predictors but not moderators or mediators. What is sorely needed are two types of biosignatures to achieve improved outcomes: 1) biosignatures to maximize the selection of optimal treatment for individual patients at beginning of treatment (moderators) and 2) biosignatures to identify indicators of eventual outcomes early in treatment (mediators). This approach has great potential to personalize treatment (thus rapidly identifying the most appropriate treatment) and to begin to characterize the biology of treatment response (thus informing future drug development). In addition, there is a growing body of evidence that more complicated treatments that have been shown to be efficacious in TRD may not be significantly more efficacious early in the course of treatment, for example, combination antidepressants as first step treatments (5) or combination of pharmacotherapy plus depression focused psychotherapy (6). Finally, these models developed to predict the various aspects of the course of illness in MDD
Figure 1. Example of differential remission patterns expected with presence of a moderator and predictor.
www.sobp.org/journal
4 BIOL PSYCHIATRY 2013;74:2–4 including treatment outcomes will need to be tested prospectively in new cohorts to be able to change clinical practice.
Summary The approach outlined by Perlis has many strengths; however, it does have some limitations for its ability to guide treatment selection and immediate clinical utility. This approach is characterized by the simplicity of the variable selection and prediction model. An electronically available calculator as developed by the author, if successful, is a particular strength. Future efforts will likely lead to the development of more complex approaches in which the results from Perlis could serve as a benchmark against which to compare results. Dr. Trivedi is or has been an advisor/consultant to the following: Abbott Laboratories, Inc., Akzo (Organon Pharmaceuticals Inc.), Alkermes, AstraZeneca, Axon Advisors, Bristol-Myers Squibb Company, Cephalon, Inc., Cerecor, Concert Pharmaceuticals, Inc., Eli Lilly & Company, Evotec, Fabre Kramer Pharmaceuticals, Inc., Forest Pharmaceuticals, GlaxoSmithKline, Janssen Global Services, LLC, Janssen Pharmaceutica Products, LP, Johnson & Johnson PRD, Libby, Lundbeck, Meade Johnson, MedAvante, Medtronic, Merck, Naurex, Neuronetics, Otsuka Pharmaceuticals, Pamlab, Parke-Davis Pharmaceuticals, Inc., Pfizer Inc., PgxHealth, Phoenix Marketing Solutions, Rexahn Pharmaceuticals, Ridge Diagnostics, Roche Products Ltd., Sepracor, SHIRE Development, Sierra, SK Life and Science, Sunovion,
www.sobp.org/journal
Commentary Takeda, Tal Medical/Puretech Venture, Targacept, Transcept, Vivus, and Wyeth-Ayerst Laboratories. In addition, he has received research support from the Agency for Healthcare Research and Quality, Corcept Therapeutics, Inc., Cyberonics, Inc., National Alliance for Research in Schizophrenia and Depression, National Institute of Mental Health, National Institute on Drug Abuse, Novartis, Pharmacia & Upjohn, Predix Pharmaceuticals (Epix), and Solvay Pharmaceuticals, Inc. 1. Perlis RH (2013): A clinical risk stratification tool for predicting treatment resistance in major depressive disorder. Biol Psychiatry 74:7–14. 2. Fava M, Nelson JC, Thase ME, Trivedi MH (2009): Easing the burden of treatment-resistant depression. J Clin Psychiatry 70:273–280. 3. Rush AJ, Trivedi MH, Wisniewski SR, Nierenberg AA, Stewart JW, Warden D, et al. (2006): Acute and longer-term outcomes in depressed outpatients who required one or several treatment steps: A STAR*D report. Am J Psychiatry 163:1905–1917. 4. Kraemer HC, Wilson GT, Fairburn CG, Agras WS (2002): Mediators and moderators of treatment effects in randomized clinical trials. Arch Gen Psychiatry 59:877–883. 5. Rush AJ, Trivedi MH, Stewart JW, Nierenberg AA, Fava M, Kurian BT, et al. (2011): Combining Medications to Enhance Depression Outcomes (CO-MED): Acute and long-term outcomes: A single-blind randomized study. Am J Psychiatry 168:689–701. 6. Kocsis JH, Gelenberg AJ, Rothbaum BO, Klein DN, Trivedi MH, Manber R, et al. (2009): Cognitive behavioral analysis system of psychotherapy and brief supportive psychotherapy for augmentation of antidepressant nonresponse in chronic depression: The REVAMP trial. Arch Gen Psychiatry 66:1178–1188.
Modeling Predictors, Moderators and Mediators of Treatment Outcome and Resistance in Depression Madhukar H. Trivedi “The art of prophecy is very difficult, especially about the future.” —Mark Twain 1886
T
he article by Perlis (1) addresses a question of major importance for treatment matching of patients with major depressive disorder (MDD): Can we identify patients seeking treatment for depression that are unlikely to respond to currently available treatments? The ability to prospectively identify patient subgroups based on their predicted outcomes especially early in the course of treatment is likely to reduce the inordinate delays in identifying the correct treatment for patients, reduce morbidity and mortality, shorten suffering, reduce rates of treatment dropouts, and likely reduce the cost burden for the patient and society (2,3). The author also suggests that early identification of treatment resistant depression (TRD) could direct patients to more complicated treatments more rapidly. The author used data available from the STAR*D sample and developed a model that is designed to address these issues. The STAR*D sample was divided into two groups for the analysis: remitters from step 1 combined with remitters from step 2 and compared with nonremitters from step 2. This sample was then divided into three cohorts: a development cohort (n = 1571), a testing cohort (n = 523), and validation cohort (n = 461). The model was developed using a combination of expert clinical judgment and an automated process using the development cohort. The final model includes a series of sociodemographic, clinical illness, and severity characteristics. To understand the usefulness of the model, it is worthwhile to examine not only the reported accuracy (.71) but also the sensitivity and specificity. In other words, how many of those with TRD are correctly identified as having TRD (sensitivity = 26%), and how many of those without TRD are correctly identified as not having TRD (specificity = 91%). The model described by Perlis is thus better at identifying those without TRD than those with TRD.
Strengths Creation of Training, Testing, and Validation Samples from the STAR*D Sample In general it is ideal to assess how well a classifier performs as a predictor in a separate data set. However, using a subset of the STAR*D sample as was done by Perlis is reasonable to test validation of the predictor model. Selection of The Initial Set of Predictors Based on the Author’s Knowledge/Experience Rather than Using the Data to Select Predictors This approach has the advantage of simplicity, selecting predictors that are readily available in practice, and selecting From the Department of Psychiatry, University of Texas Southwestern Medical Center, Dallas, Texas. Address correspondence to Madhukar H. Trivedi, M.D., Department of Psychiatry, University of Texas Southwestern Medical Center, 5323 Harry Hines Boulevard, Dallas, TX 75390-9119; E-mail: madhukar. [email protected]. Received and accepted May 14, 2013.
0006-3223/$36.00 http://dx.doi.org/10.1016/j.biopsych.2013.05.009
predictors that make sense clinically but has the disadvantage that important predictors could be overlooked. Focusing on the Preemptive Identification of Patients With MDD who are Likely to be Treatment Resistant Results from this and future efforts could lead to clinically matching patients with more complicated treatments (e.g., combination pharmacotherapy, combination pharmacotherapy, and depression focused psychotherapy or electroconvulsive therapy) rapidly to achieve the desired outcomes.
Limitations Limitations of Predictive Ability Prediction of both response and nonresponse would be best if it can be associated with specific treatments (which was not done in this case). If this is not possible, the extent to which they are related to the underlying biology of depression might be the next best alternative (which was also not possible in this data set). Need for Subsequent Testing in a New Sample The model will need to be tested in a subsequent new sample to control for characteristics that were unique to the STAR*D study. The STAR*D sample has unique disadvantages because it was designed as a large pragmatic clinical trial and was thus sparse in terms of detailed clinical phenotyping of the population under study. In addition, the clinical utility is negatively affected by the lack of information about the approximately 37% that were excluded from the analysis. The Elements That Went Into the Prediction Model (With the Exception of Quick Inventory of Depressive Symptomatology Subscale Scores and Some Illness Characteristics) are not Easily Subject to Intervention In contrast, prediction models for risk factors for cardiovascular disease, diabetes, and other chronic disorders identify factors that can either be targeted for treatment or chosen for the development of new treatments. Furthermore, if biological based markers can be associated with treatment resistance, they may provide better guidance for the development of more specific novel treatments for TRD. Modest Clinical Utility in its Current Form The calculator provided gives the probability of TRD. The presentation is useful clinically when a histogram shows the subject’s place within the sample and is also available for several samples (male subjects, female subjects, etc.). However, the results showing a 56.7% probability of TRD requires careful interpretation by the clinician. Even with this model from a practical clinical perspective the strength of the prediction is relatively weak. Also, there would need to be a consideration of the potential for harm or risk to a patient identified as TRD who was not TRD and to a patient identified as not TRD who was TRD. Models with relatively weak prediction may still be useful in situations that result in BIOL PSYCHIATRY 2013;74:2–4 & 2013 Society of Biological Psychiatry
Commentary recommendations for closer monitoring for the development of the disorder (e.g., the risk for developing breast cancer). In contrast, in cases in which classification is likely to lead to recommendations for a potentially more complicated treatment with either greater risk or burden, the need for accuracy is greater.
What Is the Next Step? The timely selection of the best treatment for patients with TRD is critical to the goal of improving remission rates. To accomplish this goal, we need to identify differential predictors of positive response as well as treatment resistance. Because of the biological heterogeneity and variable symptom presentation of depression, it is unlikely that only clinical markers can guide treatment selection even accounting for all the domains of clinical assessments. Rather, a biosignature developed from a careful exploration of a group of several clinical and biological markers is more likely to be successful. Furthermore, predictors are unable to assist with selection of specific treatments but are purely general predictors of clinical status that are unlikely to be clinically useful. Kraemer (4) has suggested that we need to look for moderators and mediators of treatment response as opposed to simple
BIOL PSYCHIATRY 2013;74:2–4 3 predictors. As shown in Figure 1, presence of a moderator would indicate that the patient should be treated with treatment B, whereas the presence of a predictor only indicates that the patient is less likely to respond to treatment, but it does not help in treatment selection. As demonstrated by Perlis, to date we have been most successful at identifying negative predictors but not moderators or mediators. What is sorely needed are two types of biosignatures to achieve improved outcomes: 1) biosignatures to maximize the selection of optimal treatment for individual patients at beginning of treatment (moderators) and 2) biosignatures to identify indicators of eventual outcomes early in treatment (mediators). This approach has great potential to personalize treatment (thus rapidly identifying the most appropriate treatment) and to begin to characterize the biology of treatment response (thus informing future drug development). In addition, there is a growing body of evidence that more complicated treatments that have been shown to be efficacious in TRD may not be significantly more efficacious early in the course of treatment, for example, combination antidepressants as first step treatments (5) or combination of pharmacotherapy plus depression focused psychotherapy (6). Finally, these models developed to predict the various aspects of the course of illness in MDD
Figure 1. Example of differential remission patterns expected with presence of a moderator and predictor.
www.sobp.org/journal
4 BIOL PSYCHIATRY 2013;74:2–4 including treatment outcomes will need to be tested prospectively in new cohorts to be able to change clinical practice.
Summary The approach outlined by Perlis has many strengths; however, it does have some limitations for its ability to guide treatment selection and immediate clinical utility. This approach is characterized by the simplicity of the variable selection and prediction model. An electronically available calculator as developed by the author, if successful, is a particular strength. Future efforts will likely lead to the development of more complex approaches in which the results from Perlis could serve as a benchmark against which to compare results. Dr. Trivedi is or has been an advisor/consultant to the following: Abbott Laboratories, Inc., Akzo (Organon Pharmaceuticals Inc.), Alkermes, AstraZeneca, Axon Advisors, Bristol-Myers Squibb Company, Cephalon, Inc., Cerecor, Concert Pharmaceuticals, Inc., Eli Lilly & Company, Evotec, Fabre Kramer Pharmaceuticals, Inc., Forest Pharmaceuticals, GlaxoSmithKline, Janssen Global Services, LLC, Janssen Pharmaceutica Products, LP, Johnson & Johnson PRD, Libby, Lundbeck, Meade Johnson, MedAvante, Medtronic, Merck, Naurex, Neuronetics, Otsuka Pharmaceuticals, Pamlab, Parke-Davis Pharmaceuticals, Inc., Pfizer Inc., PgxHealth, Phoenix Marketing Solutions, Rexahn Pharmaceuticals, Ridge Diagnostics, Roche Products Ltd., Sepracor, SHIRE Development, Sierra, SK Life and Science, Sunovion,
www.sobp.org/journal
Commentary Takeda, Tal Medical/Puretech Venture, Targacept, Transcept, Vivus, and Wyeth-Ayerst Laboratories. In addition, he has received research support from the Agency for Healthcare Research and Quality, Corcept Therapeutics, Inc., Cyberonics, Inc., National Alliance for Research in Schizophrenia and Depression, National Institute of Mental Health, National Institute on Drug Abuse, Novartis, Pharmacia & Upjohn, Predix Pharmaceuticals (Epix), and Solvay Pharmaceuticals, Inc. 1. Perlis RH (2013): A clinical risk stratification tool for predicting treatment resistance in major depressive disorder. Biol Psychiatry 74:7–14. 2. Fava M, Nelson JC, Thase ME, Trivedi MH (2009): Easing the burden of treatment-resistant depression. J Clin Psychiatry 70:273–280. 3. Rush AJ, Trivedi MH, Wisniewski SR, Nierenberg AA, Stewart JW, Warden D, et al. (2006): Acute and longer-term outcomes in depressed outpatients who required one or several treatment steps: A STAR*D report. Am J Psychiatry 163:1905–1917. 4. Kraemer HC, Wilson GT, Fairburn CG, Agras WS (2002): Mediators and moderators of treatment effects in randomized clinical trials. Arch Gen Psychiatry 59:877–883. 5. Rush AJ, Trivedi MH, Stewart JW, Nierenberg AA, Fava M, Kurian BT, et al. (2011): Combining Medications to Enhance Depression Outcomes (CO-MED): Acute and long-term outcomes: A single-blind randomized study. Am J Psychiatry 168:689–701. 6. Kocsis JH, Gelenberg AJ, Rothbaum BO, Klein DN, Trivedi MH, Manber R, et al. (2009): Cognitive behavioral analysis system of psychotherapy and brief supportive psychotherapy for augmentation of antidepressant nonresponse in chronic depression: The REVAMP trial. Arch Gen Psychiatry 66:1178–1188.