Journal of Affective Disorders 174 (2015) 281–286
Contents lists available at ScienceDirect
Journal of Affective Disorders journal homepage: www.elsevier.com/locate/jad
Research report
Mobile health monitoring to characterize depression symptom trajectories in primary care Paul N. Pfeiffer a,b,n, Kipling M. Bohnert a,b, Kara Zivin a,b, Matheos Yosef b, Marcia Valenstein a,b, James E. Aikens c, John D. Piette a,d,e a
Center for Clinical Management Research, VA Ann Arbor Health System, Ann Arbor, MI, USA Department of Psychiatry, University of Michigan Medical School, Ann Arbor, MI, USA c Department of Family Medicine, University of Michigan Medical School, Ann Arbor, MI, USA d Department of Health Behavior and Health Education, School of Public Health, University of Michigan, Ann Arbor, MI, USA e Department of Internal Medicine, University of Michigan Medical School, Ann Arbor, MI, USA b
art ic l e i nf o
a b s t r a c t
Article history: Received 12 November 2014 Accepted 20 November 2014 Available online 28 November 2014
Background: Classification of depression severity can guide treatment decisions. This study examined whether using repeated mobile health assessments to determine symptom trajectories is a potentially useful method for classifying depression severity. Methods: 344 primary care patients with depression were identified and recruited as part of a program of mobile health symptom monitoring and self-management support. Depression symptoms were measured weekly via interactive voice response (IVR) calls using the Patient Health Questionnaire (PHQ-9). Trajectory analysis of weekly IVR PHQ-9 scores from baseline through week 6 was used to subgroup patients according to similar trajectories. Multivariable linear regression was used to determine whether the trajectories predicted 12-week PHQ-9 scores after adjusting for baseline and 6-week PHQ-9 scores. Results: The optimal trajectory analysis model included 5 non-intersecting trajectories. The subgroups of patients assigned to each trajectory had mean baseline PHQ-9s of 19.7, 14.5, 9.5, 5.0, and 2.0, and respective mean decreases in PHQ-9s over six weeks of .3, 2.0, 3.6, 2.3, and 1.9. In regression analyses, each trajectory significantly predicted 12-week PHQ-9 scores (using the modal trajectory as a reference) after adjusting for both baseline and 6-week PHQ-9 scores. Limitations: Treatment history was unknown, findings may not be generalizable to new episodes of treatment. Conclusions: Depression symptom trajectories based on mobile health assessments are predictive of future depression outcomes, even after accounting for typical assessments at baseline and a single follow-up time point. Approaches to classify patients' disease status that involve multiple repeated assessments may provide more accurate and useful information for depression management compared to lower frequency monitoring. Published by Elsevier B.V.
Keywords: Depression Symptom trajectories Mobile health monitoring
1. Introduction Practice guidelines and mounting evidence support the use of standardized outcome measures to guide depression care (American Psychiatric Association (APA), 2010). Measurement-based care is a core component of collaborative care for depression, a depression care model that has been shown to improve depression outcomes for primary care patients in dozens of clinical trials (Archer et al., 2012; Gilbody et al., 2006; Woltmann et al., 2012). In fact, the use of
n Corresponding author at: North Campus Research Complex, 2800 Plymouth Road, Ann Arbor, MI 48109, United States. Tel.: þ1 734 845 3645. E-mail address:
[email protected] (P.N. Pfeiffer).
http://dx.doi.org/10.1016/j.jad.2014.11.040 0165-0327/Published by Elsevier B.V.
measurement-based care may allow patients to receive the same quality of care and outcomes in primary care settings as can be achieved in specialty mental health care settings (Gaynes et al., 2008; Ong and Rubenstein, 2009). Despite expert opinion and clinical trial evidence suggesting that measurement-based care should be implemented more widely into practice, there is no clear consensus on how frequently measures should be obtained and whether measurement frequency should be tailored to a patient's symptom trajectory (Valenstein et al., 2009). Clinical trials of measurement-based collaborative depression care have generally measured patients' depressive symptoms at least monthly during the acute phase of treatment and current American Psychiatric Association (APA) practice guidelines recommend assessment of treatment response every 4–8 weeks (American Psychiatric
282
P.N. Pfeiffer et al. / Journal of Affective Disorders 174 (2015) 281–286
Association (APA), 2010; Dietrich et al., 2004; Katon et al., 1996, 2010). Measurement practices in routine clinical practice are likely to occur less frequently or consistently than in research settings or than guidelines recommend; for example, in the Group Health Cooperative integrated health system, a leader in depression care, only 30% of patients who completed a baseline PHQ-9 PHQ-9 completed more than one subsequent PHQ-9 (Simon et al., 2013). Higher frequency monitoring (e.g., weekly), which can be achieved practically through the use of mobile health technologies, offers the potential advantage of allowing providers to use the trajectory of repeated measures to guide care rather than depending only upon a baseline and single follow-up measurement. Basing clinical decisions on a single follow-up assessment could result in possible misclassification of response and risk either over or under-treatment. For example, a patient whose overall trajectory indicated non-response but exhibits modest improvement on the day of the follow-up assessment could mistakenly be considered a treatment responder and not offered more intensive treatment. This study assessed the potential utility of high-frequency depression monitoring in a sample of primary care patients diagnosed with depression by their primary care provider (PCP) and referred to an automated depression care monitoring clinical service (Piette et al., 2013). Using weekly Patient Health Questionnaire (PHQ-9) (Kroenke et al., 2001) scores collected via automated telephone calls, we classified patients' depression symptom trajectories over six weeks and then tested the hypothesis that those trajectories would significantly predict PHQ-9 scores at 12 weeks after adjusting for lower-frequency assessments at baseline and six-weeks. We also compared sociodemographic and general health characteristics of patients assigned to each trajectory in order to develop hypotheses regarding determinants of differential patterns of treatment response.
2. Methods 2.1. Data source This study used data collected as part of a program designed to improve depression care by obtaining automated symptom and medication adherence assessments and providing feedback to patients, providers, and (if the patient wished) a family member or close friend. The program was implemented at 13 Midwestern academic and community-based primary care practices and has been described in detail elsewhere (Piette et al., 2013). 2.2. Study population Patients were eligible for the study if they had either a current depression diagnosis (major depressive disorder, depression not otherwise specified, or dysthymia) listed in their medical records or an antidepressant prescription plus a diagnosis of depression listed in billing records. Patients with bipolar disorder, schizophrenia, and other psychotic disorders, or dementia were excluded. Potential participants were mailed an introductory letter followed by a screening and recruitment telephone call, and 420 patients (29% of those contacted) provided informed consent and were enrolled in the study. 344 (82%) of enrolled patients completed their initial baseline assessments and were included in subsequent analyses. The study was approved by the institutional review boards of the University of Michigan and Ann Arbor VA Healthcare System. 2.3. Measurement and program procedures Depression symptoms were assessed using the PHQ-9 administered via an automated telephone-based interactive voice
response (IVR) system. IVR-based administration of the PHQ-9 correlate well with those of paper-and-pencil administration (Turvey et al., 2012). Patients received weekly IVR calls at times and days they indicated were convenient, and they responded to queries for clinical information using their touch-tone telephones. The system initiated calls to the patient at weekly intervals; patients with sub-threshold depression symptoms (PHQ-9 o10) for 4 consecutive calls were given the option of reducing call frequency to once per month. Each call queried patients using the same series of items with branching logic to provide personalized feedback based on item responses and changes in depressive symptoms from the prior call. For example, the IVR program suggested that patients consider consulting with their primary care provider if their PHQ-9 scores changed from under 15 to 15 and above. Patients who indicated suicidal ideation were asked about suicidal intent or plan and offered an automated transfer to the national suicide crisis line. Fax or e-mail alerts regarding treatment concerns (e.g., intention to discontinue antidepressant use due to side effects) were provided automatically to patients' care teams. For patients who opted to participate with an informal caregiver, the caregiver received weekly structured feedback via email regarding patients' medication adherence and depressive symptoms, along with suggestions for how they could support the patient's depression self-care. 2.4. Measures Depression symptoms were measured via IVR administration of the Patient Health Questionnaire (PHQ-9) (Kroenke et al., 2001), a validated and widely-used measure of depression severity in primary care. Total scale scores of 10 or greater have a sensitivity of .88 and a specificity of .88 for detecting a major depressive episode. Validated severity categories are minimal (0–4), mild (5–9), moderate (10–14), moderately severe (15–19) and severe (20–27) (Kroenke and Spitzer, 2002). General health status was assessed via IVR with the standard item, “Thinking about your overall health, how were you feeling this past week? Excellent, very good, good, fair, or poor?” Functional status was measured according to a baseline penand-paper administration of the Short Form Health Survey (SF-12). The SF-12 includes valid subscales for physical and mental healthrelated impairment (Ware et al., 1996). Medication adherence was assessed with the IVR prompt, “How often during the past week did you take your depression medication exactly as prescribed? Always, most of the time, less than half of the time, rarely or never?” Because the response distribution was strongly skewed towards adherence, we dichotomized this measure as always versus not always. Patient demographic characteristics were assessed at baseline via pen-and-paper survey and included age, gender, marital status (married or not), education (beyond high school or not), and income (above $30,000 per year or not). 2.5. Analyses The number of distinct symptom trajectories and estimates of weekly PHQ-9 scores for each trajectory were obtained using the SAS 9.3 software PROC TRAJ procedure (SAS Institute, Cary NC), which employs a group-based modeling approach to analyze longitudinal psychometric data (Jones et al., 2001). The trajectory analysis was based on each patient's first six weeks in the program. The outcome variable for the trajectory analysis was the PHQ-9 score, and the independent variable was week from program entry. Trajectories were modeled based only on observed outcomes without imputation of missing data. The optimal number of trajectories was determined using the Bayesian Information Criterion (BIC), with the largest
P.N. Pfeiffer et al. / Journal of Affective Disorders 174 (2015) 281–286
number indicating the best fit, and the log form of the Bayes factor value(2loge(B10)) in which a positive value for a number of trajectories provides support over fewer trajectories (Jones et al., 2001). After identification of the optimal number of trajectories, each patient was assigned to the trajectory for which their data indicated the highest probability of membership. Descriptive patient characteristics were calculated separately by trajectory subgroup and then compared using ANOVAs for continuous variables and chi-square tests for categorical variables. To determine whether trajectory subgroup was predictive of patients' subsequent symptoms, we fit a regression model with patients' 12-week PHQ-9 scores as the outcome, trajectory as the primary predictor, and baseline and 6-week PHQ-9 scores as covariates.
3. Results Of the 344 study patients, the mean age was 52.3 (SD 12.4); 79.3% were women, 81% completed post-high school education, and 66% had a household income greater than $30,000. Patients completed 68% of attempted monitoring calls, and call completion did not vary by baseline depression severity. Five trajectories were identified based on patients' PHQ-9 scores reported during the first six weeks of program participation (Fig. 1). The model with 5 trajectories had the largest BIC ( 4422.33 for 5 trajectories vs. 4435.77 for 4 trajectories and 4432.61 for 6 trajectories) and the 2loge(B10) was positive (26.88) for 5 trajectories and negative for 6 ( 20.56). The mean baseline PHQ-9 scores for the 5 trajectories were in order of increasing severity 2.0, 5.0, 9.5, 14.5, and 19.7. The respective 6-week PHQ-9 scores were .1, 2.7, 6.0, 12.5, and 19.3. We labeled these trajectories in order of increasing severity as minimal, mild, moderate, moderately severe, and severe, which corresponds to the nomenclature used for similar categories of cross-sectional PHQ-9 severity (Kroenke et al., 2001). The individual weekly PHQ-9 scores for each trajectory group are shown in Fig. 2 to depict the week-to-week individual-level variability within each group. The distribution of patients across trajectories was 19.1% for minimal, 30.6% for mild, 34.9% for moderate, 13.6% for moderately severe, and 1.8% for severe. There were significant differences across trajectories in marital status, income, mental health subscore of the SF-12, and overall rating of general health status (see Table 1). With increasing depression severity, patients were generally less likely to be married, have lower incomes, have lower mental-health related functioning, and be less likely to report their overall health as good or excellent.
Fig. 1. Weekly mean depression symptoms among 5 symptom trajectories. Dashed lines ¼ expected, solid lines ¼ observed.
283
In the multivariable regression model, patients' symptom trajectory was predictive of their 12-week PHQ-9 score, after adjusting for baseline and 6-week PHQ-9. Compared to the moderate trajectory, the beta coefficients for the other trajectories were 1.70 (95% CI: 3.37, .02; p ¼.047) for minimal, 1.49 (95% CI: 2.70, .28; p ¼.016) for mild, 4.06 (95% CI: 2.31, 5.82; p¼ o.0001) for moderately severe, and 8.66 (95% CI: 4.70, 12.6; p¼ o.001) for severe. Thus, given patients with the same baseline and 6-week PHQ-9, those whose longitudinal 6-week trajectory was severe would have on average an estimated 8.66 points greater PHQ-9 at week 12 than those whose trajectory was moderate. The coefficients for baseline and 6-week PHQ-9 score were .10 (95% CI: .02, .21; p ¼.09) and .30 (95% CI: .16, .45; po .0001) respectively. Thus, after accounting for trajectory and 6-week PHQ-9, baseline PHQ-9 was not a significant predictor of 12-week PHQ-9. For every PHQ-9 point at 6-weeks the score at 12 weeks is estimated to be .30 points higher after factoring in the trajectory and intercept of 2.35 (95% CI: .78, 3.92; po .01).
4. Discussion This study demonstrates the potential utility of characterizing depression status using high-frequency symptom data collected via mobile health technology (in this case, IVR). The trajectories identified by our analyses significantly predicted future depression symptom severity above and beyond severity measures at baseline and 6 weeks. This suggests that classification according to frequent automated assessments and empirical trajectory analysis may better identify underlying depression subgroups and may be a more accurate method for predicting patients' disease course than the standard approach using two time-point methods. Although the 5 severity trajectories were derived empirically, without any a priori determination of cut-points or total number of trajectories, the baseline mean PHQ-9s of the resulting trajectories closely resemble the severity thresholds proposed in the validation study of the PHQ-9 (Kroenke et al., 2001). Also similar to the findings from the initial validation work that established cross-sectional severity cut-points, we found significant monotonic declines in mental health functioning and overall health status with increasing severity of symptom trajectories. Consistent with prior studies of poor response to depression treatment, we also found that patients who were unmarried or with low income had greater symptom severity over six weeks (Carter et al., 2012; Trivedi et al., 2006). These findings provide evidence supporting the 5 trajectories as valid and discrete categories of severity. While cross-sectional and trajectory-based approaches may use similar categories to define severity, it is likely that the use of trajectories increase accuracy for predicting future symptoms because some patients' baseline and 6-week assessments may not be representative of their overall course (as exemplified in Fig. 2). Although high-frequency monitoring may result in more accurate symptom predictions, the stability of symptom means within trajectories suggests less frequent monitoring may suffice once a trajectory has been established, particularly for those with stable remission of symptoms or not undergoing treatment intensification. Whether improving the accuracy of characterizing patient symptoms and prognosis enables providers and patients to achieve better outcomes or more efficient care requires further study. Although the additional patient burden and technology costs associated with characterizing depression trajectories could represent challenges to adoption in the short-term, these barriers may decrease over time with greater integration of mobile devices into health care for a variety of conditions. Trajectory analysis may also be particularly well suited for characterizing depression treatment response in research settings, particularly when there may be
284
P.N. Pfeiffer et al. / Journal of Affective Disorders 174 (2015) 281–286
Fig. 2. Weekly individual Patient Health Questionnaire (PHQ-9) scores by trajectory group.
additional resources for monitoring, more acceptability from participants, and where accuracy of depression status may be critical to identifying associated patient characteristics (e.g., biomarkers). However, unless methods for measuring patients' treatment trajectory are standardized across studies (e.g., with the same frequency of measurement, depression measurement scale, analytic approach, and patient eligibility criteria) the resulting trajectory structure may vary substantially across studies. For these reasons, traditional methods of defining depression status and response remain important for monitoring patients' progress in clinical service delivery and in research settings. Study results may have been influenced by the fact that patients entered the monitoring program weeks after they had already been
exposed to treatment, potentially explaining the absence of a trajectory corresponding to a robust treatment response. Only 15.4% of patients had moderately severe to severe depression symptoms at baseline, and there was little mean change in PHQ-9 scores among these patients after 6 weeks. This is a lower rate of response than might be expected based on antidepressant medication trials for major depression (which generally require that all patients have moderate to severe symptoms and that patients be starting new treatments) (Rutherford et al., 2009) .Thus the treatment trajectories found here might apply to more stable treatment populations and populations that include those with less severe illness. The rate of persistent depression reported here more closely approximates the 20% of individuals in the general population who did not achieve remission
P.N. Pfeiffer et al. / Journal of Affective Disorders 174 (2015) 281–286
285
Table 1 Baseline characteristics of primary care patients with depression grouped according to 6-week symptom trajectory. Characteristics
Minimal N¼ 63
Mild N¼ 100
Moderate N¼ 116
Moderately severe N¼ 43
Severe N¼ 6
P-valuea
Baseline PHQ-9, M(S.D.) Demographic variables Female, % Age, M(S.D.) Married, % Education4high school, % Income 4$30k, %
2.0 (3.2)
5.0 (3.3)
9.5 (4.0)
14.5 (3.4)
19.7 (5.0)
o .0001
74.2 53.9 (12.3) 69.7 84.9 81.5
80.8 53.3 (13.3) 59.1 79.1 71.8
76.7 51.5 (12.2) 52.5 80.8 60.5
91.5 51.2 (11.0) 63.8 78.3 54.6
66.7 42.8 (11.6) 16.7 100.0 16.7
.16 .19 .04 .64 o .001
47.1 (10.7) 43.4 (12.6) 78.8 93.9 80.7
39.3 (11.7) 40.7 (14.1) 72.4 90.5 74.7
33.1 (10.5) 40.7 (13.9) 63.3 92.5 69.1
27.4 (10.4) 36.5 (13.4) 55.3 93.6 61.4
22.5 (14.1) 39.3 (13.3) 50.0 83.3 60.0
o .0001 .13 .04 .83 .30
Health status MCS, M(S.D.) PCS, M(S.D.) General excellent or good, % Antidepressant medication prescribed, % Adherence to medication “always as prescribed”, %
MCS ¼ mental component summary and PCS ¼physical component summary of Short Form Health Survey (SF-12). a
ANOVA for continuous variables, chi-square for dichotomous or categorical variables.
in a naturalistic epidemiologic study of major depression (Patten and Schopflocher, 2009). Contrary to longer-term surveillance studies of patients with chronic health conditions, our analyses did not identify a subgroup of patients whose depression worsened over time (Bombardier et al., 2010; Rabkin et al., 2009; Schmitz et al., 2013). This might also be explained by the fact that participants had already experienced peak symptoms and were receiving treatment at the time of program entry. The short-term follow-up of our study might also have limited the detection of patterns or worsening symptoms.
5. Limitations Diagnoses of depression were ascertained by primary care clinicians, not by standardized diagnostic interviews. While some participants may not have met more rigorous diagnostic criteria for a depressive disorder, the current data do represent real-world depression care in primary care settings. In the absence of a control group, we are unable to determine the effects of the monitoring program itself, which included feedback to patients, caregivers, and clinicians, on treatment response.
6. Conclusion In conclusion, this study demonstrates that frequent measurement of depression treatment outcomes via mobile health technology, in conjunction with trajectory analysis, could provide accurate and useful clinical information for guiding management of depression and prognosis. Whether the identified trajectories are similar across treatment populations, and whether care informed by monitoring trajectories achieves better outcomes, requires further study.
Role of funding source This study was supported by the Department of Veterans Affairs, Health Services Research and Development Service (CDA 10-036-1, CDA 11-245, and IIR 10-176), the University of Michigan Health System Faculty Group Practice, Blue Cross and Blue Shield of Michigan, and the National Institute of Mental Health (R01 MH096699). John Piette is a Senior VA Research Career Scientist.
Conflict of interest The authors have no conflicts of interest to report.
Acknowledgments This study was supported by the Department of Veterans Affairs, Health Services Research and Development Service (CDA 10-036-1, CDA 11-245, and IIR
10-176), the University of Michigan Health System Faculty Group Practice, Blue Cross and Blue Shield of Michigan (U028081-272977), and the National Institute of Mental Health (R01 MH096699). John Piette is a Senior VA Research Career Scientist.
References American Psychiatric Association (APA), 2010. Practice Guideline for the Treatment of Patients with Major Depressive Disorder Third Edition. 28. Archer, J., Bower, P., Gilbody, S., Lovell, K., Richards, D., Gask, L., Dickens, C., Coventry, P., 2012. Collaborative care for depression and anxiety problems. Cochrane Database Syst. Rev. 10, CD006525. Bombardier, C.H., Fann, J.R., Temkin, N.R., Esselman, P.C., Barber, J., Dikmen, S.S., 2010. Rates of major depressive disorder and clinical outcomes following traumatic brain injury. JAMA 303, 1938–1945. Carter, G.C., Cantrell, R.A., Zarotsky, V., Haynes, V.S., Phillips, G., Alatorre, C.I., Goetz, I., Paczkowski, R., Marangell, L.B., 2012. Comprehensive review of factors implicated in the heterogeneity of response in depression. Depress. Anxiety 29, 340–354. Dietrich, A.J., Oxman, T.E., Williams, J.W., Schulberg, H.C., Bruce, M.L., Lee, P.W., Barry, S., Raue, P.J., Lefever, J.J., Heo, M., Rost, K., Kroenke, K., Gerrity, M., Nutting, P.A., 2004. Re-engineering systems for the treatment of depression in primary care: cluster randomised controlled trial. Br. Med. J. 329, 602–605. Gaynes, B.N., Rush, A.J., Trivedi, M.H., Wisniewski, S.R., Balasubramani, G.K., McGrath, P.J., Thase, M.E., Klinkman, M., Nierenberg, A.A., Yates, W.R., Fava, M., 2008. Primary versus specialty care outcomes for depressed outpatients managed with measurement-based care: results from STARnD. J. Gen. Intern. Med. 23, 551–560. Gilbody, S., Bower, P., Fletcher, J., Richards, D., Sutton, A.J., 2006. Collaborative care for depression:a cumulative meta-analysis and review of longer-term outcomes. Arc.Intern. Med. 166, 2314–2321. Jones, B.L., Nagin, D.S., Roeder, K., 2001. A SAS procedure based on mixture models for estimating developmental trajectories. Sociol. Method Res. 29, 374–393. Katon, W., Robinson, P., Von Korff, M., Lin, E., Bush, T., Ludman, E., Simon, G., Walker, E., 1996. A multifaceted intervention to improve treatment of depression in primary care. Arch. Gen. Psychiatry 53, 924–932. Katon, W.J., Lin, E.H., Von Korff, M., Ciechanowski, P., Ludman, E.J., Young, B., Peterson, D., Rutter, C.M., McGregor, M., McCulloch, D., 2010. Collaborative care for patients with depression and chronic illnesses. N. Engl. J. Med. 363, 2611–2620. Kroenke, K., Spitzer, R.L., 2002. The PHQ-9: a new depression diagnostic and severity measure. Psychiatr. Ann. 32, 509–515. Kroenke, K., Spitzer, R.L., Williams, J.B.W., 2001. The PHQ-9: validity of a brief depression severity measure. J. Gen. Intern. Med. 16, 606–613. Ong, M.K., Rubenstein, L.V., 2009. Wishing upon a STARnD: the promise of ideal depression care by primary care providers. Psychiatr. Serv. 60, 1460–1462. Patten, S.B., Schopflocher, D., 2009. Longitudinal epidemiology of major depression as assessed by the Brief Patient Health Questionnaire (PHQ-9). Compr. Psychiatry 50, 26–33. Piette, J.D., Aikens, J.E., Trivedi, R., Parrish, D., Standiford, C., Marinec, N.S., Striplin, D., Bernstein, S.J., 2013. Depression self-management assistance using automated telephonic assessments and social support. Am. J. Manag. Care 19, 892–900. Rabkin, J.G., McElhiney, M., Moran, P., Acree, M., Folkman, S., 2009. Depression, distress and positive mood in late-stage cancer: a longitudinal study. Psychooncology 18, 79–86. Rutherford, B.R., Sneed, J.R., Roose, S.P., 2009. Does study design influence outcome? The effects of placebo control and treatment duration in antidepressant trials. Psychother. Psychosom. 78, 172–181.
286
P.N. Pfeiffer et al. / Journal of Affective Disorders 174 (2015) 281–286
Schmitz, N., Gariepy, G., Smith, K.J., Malla, A., Wang, J., Boyer, R., Strychar, I., Lesage, A., 2013. The pattern of depressive symptoms in people with type 2 diabetes: a prospective community study. J. Psychosom. Res. 74, 128–134. Simon, G.E., Rutter, C.M., Peterson, D., Oliver, M., Whiteside, U., Operskalski, B., Ludman, E.J., 2013. Does Response on the PHQ-9 Depression Questionnaire predict subsequent suicide attempt or suicide death? Psychiatr. Serv. 64, 1195–1202. Trivedi, M.H., Rush, A.J., Wisniewski, S.R., Nierenberg, A.A., Warden, D., Ritz, L., Norquist, G., Howland, R.H., Lebowitz, B., McGrath, P.J., Shores-Wilson, K., Biggs, M.M., Balasubramani, G.K., Fava, M., Team, S.S., 2006. Evaluation of outcomes with citalopram for depression using measurement-based care in STARnD: implications for clinical practice. Am. J. Psychiat. 163, 28–40.
Turvey, C., Sheeran, T., Dindo, L., Wakefield, B., Klein, D., 2012. Validity of the Patient Health Questionnaire, PHQ-9, administered through interactive-voice-response technology. J. Telemed. Telecare 18, 348–351. Valenstein, M., Adler, D.A., Berlant, J., Dixon, L.B., Dulit, R.A., Goldman, B., Hackman, A., Oslin, D.W., Siris, S.G., Sonis, W.A., 2009. Implementing standardized assessments in clinical care: now's the time. Psychiatr. Serv. 60, 1372–1375. Ware Jr., J., Kosinski, M., Keller, S.D., 1996. A 12-item Short-Form Health Survey: construction of scales and preliminary tests of reliability and validity. Med. Care 34, 220–233. Woltmann, E., Grogan-Kaylor, A., Perron, B., Georges, H., Kilbourne, A.M., Bauer, M.S., 2012. Comparative effectiveness of collaborative chronic care models for mental health conditions across primary, specialty, and behavioral health care settings: systematic review and meta-analysis. Am. J. Psychiatry 169, 790–804.