Toward a Level Playing Field: Predictive Factors for the Outcomes of Mental Health Treatment for Adolescents

SPECIAL

ARTICLE

Toward a Level Playing Field: Predictive Factors for the Outcomes of Mental Health Treatment for Adolescents SUSAN D. PHILLIPS, M.S.W., MICHAEL B. HARGIS, B.A., TERESA L. KRAMER, PH.D., SHELLY Y. LENSING, M.S., J. LYNN TAYLOR, M.D., BARBARA J. BURNS, PH.D., AND JAMES M. ROBBINS, PH.D.

ABSTRACT Objective: To understand better the effectiveness of routine treatment for emotional and behavioral problems experienced by adolescents, methods are needed to control for between-provider differences in the distribution of factors that adversely affect treatment success. Such methods are necessary to fairly compare providers’ outcomes and to aid clinicians in identifying adolescents for whom routine care may need to be altered. As a preliminary step toward developing a model to adjust treatment outcomes to account for predictive factors, findings from studies of treated samples of adolescents were reviewed to identify the factors that influence the likelihood of treatment success for this population. Method: Medline and PSYCInfo databases were searched for studies of treated adolescents that reported the association between expert-nominated predictive factors and outcomes. Thirty-four studies met inclusion criteria. Results: Significant predictors identified in these studies include diagnosis, baseline severity of symptoms and functional impairment, family dysfunction, and previous treatment. Several expert-nominated factors have not been adequately studied in treated samples. Conclusions: Much basic work is needed before a convincing body of empirical evidence can explain predictive factors for adolescent mental health treatment outcomes. Future efforts should determine a reduced set of predictive factors that can be measured with minimal burden to providers. J. Am. Acad. Child Adolesc.

Psychiatry, 2000, 39(12):1485–1495. Key Words: case mix, risk adjustment, outcomes, prognosis, adolescents.

Treatment outcomes are used to rank the relative performance of mental health treatment providers, to establish reimbursement and capitation rates, and as part of accreditation standards (Burns et al., 1999; Dewan and Carpenter, 1997; Smith et al., 1997). An unresolved methodological problem, however, limits the acceptability of using treatment outcomes for these important purposes. The problem is that differences in outcomes between groups of patients are not explained solely by differences in the quality of treatment they receive. When Accepted July 24, 2000. From the Center for Applied Research and Evaluation, Department of Pediatrics (Ms. Phillips, Mr. Hargis, Dr. Robbins, Ms. Lensing), and Center for Outcomes Research and Effectiveness, Department of Psychiatry (Drs. Kramer and Taylor), University of Arkansas for Medical Sciences, Little Rock; the Services Effectiveness Research Program, Department of Psychiatry and Behavioral Sciences, Duke University, Durham, NC (Dr. Burns); and Centers for Youth and Families, Little Rock, AR (Dr. Taylor). This work was supported by NIMH grant RO1 MH57887. Reprint requests to Dr. Robbins, Center for Applied Research and Evaluation, 4301 West Markham, Slot 512-26, Little Rock, AR 72205; e-mail: RobbinsJamesM @exchange.uams.edu. 0890-8567/00/3912-1485䉷2000 by the American Academy of Child and Adolescent Psychiatry.

patients present for an episode of care, they bring factors with them that influence their likelihood of treatment success. Differences between treatment outcomes across providers are due to differences in the distribution of these predictive factors among patients under their care as well as differences in the treatment patients receive (Burnam, 1996; Daley and Schwartz, 1994). When outcomes are compared without adjusting for prognostic factors, providers who treat a greater number of patients with characteristics associated with limited treatment success may appear to perform more poorly than providers treating fewer patients with these characteristics. Hendryx et al. (1999) demonstrated that adjusting treatment outcomes to account for differences in predictive factors alters the relative outcomes of mental health providers. In a study of public mental health clinics treating adults, severe diagnoses (e.g., major depression, bipolar disorder, schizophrenia), greater functional impairment, substance abuse, being older at intake, and lower quality of life at baseline predicted poorer functioning at followup and lower patient satisfaction. When these outcomes were adjusted to account for identified predictive factors,

J . A M . A C A D . C H I L D A D O L E S C . P S YC H I AT RY, 3 9 : 12 , D E C E M B E R 2 0 0 0

1485

PHILLIPS ET AL.

the ranking of the mental health clinics was different from rankings based on unadjusted outcomes. Methods for controlling for differences in the distribution of predictive factors among treated groups are well developed for medical and surgical care (Iezzoni, 1997). An early approach classified patients into diagnosticrelated groups (DRGs) that were thought to be homogeneous with respect to the resources required to provide treatment. Over time the definitions of these groups have been refined to improve the fairness of reimbursement to providers who treat sicker and generally more expensive patients within a given DRG (Holley et al., 1994; Iezzoni, 1997; Rigaud and Newman, 1990). In addition to applications in cost prediction and utilization review, methods of organizing patients into relatively homogeneous groups are also used in treatment planning, quality assurance, program evaluation, and provider profiling (Rigaud and Newman, 1990). Ultimately, understanding the factors that predict poorer treatment outcomes can aid providers in improving treatment. Compared with medical and surgical care, predictive factors for behavioral health care are not as well understood and methods for accounting for differences in the distribution of these factors are not as clearly defined. As a first step in developing a method to adjust outcomes of mental health care for adolescents, an extensive literature review was conducted to identify factors that affect the likelihood of treatment success for this population. This review was conducted in the context of developing the Adolescent Treatment Outcomes Module (ATOM) (Robbins and Taylor, 1998). The ATOM is a coordinated set of brief, self-administered instruments designed to measure multiple domains of outcome relevant to adolescent mental health treatment. For the ATOM to be of value in monitoring the outcomes of routine clinical care, a method was needed to adjust outcomes to account for between-group differences in the distribution of predictive factors. Two earlier reviews of predictors of treatment outcomes were identified. These, however, were based primarily on studies of children younger than 12 and focused exclusively on inpatient treatment (see Blotcky et al., 1984; Pfeiffer and Strzelecki, 1990). Because there is evidence of age-specific differences in the course of disorders (American Academy of Child and Adolescent Psychiatry, 1997) and treatment effectiveness (Weisz et al., 1995), these reviews may not accurately identify the factors affecting treatment outcomes for adolescents. This review was conducted to 1486

enable us to understand better the relevant predictive factors for adolescents. METHOD Search Strategy A panel of experts in child and adolescent mental health services research, psychotherapy research, and psychiatric epidemiology proposed a list of factors which, based on their research and clinical experience, they suspected influenced the outcomes of mental health treatment for adolescents. Factors identified by the panel included (1) psychiatric and substance abuse comorbidity, (2) severity of symptoms and behavioral problems, (3) age at onset of symptoms, (4) prior psychiatric hospitalization, (5) gender, (6) parental history of mental illness or substance abuse, (7) history of residential instability or multiple school placements, (8) family dysfunction, (9) extreme poverty, (10) history of physical or sexual abuse, and (11) exposure to violence at home or in the community. A list was compiled of words and phrases synonymous with each factor. Five common adolescent disorders were targeted: (1) major depressive disorder, (2) conduct disorder, (3) oppositional defiant disorder, (4) attention deficit disorders (ADD/ADHD), and (5) substance use disorders. Using PSYCInfo and Medline databases, we conducted a search for studies published between 1970 and 1999 that reported the effects of these predictive factors on outcomes for clinical samples of adolescents with targeted diagnoses. Searches were also conducted for more broadly defined terms (e.g., prognosis, outcomes) and for additional terms and factors that were identified during the course of the search. Each article identified through this process, as well as 10-year reviews and practice parameters published by the American Academy of Child and Adolescent Psychiatry (see, for example, American Academy of Child and Adolescent Psychiatry, 1997, and Birmaher et al., 1996), were screened for references to other studies for potential inclusion. Inclusion Criteria Studies were included in this review if (1) data were analyzed separately for adolescents aged 12 to 18, (2) there was a discernible episode of care, and (3) a relationship was reported between a factor present prior to or at the time the adolescent presented for treatment and a clinical or functional outcome. Excluded from review were studies of children and adults, studies of untreated samples of adolescents, and epidemiological or community-based samples. Also excluded from review were studies that assessed outcomes based on clinical judgment using nonstandardized instruments for which interrater reliability had not been established. Using these criteria, we excluded from the review many well-known epidemiological studies of the course of emotional and behavioral problems in children and adolescents. While factors that predict the course of mental disorders in the community are important in understanding the etiology and prevalence of disorders, it is not known whether these predictive factors apply equally to the outcomes of treated populations. Methods of the Review Of an initial pool of more than 300 articles, only 34 met inclusion criteria. Information about study design, sample size, diagnoses, racial and gender composition, treatment modality, and outcomes were abstracted from each study. Factors that were studied in relation

J . A M . A C A D . C H I L D A D O L E S C . P S YC H I AT RY, 3 9 : 12 , D E C E M B E R 2 0 0 0

TOWARD A LEVEL PLAYING FIELD

to outcomes were coded as “positive” when significantly ( p ⬍ .05) associated with better outcomes, “negative” when significantly associated with poorer outcomes, or “not significant” ( p ⬎ .05). A note was made of the outcome with which each factor was associated (e.g., better school functioning, persistence of symptoms, legal problems as an adult) and the diagnostic characteristics of the sample or subsample to which a given finding pertained. If outcome measurement occurred at discharge or the end of an episode of treatment, the follow-up period was labeled “immediate.” If the follow-up period was 1 year or less, studies were labeled “short-term” and if the period between baseline and follow-up was greater than 1 year, studies were labeled “long-term.” Evidence of effect included (1) significant regression coefficients or correlations between a factor and change in clinical or functional status, (2) increased or decreased odds of improvement given the presence of a prognostic factor, and (3) t tests or χ2 analysis demonstrating significant differences in clinical or functional status at follow-up between groups with and without a prognostic characteristic. Variables that were not present at or before baseline (e.g., relationship with peers at discharge) were not considered prognostic factors nor were elements of the treatment process (e.g., length of stay).

RESULTS Description of Studies

Of the 34 studies reviewed, the majority (76%) were published in the past decade. Studies typically used quasi-experimental designs (87%) to study adolescents receiving outpatient (42%), inpatient (35%), residential (13%), and day treatment (7%). Sufficient information to determine the specific type of treatment (e.g., cognitive-behavioral treatment, medication) was available in fewer than one third of the studies. Forty-eight percent of the studies reported outcomes for a period greater than 1 year, 17% reported short-term outcomes, 22% reported outcomes immediately after treatment, and 13% reported outcomes at multiple periods. The age range of subjects fell exclusively between the ages of 12 and 18 in 39% of the studies. Forty-six percent of the studies had samples with a mean age of 12 to 18 years but also included some subjects younger than age 12. An additional 14% included some subjects exceeding age 18. The majority of subjects (ⱖ60%) were male in 26% of the studies, the majority were female in 22%, and gender distribution was approximately equal (i.e., one gender did not equal or exceed 60% of the sample) in 30% of the studies. Gender distribution could not be determined in the remaining 22%. White subjects were predominant (ⱖ60%) in 75% of the studies that reported the racial composition of the sample. Fifty percent of the studies recruited subjects with heterogeneous

diagnoses. These were typically adolescents with various externalizing disorders or serious emotional disturbances (SED). Studies of samples with homogeneous diagnoses included studies of adolescents with depression (35%) or substance abuse disorders (15%). Conspicuously absent were studies of homogeneous groups of adolescents treated for ADHD. Studies of clinical samples of subjects with ADHD found during this review typically included patients presenting for treatment between the ages of 6 and 12 that were followed into adolescence rather than presenting for treatment during adolescence. These studies did not meet inclusion criteria. Predictive Factors

Based on their research and clinical experience, experts identified factors they expected to be important in predicting the likelihood of treatment success (see above). Additional factors were identified as studies were reviewed. These factors were (1) diagnosis, (2) duration of illness, (3) age at intake, (4) family structure, and (5) cognitive functioning. Results are presented in tabular form accompanied by a brief narrative summary. Meta-analysis of published studies was not conducted because such an analysis requires the computation of the effect of a given prognostic factor across multiple studies and it was rare that more than 3 studies addressed an identical prognostic factor over an identical outcome. Results are presented in tables that describe the specific prognostic factors examined in a given study, clinical and/or functional outcomes studied in relationship to a given prognostic factor, the diagnostic characteristics of the study sample or pertinent subsample, sample size, follow-up period, and treatment modality. For purposes of presentation, prognostic factors are organized into 4 categories: (1) clinical characteristics at presentation (e.g., diagnosis, comorbidity, severity); (2) clinical history (e.g., age at onset, previous treatment, duration of illness); (3) adolescent characteristics (e.g., age at presentation, gender, cognitive ability); and (4) family characteristics (e.g., family structure, family dysfunction, family members’ psychological and social functioning). A more explicit description of findings for individual factors is available upon request. Clinical Characteristics

Diagnosis. Studies consistently reported significant relationships between diagnosis and outcomes (Table 1).

J . A M . A C A D . C H I L D A D O L E S C . P S YC H I AT RY, 3 9 : 12 , D E C E M B E R 2 0 0 0

1487

1488 Diagnoses

Varied

Conduct disorder Substance abuse Disruptive disorders

King et al. (1997)

Protracted history of antisocial behavior; episodes of running away Pretreatment acting-out behaviors in 4 or more categories Lifetime no. of CD symptoms Baseline no. of drugs

Residual BDI scores K-SADS-P Depression at 1 year (DISC-R) in multivariate analysis Depression at 1 year based on best-estimate diagnosis Presence of CD after resolution of depression Presence of other disorders but not depression at follow-up Presence of depression or other disorders Presence of depression at follow-up Impairment at discharge; criminality, antisocial personality, and alcohol abuse at 1 year

Recurrence of depression (K-LIFE) Recurrence of depression RADS scores at follow-up Failure to achieve clinical remission at end of acute treatment Persistence of depression (K-SADS) Duration of presenting episode (K-SADS) Suicide, prison, school expulsion, welfare, alcohol or drug abuse, admission for mental health treatment, unemployed Lack of reliable change in CBCL scores Type of placement during follow-up and functioning Commission of crimes during follow-up Posttreatment substance use

Depression Depression Depression Depression Depression Depression Behavior disorders

Acting out Substance abuse/CD

Behavior disorders

Change in BDI scores Persistence of depression (DISC-R)

Depression Depression

Severity Indicators Related to Poorer Outcomes

Depression Depression MDD, CD, MDD w/CD

Depression/CD Depression/OCD Depression/CD

Baseline severity (BDI) Global functioning (SAICA) and school problems at intake Clinician, parent, and self-report depressive symptoms CDRS-R RADS scores at baseline Greater hopelessness Greater cognitive distortion Higher depression scores Severity (CGAS & SAICA) Severity of initial depressive episode Higher baseline scores on CBCL total and delinquency subscales

Depression Depression Depression Depression Depression Depression

Depression/anxiety Depression/anxiety Depression/anxiety Depression/substance abuse Depression/CD Depression/ODD

Comorbid Diagnoses Related to Poorer Outcomes

Recurrence of depression Adaptive functioning Readmission Time in prison, not attending school, receiving welfare, living with sexual partner, hospitalized for mental health, unemployed, abusing drugs Higher number of living changes after hospitalization More peer/spare-time problems Change in psychiatric symptoms and functioning

89

134

206

50 100 38

98 89 107

59

61 67

131

Long-term

6–54 months

Immediate

Immediate Immediate Long-term

Long-term Short-term Immediate

Long-term

Immediate Short-term

Immediate & long-term

Residential

Residential

Parent education

Outpatient Outpatient & inpatient Day treatment

Outpatient Inpatient Outpatient

Inpatient

Outpatient Outpatient & inpatient

Outpatient

Outpatient & inpatient Outpatient

Long-term Short-term 65 68

Outpatient

Immediate

Outpatient Outpatient Outpatient & inpatient

Inpatient

Short-term

Immediate Short-term Short-term

Inpatient Inpatient Residential Day treatment

Treatment Modality

Long-term Long-term Long-term Long-term

Follow-up

61 34 67

95

89

59 100 184 38

N

Note: BDI = Beck Depression Inventory; CBCL = Child Behavior Checklist; CD = conduct disorder; CDRS-R = Children’s Depression Rating Scale-Revised; CGAS = Children’s Global Assessment Scale; DISC-R = Diagnostic Interview Schedule for Children-Revised; K-LIFE = Kiddie Longitudinal Interval Follow-up Evaluation; K-SADS-P = Schedule for Affective Disorders and Schizophrenia for School-Age Children, Present Episode version; MDD = major depressive disorder; ODD = oppositional defiant disorder; RADS = Reynolds Adolescent Depression Scale; SAICA = Social Adjustment Inventory for Children and Adolescents; SED = serious emotional disturbance.

Crowley et al. (1998)

Nielson et al. (1982)

Ruma et al. (1996)

Jayson et al. (1998) McCauley et al. (1993) Rey et al. (1998)

Emslie et al. (1998) King et al. (1997) Brent et al. (1998)

Emslie et al. (1997a)

Clarke et al. (1992) Sanford et al. (1995)

Harrington et al. (1991)

McCauley et al. (1993) Goodyer et al. (1997a)

Clarke et al. (1992) Ryan et al. (1986) Sanford et al. (1995)

Varied

Depression Varied SED Varied

Psychosis Psychosis Thought disorder Conduct disorder

Emslie et al. (1997a) Gossett et al. (1983) Brown & Greenbaum (1994) Rey et al. (1998)

Target & Fonagy (1994)

Outcome

Diagnoses Significantly Related to Poorer Outcomes Relative to Other Diagnoses

Prognostic Factor

TABLE 1 Clinical Characteristics Significantly Related to Outcomes PHILLIPS ET AL.

J . A M . A C A D . C H I L D A D O L E S C . P S YC H I AT RY, 3 9 : 12 , D E C E M B E R 2 0 0 0

100 McCauley et al. (1993)

Better academic functioning (CBCL school-functioning scales)

Ghuman et al. (1989) Place et al. (1985)

Related to Better Outcomes

Crowley et al. (1998)

Parmelee et al. (1995)

Preadolescent CD Myers et al. (1995)

Note: BDI = Beck Depression Inventory; CBCL = Child Behavior Checklist; CD = conduct disorder; SED = serious emotional disturbance.

Depression Later age of onset

Outpatient & inpatient

Inpatient Outpatient Immediate Immediate 113 206 Placement in residential treatment Symptoms and family functioning Varied Varied

Long-term

Inpatient Short-term 90 No. of postdischarge placements

Residential Long-term 89

SED

Inpatient Long-term 131

Onset of CD symptoms before age 10 Previous psychiatric hospitalization Previous inpatient treatment Previous treatment

Outpatient Immediate 61 Later age of onset Clarke et al. (1992)

Related to Poorer Outcomes BDI scores in hierarchical block Depression regression analysis Correlated with alcohol use but not CD with substance abuse drug use at follow-up Commission of crimes Substance use with CD

Follow-up Diagnoses

Outcome

N

J . A M . A C A D . C H I L D A D O L E S C . P S YC H I AT RY, 3 9 : 12 , D E C E M B E R 2 0 0 0

Prognostic Factor

TABLE 2 Characteristics of Clinical History Significantly Related to Outcomes

For adolescents with heterogeneous disorders (Brown and Greenbaum, 1994; Gossett et al., 1983) and depressed adolescents (Emslie et al., 1997a), poorer outcomes were predicted by the presence of psychotic symptoms. Diagnoses of either conduct or substance abuse disorder predicted poorer outcomes than did diagnoses of affective, attention deficit, or oppositional defiant disorders (King et al., 1997; Rey et al., 1998; Target and Fonagy, 1994). Diagnoses of substance abuse also predicted poorer outcomes than did diagnoses of eating disorders (King et al., 1997). Psychiatric or Substance Abuse Comorbidity. Nine studies reported that comorbidity was not significantly related to outcomes (Crowley et al., 1998; Emslie et al., 1997a,b, 1998; Goodyer et al., 1997a; Harrington et al., 1991; Jayson et al., 1998; McCauley et al., 1993; Sanford et al., 1995). Five studies, however, reported a poorer recovery from depression among depressed adolescents with comorbid anxiety (Clarke et al., 1992; Ryan et al., 1986; Sanford et al., 1995), substance abuse (Sanford et al., 1995), conduct disorder (Goodyer et al., 1997a), or obsessivecompulsive disorder (Goodyer et al., 1997a) (Table 1). Other studies found that depressed adolescents who also had a diagnosis of conduct disorder (Harrington et al., 1991; McCauley et al., 1993) or oppositional defiant disorder (Goodyer et al., 1997a) were more likely than adolescents with only a diagnosis of depression to have impaired functioning or symptoms of behavioral disorders at follow-up. Baseline Symptom Severity and Functional Impairment. Severe symptoms and functional impairment at baseline were commonly associated with poorer treatment outcomes in studies of adolescents with depression (Brent et al., 1998; Clarke et al., 1992; Emslie et al., 1997a, 1998; Jayson et al., 1998; King et al., 1997; McCauley et al., 1993; Sanford et al., 1995), substance abuse (Crowley et al., 1998), and behavior disorders (Nielson et al., 1982; Rey et al., 1998; Ruma et al., 1996) (Table 1). Emslie et al. (1998), however, found that while severe depressive symptoms at baseline predicted poorer clinical outcomes, severe functional impairment did not. Age at Onset. There is mixed evidence about the prognostic importance and direction of effect of the age at onset of symptoms (Table 2). Two studies found that the early onset of conduct disorder symptoms predicted higher rates of alcohol use (Myers et al., 1995) and crime (Crowley et al., 1998) at follow-up. By comparison, multiple studies failed to find a significant relationship between

Treatment Modality

TOWARD A LEVEL PLAYING FIELD

1489

PHILLIPS ET AL.

the age at onset of depressive symptoms and clinical outcomes (Brent et al., 1998; Emslie et al., 1997a, 1998; McCauley et al., 1993; Sanford et al., 1995). There are 2 exceptions: (1) in multivariate, but not bivariate analysis, Clarke et al. (1992) found that the later onset of depressive symptoms predicted poorer clinical outcomes, and (2) McCauley et al. (1993) found that the later onset of depressive symptoms predicted better academic outcomes. Previous Treatment. Studies that examined the relationship between previous treatment and outcomes were limited to studies of adolescents with heterogeneous disorders (Ghuman et al., 1989; Parmelee et al., 1995; Place et al., 1985). These studies were consistent in reporting that previous treatment predicted poorer outcomes (Table 2). Duration of Illness. Four studies failed to find a significant relationship between the duration of illness and clinical outcomes in depressed adolescents (Brent et al., 1998; Emslie et al., 1998; Goodyer et al., 1997b; Jayson et al., 1998). Adolescent Characteristics

Gender. Three studies (McCauley et al., 1993; Ryan et al., 1986; Sanford et al., 1995) found that, for depressed adolescents, being female predicted poorer clinical outcomes. Five other studies, however, failed to find a significant relationship between being female and the outcome of depression (Emslie et al., 1997b, 1998; Goodyer et al., 1997b; Harrington et al., 1991; Jayson et al., 1998). Six studies of adolescents with substance abuse disorders or SED also failed to find a significant difference between females and males in either clinical of functional outcomes (Borduin et al., 1995; Brown and Greenbaum, 1994; Friedman et al., 1995; Gossett et al., 1983; Ralph and McMenamy, 1996; Turner et al., 1986). However, 2 studies reported that females are less likely than males to be incarcerated (Brown and Greenbaum, 1994) or have school problems (Knapp et al., 1991) at follow-up. Age at Intake. Evidence suggests that the age at which adolescents present for treatment may affect outcomes differently in different diagnostic groups. Four studies reported poorer outcomes for older adolescents with depression (Emslie et al., 1997a, 1998; Jayson et al., 1998; Sanford et al., 1995), although studies note that age was not a significant predictor of response to antidepressant medication (Emslie et al., 1997b; Ryan et al., 1986). In contrast, being older at intake was associated with better outcomes for adolescents with substance 1490

abuse disorders (Brown et al., 1994; Friedman et al., 1995; Ralph and McMenamy, 1996). For adolescents with heterogeneous disorders, being older was related to lower rates of readmission (Brown and Greenbaum, 1994) but did not predict placement (Turner et al., 1986) or incarceration (Brown and Greenbaum, 1994). Cognitive Ability. Most studies found that cognitive ability was not a significant predictor of adaptive functioning (Gossett et al., 1983), incarceration (Brown and Greenbaum, 1994), recurrence of depression (McCauley et al., 1993), or improvement in psychiatric symptoms or functioning (Friedman et al., 1995; Target and Fonagy, 1994). A study of adolescents with heterogeneous disorders, however, found that lower cognitive ability predicted higher rates of readmission following residential treatment (Brown and Greenbaum, 1994) (Table 3). Family Characteristics and Negative Life Events

Family Structure. Two studies that examined the relationship between family structure and treatment outcomes (Table 4) reported that living with parents or relatives at intake was significantly and positively related to clinical and functional outcomes (Clarke et al., 1992; Parmelee et al., 1995). Clinical outcomes, however, did not vary significantly as a function of caregiver structure (e.g., 2-parent, 1-parent, relative or out-of-home placement) (King et al., 1997). Family Functioning. Eight studies reported a significant relationship between family dysfunction and outcomes. These studies operationalized family dysfunction in diverse ways (Table 4). In studies of adolescents with depression, worse global functioning predicted poorer outcomes (Emslie et al., 1998; Goodyer et al., 1997b; King et al., 1997; McCauley et al., 1993) as did lower father–child involvement, father–child conflict, and poor response to mother’s discipline (King et al., 1997; Sanford et al., 1995). In studies of adolescents with substance abuse disorders, a greater number of problems with mothers predicted poorer clinical outcomes (Friedman et al., 1995). For SED adolescents, poorer clinical outcomes were significantly associated with lower family cohesion or adaptability (Brown and Greenbaum, 1994). Family Members’ Psychological and Social Functioning. As was the case with family functioning, investigators used a variety of measures to assess family members’ psychological and social functioning (Table 4). Studies of adolescents with heterogeneous disorders found that poorer clinical and functional outcomes were associated

J . A M . A C A D . C H I L D A D O L E S C . P S YC H I AT RY, 3 9 : 12 , D E C E M B E R 2 0 0 0

J . A M . A C A D . C H I L D A D O L E S C . P S YC H I AT RY, 3 9 : 12 , D E C E M B E R 2 0 0 0

Inpatient Outpatient Residential

Long-term Long-term Long-term

59 96 184

Recurrence Recurrence Readmission Readmission or incarceration

Depression Depression SED

Inpatient Outpatient

Residential

Not specified Long-term 6, 12, 24 months Long-term

108 176 142 184

Family role task and family conflict Abstinence Readmission

Substance use Substance use SED

Substance use

Older age at intake; age range of sample = 13–19 Older age at intake; age range of sample = 14–21 Older age at intake; age range of sample = 12–18 Older age at intake; age range of sample = 8–18

Ralph & McMenamy (1996)

Residential Outpatient Inpatient

Long-term Long-term Not specified 184 176 94

Incarceration Greater reduction in psychiatric symptoms Improvement in substance use, grades, expulsion, legal difficulties Drug use and CBCL

Related to Better Outcomes

Outpatient & inpatient

Short-term

67

Depression symptoms at 1 year

a

Note: CBCL = Child Behavior Checklist; K-SADS = Schedule for Affective Disorders and Schizophrenia for School-Age Children; SED = serious emotional disturbance. Indicates trend.

Brown & Greenbaum (1994)

Brown et al. (1994)

Outpatient & inpatient Outpatient & inpatient Outpatient Outpatient

Treatment Modality

Immediate Short-term Short-term Immediate

Follow-up

100 67 34 50

N

Depression

SED Substance use Substance use

Friedman et al. (1995)

Outcome

Related to Poorer Outcomes Longer duration of initial episode Depression Persistence of depression symptoms at 1 year Depression K-SADS Depression Persistence of symptoms Depression

Female Female Female

Female Female Female Older age at intake; age range of sample = 10–17 Older age at intake; age range of sample = 13–19 Older age at intake; age range of sample = 8–17 Older age at intake; age range of sample = 8–17 IQ; arithmetic score; reading grade level

Diagnoses

Brown & Greenbaum (1994) Friedman et al. (1995) Knapp et al. (1991)

Brown & Greenbaum (1994)

Emslie et al. (1998)

Emslie et al. (1997a)

Sanford et al. (1995)

McCauley et al. (1993) Sanford et al. (1995)a Ryan et al. (1986)a Jayson et al. (1998)

Prognostic Factor

TABLE 3 Adolescent Characteristics

TOWARD A LEVEL PLAYING FIELD

1491

1492 Living with parents Father’s higher occupational status Residing in home of parent or relative

Outpatient Residential

Immediate Long-term

206 184

Symptoms and family functioning Readmission

Varied SED

Related to Better Outcomes Posttreatment depression Depression Change in Problem Substance abuse Severity Index Placement, substance use, court Varied involvement

Inpatient

Outpatient Immediate 95

Change in psychiatric symptoms and functioning

Varied

Immediate

Inpatient Short-term 89

RADS scores following hospitalization

Varied

90

Inpatient Short-term 280

Problem Severity Index

Substance use

Outpatient Inpatient

Outpatient Long-term 176

Immediate Short-term

Outpatient Outpatient Immediate Short-term 107 49

BDI scores Child, mother, and interviewer rating of outcome Reduction in substance use

Depression Depression or anxiety

61 280

Outpatient Long-term 98

Recurrence of depression (K-LIFE)

Depression

Substance use

Outpatient & inpatient

Short-term

67

Treatment Modality Outpatient & inpatient Outpatient & inpatient Outpatient

Follow-up Immediate Immediate Short-term

N 100 100 68

Outcome

Related to Poorer Outcomes Shorter time to recurrence Depression Psychosocial competence (CBCL) Depression Less than 2 symptoms of depression Depression or other diagnosis Persistence of depression Depression

Diagnoses

Note: BDI = Beck Depression Inventory; CBCL = Child Behavior Checklist; FACES-III = Family Adaptability and Cohesion Evaluation Scale; FAD = Family Assessment Device; FGAS = Family Global Assessment Scale; K-LIFE = Kiddie Longitudinal Interval Follow-up Evaluation; RADS = Reynolds Adolescent Depression Scale; SAICA = Social Adjustment Inventory for Children and Adolescents; SED = serious emotional disturbance; SES = socioeconomic status.

Parmelee et al. (1995)

Clarke et al. (1992) Dobkin et al. (1998)

Brown & Greenbaum (1994)

Place et al. (1985)

Target & Fonagy (1994)

King et al. (1997)

Dobkin et al. (1998)

Friedman et al. (1995)

Brent et al. (1998) Goodyer et al. (1991) Greater no. of problems w/mother Father less oriented toward personal growth Poorer global functioning (FAD); low father involvement (SAICA) Past suicide attempts by mother; father has antisocial history Natural mother in care; father has criminal history Family cohesion or adaptability (FACES-III)

Low father involvement; father–child conflict; poor response to mother’s discipline Global family functioning (FGAS) Maternal depression Maternal distress

Sanford et al. (1995)

Emslie et al. (1998)

Higher SES Family environment Family functioning (FAD)

McCauley et al. (1993) McCauley et al. (1993) Goodyer et al. (1997b)

Prognostic Factor

TABLE 4 Family Characteristics Significantly Related to Outcomes

PHILLIPS ET AL.

J . A M . A C A D . C H I L D A D O L E S C . P S YC H I AT RY, 3 9 : 12 , D E C E M B E R 2 0 0 0

TOWARD A LEVEL PLAYING FIELD

with evidence of past and present maternal psychological impairment (King et al., 1997; Target and Fonagy, 1994) and paternal criminality (Place et al., 1985; Target and Fonagy, 1994). Maternal distress and maternal depressive symptoms also predicted short-term (Brent et al., 1998; Goodyer et al., 1991) but not long-term clinical outcomes (McCauley et al., 1993) in depressed adolescents. Four studies failed to find a significant relationship between the presence of mental illness in any first-degree family member and clinical outcomes (Brown and Greenbaum, 1994; Emslie et al., 1997a; Jayson et al., 1998; Turner et al., 1986). Poverty. Indicators of family socioeconomic status were often reported in describing samples, but few studies reported the relationship between these variables and outcomes (Table 4). Of the studies that did report the relationship between socioeconomic status and outcomes, 3 reported a nonsignificant relationship (Clarke et al., 1992; Jayson et al., 1998; King et al., 1997) and 2 reported significant relationships, but in opposing directions (Dobkin et al., 1998; McCauley et al. 1993). Residential Instability. The relationship between residential instability and outcomes was not examined in any studies in this review. Mundy et al. (1989), however, found that in an inpatient population, residential instability was associated with a set of characteristics that included caregiver neglect, caregiver abuse, parental separation, multiple hospitalizations, lower IQ, indices of poor impulse control, and antisocial behavior. Physical or Sexual Abuse, Domestic Violence, and Community Violence. Evidence indicates that childhood victimization and exposure to domestic or community violence influence the incidence of mental health problems in children and adolescents (Farrell and Bruce, 1997; Kaufman, 1996; Schwab-Stone et al., 1995; Walker, 1979). None of the studies meeting criteria for this review examined the relationship between these factors and treatment outcomes. Summary

Multiple studies confirm that differences in treatment outcomes are explained in part by differences that exist among adolescents when they present for treatment. Severity of symptoms and degree of functional impairment at baseline are consistently associated with poorer treatment response. Treatment success is also influenced by the presence of psychosis. Adolescents with diagnoses of conduct disorder or substance abuse are less respon-

sive to treatment than those with depression, attention deficit disorder, or oppositional defiant disorder. There is evidence that treatment for depression is less successful among youths with comorbid substance abuse, conduct disorder, or oppositional defiant disorder, but otherwise there is little convincing evidence that psychiatric comorbidity reduces treatment effectiveness. Early age at onset predicts poorer outcomes among adolescents with conduct disorders, but among depressed adolescents age at onset is not generally related to outcomes. A limited number of studies also show that duration of illness is not a significant predictor of treatment success. The influence of age and gender on treatment outcomes varies by diagnosis. Among substance-abusing or SED adolescents, males are no more or less likely to benefit from treatment than females. Depressed males, however, tend to achieve better outcomes than depressed females. Similarly, being older at intake is associated with poorer outcomes for adolescents with depression but better outcomes for adolescents with substance abuse disorders and heterogeneous disorders. Poorer treatment outcomes are also predicted by a number of family characteristics including family dysfunction, maternal psychological impairment, paternal criminality, and not living with family or relatives at intake. DISCUSSION

This review identifies clinical, adolescent, and family characteristics related to differential treatment success for adolescents with emotional and behavioral problems. While we note factors likely to be important in leveling the field upon which treatment outcomes are compared, throughout our analysis we were struck by the limited evidence base. Much basic work is needed before a convincing body of empirical evidence can explain predictive factors for adolescent mental health treatment outcomes. Many factors considered by our expert panel to be important to the success of treatment have not been adequately studied in treated samples. For instance, there is evidence that residential instability, childhood physical or sexual abuse, neglect, and exposure to domestic or community violence are associated with a higher incidence of emotional and behavioral problems in community samples. The relationship between these factors and outcomes of mental health treatment was not addressed in the studies in our review. Similarly, while certain factors were significant predictors of treat-

J . A M . A C A D . C H I L D A D O L E S C . P S YC H I AT RY, 3 9 : 12 , D E C E M B E R 2 0 0 0

1493

PHILLIPS ET AL.

ment success for a single disorder, it is not known how these factors might relate to outcomes for adolescents with other disorders. Furthermore, the most commonly investigated outcome in the reviewed studies was symptom change. Mental health professionals, however, target multiple domains of outcome. It is not known whether factors found to influence improvement in one domain of outcome (e.g., symptoms) also influence other domains (e.g., functioning, family burden). Relationships between suspected prognostic factors and outcomes of treatment for adolescents should be further explored across a range of disorders common to youths as well as a range of outcomes targeted by mental health treatment. In addition to establishing an adequate evidence base, successful models for adjusting treatment outcomes will require solutions to central methodological questions. Selecting specific factors for adjustment models will involve balancing the predictive accuracy of models against the feasibility of data collection (Iezzoni, 1997). Outcome adjustment models based on a set of simple, easily collected factors may be preferred over more complex models that create an additional burden for providers even when the more complex model optimizes predictive accuracy. Furthermore, the prognostic factors studied in this review may be highly correlated. A brief set of factors measured reliably with minimal burden to providers may capture the majority of the variation in preexisting differences in patient populations. Data on large samples of treated adolescents are needed to determine this reduced set of crucial factors. Limitations

The importance of a number of the prognostic factors proposed by our panel of experts was not supported by this review. The significance of the proposed prognostic factors may not have been detected because studies did not meet statistical criteria for power of a test. Many studies were based on small samples and may not have offered a fair assessment of the factors. Some factors may have been embedded in findings of studies and, despite rigorous efforts to scour all relevant findings, may have been missed. The findings presented in this review are based on studies with considerable differences in diagnoses, outcomes, follow-up periods and analytic strategies. The small number of studies and large differences across them precluded formal meta-analysis. A majority of the 1494

reviewed studies were based on adolescents receiving more intensive levels of treatment (e.g., inpatient, residential, day treatment). Other than this broad distinction between types of treatment, few studies provided sufficient information about the specifics of treatment to compare the effect of prognostic factors among differently treated groups of adolescents. In addition, the majority of studies in which racial or ethnic composition could be determined were studies of predominantly white subjects; therefore, it was not possible to determine the importance of race on the findings we report. Clinical Implications

It is increasingly common for outcomes to be considered in judging the quality of care provided by a treatment program or provider. This study demonstrates that factors other than treatment influence the outcomes of care. Considerable work remains if we are to adequately understand the relative merits, applications, and tradeoffs associated with models for adjusting outcomes to account for these preexisting factors. Statistically accurate models are needed to make fair between-provider comparisons, but these models may also provide information that can improve treatment. Adolescents with preexisting clinical, personal, or familial characteristics that limit treatment success present a unique challenge to clinicians. Identifying these factors gives providers the opportunity to tailor treatments to youths who may require unconventional approaches or more intensive care. An accepted methodology for adjusting outcomes will also be valuable in designing third-party reimbursement rates consistent with the level of care required for adolescents with different levels of need. REFERENCES American Academy of Child and Adolescent Psychiatry (1997), Practice parameters for the assessment and treatment of children and adolescents with conduct disorder. J Am Acad Child Adolesc Psychiatry 36:122S–139S Birmaher B, Ryan ND, Williamson DE et al. (1996), Childhood and adolescent depression: a review of the past 10 years, part I. J Am Acad Child Adolesc Psychiatry 35:1427–1439 Blotcky MJ, Dimperio TL, Gossett JT (1984), Follow-up of children treated in psychiatric hospitals: a review of studies. Am J Psychiatry 141:1499–1507 Borduin CM, Mann BJ, Cone LT et al. (1995), Multisystemic treatment of serious juvenile offenders: long-term prevention of criminality and violence. J Consult Clin Psychol 63:569–578 Brent DA, Kolko DJ, Birmaher B et al. (1998), Predictors of treatment efficacy in a clinical trial of three psychosocial treatments for adolescent depression. J Am Acad Child Adolesc Psychiatry 37:906–914 Brown EC, Greenbaum PE (1994), Reinstitutionalization after discharge from residential mental facilities: competing risks survival analysis. In: The 12th Annual Research Conference Proceedings, A System of Care for

J . A M . A C A D . C H I L D A D O L E S C . P S YC H I AT RY, 3 9 : 12 , D E C E M B E R 2 0 0 0

TOWARD A LEVEL PLAYING FIELD

Children’s Mental Health: Expanding the Research Base, Liberton CJ, Newman C, Kutash K, Friedman RM, eds. Tampa: University of South Florida, The Louis de la Parte Florida Mental Health Institute, Research and Training Center for Children’s Mental Health, pp 271–276 Brown SA, Myers MG, Mott MA, Vik PW (1994), Correlates of success following treatment for adolescent substance abuse. Appl Prev Psychol 3:61–73 Burnam MA (1996), Measuring outcomes of care for substance use and mental disorders. In: Using Client Outcomes Information to Improve Mental Health and Substance Abuse Treatment: New Directions of Mental Health Services, Steinwachs DM, Flynn M, eds. San Francisco: JosseyBass, pp 3–17 Burns BJ, Hoagwood K, Mrazek PJ (1999), Effective treatment for children and adolescents. Clin Child Fam Psychol Rev 2:199–254 Clarke G, Hops H, Lewinsohn PM, Andrews J, Seeley JR, Williams J (1992), Cognitive-behavioral group treatment of adolescent depression: prediction of outcome. Behav Ther 23:341–354 Crowley TJ, Mikulich SK, MacDonald M, Young SE, Zerbc GO (1998), Substance-dependent, conduct disordered adolescent males: severity of diagnosis predicts 2-year outcome. Drug Alcohol Depend 49:225–237 Daley J, Schwartz M (1994), Developing risk-adjustment methods. In: Risk Adjustment for Measuring Health Care Outcomes, Iezzoni L, ed. Ann Arbor, MI: Health Administration Press, pp 279–330 Dewan NA, Carpenter D (1997), Performance measurement in healthcare delivery systems. In: Annual Review of Psychiatry, Vol 16, Dickstein LJ, Riba MB, eds. Washington, DC: American Psychiatric Press, pp V81–V102 Dobkin PL, Chabot L, Maliantovitch K, Craig W (1998), Predictors of outcome in drug treatment of adolescent inpatients. Psychol Rep 83:175–186 Emslie GJ, Rush JA, Weinberg W et al. (1998), Fluoxetine in children and adolescents with depression: acute and maintenance treatment. Depress Anxiety 7:32–39 Emslie GJ, Rush JA, Weinberg W, Gullion C, Rintelmann J, Hughes C (1997a), Recurrence of major depressive disorder in hospitalized children and adolescents. J Am Acad Child Adolesc Psychiatry 36:785–792 Emslie GJ, Rush JA, Weinberg W et al. (1997b), A double-blind, randomized, placebo-controlled trial of fluoxetine in children and adolescents with depression. Arch Gen Psychiatry 54:1031–1037 Farrell AD, Bruce SE (1997), Impact of exposure to community violence on violent behavior and emotional distress among urban adolescents. J Clin Child Psychol 26:2–14 Friedman AS, Terras A, Kreisher C (1995), Family and client characteristics as predictors of outpatient outcome for adolescent drug abusers. J Subst Abuse 7:345–356 Ghuman HS, Jayaprakish S, Saidel DH, Witmarsh G (1989), Variables predictive of the need for placement in a long-term structured setting for adolescents. Psychiatr Hosp 20:31–34 Goodyer IM, Germany E, Gowrusankur J, Altham P (1991), Social influences on the course of anxious and depressive disorders in school-aged children. Br J Psychiatry 158:676–684 Goodyer IM, Herbert J, Secher SM, Pearson J (1997a), Short-term outcome of major depression, I: comorbidity and severity at presentation as predictors of persistent disorder. J Am Acad Child Adolesc Psychiatry 36:179–187 Goodyer IM, Herbert J, Tamplin A, Secher S, Pearson J (1997b), Short-term outcome of major depression, II: life events, family dysfunction, and friendship difficulties as predictors of persistent disorder. J Am Acad Child Adolesc Psychiatry 36:474–480 Gossett JT, Lewis JM, Barnhart FD (1983), To Find a Way: The Outcome of Hospital Treatment of Disturbed Adolescents. New York: Brunner/Mazel Harrington R, Fudge H, Rutter M, Pickles A, Hill J (1991), Adult outcomes of childhood and adolescent depression, II: links with antisocial disorders. J Am Acad Child Adolesc Psychiatry 30:434–439 Hendryx MS, Dyck DG, Srebnik D (1999), Risk adjusted outcome models for public mental health outpatient programs. Health Serv Res 34:171–195 Holley HL, Kulczycki G, Arbolda-Florez J (1994), Case-mix funding and legislated psychiatric care. Int J Law Psychiatry 17:377–393

Iezzoni LI (1997), Dimensions of risk. In: Risk Adjustment for Measuring Healthcare Outcomes, Iezzoni LL, ed. Chicago: Health Administration Press, pp 43–103 Jayson D, Wood A, Kroll L, Fraser J, Harrington R (1998), Which depressed patients respond to cognitive behavioral treatment? J Am Acad Child Adolesc Psychiatry 37:35–39 Kaufman J (1996), Child abuse. Curr Opin Psychiatry 9:251–256 King CA, Hovey JD, Brand E, Ghaziuddin N (1997), Prediction of positive outcomes for adolescent psychiatric inpatients. J Am Acad Child Adolesc Psychiatry 36:1434–1442 Knapp JE, Templer DI, Cannon WG, Dobson S (1991), Variables associated with success in an adolescent drug treatment program. Adolescence 26: 305–317 McCauley E, Myers K, Mitchell J, Calderon R, Schloredt K, Treder R (1993), Depression in young people: initial presentation and clinical course. J Am Acad Child Adolesc Psychiatry 32:714–722 Mundy P, Robertson J, Greenblatt M, Robertson M (1989), Residential instability in adolescent inpatients. J Am Acad Child Adolesc Psychiatry 28:176–181 Myers MG, Brown SA, Mott MA (1995), Preadolescent conduct disorder behaviors predict relapse and progression of addiction for adolescent alcohol and drug abusers. Alcohol Clin Exp Res 19:1528–1536 Nielson G, Young D, Latham S (1982), Multiply acting out adolescents: developmental correlates and response to secure treatment. Int J Offender Ther Comp Criminology 26:195–206 Parmelee D, Cohen R, Nemil M, Best AM, Cassell S, Dyson F (1995), Children and adolescents discharged from public psychiatric hospitals: evaluation of outcome in a continuum of care. J Child Fam Stud 4:43–55 Pfeiffer SI, Strzelecki SC (1990), Inpatient psychiatric treatment of children and adolescents: a review of outcome studies. J Am Acad Child Adolesc Psychiatry 29:847–853 Place M, Framrose R, Wilson C (1985), The difficult adolescents who are referred to a psychiatric unit, II: clinical features and response to treatment. Adolescence 8:307–320 Ralph N, McMenamy C (1996), Treatment outcomes in an adolescent chemical dependency program. Adolescence 31:91–107 Rey JM, Denshire E, Wever C (1998), Three-year outcome of disruptive adolescents treated in a day program. Eur Child Adolesc Psychiatry 7: 42–48 Rigaud MS, Newman FL (1990), A case mix procedure for matching the clinical characteristics of patients with dual disorders to planned treatment and treatment costs. J Ment Health Adm 17:200–206 Robbins JM, Taylor JL (1998), Testing the Adolescent Treatment Outcomes Module (ATOM). In: The 1998 Behavioral Outcomes and Guidelines Sourcebook, Coughlin KM, ed. New York: Faulkner and Gray, pp 217–222 Ruma PR, Burke RV, Thompson RW (1996), Group parent training: is it effective for children of all ages? Behav Ther 27:159–169 Ryan ND, Puig-Antich J, Cooper T et al. (1986), Imipramine in adolescent major depression: plasma level and clinical response. Acta Psychiatr Scand 73:275–288 Sanford M, Szatmari P, Spinner M et al. (1995), Predicting the one-year course of adolescent depression. J Am Acad Child Adolesc Psychiatry 34:1618–1628 Schwab-Stone ME, Ayers TS, Kasprow W et al. (1995), No safe haven: a study of violence exposure in an urban community. J Am Acad Child Adolesc Psychiatry 34:1343–1352 Smith GR, Maderscheid RW, Flynn LM, Steinwachs DM (1997), Principles for assessment of patient outcomes in mental health care. Psychiatr Serv 48:1033–1036 Target M, Fonagy P (1994), The efficacy of psychoanalysis for children: prediction of outcome in a developmental context. J Am Acad Child Adolesc Psychiatry 33:1134–1144 Turner TH, Dossetor DR, Bates RE (1986), The early outcome of admission to an adolescent unit: a report on 100 cases. Adolescence 9:367–382 Walker L (1979), The Battered Woman. New York: Harper & Row Weisz JR, Weiss B, Han SS, Granger DA, Morton T (1995), Effects of psychotherapy with children and adolescents revisited: a meta-analysis of treatment outcome. Psychol Bull 117:450–468

J . A M . A C A D . C H I L D A D O L E S C . P S YC H I AT RY, 3 9 : 12 , D E C E M B E R 2 0 0 0

1495