Outcomes of Mental Health Care for Children and Adolescents: I. A Comprehensive Conceptual Model

Outcomes of Mental Health Care for Children and Adolescents: 1. A Comprehensive Conceptual Model KIMBERLY HOAGWOOD, PH.D., PETER S. JENSEN, M.D., THEODORE PETTI, M.D., AND BARBARA J. BURNS, PH.D.

ABSTRACT Objective: Accountability for mental health care has become a standard of clinical practice. With the expansion of

managed care as a corporate response to health reform, attention to outcomes will intensify. Assessment of clinical treatment has typically focused on symptom reduction at an individual level, whereas assessment of service effectiveness

.

has more often targeted service-level change. Method: A dynamic and interactional model of outcomes is presented that broadens the range of intended consequences of care. The model comprises five domains: symptoms, functioning, . consumer perspectives, environmental contexts, and systems. Results: The model reflects the changeable interaction between children's evolving capacities and their primary environments (home, school, and community). Conclusions: As health care practices shift, attention to improved care is likely to depend increasingly on scientifically credible evidence of its impact. Greater integration between research and standard practice will be needed. Such a partnership can be strengthened by a more comprehensive view of the impact of care. J. Am. Acad. Child Ado/esc. Psychiatry,

1996, 35(8):1055-1063. Key Words: children and adolescents, outcomes, clinical efficacy, service effectiveness, managed care, health care.

As the structure and delivery of health care are reexamined in this country, numerous and sometimes conflicting proposals for organization and financing have been advanced. An element running consistently through these proposals is that providers will be held to high standards of accountability. Concerns about lack of accountability of health care delivery, especially mental health care delivery, are based on somber economic facts. In 1986 the annual direct cost of mental and substance abuse services in the United States was estimated to be $51.4 billion. Social costs-lost work days, associated with mental and substance abuse disorders-were estimated to add another $116 billion AcceptedJanuary 10, 1996. Drs. Hoagwood and [ensen are with National Institute ofMental Health, Rockville, MD; Dr. Petti is with the Department of Psychiatry, Indiana University, Indianapolis; and Dr. Burns is with the Department ofPsychiatry, Duke University, Durham, NC The authors thank Mary Schwab-Stone fOr her substantive comments about early drafts ofthe conceptualmodel and the members ofthe American Academy of Child and Adolescent Psychiatry Outcomes Task ForcefOr their original thinking about outcome models, from which the present one evolved. Reprint requests to Dr. Petti, 702 Barnhill Room 3701, Indianapolis, IN 46202-5200.

0890-8567/96/3508-I055$03.00/0©1996 by the American Academy of Child and Adolescent Psychiatry.

J. AM. ACAD. CHILD ADOLESC. PSYCHIATRY, 35:8, AUGUST 1996

annually (Rice et al., 1990). Estimates of the costs of care for children indicate that approximately $4.8 billion was spent for mental health services in 1990, accounting for about 7.1 % of the total mental health care expenditures in that year (Burns et al., 1990; Rice et al., 1990). Expansion of managed care to control health care costs likely will lead to even greater attention to outcomes as an index of accountability. Today, measuring outcomes and satisfaction has become mandatory in some localities for maintaining preferredprovider status (Plante et al., 1995). Currently, little is known about the appropriateness or the outcomes of health services delivered under managed-care arrangements (Burns, in press). Evidence for either effectiveness or cost-effectiveness of publicly funded services for children is equally rare. Mental and substance abuse disorders, particularly those affecting children, are included in many of the managed-market systems, and most states currently have reform initiatives or legislation to improve accountability in health care delivery (Frank et al., 1994; Sorian and Holmes, 1993). Congressional reports have identified barriers to insurance that limit coverage for mental illness, and several work groups have outlined

1055

HOAGWOOD ET AL.

options for coverage of most of the major mental disorders (Frank et al., 1994). The proposals for mental health coverage, however, are all based on the premise that treatment outcomes can be defined and measured. Given the increasing focus on accountability, it follows that clinicians, providers, and administrators are interested in precisely determining the outcomes of care delivered to children with mental disorders. Tools to assess the impact of treatment on adult patients have been developed for certain mental disorders, such as depression and panic disorder (Rost et al., 1992). Similar efforts are under way to construct modules with which to monitor outcomes for adolescents. In short, system accountability has become a way to preserve clinical practice. At the state level, calls for system reform and specifically for changes in the structure and governance of services for children are assuming a greater local focus (England and Cole, 1992). System reform in the next 5 years is likely to be driven by evidence that changes in service delivery will lead to improved outcomes for families and children. Such reform will necessitate new forms of community governance at the local level (Farrow et al., 1993; Friedman, 1995). Such shifts may lead to the provision of mental health services in nontraditional settings, such as school-based clinics, or in managed-care arrangements, such as health maintenance organizations, preferred-provider organizations, and independent practice associations (Burns, in press). Athough much has been written on the value of integrated systems of care for children, including provision of community-based, family-centered, and culturally appropriate services (Burns and Friedman, 1990; Duchnowski and Friedman, 1990; Friedman and Kurash, 1992; Stroul and Friedman, 1986), responsibility for such care is especially fragmented for children (Burns, 1991). Consequently, children with mental health problems are likely to be first identified and to receive mental health services through non-mental health systems such as schools (Burns et al., 1995) or, for very young children, through child welfare agencies. In addition, measures for assessing effectiveness of integrated care have been developed piecemeal. For example, only in the past few years has progress been consolidated in developing and refining instruments for reliably assessing childhood psychiatric disorders (Angold et aI., 1995; Angold and Costello, 1995; Gadow and Sprafkin, 1994; Jensen et al., 1995; Lahey

1056

et al., 1996; Schwab-Stone et al., 1993, 1994; Shaffer et al., 1993; Todd et al., 1996). Previously, assessment measures of mental disorders covered only limited numbers of childhood disorders; some failed to adapt to changes introduced by iterations in the DSM classification system. Beyond assessment of disorder, one among many potential areas in which to measure impact, there have been recent significant advances in measuring other aspects of childhood behavior and functioning, including symptoms (Achenbach, 1991a,b; Quay, 1986), impairment (Bird er al., 1990; Hodges et al., 1989; Shaffer et al., 1983), social competence (Dodge et al., 1985, 1986; Harter and Chao, 1992), and emotional regulation (Campos et aI., 1989; Cummings et al., 1981; Eisenberg and Fables, 1992; Thompson, 1989). But these assessments are rarely linked with other types of outcomes (Bickman, 1992) or understood with reference to the community and cultural norms or social contexts within which behavior occurs (Hohmann, 1995; Reid, 1994). This article argues that valid assessments of the effectiveness of care necessitate a conceptual and pragmatic linkage of multiple outcomes, reflecting interwoven levels of impact, i.e., changes at individual, familial, social, and systemic levels. Clinically, an interest in the outcomes for a particular child is a naturally occurring culmination of a sequence of therapeutic interventions, including assessment and evaluation of clinical status, development of a treatment plan, treatment, and aftercare. From a clinical standpoint, evaluation of the outcomes of care for patients is hardly novel. From a scientific standpoint, however, evaluation studies defining outcomes of interest beyond mere symptom reduction are surprisingly rare. For example, while well over 400 studies of the efficacy of psychotherapy for children in reducing symptomatology have been conducted and meta-analyses of their impact reveal that such treatment is effective (Kazdin, 1991; Kazdin et al., 1990; Weisz, 1987, 1992), fewer than a dozen outcome studies have investigated the effectiveness of psychotherapy outside of research settings. These studies show significantly poorer outcomes than the laboratory studies (Weisz et al., 1995). Unknown at this point is whether the intervention itself breaks down when deployed into other settings, whether the assessment of outcomes has not adequately captured the range of effects, or whether the model of the relationship between different types of outcomes

j, AM. ACAD. CHILD ADOLESC. PSYCHIATRY, 35:8, AUGUST 1996

MENTAL HEALTH OUTCOMES MODEL

has been conceptually limiting, thus limiting analysis itself (Hoagwood et al., 1995). This article presents a conceptual model of outcomes for children and families who receive care. It is developmentally grounded, dynamic, and interactional. In a companion report, we apply the model to recent outcome studies on clinical treatments and services for children (Jensen et al., 1996). MODELS OF OUTCOMES

In most studies on the impact of clinical interventions, the outcome of interest is usually a reduction in the problems or symptoms toward which the intervention was targeted. The concept of impact has more recently been surrounded by questions of context: impact of what, for whom, under what circumstances, in relationship to what goal? The advantage of an inclusive model of outcomes is that, if valid, it can help to clarify and refine those contextual relationships, as well as identify types of interventions in need of further development. Dangers exist in developing such models, however. Briefly, one danger is that interpretation of an outcome-its positive or negative valence-cannot be understood in isolation. An outcome per se is neutral. Its valence arises from the social, cultural, or historical context within which it is embedded. An attribution of value to an outcome cannot be made without concomitant interpretation about the conditions under which the outcomes are assessed, why they are assessed, and an understanding of the persons or groups for whom that outcome is salient. For example, a diagnosis can be positive or negative, depending on an individual's personal history. Is the diagnosis a culmination of years of distress, a final pathway of increasingly destructive behaviors, or does it represent a less serious and more treatable condition than in the past? The same diagnosis could have two entirely different meanings. Likewise, changes in placement-a common outcome in service effectiveness studies-can have different valences. Keeping a child at home with intensive in-home services rather than sending the child to a treatment facility could be viewed as a negative outcome by a clinician, who regards the child's functioning in the home as impaired, but as a positive outcome by an agency, which may be trying to reduce out-of-home placements. A second danger is that outcome models may overlook supraordinate contextual variables that assign

]. AM. ACAD. CHILD ADOLESC. PSYCHIATRY, 35:8. AUGUST 1996

meaning to any given outcome. As is well documented in cross-cultural studies, culture constitutes more than a source of variation; rather, it defines what ought to be examined, how basic questions ought to be conceptualized, and what findings mean within particular communities (Kleinman, 1988; Manson et al., 1987; RogIer, 1992). In this sense, culture determines the conditions of the possible apprehension and therefore study of mental phenomena (Habermas, 1988). Consequently, the intervention's impact can only be fully understood when measured against the norms, values, and social structures of the culture in which the intervention is embedded (e.g., the notion of risk and protective factors for childhood psychopathology varies enormously from one culture to another [Manson et al, 1987]). To fully understand impact, therefore, one must also understand the developmental pathways available to children from different cultures and the norms against which abnormal variation from a standard trajectory is defined. A third danger is that superimposing any static model on clinical practice has the potential to constrain and dictate what is practiced, what is valued, and what is paid for. Measurement can come to constitute its own institutional machinery and fail to reflect with sufficient depth and variety the changeableness of children's behavior and functioning. The ultimate value of any conceptual model will depend on the system of interpretation it constructs and the extent to which it facilitates rather than hinders professional practice. Adult Models

In 1989, the National Institute of Mental Health invited a group of research scientists, advocates, service providers, administrators, consumers, and their family members to develop a comprehensive strategy to improve services for adults with severe mental illness and their families. During the panel meetings, a model for outcome research was developed, based on the thinking of Hargreaves and Shumway (1989), that encompassed four domains of outcomes: clinical (categorical and dimensional symptoms and their severity); rehabilitative (functional adaptability, rather than illness, and social skills and competencies); humanitarian (perceptions of well-being, consumer satisfaction, and quality of life); and public safety (destructive behaviors). Discussion of outcomes for children was not included in the panel meetings because a national plan outlining a research

1057

HOAGWOOD ET AL.

agenda for children-the National Plan for Research on Child and Adolescent Mental Disorders (National Institute of Mental Health, 1990)-already existed; this plan was based on the highly influential report by the Institute of Medicine (1989). This framework for adult patients with severe mental illness was an advance over earlier models because it acknowledged that not only multiple domains, but multiple perspectives, including observations of the patient, the family, the social network, and members of the patient's social sphere, needed to be operationalized (Attkisson et al., 1992). Rosenblatt and Attkisson (1993) thoughtfully expanded the model to include both respondent and setting categories, again for adults with severe mental illness. Their putpose was to provide a categorical system for classifying assessments of outcomes, not to model a typology of outcomes. Their system proposed three dimensions of measurement: (1) type of respondent (e.g., client, family members, members of the social network, clinicians, scientists); (2) type ofsocial context (e.g., family, work or school, community); and (3) type of treatment outcome (e.g., clinical status, functional status, life satisfaction and fulfillment, and safety and welfare). This model advanced the field because it included information about the provider of data, the context within which measures were taken, and treatment outcomes. Neither of the adult models, however, is easily transferable to children, largely because they do not reflect the dynamic aspect of childhood development. For children, fluctuations in symptom expression are not only common but a sine qua non of childhood (Garber et al., 1988). The period from infancy through adolescence is characterized by rapid changes in biological and psychological capacities and resources (Arnold, 1993; Cicchetti, 1989; Cicchetti and Rizley, 1981). These changes have significant implications for understanding the relative impact of biological or environmental influences on the expression of symptoms. While it is often difficult to differentiate evanescent adjustment problems from more enduring forms of mental disorder, such variations are common and have direct implications for treatment planning, necessitating an understanding of the contributions of individual versus situational factors to the onset and course of disorders.

1058

.j

Current clinical wisdom and accumulating research evidence suggest that many childhood conditions can be traced to complex interactions between psychological, social, and biological phenomena, unfolding over time (Cicchetti, 1989; Rutter, 1989; Rutter and Hersov, 1985). To reflect this complexity, an adequate understanding of the outcomes of an intervention with a given child must be couched in the context of the progressive unfolding of that child's abilities, and, simultaneously, an awareness of increasing demands made on the child by his or her environment. Without this developmental perspective, assessment of outcomes at any given point in time may unwittingly assess only the child's current state (a poor proxy for outcomes) and fail to capture the direction and velocity of the child's evolving biological, psychological, and social capacities. The actual level of a given symptom may have much less importance to a child's eventual "outcome" than subtle shifts in the direction of symptoms. Slight reductions in symptoms or small increases in social skills may bode very well for one child's later course, given a certain history for that child. For another, such modest accomplishments may not reflect any meaningful outcome. For clinicians and researchers studying children's outcomes, primary emphasis is placed on understanding the developing transactions and processes between the individual and his or her environment; this focus provides the essential context for understanding child and adolescent mental health and disorders at molecular, intrapsychic, and interpersonal levels. It requires the conceptual separation of normal from abnormal behavior, of those for whom the behavior represents an accommodation to current environmental circumstances (but for whom long-term consequences may be minimal) from those with more enduring or recurring forms of dysfunction. This focus also requires understanding and assessment of how children negotiate critical transition points (e.g., pre- to postpuberty, where data suggest risk for depression increases twofold for girls but not boys; or from preschool to schoolage, where data suggest children with attention-deficit hyperactivity disorder or separation anxiety disorder are identified as a result of the demands entailed in starting school and mastering its unique demands [Costello, 1989]). In short, these developmental differences between adults and children suggest that assessment of outcomes

]. AM. ACAD. CHILD ADOLESC. PSYCHIATRY, 35:8, AUGUST 1996

MENTAL HEALTH OUTCOMES MODEL

for children must include attention to the biological and psychological capacities of the developing child across multiple social and environmental domains and must consider the reciprocal influence of social and environmental contexts on the development of psychopathology. Just as multimodal treatment studies are important for understanding treatment effectiveness for children and adolescents (Richters et al., 1995), likewise a multimodal and dynamic conceptualization of outcomes is needed.

TABLE 1 Model of Outcomes for Children and Adolescents Domains

Symptoms and diagnoses Distractibility, impulsivity, depression, anxiety Functioning

Capacity to adapt to the demands of home, school, and community

Consumer perspectives

Quality of life, satisfaction with care, family strain or burden

Environments

Counterpoint to functioning domain: stability of child's primary environments (marital relationships at home, classroom stability, neighborhood violence, transience, availability of social supports)

Systems

Level, type, duration, or change in use of services;change in restrictiveness of services; organizational relationships and coordination; costs and mechanisms of financing

A Model of Outcomes for Children and Adolescents

We propose a dynamic model that conceptualizes outcomes broadly to include areas of impact for children within their environmental contexts. The model proposes five outcome domains that represent significant areas in which to look for evidence of impact: symptoms, functioning, consumer perspectives, environments, and systems (SFeES model). The domains are hierarchically organized to reflect ever-widening and interactional spheres of influence (Fig. 1). The five domains represent targeted foci for the types of changes that may be anticipated to result from an intervention. Table 1 summarizes the domains. The first three domains (symptomatic/diagnostic, functional, and consumer perspectives) are used to describe a specific child's or family's outcomes, while the last two domains (environments, systems) may be applied to either aggregate-level data or data relevant to a single child or family. It is assumed that careful attention must be paid to assessing outcomes in each of the various settings (e.g., home, school, community) in which the child interacts.

Examples

The settings consist of major contexts surrounding the child. The home setting refers to the environment where the child lives. For homeless or institutionalized children, assessment in this setting may be more challenging. For school-age children, the school setting refers to the setting where his or her education is received. This setting may not apply to all children, of course, as some children are educated at home, some have dropped out, etc. The community setting refers to the larger social environment in which most children function and interact, and it includes peers, neighborhoods, and churches. Attention to those settings, as part of most children's lives, is important because to a relative degree, each of these settings imposes certain tasks and challenges for the developing child. Understanding children's outcomes necessarily must take into account that functional and symptomatic outcomes may be very different across different settings. The SFCES Model

Fig. 1 Spheres of influence on child mental health outcomes in a dynamic system.

J. AM. ACAD. CHILD ADOLESC. PSYCHIATRY, 35:8, AUGUST 1996

Symptoms/Diagnoses. Symptoms are any of the emotional or behavioral symptoms a child may exhibit in one or more settings (e.g., irritability, aggression, distractibility, oppositionality, destructiveness to property, inattention, difficulties waiting in line, impulsivity, depression, anxiety, avoidance, withdrawal, bullying, etc.). When specific patterns of symptoms occur

1059

HOAGWOOD ET AL.

(type, number, duration, etc.), criteria for a formal mental disorder or diagnosis (e.g, DSM or leD) may be met. This domain captures both dimensional and categorical aspects of childhood disorders. The severity of a given disorder may be reflected in the number, type, frequency, or duration of specific symptoms, as well as the degree to which the disorder leads to impairments in functioning. Functioning. The functional domain refers to the ability ofchildren to adapt to varying demands of home, school, peer group, or neighborhood. Functioning is a continuous variable that includes competencies, on one end, and impairment on the other. Functional outcomes must be understood and assessed across home, school, and community settings. Functional competencies are the adaptational abilities of children to respond to competing demands. These capacities include those listed in Axis V of DSM and temperamental qualities (e.g., emotional regulation). For example, participating in family chores, mastering school-related tasks, or negotiating between competing peer factions in the community are instances where a child may competently adapt to the demands placed on him or her. Impairment refers to the overall degree to which children's social/emotional/behavioral symptoms or disorder(s) interfere with their adaptation to various settings. Impaired functioning in the federal definition of serious emotional disturbance means "substantially interfering with or limiting the child's role or functioning in family, school, or community activities" (Public Law 102-321). Studies have demonstrated that the presence of a diagnosis does not in itself predict a need for services (Bird et al., 1990); consequently, assessment of impairment is important in defining treatment need because it reflects the depth or intensity with which clinical disorders may affect the child's ability to negotiate contextual demands. In our model, functional impairment includes deficits that interfere with carrying out responsibilities across settings; consequently, functional outcomes could be measured through days absent from school, academic failure, or school suspensions. Health status, because it can impact severely on a child's ability to adapt, is also a functional outcome. Besides being setting-specific, functioning is also developmentally specific (e.g., level of functioning at one age could be very inappropriate at another age), although functional assessment scales have not yet

1060

demonstrated developmental validity. For example, in the home, important functional outcomes for a child of latency age might include participation in family chores, whereas for an adolescent, acceptance of and compliance with family rules might be more important. In school, functional outcomes for elementary school children might include completion of searwork, participation and contribution during group activities, and mastery of didactic material, whereas in high school, school attendance and compliance with the teacher's authority are more relevant. In the community, salient functional outcomes for younger children could include the ability to develop and enjoy one or more close friends or the ability to participate in mutually satisfying play activities, whereas for adolescents, avoidance of gang activity or engagement in constructive social activities may be important functional outcomes. In short, outcomes are constituted by the dynamic interplay between children's development over time and their primary settings. Consumer Perspectives. This domain refers to areas not traditionally assessed in psychiatric research, yet of great relevance to patients, families, and service providers. In addition to assessments by others of the child's clinical status and functional capacities, outcomes can be assessed in terms of the experience and perspectives of the consumer-i.e., the child and family. This domain encompasses the subjective experience of the consumer, and it includes constructs such as the child or family's quality of life, their satisfaction with care, and the impact, burden, or strain on the family (e.g., the effect of the child's difficulties on the family's abilities to provide for that child). Assessment of consumer perspectives is essential for understanding the relationship between seeking care and outcomes. Inclusion of consumer perspectives as a separate domain does not imply that consumers are not also concerned about the child's symptoms or functioning, but rather that this perspective deserves separate recognition in its own right. Assessment of these perspectives is an area of increasing interest to service providers. Environments. The environmental domain is a counterpoint to the functional domain, in that, while the latter is designed to address the child's functioning visa-vis home, school, and community, the environmental domain encompasses outcomes within those contextual surrounds. Environmental outcomes refer to features of children's primary settings that are modifiable. This

J.

AM. ACAD. CHILD ADOLESC. PSYCHIATRY, 35:8, AUGUST 1996

MENTAL HEALTH OUTCOMES MODEL

domain includes changes in these environments that occur as a function of interventions targeted either toward the child or toward those primary environments. For example, the impact of an intervention specifically directed toward changing a child's behavior may also lead to changes in the marital relationships (home environment), in the classroom climate (school environment), or in neighborhood violence (community environment). Conversely, an intervention may be directly targeted toward any of these environments, but it may also directly influence the behaviors or functioning of children. For instance, a classroom behavior management program may lead to improved academic performance of a child with attentional problems, even though the intervention is not aimed directly at that particular child. In other words, a reciprocity of influence between children and their contexts is presumed to be operative. Specific environmental outcomes in the home setting may include marital and family functioning, level or intensity of family stressors and disruptions, and the family's ability to set limits, to nurture, or to provide a safe environment. In the school setting, environmental variables may include classroom stability or disruption, classroom disciplinary practices, school safety, or school climate. In the community setting, environmental factors encompass variables such as gang activities, neighborhood stability or transience, or level of violence. Systems. The various components of care within a community can be organized or fragmented; however, either way, they constitute a system. Consequently, outcomes of a particular intervention for child or family may directly influence the delivery of other forms of care. For example, focal provision of mental health services within schools may affect the organization, availability, or delivery of other mental health services within the community. System outcomes include two subtypes: service-related and organizational/costrelated. Service outcomes refer to the level, type, duration, and change in service utilization or availability (e.g., in the school, mental health, primary care, child welfare, juvenile justice, or substance abuse sectors). These outcomes can include practical aspects of service delivery, frequently measured in the aggregate, such as number ofsuspensions or arrests within a community or changes in out-of-home placements, in use of inpatient hospitalization, or in length of stay.

J. AM. ACAD. CHILD ADOLESC. PSYCHIATRY, 35:8. AUGUST 1996

System outcomes also include assessments of the relationships between or among service organizations and assessments of the costs of services. For example, the relationship between two or more service sectors, such as schools and mental health agencies, or foster care and juvenile correctional agencies, can be characterized in terms of their integration, coordination, networking, or referral patterns. Recently, many state and local communities have fundamentally altered the organization of children's services. They found that introducing new services can influence the system's structure as a whole (e.g., introducing alternative services can affect health care planning and policy within localities). Assessments of financial strucrures and funding streams of provider organizations, as well as costs of specific services to families, are clearly outcomes of interest to consumers, policymakers, and financial administrators. As new services become available in communities, financial arrangements among other service-providing agencies may change (e.g., introduction of managed care may have an impact on provision of inpatient hospital care within state hopsitals). Introducing new services may also have immediate relevance for specific families, potentially introducing shifts in copayments.

DISCUSSION

The purpose of the SFCES model is to provide an interpretational scaffolding for future empirical assessments of treatment and service effectiveness. Figure 1 depicts the model. It conveys the dynamic and interactional interplay between the child and his or her environments over time. Evolving physiological, cognitive, behavioral, and emotional structures of children unfold continuously and are inextricably intertwined with children's primary environments. A dialectic is thus created. The SFCES model permits analysis of the relations within and between children and their contexts, recognizing that these relations change as the child develops. Environmental factors reciprocally influence one another and interact with the child as the child's biological, intellective, and social-emotional capacities unfold. Small variations in any of these dimensions can lead to very different outcomes over time. But within this model, such

1061

HOAGWOOD ET AL.

Arnold LE (1993), A comparative overviewof treatment research methodology: adult vs child and adolescent, psychopharmacological vs psychosocial treatments. Psychopharmacol Bull 29:5-17 Attkisson CC, Cook J, Karno M er aI. (1992), Clinical services research. Schizophr Bull 18:561-626 Bickman L (1992), Designing outcome evaluations for children's mental health services: improving internal validity. In: Evaluating Mental Health Services for Children, Bickman L, Rog D, eds. San Francisco: jesseyBass, pp 57-68 Bird H, Yager TJ, Staghezza B, Gould M, Canino G, Rubio-Stipec M (1990), Impairment in the epidemiologic measurement of childhood psychopathology in the community. ] Am Acad Child Adolesc Psychiatry 29:796-803 Burns BJ (1991), Mental health service use by adolescents in the 1970s and 1980s. ] Am Acad Child Adolesc Psychiatry 30:144-149 Burns BJ (in press), What drives outcomes for emotional and behavioral disorders in children and adolescents? In: Using Outcomes to Improve Care: Quality AssessmentofMental Health and SubstanceAbuse Treatment, Steinwachs DM, Flynn L, Norquist G, eds. San Francisco: Jossey-Bass Burns BJ, Costello EJ, AngoldA et al. (1995), The Great Smoky Mountains Study of Youth: mental health service use across the child service system. Health AffI4:147-157 Burns BJ, Friedman R (1990), Examining the research base for child mental health services and policy. ] Ment Health Admin 17:87-98 Burns BJ, Taube CA, Taube JE (1990), Mental health services for adolescents. Contract paper prepared for the Office ofTechnology Assessment, US Congress, Washington, DC, 1990. Cited in US Congress, Office of Technology Assessment (November 1991), Adolescent Health, Vol II: Background and the E./fictiveness ofSelectedPrevention and Treatment Services (OTA-H-466). Washington, DC: US Government Printing Office Campos J, Campos R, Barrett K (1989), Emergent themes in the study of emotional development and emotion regulation. Dev Psycho biol 25:394-402 Cicchetti D (1989), Developmental psychopathology: some thoughts on its evolution. Dev Psychopatholl:I-4 Cicchetti D, Rizley R (1981), Developmental perspectives on the etiology, intergenerational transmission and sequela of child maltreatment. In: New Direetiom of Child Development, Rizley R, Cicchetti D, eds. Washington, DC: jessey-Bass, pp 31-56 Cosrello EJ (1989), Developments in child psychiatric epidemiology. ] Am Acad Child Adolesc Psychiatry 28:836-841 Cummings EM, Zahn-Waxler C, Radke-YarrowM (1981), Young children's responses to expressions of anger and affection by others in the family. Child Dev 52:1274-1282 Dodge KA, McClaskey CL, Feldman EL (1985), A situational approach to the assessment of social competence in children. ] Comult Clin PsychoI53:344-353 Dodge KA, Petit GS, McClaskey CL, Brown M (1986), Social comperence in children. Monogr Soc Res Child Dev 51(2 Serial No. 213) Duchnowski A, Friedman R (1990), Children's mental health: challenges for the nineties. ] Ment Health Admin 17:3-12 Eisenberg N, Fables R (1992), Emotion, self regulation and social competence. In: Review of Personality and Social Psychology, Vol 14, Clarke M, ed. Newbury Park, CA: Sage England MJ, Cole RF (1992), Building systems of care for youth wirh serious mental illness. Hosp Community Psychiatry 43:630-633 Farrow F, Watson S, Shorr L (1993), A framework for improving outcomes for children and families. Fam Resource Coalition Rep 12:13-15 Frank RG, Sullivan MJ, DeLeon PH (1994), Healrh care reform in the states. Am PsychoI49:855-867 Friedman R (1995), Child mental health policy. In: Mental Health Services: Public Health Perspective, Levin BL, Petrilla J, eds. Oxford, England: Oxford University Press Friedman R, Kurash K (1992), Challenges for child and adolescent mental health. Health Affll:125-136 Gadow K, Sprafkin J (1994), Child Symptom Inventories Manual. Stony Brook, NY: Checkmate Plus Ltd

variation is theoretically traceable. Given additional empirical work, it can also be operationally mapped. How well does this model fit with the current status of research on outcomes? In the companion article in this series, we review studies of clinical efficacy and service effectiveness to identify the range of outcomes that have been measured, the direction of the effects, and the areas where gaps exist (Jensen et al., 1996). The literature on outcomes of care for children classifies itself into two main categories: studies of clinical efficacy for children with specific disorders and studies of the service impact for children with heterogeneous symptoms. These parallel bodies of research have arisen in relative independence from each other, creating an artificial and unproductive split. Each of these research categories is characterized by relative emphasis on certain types of outcomes. Very few studies include comprehensive assessment of outcomes across a range of domains. Transporting and replicating findings from clinical efficacy studies into service effectiveness studies and, counter-directionally, refining field-based effectiveness studies to address issues of clinical impact will be important scientific directions for the next generation of outcomes research. The conceptual model of outcomes presented here is intended to further the goal of integrating research into clinical practice. As health care practices change to reflect more localized and corporate-driven control, both practitioners and designers of service systems will be held accountable for the care they provide. The integration of research into practice is likely to become the sine qua non of quality care. The ultimate impact of clinical practice and service delivery will be judged by the ability of service planners and clinicians alike to construct a shared meaning of improved outcomes for children.

REFERENCES Achenbach TM (199Ia), Manual for the Child Behavior Checklistl4-8 and 1991 Profile. Burlington: University of Vermont Deparrment of Psychiatty Achenbach TM (199Ib), Manualfor the Youth Self-Report and 1991 profile. Burlington: University of Vermont Department of Psychiatty Angold A, Costello EJ (1995), A test-retest reliability study of childreporred symptoms and diagnoses using the Child and Adolescent Psychopathology Assessrnel:lt (CAPA-C). PsycholMed 25:755-762 Angold A, Prendergast M, Cox A, Harrington R, Simonoff E, Rutter M (1995), The child and adolescent psychopathology assessment (CAPA). Psychol Med 25:739-753

1062

J.

AM. ACAD. CHILD ADOLESC. PSYCHIATRY, 35:8, AUGUST 1996

MENTAL HEALTH OUTCOMES MODEL

Garber J, Kriss MR, Koch M, Lindholm L (1988), Recurrent depression in adolescents: a follow-up study. JAm Acad Child Adolesc Psychiatry 27:49-54 Habermas JL (l988), The Logic of the Social Sciences. Cambridge, MA: MIT Press Hargreaves WA, Shumway M (l989), Effectiveness of services for the severely mentally ill. In: NationalInstitute ofMental Health, TheFuture ofMental Health Services Research, Taube CA, Mechanic 0, Hohmann A, eds (DHHS Publication ADM 89-1600). Washington, DC: US Government Printing Office Harter S, Chao C (l992), The role of competence in children's creation of imaginary friends. Merrill-Palmer Q 38:350-363 Hoagwood K, Hibbs E, Brent 0, Jensen P (l995), Efficacyand effectiveness studies of child and adolescent psychotherapy. J Consult Clin Psychol 63:683-687 Hodges K, Cools J, McKnew 0 (l989), Test-retest reliabiliry of a clinical research interview for children: the child assessment schedule. Psychol

Assess 1:317-322 Hohmann AA (l995), Measurement sensitiviry in clinical mental health services research. In: Outcomes Assessment in Clinical Practice, Sederer L, Dickey B, eds. Baltimore: Williams & Wilkins Institute of Medicine (l989), Research on Children and Adolescents with Mental, Behavioral and Developmental Disorders. Washington, DC: National Academy Press Jensen PS, Hoagwood K, Petti T (l996), Outcomes of mental health care for children and adolescents: II. Literature review and application of a comprehensive model. JAm AcadChildAdolesc Psychiatry 35: 1064-1 077 Jensen PS, Roper M, Fisher P et al. (l995), Test-retest reliabiliry of the Diagnostic Interview Schedule for Children (DISC 2.1): parent, child, and combined algorithms. Arch Gen Psychiatry 52:61-71 Kazdin A (l991), Effectiveness of psychotherapy with children and adolescents. J Consult Clin PsychoI59:785-798 Kazdin AE, Bass 0, Ayers WA, Rodgers A (l990), Empirical and clinical focus of child and adolescent psychotherapy research. J Consult Clin

PsychoI58:729-740 Kleinman A (1988), Rethinking Psychiatry: From CulturalCategory toPersonal Experience. New York: Free Press Lahey BB, Flagg EW, Bird HR et al. (l996), The NIMH Methods for the Epidemiology of Child and Adolescent Mental Disorders (MECA) Study: background and methodology. JAm Acad ChildAdolesc Psychia-

try 35:855-864 Manson SM, Walker RD, Kivlahan DR (l987), Psychiatric assessment and treatment of American Indians and Alaska Natives. Hosp Community

Psychiatry 38:165-173 National Institute of Mental Health (1990), National PlanfOr Research on

Child and Adolescent Mental Disorders: A Report by the US Congress (DHHS Publication ADM 90-1683). Washington, DC: US Government Printing Office Plante TG, Couchman CR, Diaz AR (1995), Measuring treatment outcome and client satisfaction among children and families. J Ment Health

Admin 22:261-277 Quay HC (l986), Institutional treatment. In: Psychopathological Disorders of Childhood, 3rd ed, Quay HC, Werry JS, eds. New York: Wiley Reid PT (l994), The real problem in the study of culture. Am Psychol 49:524-525

J. AM. ACAD. CHILD ADOLESC. PSYCHIATRY, 35:8, AUGUST 1996

Rice DP, Kelman S, Miller 15, Dunmeyer S (1990), The Economic Costs ofAlcohol and Drug Abuseand Mental Illness: 1985. Report submitted to the Office of Financing and Coverage Policy of the Alcohol, Drug Abuse, and Mental Health Administration, DHHS. San Francisco: Institute for Health and Aging, Universiry of California Richters J, Arnold LEA, Jensen PS et al. (1995), NIMH Collaborative Multisite Multimodal Treatment Study of Children with ADHD: I. Background and rationale. JAm Acad Child Adolesc Psychiatry 34:987-1000 RogIer LH (l992), The role of culture in mental health diagnosis: the need for programmatic research. J Neru Ment Dis 180:745-747 Rosenblatt A, Attkisson CC (1993), Assessing outcomes for sufferers of severemental disorder: a conceptual framework and review. EvalProgram

Plann 16:347-363 Rost K, Smith R, Burnam A, Burns B (l992), Measuring the outcomes of care for mental health' problems: the case of depressive disorders.

Med Care 30:266-273 Rutter M (l989), Isle of Wight revisited: twenty-five years of child psychiatric epidemiology. JAm Acad Child Adolesc Psychiatry 28:633-653 Rutter M, Hersov L, eds (1985), Child and Adolescent Psychiatry: Modern Approaches. Oxford, England: Blackwell Scientific Publications Schwab-Stone M, Fallon T, Briggs M, Crowther B (1994), Reliabiliry of diagnostic reporting for children aged 6-11 years: a test-retest study of the Diagnostic Interview Schedule for Children. Am J Psychiatry 151: 1048-1054 Schwab-Stone M, Fisher P, Piacentini ]C, Shaffer 0, Gioia P, Davies M (l993), The Diagnostic Interview Schedule for Children-Revised Version (DISC-R): II. Test-retest reliabiliry. J Am Acad Child Adolesc

Psychiatry 32:651-657 Shaffer 0, Gould MS, BrasicJet al. (l983), A Children's Global Assessment Scale (CGAS). Arch Gen Psychiatry 40:1228-1231 Shaffer 0, Schwab-Stone M, Fisher P er al. (l993), The Diagnostic Interview Schedule for Children-Revised Version (DISC-R): I. Preparation, field testing, and acceptability. JAm Acad Child Adolesc Psychiatry 32:643-650 Sorian RM, Holmes A (1993), The states: laboratory for health reform. In: A New DealfOr American Health Care: How Reform Will Reshape Health Care Delivery and Payment fOr a New Century, Koizume LS, ed. Washington, DC: Faulkner & Gray's Healthcare Information Center, pp 131-143 Stroul BA, Friedman RM (l986), A System ofCare fOr Severely Emotionally Disturbed Children and Youth. Washington, DC: Georgetown Universiry Child Development Center, CASSP Technical Assistance Center Thompson RA (l989), Emotion and self-regulation. In: Nebraska Symposium: Socioemotional Development, Thompson R, ed. Lincoln: Universiry of Nebraska Press Todd RD, Reich W, Petti TA et al. (l996), Psychiatric diagnoses in the child and adolescent members of extended families identified through adult bipolar affective disorder probands. J Am Acad Child Adolesc

Psychiatry 35:664-671 Weisz JR, Donenberg GR, Han SS, Weiss B (l995), Bridging the gap between lab and clinic in child and adolescent psychotherapy. J Consult

Clin PsychoI63:688-701 Weisz JR, Weiss B, Alicke MD, Klotz ML (1987), Effectiveness of psychotherapy with children and adolescents: a meta-analysis for clinicians. J

Consult Clin PsychoI55:542-549 Weisz JR, Weiss B, Donenberg GR (1992), The lab versus the clinic: effectsofchild and adolescent psychotherapy. Am Psychol47: 1578-1585

1063