The Canadian Objective Assessment of Life Skills (COALS): A new measure of functional competence in schizophrenia

Psychiatry Research 206 (2013) 302–306

Contents lists available at SciVerse ScienceDirect

Psychiatry Research journal homepage: www.elsevier.com/locate/psychres

The Canadian Objective Assessment of Life Skills (COALS): A new measure of functional competence in schizophrenia Stephanie A. McDermid Vaz a,b,n, R. Walter Heinrichs c, Ashley A. Miles c, Narmeen Ammari c, Suzanne Archie a,b, Eva Muharib c, Joel O. Goldberg c a

Cleghorn Early Intervention in Psychosis Program, St. Joseph’s Healthcare, Hamilton, Ontario, Canada Department of Psychiatry and Behavioural Neurosciences, McMaster University, Hamilton, Ontario, Canada c Department of Psychology, York University, Toronto, Ontario, Canada b

a r t i c l e i n f o

a b s t r a c t

Article history: Received 25 October 2012 Accepted 31 October 2012

This study examined the reliability and validity of a new performance-based measure of functional competence for individuals with serious mental illness, the Canadian Objective Assessment of Life Skills (COALS). The COALS assesses both routinized procedural knowledge routines (PKR) and executive operations (EXO) in order to capture functional outcome variance. The COALS was administered to 101 outpatients with schizophrenia and schizoaffective disorder and 80 non-psychiatric controls. One month later, 95 patients and 63 controls completed a follow-up assessment. Measures of psychopathology, neurocognition, functionality and community adjustment were also administered. Results indicated that the COALS summary scores had good test–retest reliability for patient data. Further, the COALS correlated with other measures of functionality and with negative symptoms, but was independent of positive symptoms, demonstrating concurrent and discriminant validity. The overall COALS summary score added incremental validity to the prediction of community independence over and above the contribution of symptoms, intellectual ability and neurocognitive performance. Inclusion of EXO scores provided incremental validity not available with PKR scores alone. The COALS increases the number of functional competence instruments and offers the advantage of specific validity while incorporating important distinctions in cognitive performance. & 2012 Elsevier Ireland Ltd. All rights reserved.

Keywords: Schizophrenia Functioning Assessment Neurocognition Functional competence

1. Introduction Cognitive performance predicts and may mediate important aspects of functional outcome in the schizophrenia population (Matza et al., 2006). Hence, the need to enhance functional status and outcomes in people with serious mental illness has spurred a search for medications and behavioral interventions to improve impaired cognitive abilities. Performance-based measures of functionality have advanced the field by providing an objective, focused assessment of practical thinking and skills needed for key aspects of daily life (Harvey et al., 2007). However, the role-play scenarios and simulations used by these measures may not represent adequately the complexity of real-life situations as skill application in the real world is influenced by varied personal, social and environmental factors (Bromley and Brekke, 2010). Nonetheless, the practical cognition indexed by functional

n Corresponding author at: Cleghorn Early Intervention in Psychosis Program, St. Joseph’s Healthcare, 25 Charlton Avenue East, Suite 703, Hamilton, Ontario, Canada L8N 1Y2. Tel.: þ 1 905 540 6586; fax: þ 1 905 525 2805. E-mail address: [email protected] (S.A. McDermid Vaz).

0165-1781/$ - see front matter & 2012 Elsevier Ireland Ltd. All rights reserved. http://dx.doi.org/10.1016/j.psychres.2012.10.020

capacity measures may be a necessary prerequisite for effective performance in real-life situations, even though skill proficiency is no guarantee that such performance will actually occur (Gupta et al., 2012). Two major challenges for functional capacity measures are (1) indexing cognitive skills and processes required by real-life situations in a sophisticated and sensitive way and, (2) demonstrating new or incremental validity relative to standard neurocognitive tasks. The most widely used functional capacity measure, the University of California San Diego PerformanceBased Skills Assessment (UPSA; Patterson et al., 2001), provides a limited sample of the kinds of problem identification and initiation skills that may be adaptive in daily life. Hence, more recent instruments incorporate stimulus items that engage executive abilities required for effective functioning (Velligan et al., 2007). However, there is little evidence to confirm that functional capacity measures add new validity to the prediction of real world outcome, validity not already provided by standard neurocognitive tests (Heinrichs et al., 2010). There is a growing literature describing the success of functional capacity measures in terms of their psychometric properties (Green et al., 2011) and

S.A. McDermid Vaz et al. / Psychiatry Research 206 (2013) 302–306

in predicting functional outcomes across subpopulations of psychotic patients (Cardenas et al., 2008; Gould et al., 2012) and different outcome indicators (Mausbach et al., 2008). Yet it remains unclear whether these measures capture the cognitive demands of real-life problem solving or yield new information not already available from standard neuropsychological tasks. These considerations of construct and incremental validity prompted our lab to begin development of the Canadian Objective Assessment of Life Skills (COALS). We view functional competence as a psychological construct underpinned by neurocognitive mechanisms and systems and mediated by biosocial and broader sociocultural influences. Daily living skills are in part behavior sequences and transactions that achieve pragmatic goals such as food preparation, arrival at travel destinations and appointments or management of a medication regimen. These behavior sequences can be understood and described as procedural knowledge routines (PKRs) or ‘‘knowing how’’ to carry out an adaptive action and activity as well as in terms of executive operations (EXOs), which reflect ‘‘knowing what to do and when to do it’’. PKRs are basic cognitive and behavioral skills refined and structured for specific life tasks. For example, preparing a meal from a recipe requires reading comprehension and instructionfollowing praxis with domain-specific content related to cooking. In contrast, answering questions in an employment interview can require social cognition as well as working and episodic memory, comprehension and instruction-following with highly specialized content. Thus PKRs are poly-factorial and vary in terms of their component cognitive processes and in terms of the domain knowledge required for successful action. However, supervisory control, decision-making and regulation are also required for adequate functional competence (Koren et al., 2006). Thus, preparing a meal from a recipe in real life may require more than a cluster of content-related PKRs. Successful food preparation may require the ability to determine if appropriate ingredients are at hand and, if not, the ability to provide a way of obtaining them. Moreover, time constraints and time regulation are often involved in meal preparation. Furthermore, real life may impose unanticipated complications or the need for adjustments to sudden situational changes as when, say, expected guests fail to arrive or additional and unexpected guests suddenly materialize. Without EXOs and component supervisory abilities in problem identification, initiation, solution and management, meal preparation may be unsuccessful despite the presence of relevant PKRs in the skill repertoire. This study examines the reliability and validity of the COALS in a sample of patients with schizophrenia and in non-psychiatric control participants. We report statistics indexing test–retest reliability and correlations between COALS data and symptom severity, neurocognitive performance and community adjustment. Additionally, we test the incremental validity of COALS performance relative to symptom severity and standard cognitive data. Results will provide a preliminary indication of the value of this new instrument in the assessment of functionality.

303

endocrine disorder; (3) no concurrent DSM-IV diagnosis of substance abuse or substance dependence; and (4) willingness and ability to sign informed consent. Patients were recruited from outpatient settings in south central Ontario, Canada. Non-psychiatric controls were recruited by postings and advertisements for paid research participation in community newspapers and internet-based classified advertisements. Potential participants were screened for medical and psychiatric illness and history of substance abuse. All participants signed informed consent and were paid for their time. The project was approved by the Research Ethics Board at St. Joseph’s Healthcare, Hamilton, and by the institutional review board at each research site. 2.2. Measures 2.2.1. COALS The test items, content, and structure for the COALS were generated through focus groups and feedback sessions with clinicians including psychiatrists, psychologists, care coordinators/case managers, occupational therapists, and peer support workers. We also conducted a review and content analysis of existing instruments and used patient feedback and clinical observations during the administration of other functional outcome measures in previous studies conducted by our research lab (Heinrichs et al., 2006). The COALS takes approximately 25 min to administer and is a structured roleplay demonstration of skills in five domains relevant to independent functioning in the community: (1) Health and Hygiene, (2) Time Management, (3) Transportation, (4) Crisis Management and (5) Domestic Activities. In each of the domains participants are presented with situations and stimulus material and instructed to role-play tasks or respond to scenarios designed to test procedural knowledge routines (PKR) and executive operations (EXO) important for independent living.

2.1. Participants

(1) Health and Hygiene Domain: in the first section the participant is presented with two types of medications and a dosette and told that they will be going on a trip and must plan to take their medications with them. The task requires the participant to sort varied doses of medication (PKR) and problem-solve a situation wherein the medication supply is insufficient for their trip (EXO). In the second section the participant is presented with several medications indicating specific instructions (i.e., to be taken with food, should not operate a motor vehicle) and then asked a series of scenario questions to assess proper medication management (EXO). They are then asked to prioritize with appropriate explanations the purchasing of personal hygiene items for the coming week (EXO). (2) Time Management Domain: a simulated phone message describing details for scheduling a job interview is presented to the participant. The participant is asked a series of questions regarding the details of the message (PKR). They are then instructed to find an appropriate interview time based on their scheduled activities as outlined in a week-long calendar that is presented to them (PKR). A problem situation arises in that possible appointment times and scheduled activities conflict. The participant must identify the problem, generate a solution and simulate leaving a phone message confirming their appointment with all of the relevant details (EXO). (3) Transportation Domain: the participant is instructed to read a flyer detailing a Cultural Festival being held in the city, and is asked a series of questions regarding the details (PKR). They are then presented with a bus schedule and a route map, and asked to work through a series of problem-solving scenarios involving trips to several events occurring at varied times throughout the day (EXO). (4) Crisis Management Domain: the participant is presented with a problemsolving scenario in which the lights in their home have just gone out. They are given the opportunity to propose responses to the situation (EXO). They are then presented with a simulated fuse box and accompanying instructions and a flashlight and are asked questions about the outlined procedure (PKR). The lights are turned off (with the participant’s permission) and the participant is asked a series of questions to determine their response repertoire in dealing with the situation (EXO) and the steps needed to change the fuses (PKR). (5) Domestic Activities Domain: this involves presenting the participant with two recipe cards, a life-size picture of pantry items, and instructions to work through a number of questions and scenarios (e.g., which recipe to make, which items needed to buy) as they plan an appropriate meal (PKR). They are then faced with a situation where they have to revise their plans to accommodate additional guests (EXO).

The sample consisted of 101 patients (23 females, 78 males) who met diagnostic criteria for schizophrenia (79) or schizoaffective disorder (22) and 80 non-psychiatric controls (26 females, 54 males). The mean interval between index and follow-up assessment was 34.7 days (S.D. ¼ 9.92) and 95 patients (23 males, 72 females) and 63 non-psychiatric controls (22 females, 41 males) completed the follow-up assessment. Diagnosis was confirmed by the Structured Clinical Interview for DSM IV-TR Axis I Disorders, Research Version, Patient Edition (SCID-I/P; First et al., 2002). Patients were included if they met the following criteria: (1) age 18–65 years; (2) no history of developmental disability or serious neurological or

2.2.2. Psychopathology, neurocognitive and functionality measures Symptom, cognitive, functional competence and real world outcome measures were administered to patient and non-psychiatric participants at index and 1-month follow-up. Current symptoms were evaluated with the Positive, Negative, and General Psychopathology subscales of the Positive and Negative Syndrome Scale (PANSS; Kay et al., 1999). Neurocognitive assessment included the MATRICS Consensus Cognitive Battery (MCCB; Nuechterlein et al., 2008) with the MCCB Composite score representing an overall measure of cognitive performance.

2. Methods

304

S.A. McDermid Vaz et al. / Psychiatry Research 206 (2013) 302–306

The Wechsler Abbreviated Scale of Intelligence (WASI; Wechsler, 1999) provided a brief measure of general intellectual ability. Premorbid ability was estimated with the Wide Range Achievement Test-4 (WRAT-4; Wilkinson and Robertson, 2006) Reading subtest. The UPSA (Patterson et al., 2001) was administered to determine the concurrent validity of the COALS. Real-world outcome and community independence were measured with the overall global rating from the Multidimensional Scale of Independent Functioning (MSIF; Jaeger et al., 2003) in both patients and nonpsychiatric controls, and provided data on criterion validity. The MSIF global rating reflects the level of functioning across residential, work, and educational settings while considering role position, level of performance, and degree of assistance required. Scores range from one to seven, with higher scores indicating poorer function.

Table 2 Comparison of patients’ and non-psychiatric controls’ index COALS performance. COALS scale total (max score)

Patients (n¼101) M (S.D.)

Non-psychiatric controls (n¼ 80) M (S.D.)

p

Effect size (Cohen’s d)

Time Management (20) Domestic Activities (20) Trip Planning (20) Health and Hygiene (18) Crisis Management (18) PKR Total (46) EXO Total (50) COALS Total (96)

15.1 11.8 8.5 13.9 12.4 30.4 31.2 61.7

18.0 15.0 12.1 15.7 13.7 37.4 37.1 73.8

0.000 0.000 0.000 0.000 0.006 0.000 0.000 0.000

0.83 0.77 0.87 0.70 0.41 1.03 0.87 0.91

(4.2) (4.6) (4.3) (2.8) (3.4) (7.8) (7.6) (14.3)

(2.6) (3.7) (4.0) (2.3) (3.0) (5.6) (5.8) (12.2)

3. Results Descriptive statistics for the patient and control group are presented in Table 1. The groups did not differ significantly in age, sex, parental education, or frequency of English as the first language. However, patients had less education (t179 ¼5.73, po0.001) and lower rates of full-time employment (w2 (2)¼38.42, po0.001) than the non-psychiatric participants. In addition, there was a significant group difference in estimated premorbid ability (t179 ¼4.68, po0.001) and IQ (t179 ¼4.04, po0.001); however, values for both groups on these indices fell within what would be considered the ‘‘average’’ range according to the current clinical practice (Wechsler, 1999; Wilkinson and Robertson, 2006). Overall global MSIF scores in patients (M¼ 4.12, S.D. ¼1.1) and healthy comparison samples (M¼2.05, S.D.¼1.1) differed significantly (t174 ¼12.38, po0.001) and correspond to an effect size (standardized mean difference; ES) of 1.88. The mean patient score reflects a rating of ‘‘moderate disability’’ in terms of independent functioning across residential, work and educational settings corrected for level of support and performance success. This mean score indicates significant difficulties in role functioning in mainstream community environments without support services. The control group mean is consistent with the low end of the normal range in terms of independent role functioning (Jaeger et al., 2003). Table 2 presents the mean COALS total and subscale scores for patients and controls obtained at index. Patients with schizophrenia performed significantly worse than controls in all domains of functioning and yielded large effect sizes for all scales, the only exception being the Crisis Management subscale (ES¼0.41) which Table 1 Demographic characteristics of patients and non-psychiatric controls.

Age in years, M (S.D.) Education in years, M (S.D.) Gender, n (% males) First language, n, % English

Patients (n¼ 101)

Controls (n¼ 80)

41.76 12.99 78 88

41.40 14.99 54 61

(10.27) (2.35) (77) (87)

(14.63) (2.31) (68) (76)

Employment, n (%) Full/part-time Unemployed Student

30 (30) 68 (67) 3 (3)

48 (60) 18 (23) 14 (18)

Parental education, n (%)a Grade six or less Grade from 7 to 12 Graduated high schoolb Part college Graduated 2 year college Graduated 4 year college Graduate/professional school

8 15 31 2 15 17 8

6 8 25 0 14 18 7

WASI IQ estimate, M (S.D.) WRAT-4 reading, M (S.D.) a b

Patients n¼ 96, controls n¼ 78. Or equivalent.

(8) (16) (32) (2) (16) (18) (8)

96.3 (15.8) 97.2 (15.6)

(8) (10) (32) (0) (18) (23) (9)

106.4 (17.8) 108.2 (15.7)

approximates a ‘‘medium’’ value (Cohen, 1992). The PKR subscale generated the largest effect size (ES¼1.03), but both the EXO subscale (ES¼0.87) and the COALS Total score (ES¼0.91) resulted in large group differences. Intra-class correlation coefficients (ICC) were calculated between initial COALS summary scores and scores obtained one month later to determine test–retest reliability. Using the convention of ICC¼0.70 as a cutoff for acceptable reliability, each summary index exceeded the criterion (COALS Total ICC¼0.82, EXO ICC¼0.75, PKR ICC¼0.75) in the patient data. However, calculation of this reliability statistic for the healthy control participants yielded somewhat more modest findings (COALS Total ICC¼ 0.60, EXO¼0.68, PKR¼0.73). 3.1. Validity Concurrent validity was assessed by examining relationships between the COALS, the UPSA and the MCCB. The UPSA Total score was strongly and positively correlated with the COALS Total (r ¼0.70, p o0.001), EXO (r ¼0.65, po0.001) and PKR (r¼ 0.66, po0.001) scores. The summary COALS scores also varied significantly with the composite MCCB score (Total r ¼0.75, po0.001; EXO r ¼0.69, p o0.001; PKR r ¼0.70, po0.001). Discriminant validity was evaluated by examining relationships between COALS performance and symptom severity. There were modest, but significant negative correlations between performance on the COALS and the PANSS index of negative symptoms (Total r¼  0.29, p o0.01; EXO r ¼  0.30, po0.01; PKR r ¼  0.24, po0.05). General psychopathology scores were also negatively correlated with higher functioning on the COALS Total (r ¼  0.23, po0.05), EXO (r ¼ .21, p o0.05) and PKR (r ¼  0.20, po0.05) scores. There were no significant relationships between positive symptoms and performance on the COALS. Criterion validity was assessed by correlating each COALS summary score with the MSIF overall global score, an index of real world functional status. This yielded significant negative relationships for Total (r ¼  0.36, p o0.001), EXO (r ¼  0.40, po0.001) and PKR (r ¼  0.25, p o0.05) scores. Accordingly, higher community support requirements associate with lower performance on each COALS index. Incremental validity of the COALS relative to symptom and standard neurocognitive data was evaluated with hierarchical regression. First, the global MSIF score was regressed on a block of symptom variables comprising the three PANSS scores (i.e., total scores for positive, negative, and general psychopathology domains). This yielded a significant F ratio (F (93)¼4.3, po0.01) and accounted for 12% of the variance in MSIF scores. Next, an IQ estimate based on the WASI was entered and an Fchange ratio calculated for the increase in variance accounted for by the model. The addition of the IQ term failed to add significantly to validity (Fchange (92) ¼2.0, n.s.). The composite MCCB score was then added and an F on the change in validity calculated. This

S.A. McDermid Vaz et al. / Psychiatry Research 206 (2013) 302–306

addition also failed to reach significance (Fchange (91)¼0.0, n.s.). Finally, the procedure was repeated with entry of the Total COALS score, which produced a significant increase in validity (Fchange (90) ¼7.6, p o0.01). The complete model accounted for 21% of the variance in MSIF ratings. The same procedure was then repeated using index data to predict MSIF ratings completed at the 1-month follow-up assessment. The model based on PANSS ratings accounted for 15% of the variance in follow-up MSIF ratings (F (88)¼5.2, po0.01). Inclusion of an IQ estimate increased validity to 19% with a marginally significant F ratio (F (87)¼3.8, p¼0.055). However, addition of the MCCB term failed to yield new validity (F (86)¼0.4, n.s.). Inclusion of the Total COALS score resulted in an increase in validity to 24% (F (85)¼ 6.2, po0.05). To assess whether EXO scores provide new validity not available with PKR scores alone, the index MSIF global rating was regressed on the PKR score. On its own PKR accounted for 6% of the variance in MSIF ratings (F (99)¼ 6.5, po0.05). Addition of the EXO term significantly increased validity to 17% (Fchange (98)¼ 12.74, p o0.01). In contrast, when the MSIF rating was regressed on only the EXO score, the model accounted for 16% of the variance (F(99)¼ 19.35, po0.001) whereas addition of the PKR score failed to add new validity (Fchange (98) ¼0.73, n.s.). This procedure was then repeated for the 1-month follow-up MSIF ratings. On its own PKR accounted for 7% of the variance in follow-up MSIF ratings (F (93)¼7.47, po0.01). Nonetheless, addition of the EXO term significantly increased validity to 19% (F (92)¼12.57, po0.01). When the MSIF follow-up rating was regressed first on EXO, the equation accounted for 18% of the variance (F (93)¼20.70, po0.001), but addition of the PKR term failed to yield a significant increment in validity (F (92)¼ 0.40, n.s.).

4. Discussion This paper describes the development of a new performancebased functional competence measure (COALS) for people with serious mental illness. Our findings provide preliminary evidence that summary scores derived from the COALS have 1-month temporal stability and differ significantly and substantially in schizophrenia patients and non-psychiatric participants. Additionally, although COALS scores vary with performance on standard neurocognitive measures, this new functional competence instrument provides additional predictive validity not available with standard measures. Moreover, the COALS was designed to incorporate a distinction between routinized knowledge (PKR) and supervisory processes (EXO). Our evidence suggests that indexing these supervisory processes provides unique validity in the prediction of real world outcomes in the schizophrenia population. Hence the COALS may have advantages relative to current assessments of functional competence. As expected, our sample of patients with schizophrenia performed significantly worse than non-psychiatric controls in all five domains of the COALS and on each of the summary scores. Group comparisons yielded large effect sizes for the COALS Total score, the PKR Total score and the EXO Total score. The magnitudes of group difference are similar to those reported for the UPSA in an earlier study conducted in similar clinical settings in Canada (Heinrichs et al., 2006). These group differences may be smaller than those in the original U.S.-based validation studies of the UPSA (Patterson et al., 2001) and the TABS (Velligan et al., 2007). However, in view of differences in demographic, sociocultural and clinical sample characteristics across studies and the relatively infrequent reporting of healthy control data, the interpretation of our somewhat smaller group effects is unclear. Nonetheless, evidence of both functional (Miles et al., 2011) and cognitive (Wexler et al., 2009) near-normality in minorities of

305

patients with schizophrenia as well as meta-analytic findings (Heinrichs and Zakzanis, 1998) imply that joint patient–control distributions overlap. Therefore, complete separation of patient and control groups in terms of cognitive performance or functional competence seems unlikely. Moreover, our IQ data suggest that the clinical sample was comprised of relatively highfunctioning patients, which may have been reflected in the magnitude of group differences in the study variables. The test–retest reliability of the COALS summary scores, indexed by intra-class correlations, is acceptable by convention and compares favorably with statistics reported for summary indices of the TABS (Velligan et al., 2007), the UPSA (Green et al., 2011) and many neurocognitive measures (Nuechterlein et al., 2008). The slightly lower reliabilities derived from control participants also occur with the UPSA (Leifker et al., 2010) and may reflect the restricted variance and performance ceiling observed in healthy community-dwelling populations (Heinrichs et al., 2006). The COALS was highly correlated with summary scores from both the most commonly used performance-based measure of functional competence (UPSA) and with a widely used and specifically designed battery of neurocognitive measures (MCCB), thereby providing evidence of concurrent validity. Elevated performance on the COALS was also associated with better illness status overall and with less severe negative symptoms, but there was no relationship with positive symptoms. This echoes the pattern of findings observed with the TABS (Velligan et al., 2008), whereas relations between symptoms and UPSA performance levels are more variable, at least in terms of older patients (Leifker et al., 2009). In addition, a key aspect of validity is the ability of functional competence measures to predict real-world adjustment (Bromley and Brekke, 2010; Green et al., 2011). The significant association between the COALS and community independence (MSIF) indicates that patients with lower life skills performance also tend to be those with higher rates of disability and greater overall support needs. The incremental validity of functional competence measures in predicting community adjustment relative to standard neurocognitive measures has received scant research attention (Heinrichs et al., 2010). It is known that standard neurocognitive measures predict varied real-world outcomes in schizophrenia (Fett et al., 2011). Functional competence measures differ from neurocognitive tests by simulating everyday role-play activities and through use of practical and functionally relevant rather than abstract or generic stimulus material. Nevertheless, both competence measures and neurocognitive tests index behavior and cognitive operations with psychometric methods. Therefore, it is not surprising that neurocognitive test batteries and functional competence measures share variance (Green et al., 2008). But do measures of functional competence also capture new outcome and functional status validity not already captured with standard neurocognitive tests? Our data show that the overall COALS summary score adds new validity over and above the contribution of symptoms, intellectual ability and composite neurocognitive performance. In addition, this was demonstrated with both crosssectional and 1-month follow-up outcome data. A caution on this finding is the failure of the neurocognitive test battery (MCCB) to contribute validity to community adjustment prediction in our data. It should be noted that the MCCB has also demonstrated low validity in this regard in other studies (Nuechterlein et al., 2008; Shamsi et al., 2011). Another question we addressed was the potential value of the distinction between routinized knowledge and supervisory or executive abilities in predicting community adjustment. Our data suggest that executive components of COALS performance (EXO) provide incremental and therefore unique validity to prediction, over and above the contribution of routinized

306

S.A. McDermid Vaz et al. / Psychiatry Research 206 (2013) 302–306

knowledge and skill (PKR). This finding underscores recent arguments for the integration of meta-cognitive processes such as self-regulation and self-monitoring into competence measures because these processes may be essential for adaptive real-world functioning (Koren et al., 2006; Lysaker et al., 2011). At the same time, the construct validity of the EXO items as estimators of meta-cognition requires attention in future research. Development of these items for the COALS was informed by theoretical considerations and must be supplemented with convergent evidence from independent executive ability tasks in order to confirm the accuracy of the EXO/PKR distinction. The reported data and their limitations should be considered in light of the challenges faced in the formulation and validation of any new instrument. Thus examination of COALS retest data at intervals longer than 1 month, development of a brief version, suitability for use as a change measure, and application to distinct subpopulations including first episode psychosis patients are required. In addition, self-report-based criterion measures of real-world outcome (e.g., MSIF) have been criticized and should be supplemented and compared with functionality estimates derived from informed observers (Sabbag et al., 2012). It is also noteworthy that less than a quarter of functional outcome variance was accounted for by the aggregate of clinical status and neurocognitive and COALS performance. The relatively modest validity underscores recent assertions that understanding the discrepancy between competence and real-world outcomes is a complex and increasingly important research problem in its own right (Gupta et al., 2012). The COALS is a new measure of functional competence that merits further refinement and application to this key area of investigation.

Acknowledgments This work was supported by the Ontario Mental Health Foundation. We express our appreciation to Ashley Oman (B.A.), Melissa Parlar (B.Sc.), and Natalie Michel (M.A.) as well as the clinicians and staff of the Cleghorn Early Intervention in Psychosis Program, the Community Schizophrenia Service, St. Joseph’s Healthcare, Hamilton, the Hamilton Program for Schizophrenia, and Homewood for their assistance. The authors report no competing interests. References Bromley, E., Brekke, J.S., 2010. Assessing function and functional outcome in schizophrenia. Current Topics in Behavioral Neurosciences 4, 3–21. Cardenas, V., Mausbach, B.T., Barrio, C., Bucardo, J., Jeste, D., Patterson, T., 2008. The relationship between functional capacity and community responsibilities in middle-aged and older Latinos of Mexican origin with chronic psychosis. Schizophrenia Research 98, 209–216. Cohen, J., 1992. A power primer. Psychological Bulletin 112, 155–159. Fett, A.K., Viechtbauer, W., Dominguez, M.G., Penn, D.L., van Os, J., Krabbendam, L., 2011. The relationship between neurocognition and social cognition with functional outcomes in schizophrenia: a meta-analysis. Neuroscience and Biobehavioral Reviews 35, 573–588. First, M.B., Spitzer, R.L., Gibbon, M., Williams, J.B.W., 2002. Structured Clinical Interview for DSM-IV-TR Axis I Disorders, Research Version, Patient Edition. (SCID-I/P). Biometrics Research, New York State Psychiatric Institute, New York, NY. Gould, F., Bowie, C.R., Harvey, P.D., 2012. The influence of demographic factors on functional capacity and everyday functional outcomes in schizophrenia. Journal of Clinical and Experimental Neuropsychology 34, 467–475. Green, M.F., Nuechterlein, K.H., Kern, R.S., Baade, L.E., Fenton, W.S., Gold, J.M., Keefe, R.S., Mesholam-Gately, R., Seidman, L.J., Stover, E., Marder, S.R., 2008. Functional co-primary measures for clinical trials in schizophrenia: results from the MATRICS psychometric and standardization study. American Journal of Psychiatry 165, 221–228. Green, M.F., Schooler, N.R., Kern, R.S., Frese, F.J., Granberry, W., Harvey, P.D., Karson, C.N., Peters, N., Stewart, M., Seidman, L.J., Sonnenberg, J., Stone, W.S.,

Walling, S.R., Stover, E., Marder, S.R., 2011. Evaluation of functionally meaningful measures for clinical trials of cognition enhancement in schizophrenia. American Journal of Psychiatry 168, 400–407. Gupta, M., Bassett, E., Iftene, F., Bowie, C.R., 2012. Functional outcomes in schizophrenia: understanding the competence–performance discrepancy. Journal of Psychiatric Research 46, 205–211. Harvey, P.D., Velligan, D.I., Bellack, A.S., 2007. Performance-based measures of functional skills: usefulness in treatment studies. Schizophrenia Bulletin 33, 1138–1148. Heinrichs, R.W., Ammari, N., Miles, A.A., McDermid Vaz, S., 2010. Cognitive performance and functional competence as predictors of community independence in schizophrenia. Schizophrenia Bulletin 36, 381–387. Heinrichs, R.W., Statucka, M., Goldberg, J., McDermid Vaz, S., 2006. The University of California Performance Skills Assessment (UPSA) in schizophrenia. Schizophrenia Research 88, 135–141. Heinrichs, R.W., Zakzanis, K.K., 1998. Neurocognitive deficit in schizophrenia: a quantitative review of the evidence. Neuropsychology 12, 426–445. Jaeger, J., Berns, S.M., Czobor, P., 2003. The multidimensional scale of independent functioning: a new instrument for measuring functional disability in psychiatric populations. Schizophrenia Bulletin 29, 153–167. Kay, S.R., Opler, L.A., Fiszbein, A., 1999. Structured Clinical Interview: The Positive and Negative Syndrome Scale (SCI-PANSS). Multi-Health Systems Inc, North Tonawanda, NY. Koren, D., Seidman, L.J., Goldsmith, M., Harvey, P.D., 2006. Real-world cognitive—and metacognitive—dysfunction in schizophrenia: a new approach for measuring (and remediating) more ‘‘right stuff’’. Schizophrenia Bulletin 32, 310–326. Leifker, F.R., Bowie, C.R., Harvey, P.D., 2009. Determinants of everyday outcomes in schizophrenia: the influences of cognitive impairment, functional capacity, and symptoms. Schizophrenia Research 115, 82–87. Leifker, F.R., Patterson, T.L., Bowie, C.R., Mausbach, B.T., Harvey, P.D., 2010. Psychometric properties of performance-based measurements of functional capacity: test–retest reliability, practice effects, and potential sensitivity to change. Schizophrenia Research 119, 246–252. Lysaker, P.H., McCormick, B.P., Snethen, G., Buck, K.D., Hamm, J.A., Grant, M., Nicolo, G., Dimaggio, G., 2011. Metacognition and social function in schizophrenia: associations of mastery with functional skills competence. Schizophrenia Research 131, 214–218. Matza, L.S., Buchanan, R., Purdon, S., Brewster-Jordan, J., Zhao, Y., Revicki, D.A., 2006. Measuring changes in functional status among patients with schizophrenia: the link with cognitive impairment. Schizophrenia Bulletin 32, 666–678. Mausbach, B.T., Depp, C.A., Cardenas, V., Jeste, D.V., Patterson, T.L., 2008. Relationship between functional capacity and community responsibility in patients with schizophrenia: differences between independent and assisted living settings. Community Mental Health Journal 44, 385–391. Miles, A.A., Heinrichs, R.W., Ammari, N., 2011. ‘‘Real world’’ functioning in schizophrenia patients and healthy adults: assessing validity of the multidimensional scale of independent functioning. Journal of Psychiatry Research 186, 123–127. Nuechterlein, K.H., Green, M.F., Kern, R.S., Baade, L.E., Barch, D.M., Cohen, J.D., Essock, S., Fenton, W.S., Frese 3rd, F.J., Gold, J.M., Goldberg, T., Heaton, R.K., Keefe, R.S., Kraemer, H., Mesholam-Gately, R., Seidman, L.J., Stover, E., Weinberger, D.R., Young, A.S., Zalcman, S., Marder, S.R., 2008. The MATRICS consensus cognitive battery, part 1: test selection, reliability, and validity. American Journal of Psychiatry 165, 203–213. Patterson, T.L., Goldman, S., McKibbin, C.L., Hughs, T., Jeste, D.V., 2001. UCSD performance-based skills assessment: development of a new measure of everyday functioning for severely mentally ill adults. Schizophrenia Bulletin 27, 235–245. Sabbag, S., Twamley, E.W., Vella, L., Heaton, R.K., Patterson, T.L., Harvey, P.D., 2012. Predictors of the accuracy of self assessment of everyday functioning in people with schizophrenia. Schizophrenia Research 137, 190–195. Shamsi, S., Lau, A., Lencz, T., Burdick, K.E., DeRosse, P., Brenner, R., Lindenmayer, J.P., Malhotra, A.K., 2011. Cognitive and symptomatic predictors of functional disability in schizophrenia. Schizophrenia Research 126, 257–264. Velligan, D.I., Diamond, P.M., Glahn, D.C., Ritch, J., Maples, N., Castillo, D., Miller, A.L., 2007. The reliability and validity of the Test of Adaptive Behavior in schizophrenia (TABS). Psychiatry Research 151, 55–66. Velligan, D.I., Diamond, P.M., Mintz, J., Maples, N., Li, X., Zeber, J., Ereshefsky, L., Lam, Y.W., Castillo, D., Miller, A.L., 2008. The use of individually tailored environmental supports to improve medication adherence and outcomes in schizophrenia. Schizophrenia Bulletin 34, 483–493. Wechsler, D., 1999. Wechsler Abbreviated Scale of Intelligence. The Psychological Corporation, New York. Wexler, B.E., Zhu, H., Bell, M.D., Nicholls, S.S., Fulbright, R.K., Gore, J.C., Colibazzi, T., Amat, J., Bansal, R., Peterson, B.S., 2009. Neuropsychological near normality and brain structure abnormality in schizophrenia. American Journal of Psychiatry 166, 189–195. Wilkinson, G.S., Robertson, G., 2006. WRAT-4: The Wide Range Achievement Test Administration Manual, 4th ed. Wide Range, Wilmington, DE.