Child Adjustment and Parent Efficacy Scale-Developmental Disability (CAPES-DD): First psychometric evaluation of a new child and parenting assessment tool for children with a developmental disability

Research in Developmental Disabilities 53–54 (2016) 158–177

Contents lists available at ScienceDirect

Research in Developmental Disabilities

Child Adjustment and Parent Efficacy Scale-Developmental Disability (CAPES-DD): First psychometric evaluation of a new child and parenting assessment tool for children with a developmental disability Theresa S. Emser a,*, Trevor G. Mazzucchelli b,c, Hanna Christiansen a, Matthew R. Sanders c a

Department of Psychology, Child and Adolescent Psychology, Philipps-Universita¨t Marburg, Gutenbergstr. 18, 35032 Marburg, Germany Child and Family Research Group and Brain Behaviour and Mental Health Research Group, School of Psychology and Speech Pathology, Curtin University, Kent St, Bentley, WA 6102, Australia c Parenting and Family Support Centre, School of Psychology, The University of Queensland, Brisbane, QLD 4072, Australia b

A R T I C L E I N F O

A B S T R A C T

Article history: Received 23 March 2015 Received in revised form 23 July 2015 Accepted 13 September 2015 Available online

This study examined the psychometric properties of the Child Adjustment and Parent Efficacy Scale-Developmental Disability (CAPES-DD), a brief inventory for assessing emotional and behavioral problems of children with developmental disabilities aged 2to 16-years, as well as caregivers’ self-efficacy in managing these problems. A sample of 636 parents participated in the study. Children’s ages ranged from 2 to 15. Exploratory and confirmatory factor analyses supported a 21-item, three-factor model of CAPES-DD child adjustment with 13 items describing behavioral (10 items) and emotional (3 items) problems and 8 items describing prosocial behavior. Three additional items were included due to their clinical usefulness and contributed to a Total Problem Score. Factor analyses also supported a 16-item, one factor model of CAPES-DD self-efficacy. Psychometric evaluation of the CAPES-DD revealed scales had satisfactory to very good internal consistency, as well as very good convergent and predictive validity. The instrument is to be in the public domain and free for practitioners and researchers to use. Potential uses of the measure and implications for future validation studies are discussed. ß 2015 Elsevier Ltd. All rights reserved.

Keywords: Rating scale Developmental disability Emotional and behavioral problems Self-efficacy Psychometric properties Factor analysis

1. Introduction Developmental disability (DD) is a term that describes a group of rather heterogeneous and diverse conditions. They range from intellectual (e.g., autism spectrum disorder) to physical (e.g., cerebral palsy) impairments and can be of genetic, environmental or trauma-related origin. The impairments must affect at least three major areas of life activities, such as

* Corresponding author. E-mail addresses: [email protected] (T.S. Emser), [email protected] (T.G. Mazzucchelli), [email protected] (H. Christiansen), [email protected] (M.R. Sanders). http://dx.doi.org/10.1016/j.ridd.2015.09.006 0891-4222/ß 2015 Elsevier Ltd. All rights reserved.

T.S. Emser et al. / Research in Developmental Disabilities 53–54 (2016) 158–177

159

communication, living independently, self-care, mobility or learning (United States of America Developmental Disabilities Act of 1984). With an overall prevalence rate of about 14% for children aged of 3- to 17-years they concern a significant number of families (Boyle et al., 2011). Children with a DD have an approximately three to four times elevated risk of presenting with behavioral and emotional problems, not uncommonly resulting in serious psychiatric disorders (Einfeld & Tonge, 1996; Emerson, 2003b). Furthermore, Einfeld et al. (2006) found that these problems tend to persist into adulthood. As these problems increase the risk for parental stress, they not only have an impact on the wellbeing and life of the child, but are a burden for the whole family (Roberts, Mazzucchelli, Taylor, & Reid, 2003). It is therefore important to provide parents with useful parenting tools and advice as well as to support them in handling these problems. Parenting programmes such as Stepping Stones Triple P (SSTP; Mazzucchelli & Sanders, 2011; Sanders, Mazzucchelli, & Studman, 2004) represent useful methods for doing this. However, it is not sufficient to restrict these interventions to clinical settings, instead it is necessary to provide wide reaching evidence-based support to every family in need. In order to offer the most appropriate support and to evaluate the usefulness of such support, it is necessary to assess child behavior problems in a reliable and valid way. At present there are several instruments which are commonly used for the assessment of emotional and behavioral problems of children with DD such as the Child Behavior Checklist (CBCL; Achenbach & Rescorla, 2001), the Strengths and Difficulties Questionnaire (SDQ; Goodman, 1997), and the Developmental Behaviour Checklist (DBC; Einfeld & Tonge, 2002). However, these measures show several weaknesses in assessing child adjustment, particularly in the context of DDs. The CBCL, for example, is a very long and time-consuming instrument, which can be burdensome for parents and caregivers to complete (Goodman & Scott, 1999). It also incurs a fee, which can make it problematic to use for large population studies. Furthermore, the CBCL does not provide any norms regarding children with a DD. The SDQ correlates highly with the CBCL (Goodman & Scott, 1999) and is significantly shorter with only 25 items. In spite of this, it shows some weaknesses regarding the internal consistency of individual subscales (Smedje, Broman, Hetta, & Von Knorring, 1999), and its online use is restricted. In addition, the SDQ was developed for typically developing children and there has been limited research regarding its suitability for children with a disability. The DBC has sound psychometric properties, but was specifically developed for children with an ID and consequently may lack applicability for children having just a DD. The DBC must be purchased for use and is not very change sensitive. Thus, an economical instrument for the assessment of emotional and behavioral problems of children with a DD that is in the public domain, change sensitive, and has good psychometric properties is still needed. Additionally, increasing research is emerging regarding the construct of parental self-efficacy (PSE) that ‘‘can be broadly defined as the expectation caregivers hold about their ability to parent successfully’’ (Jones & Prinz, 2005, p. 342). PSE is very important in the context of parenting and parenting interventions. In their review, Coleman and Karraker (1998) link maternal self-efficacy to more adaptive parenting strategies, to parental adjustment as well as to child difficulties. The Child Adjustment and Parent Efficacy Scale-Developmental Disability (CAPES-DD; Mazzucchelli, Sanders, & Morawska, 2011) being evaluated in the present study provides a newly developed scale measuring PSE in relation to the parallel assessed emotional and behavioral problems presented by the children. This not only establishes a direct link between the efficacy beliefs of the parents and the specific demands claimed by their children but also decreases the assessment burden on families by only using one instrument instead of two. The CAPES-DD was developed to assess different behavior domains (i.e., externalizing, internalizing and prosocial) in children with a DD, and to also assess parental confidence in handling those problems. The authors wanted it to be suitable for children aged 2- to 16-years and to be appropriate for a range of respondents (e.g., parents, carers, teachers). Furthermore it was desired to be relatively brief (a maximum of 30 items), change sensitive, and in the public domain. In correspondence to the already developed Child Adjustment and Parent Efficacy Scale (CAPES; Morawska & Sanders, 2010), Sanders and Mazzucchelli adopted the same structure regarding the number of items, response format and scales, namely an Intensity scale measuring child adjustment and a Self-Efficacy scale measuring PSE. The CAPES-DD items were selected in a way so that they reflect the full range of behaviors that a child with a disability might present, including problems and strengths. These items were generated based on the authors’ own experience as well as by considering items from existing scales such as: Aberrant Behavior Checklist – Community (Aman & Singh, 1994), the Maladaptive Behavior Scale of the Scales of Independent Behavior-R (Bruininks, Woodcock, Weatherman, & Hill, 1984) and the DBC (Einfeld & Tonge, 2002). 1.1. The current study This study is the first to examine the psychometric properties of the newly developed CAPES-DD. Specifically, it aimed to investigate: (a) the item properties, (b) the construct validity with a focus on the exploration of the dimensional structure of the Intensity scale, (c) the concurrent validity, (d) the predictive validity as well as (e) the reliability of the instrument.

160

T.S. Emser et al. / Research in Developmental Disabilities 53–54 (2016) 158–177 Table 1 Socio-demographic characteristics of the sample.

Gender child Male Female Relationship to child Mother Father Stepmother Foster Mother Foster Father Grandmother Grandfather Other Working status Fulltime Part time Unemployed and looking for work Not in paid employment Employed casually Full time student Employed, but on maternity leave Home based paid work Household Original family Step family Sole parent family Other Marital status Married Divorced/separated Single Cohabiting/de facto Widow/er

My Say Survey (n = 559) %

Curtin Study (n = 77) %

73.6 26.4

67.5 32.5

91.9 3 0.4 1.4 0.2 1.6 0.4 1.1

87.7 8.2 N/A 1.4 N/A 1.4 N/A 2.7

15.3 20.5 6.3 46.7 6.9 3.4 0.8 N/A

21.1 39.4 4.2 32.4 N/A N/A N/A 2.8

72.4 3.9 18 5.7

80.8 1.4 9.6 8.2

69.4 11 9.8 8.5 0.9

74 13.7 2.7 8.2 1.4

Note. N/A = not available.

2. Method 2.1. Participants The participants consisted of parents and carers recruited for two different projects. The first project, conducted in 2012 at Curtin University, recruited parents (n = 77) of a child with a DD from several Australian states and territories (Western Australia, Northern Territory, South Australia, Australian Capital Territory, and Tasmania). Disability agencies as well as education support schools and centres were contacted and asked to forward information about the project to their clients. The second part (n = 559) was gathered via the My Say Survey that was part of the SSTP project, a programme of research funded by Australia’s National Health and Medical Research Council. The My Say Survey was an online survey that took place from October 2012 to June 2013 in the Australian state of Queensland. The advertisement for the survey consisted of electronic and paper flyers as well as posters that were distributed in agencies, clinics and practices. Furthermore, the research team of the SSTP project contacted almost all of the special schools and agencies in Queensland supporting children with a disability in order to raise awareness of the project and to request that information about the project be forwarded to suitable parents. Finally the project was promoted through broadcast media such as television and radio. Child diagnoses were provided by caregivers. The study permitted the reporting of comorbid disabilities by allowing caregivers to indicate that their child had more than one diagnosis. An overview of the socio-demographic characteristics of these samples is provided in Table 1. In both studies the most prevalent disabilities were the pervasive developmental disorders (PDD) or autism spectrum disorder (ASD; My Say: 91.5%, Curtin Study: 57.1%) including Asperger’s syndrome, autism, PDD-not otherwise specified, Rett syndrome and childhood disintegrative disorder. This category was followed by intellectual disability or developmental delay (My Say: 52.1%, Curtin Study: 58.9%). For a detailed overview over the distribution of disabilities refer to Figs. 1 and 2.

[(Fig._1)TD$IG]

T.S. Emser et al. / Research in Developmental Disabilities 53–54 (2016) 158–177

161

None of the above/unknown Other Physical Disability Other Neurological Disability Other chromosome or genetic disorder Tuberous Sclerosis Williams Syndrome Velo Cardio Facial Syndrome Multiple Sclerosis Prader Willi Syndrome Muscular Distrophy Motor Neurone Disease Rett Syndrome Foetal Alcohol Syndrome Childhood disintegrative Disorder Fragile X Syndrome Para-/Quadri-/Tetra-Hemiplegia Acquired Brain Injury Blind/Vision Impairment Deaf/Hearing Impairment Down Syndrome Cerebral Palsy Epilepsy PDD - NOS Autism Asperger's Syndrome IntellectualDisability or Developmental Delay Autism Spectrum Disorder Language Delay 0

5

10

15

20

25

30

percentage Fig. 1. Developmental disabilities indicated by the caregivers participating in the My Say Survey (N = 559).

[(Fig._2)TD$IG] Psychiatric Disability Other Physical Disability Other Neurological Disability Other Sensory Disability Muscular Distrophy Multiple Sklerosis Motor Neurone Disease Acquired brain injury Para-/Quadri-/Tetra-Hemiplegia Cerebral Palsy Blind/Vision Impairment Deaf/Hearing Impairment Spina Bifida Epilepsy Developmental Delay Specific Learning Difficulty Intellectual Disability Pervasive Developmental Disorder 0

10

20

30

40

50

60

percentage Fig. 2. Developmental disabilities indicated by the caregivers participating in the Curtin Study (N = 77).

2.2. Measures The Child Adjustment and Parent Efficacy Scale-Developmental Disability (Mazzucchelli et al., 2011). The CAPES-DD consists of 30 items and assesses emotional and behavioral problems of children with disabilities (Intensity scale) as well as the caregiver’s confidence in being able to handle these problems (Self-Efficacy scale). It can be completed by any caregiver of the child including parents, grandparents, and/or teachers. The 30 items of the Intensity scale ask about difficulties

162

T.S. Emser et al. / Research in Developmental Disabilities 53–54 (2016) 158–177

(e.g., ‘‘Loses their temper’’), but also about strengths (e.g., ‘‘Expresses feelings appropriately’’; see Appendix A). The caregivers rate each item from 0 (‘‘Not true of my child at all’’) to 3 (‘‘True of my child very much, or most of the time’’) taking into account their child’s behavior over the past four weeks. The authors proposed two subscales of the Intensity scale, one describing externalizing problems (18 items) and one describing internalizing problems (12 items). The authors suggested that the items asking about strengths be reverse scored and included in the Intensity scale. For the Self-Efficacy scale, respondents’ confidence is rated for the difficulties from 1 (‘‘Certain I can’t manage it’’) to 10 (‘‘Certain I can manage it’’). The Developmental Behaviour Checklist – Primary Carer version and version for children under the age of four (DBC-P and DBCU4; Einfeld & Tonge, 2002). The DBC-P assesses behavioral and emotional disturbance in children and adolescents with an ID between the ages of 4- and 18-years. It consists of 96 items that are answered by a primary carer on a three-point scale (not true as far as you know, sometimes or somewhat true, often true or very true). It is divided into five subscales (disruptive/ antisocial, self-absorbed, communication disturbance, anxiety, social relating) whose individual scores can be summed up to a Total Behaviour Problem Score (TBPS). The DBC-P-U4 is a parallel version for children under the age of four that is still under psychometric evaluation. The DBC-P has been shown to have sound psychometric properties with an internal consistency of a = .94 for the TBPS and is widely used as an assessment tool for children with ID (Einfeld et al., 2010). In this study the DBC-U4 showed an internal consistency of a = .97, Cronbach’s a of the DBC-P was .95. The Parenting and Family Adjustment Scales (PAFAS). The PAFAS (Sanders & Morawska, 2010) is a brief inventory of 30 items with one scale assessing parenting (18 items measuring parenting practices) and one assessing family adjustment (12 items measuring parent and family adjustment). The Parenting Scale consists of four subscales: Parental Consistency, Coercive Parenting, Positive Encouragement, and Parent-Child Relationship. The Family Adjustment Scale consists of three subscales: Parental Adjustment, Family relationships, and Parental Teamwork. Each item is rated on a 4-point Likert-type scale and answers are summed for each individual subscale with higher scores indicating higher dysfunction. A validation study (Sanders, Morawska, Haslam, Filus, & Fletcher, 2013) reported good internal consistency ranging from .70 (Parental Consistency) to .87 (Parental Adjustment) for the subscales as well as satisfactory construct and predictive validity. In this study Cronbach’s a of the PAFAS subscales ranged from .59 (Parental Consistency) to .82 (Parent-Child Relationship). 2.3. Data analysis 2.3.1. Item properties For the analysis of the item properties, 526 respondents (n = 457 from the My Say survey and n = 69 from the Curtin project) were used for the Intensity scale and 511 (n = 448 from the My Say Survey and n = 63 from the Curtin project) for the Self-Efficacy scale. Respondents were excluded because they had not completed any items of the respective scale. Means and standard deviations of every item were investigated. For items 21–30 this was done for their original form and after reversing them. In order to validate the goodness of the items and how well they can differentiate between individual respondents, their difficulty and their discriminatory power were calculated. 2.3.2. Construct validity The construct validity of the Intensity scale of the CAPES-DD was assessed using exploratory factor analysis (EFA) as a first step and confirming its results via confirmatory factor analysis (CFA) as a second step (Fabrigar, Wegener, MacCallum, & Strahan, 1999). In order to do this, the sample (N = 526) was randomly split in half and each analysis was conducted with one half (n = 263 for the EFA and n = 263 for the CFA). As there were strong assumptions regarding the structure of the SelfEfficacy scale, this was only analyzed via CFA (N = 511). For the EFA and CFA the Mplus version 6.0 (Muthe`n & Muthe`n, 1998– 2012) was employed. It was decided to start with an EFA because this study is the first to examine the dimensional structure and the intention was to explore the data without any assumptions in the first instance. For the estimation of the model, Mplus uses the maximum likelihood (ML) estimator as default. The CAPES-DD variables are ordinal and not normally distributed, furthermore the data set has some missing values (see Section 3.2.1). Therefore, the robust maximum likelihood estimator (MLR) was employed, which produces standard errors and fit indices that are robust in relation to observations that are not normally distributed (Beauducel & Herzberg, 2006; Christ & Schlu¨ter, 2012). When using Mplus to do an EFA one has to specify the minimum and the maximum number of factors to be extracted. This decision was based on a scree-test (Cattell, 1966) and a parallel analysis (Horn, 1965) as well as previous analyses regarding the factor structure of the CAPES, the corresponding instrument for normally developed children. It was assumed that the identified factors found would correlate as the Intensity scale is supposed to assess behavioral and emotional problems in general with different kinds of problems as secondary dimensions. So an oblique rotation was applied (Preacher & MacCallum, 2003; the Mplus default is Geomin rotation which was not altered). The final decision for the model to be confirmed via CFA was based on the inspection of fit indices. Therefore the Tucker Lewis Index (TLI; Tucker & Lewis, 1973), the Comparative Fit Index (CFI; Bentler, 1990), the Standardized Root Mean Square Residual (SRMR; Bentler, 1995), and the Root Mean Square Error (RMSEA; Steiger & Lind, 1980) with a 90% confidence interval were inspected and compared between the different solutions in order to identify the model displaying the best fit. Hu and Bentler (1999) suggest that values .95 for the TLI and the CFI indicate good model fit and SRMR values .06 are acceptable. Regarding the RMSEA, values smaller than .05 indicate good fit, values between .05 and .08 represent acceptable fit and if the model shows a value bigger than .10 the fit can be called poor (Browne & Cudeck, 1992). Furthermore, the different factor solutions were compared via a chi-square difference test (Dx2) to see if the

T.S. Emser et al. / Research in Developmental Disabilities 53–54 (2016) 158–177

163

improvement of model fit in the less restrictive model was significant. Because the MLR was used, a correction developed by Satorra and Bentler (2001) had to be applied. Factor loadings smaller than .40 were considered to be too small to be relevant (Preacher & MacCallum, 2003) and were therefore dropped for further analysis. The final model for the Intensity scale derived from the EFA as well as a one-factor model of the Self-Efficacy scale was then tested via CFA. Just as it was done in the EFA, the MLR was employed. The TLI, the CFI, the SRMR and the RMSEA with 90% confidence interval were used to evaluate model fit. The values used as estimators for the goodness of the model fit are described above. Models were re-specified based on Modification Indices, inspection of standardized residuals and theoretical considerations (Byrne, 2012; Kline, 2011). To assess the extent to which a newly specified model exhibits an improvement over its predecessor, again the Satorra and Bentler Dx2-test (2001) was employed. 2.3.3. Concurrent validity In order to determine concurrent validity the associations between the Intensity scale of the CAPES-DD and two other instruments assessing behavioral and emotional problems in children and adolescents (DBC-P and DBC-U4) were examined. To do so, Pearson product moment correlations were calculated. 2.3.4. Predictive validity For assessing predictive validity, associations between the Intensity and the Self-Efficacy scale of the CAPES-DD and another instrument measuring parenting and parental as well as family adjustment (PAFAS) were examined. Furthermore the relation between the Self-Efficacy scale and the TBPS of the DBC-P was studied. For that, Pearson product moment correlations were calculated. 2.3.5. Reliability As a parameter for reliability, the internal consistency of the different scales presented in the factor analyses were calculated using the Cronbach’s alpha coefficient. In the applied research context, a minimum value of .80 should be targeted (Lance, Butts, & Michels, 2006). But the interpretation of the alpha coefficient always has to take the number of items used for the calculation into account (Cortina, 1993).

3. Results 3.1. Item properties 3.1.1. Intensity scale First, the analysis of the item properties was done based on the model suggested by the authors that intended to reverse the positively formulated items asking about strengths and to include them into two subscales describing externalizing and internalizing problems. But as factor analyses of the Intensity scale revealed a different structure (namely a three factor structure with one subscale assessing behavioral problems, one assessing emotional problems, and one assessing prosocial skills) they were redone based on this revised model. The report of the results regarding item properties is restricted to these second analyses. Means of the Items 1–20 ranged from M = 0.29 (SD = 0.63; Item 16: ‘‘Is inactive, listless’’) to M = 2.02 (SD = 0.83; Item 26: ‘‘Gets along with adults’’) with a possible maximum of 3.00. The item difficulties ranged from Pi = 9.67 (Item 16) to Pi = 67.33 (Item 26) with an average of P 130 ¼ 38:04: Except for Items 16 (9.67) and 4 (17.33), all of the items were within the middle range of 20–80. When Items 1–20 built their own scale (as suggested by EFA and CFA), their discriminatory powers ranged from rit(i) = 0.25 (Item 16) to rit(i) = 0.68 (Item 8) with an average of r itð120Þ ¼ 0:49: Four of the items (2, 5, 9 and 16) displayed values showing only weak discriminatory power (<.40). The discriminatory powers of Items 21–30 ranged from rit(i) = 0.45 (Item 24: ‘‘Keeps busy without adult attention’’) to rit(i) = 0.64 (Item 22: ‘‘Expresses feelings appropriately’’) with an average of r itð2130Þ ¼ 0:56: All of the items displayed discriminatory powers that can be considered as good. As Item 16 showed an item difficulty suggesting it is too difficult, as well as a discriminatory power that can be considered as poor, it was decided to drop this item and exclude it from further analyses. All of the other items showing problematic values (Intensity scale: items 2, 5, 9, 14) in any of the properties were left in the item pool for the following analyses because they only showed weaknesses in one indicator or because they were very close to the cut-off. 3.1.2. Self-Efficacy scale The means of the Items 1–20 ranged from M = 6.19 (SD = 2.54; Item 17: ‘‘Does not cooperate with requests’’) to M = 7.86 (SD = 2.48; Item 16: ‘‘Is inactive, listless’’) with a possible maximum of 10.00. Item difficulties ranged from Pi = 57.67 (Item 17) to Pi = 76.22 (Item 16) with an average of P 120 ¼ 66:200: All of the difficulties fell into the middle range. The discriminatory powers ranged from rit(i) = 0.52 (Item 2: ‘‘Spends too much time on their own’’) to rit(i) = 0.75 (Item 12: ‘‘Becomes upset over changes to routines or surroundings’’) with an average of r itð120Þ ¼ 0:65: Six out of the 20 items (8, 12, 15, 17, 18, and 19) showed a discriminatory power bigger than .70 which indicates poor discriminatory power. Nevertheless, these items were left in the item pool because they only showed weaknesses in one indicator or because they were very close to the cut-off.

164

T.S. Emser et al. / Research in Developmental Disabilities 53–54 (2016) 158–177

3.2. Construct validity 3.2.1. Preliminary analysis Regarding the Intensity scale, 0.6% of the values were missing with no item missing in more than two percent of cases. Items 13 (‘‘Breaks or destroys things’’) and 10 (‘‘Demands attention’’) were missing most often (1.5%). Little’s MCAR test was not significant (x2 (787) = 772.44, p = .638) indicating that the data was missing completely at random (MCAR). In contrast, 7.9% of the values of the Self-Efficacy scale were missing with 19 out of the 20 items missing more often than in two percent of the cases. Item 16 had the highest number of missing values (13.5%). Nevertheless, Little’s MCAR test was not significant (x2 (1212) = 1234.35, p = .312) indicating that the data was MCAR. Therefore the Mplus full information procedure (FIML) was used to handle missing data, which has been shown to outperform traditional approaches for handling missing data when the data are MCAR or missing at random (MAR; Enders, 2001; Graham, 2009). For the analysis of univariate and multivariate normality, the items 21–30 of the Intensity scale were left in their original form and not reversed, as a reversed sign of the skewness values was the only difference resulting for reversing them. Five out of the 30 items of the Intensity scale showed values of skewness bigger than I1I ðx ¼ :56Þ; and the same accounts for nine items regarding kurtosis ðx ¼ :28Þ: For 18 out of the 30 items skew was significant and 21 items showed significant kurtosis. Regarding the SelfEfficacy scale, two out of 20 items presented themselves with values of skewness bigger than I1I ðx ¼ :64Þ and the kurtosis of one item exceeded the value of I1I ðx ¼ :53Þ: Nineteen of the items showed significant skew and nine showed significant kurtosis. Inspection of the stem-leaf diagrams revealed 261 (1.66%) extreme data points for the Intensity scale as well as 33 (0.35%) for the Self-Efficacy scale. These were transformed by recoding values into the next lower/higher (not extreme) value (Intensity scale: 3, 4, 5, 6, 13, 18, 20: 3 ! 2; 29: 0 ! 1; Self-Efficacy scale: 5, 16: 1 ! 2 and 2 ! 3). A review of D2 showed minimal evidence for serious multivariate outliers. 3.2.2. Exploratory factor analysis Before the EFA was applied, a scree-test (Cattell, 1966) and a parallel analysis (Horn, 1965) were performed in order to determine the number of factors to be extracted. In the scree-plot there were three eigenvalues falling before the last substantial drop. For the parallel analysis, five eigenvalues lay above the intersection point of the line representing the sample and the one derived from random data. However, the last two eigenvalues before the intersection point were very close to the randomly created plot. Based on these results and on the original idea of two factors, it was decided to set the minimal number of factors to one (which is always good as a baseline model for comparisons) and a maximum of three. First, the EFA was started using items 21–30 in their reversed form because the original idea of the authors was that they would form individual factors together with the other items of the Intensity scale. But the inspection of the data indicated that the reversed items seemed to build their own factor. Therefore the analysis was repeated with the original items. The Satorra and Bentler chi square difference test between the two factor and the three factor solution was significant (Dx2 (30) = 204.84, p < .001), indicating that the three factor solution represents the data better. So it was decided to proceed with a three-factor model. In the following, all the items showing loadings smaller than .40 were eliminated (2, 4, 5, 7, 9, 14; Preacher & MacCallum, 2003). Items 23 and 29 were also eliminated because they loaded highly on more than one factor. Item 16 was eliminated because of its problematic item properties. Another EFA was conducted with the resulting item pool. Table 2 shows the factor loadings of the so generated three-factor solution. Table 2 Factor loadings of the three factor solution exploratory factor analysis (n = 263). Item

Factor 1

Factor 2

Factor 3

1 6 8 10 11 13 15 17 19 3 12 18 20 21 22 24 25 26 27 28 30

.89 .58 .79 .66 .72 .63 .54 .58 .65 .23 .54 .36 .40 .05 .01 .01 .09 .02 .23 .00 .19

.03 .02 .02 .18 .09 .13 .34 .07 .18 .48 .17 .50 .49 .18 .21 .27 .04 .00 .11 .01 .01

.10 .13 .01 .03 .09 .02 .02 .14 .03 .01 .01 .10 .02 .57 .66 .46 .67 .58 .62 .71 .60

Note. Factor loadings .40 are in boldface indicating which item loads on which factor.

T.S. Emser et al. / Research in Developmental Disabilities 53–54 (2016) 158–177

165

This model showed satisfactory to good model fit with RMSEA = 0.06, CFI = 0.92, TLI = 0.88 and SRMR = 0.04. Therefore it was decided to use it as the one to be confirmed via CFA. In contrast to the original model proposed by the authors of the CAPES-DD, the data revealed three factors with the positively worded items building one individual factor. Opposed to the original idea, item 12 showed higher loadings on the first factor than on the second. Therefore it was relocated for following analyses. 3.2.3. Confirmatory factor analysis The CFA of the Intensity scale started with the testing of a single factor model (Model A) which served as a baseline model for further comparisons with a two factor model (Model B) and the three-factor model (Model C) derived from EFA. As Table 3 shows, the fit of Model B was significantly better than the one of Model A, Dx2 (22) = 419.79, p < .001. Furthermore, Model C displayed an even better fit than Model B which also turned out to be significant, Dx2 (2) = 12.83, p < .001. So it was decided to regard the three factor model as the one most appropriate in representing the data. Model C already showed acceptable fit in all of the fit indices. However, an attempt was made to further improve the fit. The inspection of the MIs revealed that allowing the error terms of items 11 and 13 as well as the error terms of items 3 and 20 to correlate could further improve model fit. Changes were made one at a time. Results for that model (Model C1) can also be seen in Table 3. Again, Model C1 displayed a significantly better fit than Model C, Dx2 (2) = 116.53, p < .001. Item 11 (‘‘Hurts me or others [e.g., hits, bites, scratches, pinches, pushes]’’) and item 13 (‘‘Breaks or destroys things’’) both refer to behavior that is connected to hurting or destroying. Item 3 (‘‘Seems fearful and scared’’) and item 20 (‘‘Seems unhappy or sad’’) both describe negative feelings the child can experience. Thus, the correlations are theoretically sensible. Intercorrelations of the three factors were all significant with r = 0.87 for the Emotional problems and the Behavioural problems scale. The Prosocial behaviour scale showed a correlation of r = 0.39 with the Behavioural problems scale and r = 0.47 with the Emotional problems scale (for details please refer to Table 4). Although items 4 (‘‘Hurts themselves [e.g., hits, bites, scratches, pinches]), 7 (‘‘Becomes upset when separated from familiar people’’) and 14 (‘‘Fusses over food or refuses to eat’’) loaded less than .40 on the factors, the authors considered them to be highly clinically relevant. Meaning, it would be very useful for the clinician to know if the child exhibited these problems. Therefore they decided to leave them in the item pool contributing to a Total Problem Score but not to include them into the subscales. To adjust the Self-Efficacy scale to the new version of the Intensity scale as it turned out in the EFA, the number of items of the Self-Efficacy scale was shortened as well by eliminating the respective items (2, 5, 9, 16) but leaving those three items (4, 7, 14) considered as clinically relevant in the scale resulting in 16 items. The CFA of the Self-Efficacy scale only contained the testing of a single factor model (Model D), because strong assumptions existed regarding its

Table 3 Fit Indices of the confirmatory factor analysis of the CAPES-DD. Model

x2

df

Dx2

Ddf

CFI

TLI

SRMR

RMSEA

Intensity scale A: Single factor model B: Two factor model

924.75 487.78

210 188

419.79***

22

.62 .84

.59 .82

.12 .06

.11 .08

2

.85

.83

.06

.08

2

.88

.87

.06

.07

1

.91 .92

.90 .91

.05 .05

.07 .07

(A-B)

C: Three factor model

468.86

186

12.83*** (B-C)

C1: Three factor model with correlated errors between Items 11 and 13 and Items 3 and 20 Self-Efficacy scale D: Single factor model D1: Single factor model with correlated errors between Items 3 and 20

400.96

184

116.53*** (C-C1)

361.16 323.228

104 103

32.16*** (D-D1)

Note. x2 = Satorra–Bentler scaled chi-square, df = degrees of freedom, CFI = comparative fit index, SRMR = standardized root mean square residual, RMSEA = root mean square error of approximation, CI = confidence interval. Models of the Intensity scale based on n = 263, models of the Self-efficacy scale based on N = 511. *** p < .001.

Table 4 Correlations between the three factors of Model C1 of the CFA.

1. Behavioural problems scale 2. Emotional Problems scale 3. Prosocial behaviour scale Note. *** p < .001.

1

2

3

– .87*** .39***

– .47***

–

[(Fig._3)TD$IG]

166

T.S. Emser et al. / Research in Developmental Disabilities 53–54 (2016) 158–177

1. Loses their temper 3. Makes rude noises or says rude words .765***

.564***

4. Yells, shouts or screams

.797***

5. Demands attention 6. Hurts me or others (e.g., hits, bites, scratches, pinches, pushes)

.671*** .683***

Behavioural problems

.597*** .413***

7. Becomes upset over changes to routines or surroundings 8. Breaks or destroys things

.601*** .696*** .556***

9. Whines or complains

.608*** .874***

10. Does not cooperate with requests

12. Is overactive or restless

2. Seems fearful and scared .379***

11. Cries easily for no apparent reason

13. Seems unhappy or sad

.459*** -.393***

.663***

Emotional problems

.511***

16. Cooperates with self-care routines (e.g., getting dressed) .488***

17. Expresses feelings appropriately .718***

18. Keeps busy without adult attention

19. Comforts others who are upset or hurt 14. Gets along with adults 20. Shares with others 15. Makes requests appropriately

-.470**

.461*** .584***

Prosocial behaviour

.561*** .686*** .729*** .611***

21. Gets along with peers Fig. 3. Factor structure of the Intensity scale with one correlated error term, intercorrelations of the factors and standardized estimates. **p < .05 ***p < .001. Numbers according to item numbers in the final version of the CAPES-DD (Appendix B).

[(Fig._4)TD$IG]

T.S. Emser et al. / Research in Developmental Disabilities 53–54 (2016) 158–177

167

1. Loses their temper 3. Makes rude noises or says rude words 4. Yells, shouts or screams 5. Demands attention 6. Hurts me or others (e.g., hits, bites, scratches, pinches, pushes)

.697** .660** .798** .741** .698**

7. Becomes upset over changes to routines or surroundings

.789**

8. Breaks or destroys things

.737**

9. Whines or complains

.778** .812**

10. Does not cooperate with requests 11. Cries easily for no apparent reason 12. Is overactive or restless

Parental SelfEfficacy

.746** .761** .592** .632**

2. Seems fearful and scared .603**

.369**

13. Seems unhappy or sad

.687**

14. Hurts themselves

.569**

15. Becomes upset when separated from familiar people 16. Fusses over food or refuses to eat Fig. 4. Factor structure of the Self-Efficacy scale with one correlated error term and standardized estimates. **p < .001. Numbers according to item numbers in the final version of the CAPES-DD (Appendix B).

dimensional structure. Model D showed an acceptable fit for the CFI (.91), TLI (.90) and RMSEA (.07) as well as a good fit for the SRMR (.05). However, an attempt was made to further improve the fit. The inspection of the MIs revealed that allowing the error terms of items 3 and 20 to correlate could improve the model fit (Model D1). Model D1 demonstrated a significant improvement compared to Model D, Dx2 (1) = 32.16, p < .001. Fit indices for this model are also included in Table 4. Item 3 (‘‘Seems fearful and scared’’) and item 20 (‘‘Seems unhappy or sad’’) both refer to problems that can be considered as emotional problems and those are items representing the Emotional problems scale of the Intensity scale which makes the correlation theoretically sensible. Graphical illustrations of Models C1 and D1 are presented in Figs. 3 and 4. 3.2.4. Concurrent validity All of the correlations between the subscales of the CAPES-DD and the ones of the DBC-P were significant at least at an alpha-level of .05. As the evaluation of the factor structure of the DBC-U4 is still ongoing at the moment, correlations between this scale and the CAPES-DD were restricted to the TBPS only. As expected, CAPES-DD subscales indicating problems all correlated positively with the subscales of the DBC-P and the TBPS of the DBC-U4 ranging from 0.39 (CAPES-DD Emotional problems and DBC-P Self-absorbed) to 0.85 (CAPES-DD Behavioural problems and DBC-P Disruptive/antisocial and DBC-U4 TBPS). The CAPES-DD subscale assessing prosocial behavior correlated negatively with all of the DBC-P subscales as well as the TBPS of the DBC-U4 ranging from 0.16 (DBC-P Anxiety) to 0.38 (DBC-U4 TBPS). All of the correlations can be found in Table 5.

T.S. Emser et al. / Research in Developmental Disabilities 53–54 (2016) 158–177

168

Table 5 Pearson Correlations among the Subscales of the CAPES-DD and the DBC-P (N = 383)/DBC-U4 (N = 73). 1 1. CAPES-DD Total problems 2. CAPES-DD Behavioural problems 3. CAPES-DD Emotional problems 4. CAPES-DD Prosocial behaviour 5. DBC-P TBPS 6. DBC-P Disruptive/antisocial 7. DBC-P Self-absorbed 8. DBC-P Communication disturbance 9. DBC-P Anxiety 10. DBC-P Social relating 11. DBC-U4 TBPS M SD

2

3

4

5

6

7

8

9

10

11

– .96**

–

**

.59**

.34**

.34**

.19**

.85**

.80**

.62**

.37**

**

**

**

.57

**

.30

.87**

–

.75

– – –

.83

.85

.64**

.61**

.39**

.36**

.87**

.61**

–

.63**

.59**

.48**

.19**

.81**

.61**

.66**

**

**

**

**

.71

**

**

**

.58**

**

**

**

.68

**

.52

**

.58

.50

.86**

.85**

18.45 10.02

12.94 7.17

.65

**

.16

**

.55

.35

.71

.58**

.38**

2.44 2.18

11.31 4.43

.56

**

.47

– – .57**

–

N/A

N/A

N/A

–

8.41 4.90

7.37 4.06

6.27 3.47

79.51 49.80

.50

.57

.59

N/A

N/A

N/A

65.97 30.07

21.11 11.41

20.70 11.25

Note. N/A = not available. ** p < .01, two-tailed.

3.2.5. Predictive validity As expected, in most cases the CAPES-DD subscales assessing child maladjustment correlated significantly and positively with all of the PAFAS constructs indicating less effective parenting strategies as well as family maladjustment ranging from 0.11 (CAPES-DD Total Problems and PAFAS Parenting – Parent-Child Relationship) to 0.42 (CAPES-DD Total Problems and PAFAS Parenting – Coercive Parenting). Positive encouragement failed to produce significant correlations with the CAPES-DD constructs on the significance level <0.01. Also CAPES-DD Emotional Problems did not correlate significantly with all of the PAFAS constructs, probably due to its brevity. The CAPES-DD Prosocial behaviour as well as CAPES-DD Self-Efficacy correlated significantly and negatively with the TBPS of the DBC-P (r = 0.37) and with most of the PAFAS subscales ranging from 0.14 (CAPES-DD Prosocial Behaviour and PAFAS Family Adjustment – Family relationships) to 0.41 (CAPES-DD SelfEfficacy and PAFAS Family Adjustment – Parental Adjustment). See Table 6 for these correlations. 3.2.6. Reliability Reliability analyses were done based on the original model suggested by the authors (Model 1) as well as for the model derived from EFA and CFA (Model 2). Regarding the Self-Efficacy scale there is one alpha coefficient presented for the original version and one for the abridged version as a result of the factor analyses (see Sections 3.2.2 and 3.2.3). The results are displayed in Table 7. The internal consistency of the Intensity scale based on Model 1 is .91 with a = .88 for the Externalizing subscale and a = .81 for the Internalizing subscale. Analyses based on Model 2 show an alpha coefficient of .89 for the Behavioural problems scale and a = .71 for the Emotional Problems scale. Cronbach’s a of the Prosocial behaviour scale is .82. Reliability analyses of the Total Problems scale after including the three items which were excluded from the subscales (4, 7 and 14) showed the same Cronbach’s alpha as they did without them (a = .90). The same accounts for the abridged version of the Self-Efficacy scale (a = .94 with or without the items). 4. Discussion The present study established that the CAPES-DD shows a three-factor structure assessing emotional and behavioral problems as well as prosocial skills. The Total Problems scale, Behavioural Problems and Emotional Problems subscales, Prosocial Behaviour scale, and the Self-Efficacy scale all turned out to have very good convergent and predictive validity as well as satisfactory to good internal consistency. Thorough and explicit analyses were also performed in order to establish construct validity. The final 24-item CAPES-DD inventory is outlined below (See Appendix B). The analysis of item properties as well as factorial analyses suggested the elimination of nine of the initial items. Item 16 (‘‘Is inactive, listless’’) showed problematic item properties in every variable (mean, SD, difficulty and discriminatory power) assessed. EFA revealed that another eight items (Items 2, 4, 5, 7, 9, 14, 23, and 29) should be removed due to their small loadings in order to make the scale more valid. Those items showed several weaknesses regarding their item properties which could have led to the small loadings. Items 2, 5, and 9 all have rather low means,

T.S. Emser et al. / Research in Developmental Disabilities 53–54 (2016) 158–177

169

Table 6 Pearson Correlations among the Subscales of the CAPES-DD and the PAFAS (N = 382)/the TBPS of the DBC-P (N = 383). 1 1. CAPES-DD Total problems 2. CAPES-DD Behavioural problems 3. CAPES-DD Emotional problems 4. CAPES-DD Prosocial behaviour 5. CAPES-DD Self-efficacy 6. PAFAS parenting Parental consistency 7. PAFAS parenting Coercive parenting 8. PAFAS parenting Positive encouragement 9. PAFAS parenting Parent–child relationship 10. PAFAS family adjustment Parental adjustment 11. PAFAS Family adjustment Family relationships 12. PAFAS family adjustment Parental teamwork 13. DBC-P TBPS M SD

2

3

4

5

6

7

8

9

10

11

12

13

– .96**

–

**

.59**

.34**

.34**

.19**

.50**

.48**

.42**

**

**

.75

–

.16

.06

.39**

.42**

.29**

*

.11

**

.10* **

.16

**

.35** **

.16

.13*

–

.21

.28**

–

.00

.39**

.25**

–

.10

.14**

.01

.12*

.08

**

.03

.24

**

.25

.41

**

.20 **

–

**

*

–

**

.33**

–

.21

**

.36

.02

.29**

–

.13

**

.34

.32

.31

.28

.35**

.33**

.29**

.14*

.31**

.21**

.36**

.08

.23**

.44**

–

.29**

.29**

.20**

.10

.17**

.23**

.24**

.02

.14*

.49**

.55**

–

N/A

N/A

N/A

N/A

N/A

N/A

N/A

–

4.97 2.60

4.50 2.66

2.01 1.61

1.98 2.35

6.92 3.10

4.16 2.75

2.75 2.03

65.97 30.07

.37 18.45 10.02

12.94 7.17

2.44 2.18

11.31 4.43

110.31 31.21

**

Note: N/A = not available. * p < .05, two-tailed. ** p < .01, two-tailed.

Table 7 Cronbach’s Alpha Coefficients for the Original Model (Model 1) and the Model derived from the Factor Analyses (Model 2) (N = 526). Cronbach’s a Model 1 Intensity scale Externalizing subscale Internalizing subscale Model 2 Total Problems scalea Behavioural Problems scale Emotional Problems scale Prosocial Behaviour scale Self-Efficacy scale, original Self-Efficacy scale, abridged versiona

.91 .88 .81 .90 .89 .71 .82 .94 .94

Note. a Including the 3 additional items.

so parents did not report those problems as very prevalent. Items 2 (‘‘Spends too much time on their own’’), 5 (‘‘Stares at objects or into space’’) and 9 (‘‘Makes repeated hand or body movement [e.g., hand flapping, rocking]’’) additionally show problematic discriminatory powers indicating that these items are not able to differentiate between people with low or high values well. Item 23 (‘‘Seems to feel good about themselves’’) showed a high cross-loading on the Emotional problems scale. Regarding its content it seems like a positive version of Item 20 (‘‘Seems unhappy or sad’’) which loads on the Emotional Problems subscale. This could explain the cross-loading and make Item 23 redundant. Item 29 showed a high cross-loading on the Behavioural Problems scale. Thus, it is not possible to distinctly allocate it to one or the other scale and therefore it had to be eliminated from the item pool. Items 4 (‘‘Hurts themselves [e.g., hits, bites, scratches, pinches]’’), 7 (‘‘Becomes upset when separated from familiar people’’) and 14 (‘‘Fusses over food or refuses to eat’’) were eliminated mainly because they were not reported very often and did not load highly enough on the factors. Nevertheless the authors consider them as highly clinically relevant, meaning that even if those problems are not very

170

T.S. Emser et al. / Research in Developmental Disabilities 53–54 (2016) 158–177

prevalent it is important to know about their existence for those children who show them. Following Arnold, O’Leary, Wolff, and Acker (1993) those items were included into the Total Problems scale but leaving them out of the subscales. Reliability analyses of the Total Problems scale with or without those items did not show any differences, therefore the authors decided to leave them in the item pool. EFA and CFA of the Intensity scale thus supported a three-factor structure with 13 items describing behavioral (Behavioural Problems subscale, ten items) and emotional problems (Emotional Problems subscale, three items) and eight items describing prosocial skills (Prosocial Behaviour scale). With the initially surprising additional factor, strengths rather than problems exhibited by the children can be assessed. This is indeed helpful if one chooses a therapeutic approach that focuses on the existing skills a child has in his or her behavioral repertoire and which can be used for solving current problems. This resource-oriented approach is becoming increasingly popular in therapy approaches (Priebe, Omer, Giacco, & Slade, 2014). Furthermore, it can be very encouraging for parents to hear about what their children can do and not only talk about problems. This is also consistent with parenting principles like the importance of reinforcing desirable behavior. In order to adapt the Self-Efficacy scale to the length of the Intensity scale, corresponding items (Items 2, 5, 9, and 16) were eliminated but leaving the clinically relevant items (4, 7, and 14). Regarding this abridged version, CFA supported a 16-item, one-factor structure. Again, there was no difference in the internal consistency of the scale with or without those items. The CAPES-DD showed very good convergent and predictive validity investigated via correlations with corresponding instruments. Also, the analysis of the internal consistency provided support for satisfactory to very good reliability of the CAPES-DD. However, the internal consistency of the Emotional Problems subscale cannot be interpreted as sufficient for the applied research context probably due to the small number of items (Items 3, 18 and 20) representing it. The number of variables representing one common factor should be at least three to five (Fabrigar et al., 1999; MacCallum, Widaman, Zhang, & Hong, 1999). Furthermore the intercorrelation of the two factors describing problems can be considered as high (r = 0.87) and two items (Items 18 and 20) have crossloadings with factor one, which is also an indicator for their similarity and which could suggest that it would be better to regard them as one factor. Nevertheless, in the EFA as well as the CFA the three-factor solution showed a significantly better fit than the two factor solution that considered emotional and behavioral problems as representing one factor. Furthermore, retaining the Emotional Problems subscale has clinical utility and is consistent with research indicating two broad behavior problem factors (i.e., internalizing/externalizing; Achenbach, 1966). Thus, the idea of dividing the problems into two different kinds of problems seems reasonable but the inclusion of some additional items assessing emotional problems has to be considered. Further studies should be conducted to expand the type of disability and age-range represented. As the target group of children with developmental disabilities is a very heterogeneous one, it may be necessary to evaluate the CAPES-DD for different disability groups individually. In this study, children mainly had ASD (My Say: 91.5%; Curtin Study: 57.1%), some disabilities are only rarely represented (like cerebral palsy), and some like multiple sclerosis are not represented at all. Thus, the dominance of children with ASD made it impossible to compare disability groups in this study. The CAPES-DD was intended for children aged from two to 16 years, but in the My Say survey the oldest child was eleven and about 90% were nine years of age or younger. Although the full range of ages was tapped in the Curtin Study, 87% of the children were 12 or younger. As the prevalence and kind of problems varies over the course of time (Horiuchi et al., 2014; Lahey et al., 2000), a wide range of ages is necessary for reliable analyses of individual age groups. Therefore, future studies should aim to collect data of children within the older age range. Furthermore, the CAPES-DD is supposed to be appropriate for a range of respondents (e.g., parents, carers, teachers). Unfortunately, the sample of this study mainly consists of mothers (My Say: 91.8%; Curtin Study: 83.1%). Thus, it is not possible to make substantial statements about the reliability and validity of the instrument for different respondent groups. Taking into account that there is evidence for informant differences regarding the report of problem behavior (Van der Ende & Verhulst, 2005), it seems even more important to target other caregivers than mothers in future studies. The goal in developing the CAPES-DD was to establish an instrument with advantages over existing measures. The CAPESDD is very short and may therefore be a more preferable choice than time consuming instruments like the CBCL and DBC-P. In contrast to the SDQ, the CAPES-DD showed satisfactory to very good internal consistency which could probably be improved even more once additional items are added to the Emotional problems subscale. Now that it contains a factor additionally assessing strengths, the CAPES-DD may be more able to provide a comprehensive picture of the children and some basics for a resource oriented therapeutic approach than the DBC that completely focuses on the problems children exhibit. Finally, none of the existing measures has a scale assessing PSE directly linked to the problems shown by the children. By including a scale assessing PSE the CAPES-DD decreases the assessment burden on families but still provides information about two different constructs. The Self-Efficacy scale of the CAPES-DD turned out to be reliable and valid and therefore represents a sensible and important addition to the Intensity scale. Nevertheless, further research is needed regarding the psychometric value of the inventory. Specifically it should aim to develop norms for individual disability groups, age norms and to investigate gender differences, as well as differences among raters. Another limitation of this study is that it permitted the reporting of comorbid disabilities by allowing caregivers to indicate that their child had more than one diagnosis which made it difficult to differentiate exactly between disabilities. However, this represents the reality of children with DD where it is the rule rather than the exception to have more than one disability. Furthermore, taking into account that several already existing measures are not very sensitive to change, future research should include analyses of the CAPES-DD’s change sensitivity.

T.S. Emser et al. / Research in Developmental Disabilities 53–54 (2016) 158–177

171

5. Conclusions This first psychometric evaluation of the CAPES-DD provides substantial support for the reliability and validity of the instrument. With its very economical length of now 24 items, the CAPES-DD has advantages for parents to complete and for its administration in large population studies. The addition of a factor that assesses prosocial behavior and skills improves the ability of the CAPES-DD in providing a comprehensive picture of the assessed child. The Self-Efficacy scale assessing a construct that is highly important in the context of parenting and parenting interventions turned out to be a reliable and valid addition to the instrument. The CAPES-DD is to be in the public domain and free for practitioners and researchers to use. Conflict of interest The Triple P-Positive Parenting Program is owned by The University of Queensland. The University, through its technology transfer company Uniquest Pty Ltd., has licensed Triple P International Pty Ltd to disseminate the program worldwide. Royalties stemming from this dissemination work are distributed to the Faculty of Health and Behavioural Sciences, the School of Psychology, Parenting and Family Support Centre, and contributory authors in accordance with the University’s intellectual property policy. No author has any share or ownership in Triple P International. M. S. is the founder and lead author of the Triple P-Positive Parenting Program and is a consultant to Triple P International. T. G. M. is a co-author of Stepping Stones Triple P. Acknowledgements We wish to acknowledge funding support from the Australian Government’s National Health and Medical Research Council as the major funding body, whose support made this research possible. This research was also funded in part by a grant of the Faculty of Health Sciences funded through the Dean of Research and School of Psychology and Speech Pathology, Curtin University. We thank Associate Professor Kate Sofronoff, Dr. Julie Hodges, and Ms. Martha Schoch who lead and coordinate the Stepping Stones Triple P project at the Parenting and Family Support Centre of the University of Queensland. We also thank Dr. Ania Filus for several fruitful discussions about statistics.

Appendix A. Child Adjustment and Parent Efficacy Scale-Developmental Disability (CAPES-DD) – A list of the original 30 items Item 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23. 24. 25. 26 27. 28. 29. 30.

Loses their temper Spends too much time on their own Seems fearful and scared Hurts themselves (e.g., hits, bites, scratches, pinches) Stares at objects or into space Makes rude noises or says rude words Becomes upset when separated from familiar people Yells, shouts or screams Makes repeated hand or body movements (e.g., hand flapping, rocking) Demands attention Hurts me or others (e.g., hits, bites, scratches, pinches, pushes) Becomes upset over changes to routines or surroundings Breaks or destroys things Fusses over food or refuses to eat Whines or complains Is inactive, listless Does not cooperate with requests Cries easily for no apparent reason Is overactive or restless Seems unhappy or sad Cooperates with self-care routines (e.g., getting dressed) Expresses feelings appropriately Seems to feel good about themselves Keeps busy without adult attention Comforts others who are upset or hurt Gets along with adults Shares with others Makes requests appropriately Waits patiently for what they want Gets along with peers

172

T.S. Emser et al. / Research in Developmental Disabilities 53–54 (2016) 158–177

Appendix B. Child Adjustment and Parent Efficacy Scale-Developmental Disability (CAPES-DD) – Final version [TD$INLE]

Please read each statement and circle a number on the scale to show how true the statement was of your child over the past 4 weeks. Then, using the scale provided, write down the number next to each item that best describes how confident you are that you can successfully manage that behaviour, even if it is a behaviour that rarely occurs or does not concern you. There are no right or wrong answers. Do not spend too much time on any statement. Example: How true is this of your child?

Rate your confidence

My child: Has temper tantrums

0

1

2

3

7

The rating scale is as follows: 0. Not true of my child at all 1. True of my child a little, or some of the time 2. True of my child quite a lot, or a good part of the time 3. True of my child very much, or most of the time How true is this of your child? 0 1 2 3 My child:

Not at all

A little

Quite a lot

Very much

1.

Loses their temper

0

1

2

3

2.

Seems fearful and scared

0

1

2

3

3.

Makes rude noises or says rude words

0

1

2

3

4.

Yells, shouts or screams

0

1

2

3

5.

Demands attention

0

1

2

3

6.

Hurts me or others (e.g., hits, bites, scratches, pinches, pushes)

0

1

2

3

Becomes upset over changes to routines or surroundings

0

1

2

3

8.

Breaks or destroys things

0

1

2

3

9.

Whines or complains

0

1

2

3

10.

Does not cooperate with requests

0

1

2

3

11.

Cries easily for no apparent reason

0

1

2

3

12.

Is overactive or restless

0

1

2

3

7.

Rate your confidence (from 1-10) 1 – Certain I can’t manage it to 10 – Certain I can manage it

T.S. Emser et al. / Research in Developmental Disabilities 53–54 (2016) 158–177

[TD$INLE]

13.

Seems unhappy or sad

0

1

2

3

14.

Hurts themselves (e.g., hits, bites, scratches, pinches)

0

1

2

3

15.

Becomes upset when separated from familiar people

0

1

2

3

16.

Fusses over food or refuses to eat

0

1

2

3

How true is this of your child? 0 1 2 3 My child:

Not at all

A little

Quite a lot

Very much

17.

Gets along with adults

0

1

2

3

18.

Makes requests appropriately

0

1

2

3

19.

Cooperates with self-care routines (e.g., getting dressed)

0

1

2

3

20.

Expresses feelings appropriately

0

1

2

3

21.

Keeps busy without adult attention

0

1

2

3

22.

Comforts others who are upset or hurt

0

1

2

3

23.

Shares with others

0

1

2

3

24.

Gets along with peers

0

1

2

3

173

174

T.S. Emser et al. / Research in Developmental Disabilities 53–54 (2016) 158–177

[TD$INLE]

Scoring Key for the CAPES-DD To obtain a Behavioural Problems subscale score, sum “how true” ratings for items 1, 3, 4, 5, 6, 7, 8, 9, 10 and 12, with a possible range from 0-30. To obtain an Emotional Problems subscale score, sum “how true” ratings for items 2, 11, and 13 with a possible range from 0-9. To obtain a Total Problems scale score, sum the Behavioural Problems subscale score, the Emotional Problems subscale score and the “how true” ratings for the three additional items (14, 15, 16), with a possible total range from 0-48. Higher scores indicate greater levels of child emotional or behavioural problems. To obtain a Prosocial Behaviour scale score, sum "how true" ratings for items 17, 18, 19, 20, 21, 22, 23 and 24, with a possible range from 0-24. Self-Efficacy Scale: sum all parent confidence ratings for the emotional or behavioural problems (rating scale 110). Note that there are no parent confidence ratings for the prosocial behaviour scale. Possible range for the total score is 16-160 with higher scores indicating greater levels of parent self-efficacy.

Enter “how true” ratings

Enter confidence ratings

Behavioural Problems Subscale

Self-Efficacy Scale

1.

Temper

1.

3.

Rude

3.

[TD$INLE] 4.

Yells, shouts, screams

4.

5.

Demands attention

5.

6.

Hurts others

6.

7.

Upset over changes

7.

8.

Breaks or destroys things

8.

9.

Whines or complains

9.

10.

Does not cooperate with requests

10.

12.

Overactive or restless

12.

Total

Emotional Problems Subscale

[TD$INLE]

T.S. Emser et al. / Research in Developmental Disabilities 53–54 (2016) 158–177 2.

11.

13.

Fearful and scared

2.

Cries easily

11.

Unhappy or sad

13.

Total

Additional Items 14.

Hurts themselves

14.

15.

Upset when separated

15.

16.

Fusses or refuses to eat

16.

Total

Total

Total Problems Scale (Sum of Behavioural Problems, Emotional Problems, and Additional Items)

[TD$INLE]

Total

Prosocial Behaviour Scale 17.

Gets along with adults

18.

Makes requests appropriately

19.

Cooperates with self-care routines

20.

Expresses feelings appropriately

21.

Keeps busy

22.

Comforts others

23.

Shares with others

24.

Gets along with peers

Total

175

176

T.S. Emser et al. / Research in Developmental Disabilities 53–54 (2016) 158–177

References Achenbach, T. M. (1966). The classification of children’s psychiatric symptoms: A factor-analytic study. Psychological Monographs: General and Applied, 80, 1–37. Achenbach, T. M., & Rescorla, L. A. (2001). Manual for the ASEBA School-Age Forms and Profiles. Burlington, VT: University of Vermont, Research Center for Children, Youth & Families. Aman, M. G., & Singh, N. N. (1994). Aberrant Behavior Checklist – Community, Supplementary Manual. East Aurora, NY: Slosson Educational Publications. Arnold, D. S., O’Leary, S. G., Wolff, L. S., & Acker, M. M. (1993). The Parenting Scale: A measure of dysfunctional parenting in discipline situations. Psychological Assessment, 5, 137–144. http://dx.doi.org/10.1037/1040-3590.5.2.137 Beauducel, A., & Herzberg, P. Y. (2006). On the performance of maximum likelihood versus means and variance adjusted weighted least squares estimation in CFA. Structural Equation Modeling: A Multidisciplinary Journal, 13, 186–203. http://dx.doi.org/10.1207/s15328007sem1302_2 Bentler, P. M. (1990). Comparative fit indexes in structural models. Psychological Bulletin, 107, 238–246. http://dx.doi.org/10.1037//0033-2909.107.2.238 Bentler, P. M. (1995). EQS structural equations program manual. Encino, CA: Multivariate Software. Boyle, C. A., Boulet, S., Schieve, L. A., Cohen, R. A., Blumberg, S. J., Yeargin-Allsopp, M., et al. (2011). Trends in the prevalence of developmental disabilities in US children, 1997–2008. Pediatrics, 127, 1034–1042. http://dx.doi.org/10.1542/peds.2010-2989 Browne, M. W., & Cudeck, R. (1992). Alternative ways of assessing model fit. Sociological Methods and Research, 21, 230–258. http://dx.doi.org/10.1177/ 0049124192021002005 Bruininks, R. H., Woodcock, R. W., Weatherman, R. F., & Hill, B. K. (1984). Scales of independent behavior. Allen, TX: DLM Teaching Resources. Byrne, B. M. (2012). Structural equation modeling with Mplus: Basic concepts, applications, and programming. New York, NY: Routledge. Cattell, R. B. (1966). The scree test for the number of factors. Multivariate Behavioral Research, 1, 245–276. Christ, O., & Schlu¨ter, E. (2012). Strukturgleichungsmodelle mit Mplus, eine praktische Einfu¨hrung. Mu¨nchen: Oldenbourg Verlag. Coleman, P. K., & Karraker, K. H. (1998). Self-efficacy and parenting quality: Findings and future applications. Developmental Review, 18(1), 47–85. Cortina, J. M. (1993). What is coefficient alpha? An examination of theory and applications. Journal of Applied Psychology, 78(1), 98–104. http://dx.doi.org/ 10.1037//0021-9010.78.1.98 Einfeld, S. L., & Tonge, B. J. (1996). Population prevalence of psychopathology in children and adolescents with intellectual disability. II: Epidemiological findings. Journal of Intellectual Disability Research, 40, 99–109. http://dx.doi.org/10.1046/j.1365-2788.1996.768768.x Einfeld, S. L., & Tonge, B. J. (2002). Manual for the Developmental Behaviour Checklist: Primary Carer Version (DBC-P) & Teacher Version (DBC-T) (2nd ed.). Clayton, Australia: Monash University Centre for Developmental Psychiatry and Psychology. Einfeld, S. L., Piccinin, A. M., Mackinnon, A., Hofer, S. M., Taffe, J., Gray, K. M., et al. (2006). Psychopathology in young people with intellectual disability. The Journal of the American Medical Association, 296, 1981–1989. http://dx.doi.org/10.1001/jama.296.16.1981 Einfeld, S. L., Ellis, L. A., Doran, C. M., Emerson, E., Horstead, S. K., Madden, R. H., et al. (2010). Behavior problems increase costs of care of children with intellectual disabilities. Journal of Mental Health Research in Intellectual Disabilities, 3(4), 202–209. Emerson, E. (2003). Prevalence of psychiatric disorders in children and adolescents with and without intellectual disability. Journal of Intellectual Disability Research, 47, 51–58. http://dx.doi.org/10.1046/j.1365-2788.2003.00464.x Enders, C. K. (2001). A primer on maximum likelihood algorithms available for use with missing data. Teacher’s Corner. Structural Equation Modeling, 8(1), 128–141. Fabrigar, L. R., Wegener, D. T., MacCallum, R. C., & Strahan, E. J. (1999). Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods, 4, 272–299. http://dx.doi.org/10.1007/s10803-009-0816-2 Goodman, R. (1997). The Strengths and Difficulties Questionnaire: A research note. Journal of Child Psychology and Psychiatry, 38(5), 581–586. Goodman, R., & Scott, S. (1999). Comparing the Strengths and Difficulties Questionnaire and the Child Behavior Checklist: Is small beautiful? Journal of Abnormal Child Psychology, 27, 17–24. http://dx.doi.org/10.1023/A:1022658222914 Graham, J. W. (2009). Missing data analysis: Making it work in the real world. Annual Review of Psychology, 60, 549–576. http://dx.doi.org/10.1146/ annurev.psych.58.110405.085530 Horiuchi, F., Oka, Y., Uno, H., Kawabe, K., Okada, F., Saito, I., et al. (2014). Age- and sex-related emotional and behavioral problems in children with autism spectrum disorders: Comparison with control children. Psychiatry and Clinical Neurosciences, 68, 542–550. http://dx.doi.org/10.1111/pcn.12164 Horn, J. L. (1965). A rationale and test for estimating the number of factors in factor analysis. Psychometrika, 30, 179–185. http://dx.doi.org/10.1007/ BF02289447 Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1–55. Jones, T. L., & Prinz, R. J. (2005). Potential roles of parental self-efficacy in parent and child adjustment: A review. Clinical Psychology Review, 25, 341–363. http://dx.doi.org/10.1016/j.cpr.2004.12.004 Kline, R. B. (2011). Principles and practice of structural equation modeling (3rd ed.). New York, NY: Guilford Press. Lahey, B. B., Schwab-Stone, M., Goodman, S. H., Wladman, I. D., Canino, G., Rathouz, P. J., et al. (2000). Age and gender differences in oppositional behaviour and conduct problems: A cross-sectional household study of middle childhood and adolescence. Journal of Abnormal Psychology, 109, 488– 503. http://dx.doi.org/10.1037/0021-843X.109.3.488 Lance, C. E., Butts, M. M., & Michels, L. C. (2006). The sources of four commonly reported cutoff criteria: What did they really say? Organizational Research Methods, 9, 202–220. http://dx.doi.org/10.1177/1094428105284919 MacCallum, R. C., Widaman, K. F., Zhang, S., & Hong, S. (1999). Sample size in factor analysis. Psychological Methods, 4, 84–99. Mazzucchelli, T. G., & Sanders, M. R. (2011). Preventing behavioural and emotional problems in children who have a developmental disability: A public health approach. Research in Developmental Disabilities, 32, 2148–2156. http://dx.doi.org/10.1016/j.ridd.2011.07.022 Mazzucchelli, T. G., Sanders, M. R., & Morawska, A. (2011). Child Adjustment and Parent Efficacy Scale-Developmental Disability (CAPES-DD). Brisbane, Australia: Parenting and Family Support Centre, University of Queensland. Morawska, A., & Sanders, M. R. (2010). Child Adjustment and Parent Efficacy Scale (CAPES). Brisbane, Australia: Parenting and Family Support Centre, University of Queensland. Muthe`n, L. K., & Muthe`n, B. O. (1998–2012). Mplus user’s guide. Statistical analysis with latent variables. Los Angeles, CA: Muthe`n & Muthe`n. Preacher, K. J., & MacCallum, R. C. (2003). Repairing Tom Swift’s electric factor analysis machine. Understanding Statistics, 2(1), 13–43. Priebe, S., Omer, S., Giacco, D., & Slade, M. (2014). Resource-oriented therapeutic models in psychiatry: Conceptual review. The British Journal of Psychiatry, 204, 256–261. http://dx.doi.org/10.1192/bjp.bp.113.135038 Roberts, C., Mazzucchelli, T., Taylor, K., & Reid, R. (2003). Early intervention for behaviour problems in young children with developmental disabilities. International Journal of Disability, Development and Education, 50, 275–292. http://dx.doi.org/10.1080/1034912032000120453 Sanders, M. R., & Morawska, A. (2010). Parenting and Family Adjustment Scale. Brisbane, Australia: Parenting and Family Support Centre, The University of Queensland. Sanders, M. R., Mazzucchelli, T. G., & Studman, L. J. (2004). Stepping Stones Triple P: The theoretical basis and development of an evidence-based positive parenting program for families with a child who has a disability. Journal of Intellectual and Developmental Disability, 29, 265–283. http://dx.doi.org/ 10.1080/13668250412331285127 Sanders, M. R., Morawska, A., Haslam, D., Filus, A., & Fletcher, R. (2013). Parenting and Family Adjustment Scale (PAFAS): Validation of a brief parent-report measure for use in assessment of parenting skills and family relationships. Child Psychiatry and Human Development, 45, 255–272. http://dx.doi.org/ 10.1007/s10578-013-0397-3

T.S. Emser et al. / Research in Developmental Disabilities 53–54 (2016) 158–177

177

Satorra, A., & Bentler, P. M. (2001). A scaled difference chi-square test statistic for moment structure analysis. Psychometrika, 66, 507–514. http://dx.doi.org/ 10.1007/BF02296192 Smedje, H., Broman, J.-E., Hetta, J., & Von Knorring, A.-L. (1999). Psychometric properties of a Swedish version of the ‘‘Strengths and Difficulties Questionnaire. European Child and Adolescent Psychiatry, 8, 63–70. http://dx.doi.org/10.1007/s007870050086 Steiger, J. H., & Lind, J. M. (1980). Statistically based tests for the number of factors. Paper presented at the annual meeting of the Psychometric Society. Tucker, L. R., & Lewis, C. (1973). A reliability coefficient for maximum likelihood factor analysis. Psychometrika, 38, 1–10. http://dx.doi.org/10.1007/ BF02291170 United States of America Developmental Disabilities Act of 1984, S. 2662, 98th Cong., § 102 (1984). . Van der Ende, J., & Verhulst, F. C. (2005). Informant, gender and age differences in ratings of adolescent problem behaviour. European Child and Adolescent Psychiatry, 14(3), 117–126.