Validity and reliability analysis of the Chinese parent version of the Autism Spectrum Rating Scale (6–18 years)

Psychiatry Research ∎ (∎∎∎∎) ∎∎∎–∎∎∎

Contents lists available at ScienceDirect

Psychiatry Research journal homepage: www.elsevier.com/locate/psychres

Validity and reliability analysis of the Chinese parent version of the Autism Spectrum Rating Scale (6–18 years) Hao Zhou a, Lili Zhang a, Lijie Wu b, Xiaobing Zou c, Xuerong Luo d, Kun Xia e, Yimin Wang a, Xiu Xu f, Xiaoling Ge g, Caihong Sun b, Hongzhu Deng c, Eric Fombonne h, Yong-Hui Jiang i, Weili Yan j,n, Yi Wang a,nn a

No. 399, Wanyuan Road, Minhang District, Division of Neurology, Children's Hospital of Fudan University, Shanghai, China School of Public Health, Harbin Medical University, Harbin, China c Child Development Center, The Third Affiliated Hospital, Sun Yat-Sen University, Guangzhou, China d Department of Psychiatry, The Second Xiangya Hospital of Central South University, Changsha, China e State Key Laboratory of Medical Genetics, Changsha, China f Department of Child Healthcare, Children's Hospital of Fudan University, Shanghai, China g Children’s Hospital of Fudan University, Shanghai, China h Oregon Health & Science University, Portland, OR, USA i Division of Medical Genetics, Department of Pediatrics and Neurobiology, Duke University School of Medicine, USA j Department of Clinical Epidemiology, Children's Hospital of Fudan University, Shanghai, China b

art ic l e i nf o

a b s t r a c t

Article history: Received 24 February 2015 Received in revised form 31 August 2015 Accepted 3 September 2015

This study aimed to investigate the validity and reliability of the Chinese parent version of the Autism Spectrum Rating Scale (ASRS, 6–18 years) for a general sample of Chinese children. The study involved assessing 1625 community-based subjects aged 6–12 years from four sites (Shanghai, Guangzhou, Changsha, and Harbin city) in China and 211 clinic-based participants aged 6–18 with a confirmed diagnosis of autism spectrum disorders (ASDs). The internal consistency (Cronbach’s alpha) ranged from 0.585 to 0.929, and the test–retest reliability (interclass correlations) ranged from 0.542 to 0.749, indicating no significant difference between the two tests at an interval of 2–4 weeks. The construct validity was relatively excellent, and the concurrent validity with the Social Responsiveness Scale (SRS) (Pearson correlations) was 0.732 between the two total scores. Receiver operating characteristics (ROC) analyses showed excellent and comparable discriminant validity of the ASRS with respect to the SRS, with an area under the curve (AUC) of 0.9507 (95% CI: 0.93–0.97) versus 0.9703 (95% CI: 0.96–0.98), respectively. Our data suggested a cutoff Z60 for the Chinese version of the ASRS, with good accuracy in screening autism symptoms (sensitivity ¼94.2%, specificity ¼77%). The Chinese parent version of the ASRS is therefore a reliable and valid tool for screening autistic symptoms in Chinese children in general. & 2015 Elsevier Ireland Ltd. All rights reserved.

Keywords: Epidemiology Autism Screening Scale Validation Child

1. Introduction Autism spectrum disorders (ASDs) are a group of neurodevelopment disorders characterized by diverse clinical phenotypes and varying levels of impairment in social interaction and reciprocal communication, as well as restricted and repetitive interests and behaviors (Wills, 2014). The term “ASDs” is an umbrella term in the DSM-V. Symptoms are typically notable before n Correspondence to: No. 399, Wanyuan Road, Minhang District, Division of Neurology, Children's Hospital of Fudan University, Shanghai, China. Fax: 86 21 64931901. nn Corresponding author. Fax: 86 21 64931901 E-mail addresses: [email protected] (W. Yan), [email protected] (Y. Wang).

the age of 3 years old, and numerous studies have indicated a high level of functional limitations and poor life quality in affected children (Eaves and Ho, 2008; Barneveld et al., 2014). The disease burden of ASD for families and society is remarkable (Horlin et al., 2014). ASD is one of the fastest-growing disorders worldwide, with many countries showing significantly increased incidence in recent decades (Williams et al., 2006; Elsabbagh et al., 2012; Blumberg et al., 2013). Increased public awareness of ASDs has recently resulted in more scientific epidemiological surveys of ASDs in many areas (Idring et al., 2014; Ouellette-Kuntz et al., 2014; Zahorodny et al., 2014). To conduct a national epidemiology survey of ASDs in any population, the usual method is to screen a representative sample of children to identify those who may have ASDs and conduct further clinical assessment with more in-depth approaches to

http://dx.doi.org/10.1016/j.psychres.2015.09.003 0165-1781/& 2015 Elsevier Ireland Ltd. All rights reserved.

Please cite this article as: Zhou, H., et al., Validity and reliability analysis of the Chinese parent version of the Autism Spectrum Rating Scale (6–18 years). Psychiatry Research (2015), http://dx.doi.org/10.1016/j.psychres.2015.09.003i

2

H. Zhou et al. / Psychiatry Research ∎ (∎∎∎∎) ∎∎∎–∎∎∎

derive unbiased prevalence estimations. A questionnaire-based screening is an ideal approach for conducting ASD screening because of the easy administration and use. In recent decades, the epidemiology of autism in children has made great progress in the development of valid and reliable screening instruments for diverse participants, such as the Checklist for Autism in Toddlers (CHAT), the Modified Checklist for Autism in Toddlers (M-CHAT), and the Checklist for Autism in Toddler-23 (CHAT-23) recommended for use in very young children (Robins et al., 2014). The Pervasive Developmental Disorders Screening Test (PDDST) is a 40-item scale that can also be used for children from birth to 3 years old (Ferreri, 2014). The Social Communication Questionnaire (SCQ) is a 40-item scale derived from a diagnostic tool (ADI-R) for use with children ages 4 years and older (Mulligan et al., 2009). The Social Responsiveness Scale (SRS) is a 65-item scale for children ages 4–18 years (Wigham et al., 2012). These screening instruments are usually completed within 10–20 min and are designed to estimate the risk of autistic condition. However, reviews of autism epidemiological studies have indicated that screening instruments have been very heterogeneous in population surveys (Matson et al., 2007). Such studies have rarely been based on the use of criterion instruments of known reliability and validity applied to representative samples in the general population. In a way, the variability in questionnaires reflects differences in the culture and representative samples of the surveyed areas. Besides, the lack of properly validated and reliable instruments to screen for autistic behaviors in the general population has been a significant barrier to epidemiologic studies of these conditions. Research on ASDs has only recently begun in China, and the exact national prevalence of ASDs in China remains unknown (Li et al., 2011). One year ago, an important initiative program funded by the Ministry of Health was launched to boost autism awareness among clinical experts and researchers. As part of this program, a national epidemiological survey of ASD among the school-aged (6–12 years) population of China was designed. To conduct the screening phase of this survey, an instrument that is properly validated for the Chinese population is necessary. Currently, there is no published autism screening tool that has been developed in Chinese. Several English versions of autism screening tools have been adapted to Chinese. For example, the Chinese version of the CHAT-23, which is designed for use in primary care settings at the 18-month visit, combines 23 items from parental reports with 2 items from the direct observation of the child. The CHAT-23 has excellent screening performance in representative Hong Kong samples (Wong et al., 2004). A Mandarin Chinese version of the CAST (M-CAST) was translated and developed by Dr. Sun Xiang, and it has good psychometric properties for Chinese children aged 4–11 years (Sun et al., 2014). Moreover, the Chinese version of the Social Responsiveness Scale (SRS) and the Social Communication Questionnaire (SCQ) were shown to have good reliability and validity in Taiwanese children (Gau et al., 2011, 2013). However, those existing Chinese versions of screening instruments were applied in populations that either lacked regional diversity or were not matched to the age range of our studied population. Therefore, it is necessary to develop a screening tool appropriate for the age range and population diversity of this national ASD prevalence survey. Among recently developed instruments designed to detect autistic symptomatology in both clinical and nonclinical samples, we specially considered the Autism Spectrum Rating Scale (ASRS) (Goldstein and Naglieri, 2009) for evaluation. The ASRS was designed for both young children aged 2–5 years and children aged 6–18 years, and it has excellent reliability and validity in the U.S. population. This scale has both full-length and short versions, with the former comprising 70/71 items and the latter containing 15

items. Separate forms are completed by parents (ASRS Parent Rating) or teachers (ASRS Teacher Rating). Both have the same items and structure. The full-length ASRS was designed not only as an screening tool to guide diagnostic decisions but also as a tool for treatment planning based on ongoing monitoring of the response to intervention and evaluation; by contrast, the short version is suitable only for screening. Comparisons with other instruments are easy because of the availability of standard scores. Therefore, according to the task of the epidemiologic surveys and further ongoing monitoring for ASD children, we evaluated the full-length ASRS (6–18 years) as the candidate screening tool for the national epidemiological survey, with the following aims: (1) to examine the psychometric properties of the Chinese parent version of the ASRS in a Chinese sample aged 6–12 years and (2) to measure the discriminant validity of the Chinese parent version of the ASRS as a screening instrument for investigate the national prevalence of ASDs.

2. Methods 2.1. Study populations 2.1.1. General sample The ASRS was developed for the general population in the U.S. (Goldstein and Naglieri, 2009). Thus, we selected the communitybased population (aged 6–12 years) as the general sample to ensure its representativeness by using convenient cluster sampling. Four community-based samples were selected in Shanghai, Harbin, Guangzhou, and Changsha. All children with local residency or children who have been living in the community for more than six months were eligible to participate in this study, with a total number of 2053. 2.1.2. Clinically diagnosed ASD cases To analyze concurrent and discriminant validity, children who had been clinically diagnosed with ASDs were recruited from the local autism rehabilitation center. The clinical diagnosis of ASD was made by senior child psychiatrists who have extensive clinical and research experience in the assessment and treatment of children with ASD according to DSM-V criteria, confirmed using the autism diagnostic interview-revised (ADI-R). In total, 211 ASD cases aged 6–18 years were recruited. 2.2. Measures 2.2.1. ASRS Chinese parent version The Chinese parent version of the full-length ASRS (6–18 years) was used in this study. The ASRS questionnaire evaluates the frequency of each behavior (‘0’ for never and ‘4’ for Very Frequently) to quantify autistic features. According to different study purposes, the ASRS (for children 6–18 years old) can be referred to as the ASRS scales, DSM-IV-TR scales, and treatment scales. A three-factor solution was the most suitable parent version of the ASRS scales for the 6–18 age group. Three factors constituting 60 of the total 71 items are generated for screening: one factor related to Social/Communication (19 items); another factor with items related to Unusual behaviors (24 items); and the third factor SelfRegulation (17 items) that included items primarily related to attention problems, impulsivity, and compliance. These 3 scales are combined into a single composite score, the T-score, which was developed for screening purposes. High scores on the ASRS scales suggest that additional evaluation is required. The DSM-IV-TR scale is derived from 34 items based on experience from the total 71 items, some of which also contribute to the diagnosis scales. Finally, the treatment scales consist of 8 subscales that can be used

Please cite this article as: Zhou, H., et al., Validity and reliability analysis of the Chinese parent version of the Autism Spectrum Rating Scale (6–18 years). Psychiatry Research (2015), http://dx.doi.org/10.1016/j.psychres.2015.09.003i

H. Zhou et al. / Psychiatry Research ∎ (∎∎∎∎) ∎∎∎–∎∎∎

3

to monitor treatment responses: Peer Socialization (9 items, PS), Adult Socialization (6 items, AS), Social/Emotional Reciprocity (13 items, SER), Atypical Language (6 items, AL), Stereotypy (5 items, ST), Behavioral Rigidity (8 items, BR), Sensory Sensitivity (6 items, SS), and Attention (11 items, AT). This scale can be used as an ongoing tool for monitoring the clinical assessment of children with ASD. All original scales are set to the T-score metric, which has a normative mean of 50 and a standard deviation of 10. The ASRS has good inter-rater reliability, test–retest reliability, construct validity, and content validity (Goldstein and Naglieri, 2009). In view of our research purpose, the study primarily focuses on the ASRS scales. The Chinese version of the ASRS was developed by the team of the Research Special Fund for the Public Welfare Industry of Health of China using standard translation and back-translation procedures (Harkness and Schoua-Glusberg, 1998) with permission from Goldstein and Naglieri and approval from the MultiHealth System. The Chinese ASRS was initially piloted with a small sample containing five Chinese parents whose children were 6–12 years old; they were selected from a local autism rehabilitation center in Shanghai. We retained all 71 items that were translated using culturally equivalent expressions in the Chinese environment for further practice.

fit index (GFI), the adjusted goodness-of-fit index (AGFI), the comparative fit index (CFI), the non-normed fit index (NNFI), and the root mean square error of approximation (RMSEA). The test– retest reliability was evaluated using the intra-class correlations and paired t-tests (n ¼131) (Brown et al., 2004). The internal consistency of each subscale was tested by using the Cronbach’s alpha (n ¼1625), and the concurrent validity of the SRS (n ¼211) was tested using the Pearson correlation coefficient (Sen, 1993). To measure the discriminant validity, we compared the mean scores of the Chinese ASRS between the clinical sample aged 6–12 years (n ¼190) and the community-based sample (n ¼ 1625) by using Student’s t-test. Between-group differences (effect size) were analyzed using Cohen’s d (Larner, 2014). Age and gender effects on the subscale scores and T-scores were analyzed by using multiple linear regressions. Receiver operating characteristic (ROC) analysis was conducted, and the area under the curve (AUC) and 95% confidence interval (CI) were computed to evaluate the overall discriminant validity of the Chinese parent version of the ASRS and compared with the original SRS. All tests were two-tailed, and a p-value of 0.05 was retained as the level of statistical significance.

2.2.2. Social responsiveness scale (SRS) as the criterion instrument The SRS was designed to quantify autistic behaviors. It comprises 65 items that are scored to measure the severity (mild to severe) and frequency (‘0’ for never true and ‘3’ for always true) of each behavior, resulting in a total score and five subscale scores (i.e., social awareness, social cognition, social communication, social motivation, and autistic mannerisms). The survey can be completed by parents, caregivers, or teachers in 15–20 minutes. The SRS has been used in a variety of studies described elsewhere (Constantino and Gruber, 2002). Different versions of the SRS have been shown to have good psychometric properties (Bölte et al., 2008; Fombonne et al., 2012). The Chinese SRS has demonstrated good reliability and validity with regard to internal consistency, test–retest reliability, concurrent validity with ADI-R, and discriminant validity (Gau et al., 2013). Because of these advantages, we used the Chinese version of the SRS as the criterion instrument to establish the concurrent and discriminant validity of the ASRS in this study (Umphress et al., 1997; Howard and Hughes, 2012).

3. Results

2.3. Protocol Parents of children in both the community and local autism rehabilitation centers were informed of the details of this study by the local authority and were then invited to participate in the study. The parents who consented were then asked to complete the parent version of ASRS questionnaire at home. Parents of children with ASD were asked to complete the ASRS and SRS questionnaires while receiving the ADI-R interview at the hospital. In total, we received 1684 returned ASRS questionnaires from the community, with a response rate of 83.97%. The study was approved by the Ethics Review Board of the Children’s Hospital of Fudan University ([2012] No.185).

3.1. Demographic characteristics of the participants Of the 2053 children eligible for the general sample, 1684 (83.97%) were contacted and given questionnaires to complete. After the exclusion of questionnaires because of various errors, 1625 participants (mean age: 8.85 71.78 years) including 830 boys (51.1%) were included in the analysis. We randomly selected 131 participants (approximately 10% of the total sample) to complete the same questionnaire again at an interval of 2–4 weeks to measure test–retest reliability. Of the 211 children aged 6–18 years with ASDs (8.9 71.7 years) who were recruited from the local autism rehabilitation center, 87.7% were male (the male-to-female ratio was 7.8:1). The ADI-R interview results are consistent with clinical diagnosis. 3.2. Chinese version of the ASRS scale scores A preliminary analysis of the general sample obtained with the parent version of the ASRS was performed. Boys in the community-based sample had significantly higher T-standardized scoring in the areas of Social Communication (SC), Unusual Behavior (UB), and Self-Regulation (SR), and in their standardized total score (Tscore) by a difference of 2–3 points relative to the scores for girls, Pso0.001 (Fig. 1). Fig. 2 shows slight site differences among cities for the SC, UB, SR and T-scores. Minor age effects on the ASRS

2.4. Data analysis The data analysis was conducted by statisticians using the Stata statistical package program (version 11.0, College Station, Texas 77845, United States). We performed a confirmatory factor analysis (CFA) by using the three-factor structural model (60 items) based on an exploratory factor analysis (EFA) of the technical manual of ASRS scales (Harrington, 2008). Several model-fitting indices were employed as structural parameters: the goodness-of-

Fig. 1. Slight difference between males and females for SC (Social Communication), UB (Unusual Behaviors), SR (Self-Regulation), and T-score (Total score).

Please cite this article as: Zhou, H., et al., Validity and reliability analysis of the Chinese parent version of the Autism Spectrum Rating Scale (6–18 years). Psychiatry Research (2015), http://dx.doi.org/10.1016/j.psychres.2015.09.003i

H. Zhou et al. / Psychiatry Research ∎ (∎∎∎∎) ∎∎∎–∎∎∎

4

Table 2 Internal consistency of the Chinese parent version of the ASRS (n¼ 1625). Scales T-score ASRS scales

DSM-IV-TR scale Treatment scales

Fig. 2. Slight site difference among cities for SC (Social Communication), UB (Unusual Behaviors), SR (Self-Regulation), and T-score (Total score).

SC UB SR T-score

Clinical case Male (n¼ 184)

Female (n¼ 27)

45.9 7 12.14 44.107 13.77 34.26 7 10.18 68.977 6.39

50.117 10.22 43.88 7 10.75 38.127 8.71 71.46 7 4.42

t

P

 1.9 0.093  2.06  2.53

0.065 0.926 0.046* 0.015*

SC, Social/Communication; UB, Unusual Behaviors; SR, Self-Regulation; T-score, Total Score. Data are represented by the mean and standard deviation. *

PS AS SER AL ST BR SS AT

Cronbach’s alpha

60 19 24 17 34 9 6 13 6 5 8 6 11

0.91 0.88 0.84 0.85 0.85 0.69 0.64 0.81 0.69 0.59 0.68 0.62 0.80

SC, Social/Communication; UB, Unusual Behaviors; SR, Self-Regulation; PS, Peer Socialization; AS, Adult Socialization; SER, Social/Emotional Reciprocity; AL, Atypical Language; ST, Stereotypy; BR, Behavioral Rigidity; SS, Sensory Sensitivity; AT, Attention; T-score, Total Score.

Table 1 Gender differences for ASRS scale scores in clinical cases. ASRS scales

SC UB SR

Number of items

Po 0.001.

scales scores were found for boys and girls (r¼  0.01–0.07). Among the cases, girls had higher scores than the boys did, with mean differences of 0.4 SD for SC, 0.4 SD for UB, and 0.3 SD for the T-score (Table 1). 3.3. Item reliability The screening scales had high Cronbach’s alpha values (i.e., Social/Communication 0.880, Unusual Behaviors 0.841, Self-Regulation 0.851, and T-score 0.910). The Cronbach’s alpha of the DSM-IV-TR scale was 0.854. There were eight treatment subscales with Cronbach’s alpha values ranging from 0.58 (ST) to 0.81 (SER). These subscales were involved in treatment evaluation. Notably, the treatment subscales had 9 items or fewer, which probably explains the lower range of alpha coefficients. All subscales with alpha values 40.80 had a higher number of items (11 items or more) (see Table 2). 3.4. Test–retest reliability Two to four weeks after the initial questionnaire administration, we asked 131 randomly selected community participants (approximately 10% of the total sample) to repeat the Chinese parent version of the ASRS. Only the scoring of UB, AL, and ST showed statistically significant difference between the two tests at an interval of 2–4 weeks; the others showed no statistically significant difference. However, Table 2 shows that the Chinese ASRS had good intra-class correlations (ICC ¼ 0.542–0.749), which indicate that the stability of this questionnaire is acceptable over time (Table 3). 3.5. Construct validity The Kaiser–Meyer–Olkin (KMO) measure (0.945) and Bartlett’s test (0.000) demonstrated that the data were suitable for factor

analysis. We initially conducted confirmatory factor analysis based on the EFA results of the original ASRS (60 items). In the CFA modeling, the RMSEA value was 0.043 (o0.05). We obtained the values of 0.818 for CFI, 0.856 for GFI, 0.844 for AGFI, and 0.767 for NNFI, all of which were acceptable. The CFI, GFI, AGFI, NNFI, and RMSEA values indicated a relatively good model fit. The results showed that the Chinese ASRS parent version has relatively good construct validity. 3.6. Concurrent validity To test for concurrent validity, we used the SRS as the external criterion and estimated its correlation with the ASRS in the clinical sample. The results for the correlations between the subscales and the total scores of the ASRS scales as well as the SRS are shown in Table 4. The high correlation between the total scores (r ¼0.732) of the two scales shows that the 2 questionnaires are robust and comparable in measuring autistic symptomatology. However, some subscales of the two instruments had weak correlations, such as the UB and social awareness subscales, for which the correlation index was 0.152. 3.7. Discriminant validity To measure the discriminant ability of the ASRS, we compared the scores for the ASRS scales from the community-based participants (n ¼1625) with those for the clinic cases (n ¼190) within the same age range (6–12 years old). Multiple linear regressions showed a slight age effect (b¼  0.05 for the general sample, b¼0.12 for clinical cases, Ps ¼0.260, 0.070) and a gender effect on the T-score (b¼  0.13 for the general sample, b¼0.09 for clinical cases, Ps ¼0.000, 0.224). There were significant differences between the two groups in the two ASRS subscale scores (Table 5). The clinical sample showed significantly higher T-scores and SC, UB and SR scores compared with the general sample (Cohen’s d from 1.14 to 2.27). The ROC analysis is shown in Fig. 3. Both versions with an AUC over 0.9 indicated equally excellent discriminant validity in the screening of ASD cases, with the AUC of the total scores equaling 0.9507 (95% CI: 0.93–0.97) versus 0.9703 (95% CI: 0.96–0.98) for the ASRS and SRS, respectively. We further performed the same analysis separately in both genders, and the results indicated even better performance for girls (AUC¼0.9958; 95% CI: 0.9914–1.0000) on the ASRS. According to the approach that maximizes the sum of sensitivity and specificity, our data suggested a cutoff T scoreZ60 for the ASRS to achieve a sensitivity of 94.2% and a specificity of

Please cite this article as: Zhou, H., et al., Validity and reliability analysis of the Chinese parent version of the Autism Spectrum Rating Scale (6–18 years). Psychiatry Research (2015), http://dx.doi.org/10.1016/j.psychres.2015.09.003i

H. Zhou et al. / Psychiatry Research ∎ (∎∎∎∎) ∎∎∎–∎∎∎

5

Table 3 Test–retest reliability of the Chinese parent version of the ASRS. Scales

T-Score ASRS Scales

DSM-IV-TR Scale Treatment Scales

Test-retest reliability (n=131)

SC UB SR PS AS SER AL ST BR SS AT

ICC

95%CI

Initial

Re-test

p

r

0.739 0.702 0.669 0.691 0.749 0.611 0.705 0.716 0.597 0.542 0.673 0.577 0.737

0.63–0.81 0.58–0.79 0.53–0.77 0.56–0.78 0.65–0.82 0.45–0.72 0.58–0.78 0.60–0.80 0.43–0.72 0.35–0.68 0.54–0.77 0.40–0.70 0.63–0.81

57.69±6.22 59.45±9.66 59.93±5.41 50.65±7.43 59.32±5.60 54.70±9.09 52.57±7.93 58.60±8.58 56.80±7.32 57.50±6.24 58.34±6.71 60.99±8.67 52.66±7.46

56.97±6.98 59.67±10.09 58.50±6.48 49.90±7.83 58.78±6.12 54.38±9.37 51.79±8.34 58.49±9.45 55.21±7.78 55.97±6.50 57.37±7.19 59.50±8.96 51.96±8.22

0.174 0.790 0.006 0.255 0.239 0.709 0.248 0.878 0.025 0.014 0.101 0.075 0.269

0.591* 0.539* 0.530* 0.529* 0.603* 0.438* 0.546* 0.558* 0.439* 0.389* 0.513* 0.412* 0.586*

SC, Social/Communication; UB, Unusual Behaviors; SR, Self-Regulation; PS, Peer Socialization; AS, Adult Socialization; SER, Social/Emotional Reciprocity; AL, Atypical Language; ST, Stereotypy; BR, Behavioral Rigidity; SS, Sensory Sensitivity; AT, Attention; ICC, Intra-Class Correlation; T-score, Total Score. Data are represented by the mean and standard deviation. *

All r values are moderate correlations.

77% in the current study sample. The data indicated good discriminant ability of the Chinese parent version of the ASRS.

4. Discussion In this article, we assessed the validity and reliability of the Chinese parent version of the ASRS (for children 6–18 years old). All original items remained in the Chinese parent version of the ASRS, and the scale factor structure remained the same regardless of whether the general or clinical case sample were considered in this study. Our findings show that the Chinese parent version of the ASRS has good validity and reliability, with the same items and factor structure as the original ASRS. As predicted from previous studies, the ASRS scale scores for boys in the general sample were significantly higher than those for girls. Previous studies have indicated that boys and girls have different profiles of social and communication developmental trajectories, particularly a higher rate of difficulties in social and communicational skills for boys (Leaper, 1994; Gagnon and Simon, 2011). Social communication dysfunction is one of the core symptoms of ASDs. Epidemiological research also shows that the prevalence of ASDs dramatically differs between males and females (the male–female ratio is approximately 4:1) (Simonoff et al., 2008; Kim et al., 2011; Mandy et al., 2012). One possible explanation is the gender difference in neurodevelopment (O’Connor and Joffe, 2014). In this study, the mean subscale raw scores and T-scores slightly differed among the 4 cities, which were selected in regions that differ slightly in their cultural background and level of economic development; it is likely that these differences reflect true variability in the population that was

appropriately reflected in our representative sample. Interestingly, we find a contrary result that girls in the clinical sample have significantly higher SC and UB scores and T-scores than boys do. To our knowledge, the prevalence of ASDs is higher in boys than in girls, but the symptoms experienced by girls are more severe than those experienced by boys (Bejerot and Eriksson, 2014). The screening subscales demonstrated high internal reliability coefficients. There were eight treatment subscales with Cronbach’s alphas ranging from 0.58 to 0.81. Most subscales had acceptable internal consistency (alpha 40.60); alpha coefficients are influenced by the average inter-item correlation in a composite form and by the number of items pertaining to this composite value. Lower values for internal consistency were obtained for subscales that had relatively fewer items, whereas all scales with 10 or more items had alpha coefficients above 0.80. The questionnaire’s Cronbach’s alpha value 40.8 is suitable for scientific research (Connelly, 2011). Therefore, the results of our study demonstrate that the Chinese parent version of the ASRS has good internal consistency. The test–retest reliability reflects the stability of the scale; as in previous studies, the test–retest reliability assessment shows that the Chinese parent version of the ASRS has good test– retest reliability and an acceptable (low) level of measurement error. The English parent version of the ASRS also has good reliability (Goldstein and Naglieri, 2009). Of the several types of validity, we evaluate the construct validity, concurrent validity and discriminant validity of the Chinese parent version of the ASRS. The results (KMO 40.9, Bartlett’s test o0.01) demonstrated that the data are suitable for factor analysis (Velicer et al., 2000). The results for construct validity showed the fitting indices (CFI, GFI, AGFI, and NNFI) based on CFA, supporting the 3-factor structure proposed initially for the

Table 4 Concurrent validity of the Chinese parent version of the ASRS compared with the SRS. Pearson’s correlations between the subscales of the Chinese ASRS and SRS (n¼ 211). ASRS Scales

SC UB SR T-score

SRS Social awareness

Social cognition

Social communication

Social motivation

Autistic mannerisms

Total score

0.467 0.152 0.518 0.556

0.553 0.363 0.347 0.619

0.596 0.305 0.505 0.694

0.511 0.385 0.355 0.609

0.261 0.429 0.421 0.521

0.581 0.404 0.519 0.732

SC, Social/Communication; UB, Unusual Behaviors; SR, Self-Regulation; T-score, Total Score.

Please cite this article as: Zhou, H., et al., Validity and reliability analysis of the Chinese parent version of the Autism Spectrum Rating Scale (6–18 years). Psychiatry Research (2015), http://dx.doi.org/10.1016/j.psychres.2015.09.003i

H. Zhou et al. / Psychiatry Research ∎ (∎∎∎∎) ∎∎∎–∎∎∎

6

Table 5 Discriminant validity of the Chinese parent version of the ASRS. Scales

Community sample (n ¼1625)

54.63 7 6.92 SC 56.377 9.39 UB 58.067 6.27 SR 47.147 8.29 DSM-IV-TR scale 56.81 7 5.95 Treatment PS 52.02 7 8.38 scales AS 49.237 8.64 SER 71.25 7 7.69 AL 53.53 7 7.84 ST 55.59 7 6.93 BR 57.32 7 6.94 SS 59.147 9.46 AT 49.367 8.41

T-score ASRS scales

Clinical sample

t Value

Cohen’s d

 29.00  26.95  14.82  20.91  27.72  35.82  24.23

2.23 2.27 1.14 1.72 2.43 3.78 1.83

2.08  19.52  9.22  5.94  8.11  20.91

1.40 0.66 0.43 0.58 1.54

(n ¼190) 69.177 6.38 74.58 7 8.02 64.817 5.92 59.647 7.28 68.637 4.87 73.26 7 5.63 64.457 8.31 55.277 8.74  28.00 66.777 9.46 61.047 8.21 60.767 8.07 64.767 9.75 61.577 7.95

SC, Social/Communication; UB, Unusual Behaviors; SR, Self-Regulation; PS, Peer Socialization; AS, Adult Socialization; SER, Social/Emotional Reciprocity; AL, Atypical Language; ST, Stereotypy; BR, Behavioral Rigidity; SS, Sensory Sensitivity; AT, Attention; T-score, Total Score. Data are represented by the mean and standard deviation. All P values are significant at the 0.001 level.

the ASRS, which achieves a sensitivity of 94.7% and a specificity of 77% in distinguishing cases from non-cases in the study sample. All the data indicated that the ASRS has excellent discriminant validity and is an effective instrument for screening for ASD in a Chinese general sample. 4.1. Limitations To the best of our knowledge, this is the first multi-site and large-scale study using both general sample and clinic cases to validate the Chinese parent version of the ASRS by performing validity and reliability analyses. Our results demonstrate that the Chinese parent version of the ASRS is a useful and reliable national screening tool for autistic behavior. However, the limitation of this study is the representativeness of the samples; because our sample was obtained from four cities, these results may not be generalized to the entire population in mainland China. Another limitation is that the CFA results indicate a relatively good model fit. In general, fitting indices 40.9 indicate fairly good construct validity of a questionnaire (Beauducel and Rabe, 2009). In particular, the ASRS is a newly developed assessment instrument. Because of the lack of studies conducting CFA on the ASRS in different countries, a comparison of the present results and previous findings cannot be made with respect to these issues. Therefore, comprehensive factor analysis is necessary in further research to reassess the scale items and structure based on the Chinese culture.

5. Conclusion In summary, we developed the Chinese parent version of the ASRS for children aged 6–12 years. The psychometric properties of this version are excellent, as shown by the high internal consistency, good test–retest reliability, and discriminant and concurrent validity. The Chinese parent version of the ASRS is therefore a reliable and valid instrument for national ASD screening in the general population of children in China. Future studies should investigate the validity and reliability of other forms of the ASRS (2–5 years old). Fig. 3. Receiver operating characteristics (ROC) for ASRS and SRS; ASRS, Autism spectrum rating scale; SRS, Social Responsiveness Scale.

instrument. The fit indices indicate that the ASRS has relatively good construct validity. Previous studies have also shown that the cultural setting can affect performance on the scales (McCrae et al., 1998). To test the concurrent validity, we use the SRS as the external criterion. The high correlation between the total scores (r ¼0.732) of the two scales show the homogeneity of these measurements. The fact that some subscales have low correlations is expected because the factor structure differs between the two instruments. Thus, the lowest correlation observed (0.152) is between 2 subscales that measure different constructs and that have component items that are not comparable. It is necessary for the screening tool to have good discriminant ability between a general sample and clinical cases (Lucas et al., 1996). The comparison between the general sample and clinical case sample shows robust mean differences in all scores; in particular, the effective size of the three screening scales and T-scores was greater than 1, supporting the discriminating ability of the Chinese parent version of the ASRS. The excellent AUC estimation from the ROC analyses indicates strong and equally high discriminant validity of the total ASRS scores compared with the SRS. No gender difference was found for the ASRS. Our data suggest an optimal cutoff of 60 for

Conflict of interest The authors declare that they have no competing interests.

Contributions Dr. Hao Zhou, who wrote the manuscript, is a Ph.D. student at the Children’s Hospital of Fudan University. This study is part of her dissertation work. Xiaoling Ge, Lili Zhang, Caihong Sun, and Hongzhu Deng collected the data. Hao Zhou and Weili Yan completed the data analysis. Yimin Wang, Weili Yan, Eric Fombonne, Yong-Hui Jiang and Yi Wang revised the manuscript. Lijie Wu, Xiaobing Zou, Xuerong Luo, Kun Xia, Xiu Xu, Weili Yan, Eric Fombonne, Yong-Hui Jiang, and Yi Wang conducted and designed the study.

Acknowledgments This study was supported by the National Health and Family Planning Commission of the People’s Republic of China (Grant number: 201302002; ClinicalTrials.gov number NCT 02200679).

Please cite this article as: Zhou, H., et al., Validity and reliability analysis of the Chinese parent version of the Autism Spectrum Rating Scale (6–18 years). Psychiatry Research (2015), http://dx.doi.org/10.1016/j.psychres.2015.09.003i

H. Zhou et al. / Psychiatry Research ∎ (∎∎∎∎) ∎∎∎–∎∎∎

We would like to thank all of the parents who participated in this project.

References Barneveld, P.S., Swaab, H., Fagel, S., van Engeland, H., de Sonneville, L.M., 2014. Quality of life: a case-controlled long-term follow-up study, comparing young high-functioning adults with autism spectrum disorders with adults with other psychiatric disorders diagnosed in childhood. Compr. Psychiatry 55, 302–310. Beauducel, A., Rabe, S., 2009. Model-related factor score predictors for confirmatory factor analysis. Br. J. Math. Stat. Psychol. 62, 489–506. Bejerot, S., Eriksson, J.M., 2014. Sexuality and gender role in autism spectrum disorder: a case control study. PLoS One 9, e87961. Blumberg, S.J., Bramlett, M.D., Kogan, M.D., Schieve, L.A., Jones, J.R., Lu, M.C., 2013. Changes in prevalence of parent-reported autism spectrum disorder in schoolaged U.S. children: 2007 to 2011–2012. Natl. Health Stat. Rep., 1–11. Bölte, S., Poustka, F., Constantino, J.N., 2008. Assessing autistic traits: cross‐cultural validation of the social responsiveness scale (SRS). Autism Res. 1, 354–363. Brown, W.J., Trost, S.G., Bauman, A., Mummery, K., Owen, N., 2004. Test–retest reliability of four physical activity measures used in population surveys. J. Sci. Med. Sport 7, 205–215. Connelly, L.M., 2011. Cronbach’s alpha. Medsurg Nurs. 20 (45), 44. Constantino, J.N., Gruber, C.P., 2002. The Social Responsiveness Scale. Western Psychological Services, Los Angeles. Eaves, L.C., Ho, H.H., 2008. Young adult outcome of autism spectrum disorders. J. Autism Dev. Disord. 38, 739–747. Elsabbagh, M., Divan, G., Koh, Y.J., Kim, Y.S., Kauchali, S., Marcin, C., Montiel-Nava, C., Patel, V., Paula, C.S., Wang, C., Yasamy, M.T., Fombonne, E., 2012. Global prevalence of autism and other pervasive developmental disorders. Autism Res. 5, 160–179. Ferreri, M., 2014. Pervasive developmental disorders. Rev. Prat. 64, 481–486. Fombonne, E., Marcin, C., Bruno, R., Tinoco, C.M., Marquez, C.D., 2012. Screening for autism in Mexico. Autism Res. 5, 180–189. Gagnon, J.H., Simon, W., 2011. Sexual Conduct: The Social Sources of Human Sexuality. Transaction Publishers. Gau, S.S., Lee, C., Lai, M., Chiu, Y., Huang, Y., Kao, J., Wu, Y., 2011. Psychometric properties of the Chinese version of the Social Communication Questionnaire. Res. Autism Spectr. Disord. 5, 809–818. Gau, S.S., Liu, L., Wu, Y., Chiu, Y., Tsai, W., 2013. Psychometric properties of the Chinese version of the social responsiveness scale. Res. Autism Spectr. Disord. 7, 349–360. Goldstein, S., Naglieri, J.A., 2009. Autism Spectrum Rating Scales (ASRS™). MultiHealth Systems, Tonawanda, NY. Harkness, J.A., Schoua-Glusberg, A., 1998. Questionnaires in translation. ZUMANachrichten Spezial 3, 87–127. Harrington, D., 2008. Confirmatory Factor Analysis. Oxford University Press, New York. Horlin, C., Falkmer, M., Parsons, R., Albrecht, M.A., Falkmer, T., 2014. The cost of autism spectrum disorders. PLoS One 9, e106552. Howard, S., Hughes, B.M., 2012. Construct, concurrent and discriminant validity of Type D personality in the general population: associations with anxiety, depression, stress and cardiac output. Psychol. Health 27, 242–258. Idring, S., Lundberg, M., Sturm, H., Dalman, C., Gumpert, C., Rai, D., Lee, B.K., Magnusson, C., 2014. Changes in prevalence of autism spectrum disorders in 2001–2011: findings from the Stockholm Youth Cohort. J. Autism Dev. Disord., 1–8. Kim, Y.S., Leventhal, B.L., Koh, Y., Fombonne, E., Laska, E., Lim, E., Cheon, K., Kim, S., Kim, Y., Lee, H., 2011. Prevalence of autism spectrum disorders in a total population sample. Am. J. Psychiatry 168, 904–912.

7

Larner, A.J., 2014. Effect size (Cohen’s d) of cognitive screening instruments examined in pragmatic diagnostic accuracy studies. Dement. Geriatr. Cogn. Disord. Extra 4, 236–241. Leaper, C., 1994. Exploring the consequences of gender segregation on social relationships. New Dir. Child Adolesc. Dev. 1994, 67–86. Li, N., Chen, G., Song, X., Du, W., Zheng, X., 2011. Prevalence of autism-caused disability among Chinese children: a national population-based survey. Epilepsy Behav. 22, 786–789. Lucas, R.E., Diener, E., Suh, E., 1996. Discriminant validity of well-being measures. J. Pers. Soc. Psychol. 71, 616. Mandy, W., Chilvers, R., Chowdhury, U., Salter, G., Seigal, A., Skuse, D., 2012. Sex differences in autism spectrum disorder: evidence from a large sample of children and adolescents. J. Autism Dev. Disord. 42, 1304–1313. Matson, J.L., Nebel-Schwalm, M., Matson, M.L., 2007. A review of methodological issues in the differential diagnosis of autism spectrum disorders in children. Res. Autism Spectr. Disord. 1, 38–54. McCrae, R.R., Costa, P.T., Del Pilar, G.H., Rolland, J., Parker, W.D., 1998. Cross-cultural assessment of the five-factor model. The revised NEO personality inventory. J. Cross-Cult. Psychol. 29, 171–188. Mulligan, A., Richardson, T., Anney, R.J., Gill, M., 2009. The Social Communication Questionnaire in a sample of the general population of school-going children. Ir. J. Med. Sci. 178, 193–199. O’Connor, C., Joffe, H., 2014. Gender on the brain: a case study of science communication in the new media environment. PLoS One 9, e110830. Ouellette-Kuntz, H., Coo, H., Lam, M., Breitenbach, M.M., Hennessey, P.E., Jackman, P.D., Lewis, M.S., Dewey, D., Bernier, F.P., Chung, A.M., 2014. The changing prevalence of autism in three regions of Canada. J. Autism Dev. Disord. 44, 120–136. Robins, D.L., Casagrande, K., Barton, M., Chen, C.M., Dumont-Mathieu, T., Fein, D., 2014. Validation of the modified checklist for autism in toddlers, revised with follow-up (M-CHAT-R/F). Pediatrics 133, 37–45. Sen, A., 1993. Internal consistency of choice. Econom.: J. Econom. Soc., 495–521. Simonoff, E., Pickles, A., Charman, T., Chandler, S., Loucas, T., Baird, G., 2008. Psychiatric disorders in children with autism spectrum disorders: prevalence, comorbidity, and associated factors in a population-derived sample. J. Am. Acad. Child Adolesc. Psychiatry 47, 921–929. Sun, X., Allison, C., Auyeung, B., Matthews, F.E., Norton, S., Baron-Cohen, S., Brayne, C., 2014. Psychometric properties of the Mandarin version of the Childhood Autism Spectrum Test (CAST): an exploratory study. J. Autism Dev. Disord. 44, 1565–1576. Umphress, V.J., Lambert, M.J., Smart, D.W., Barlow, S.H., Clouse, G., 1997. Concurrent and construct validity of the Outcome Questionnaire. J. Psychoeduc. Assess. 15, 40–55. Velicer, W.F., Eaton, C.A., Fava, J.L., 2000. Construct explication through factor or component analysis: a review and evaluation of alternative procedures for determining the number of factors or components, Problems and Solutions Human Assessment. Springer, pp. 41–71. Wigham, S., McConachie, H., Tandos, J., Le Couteur, A.S., 2012. The reliability and validity of the Social Responsiveness Scale in a UK general child population. Res. Dev. Disabil. 33, 944–950. Williams, J.G., Higgins, J.P., Brayne, C.E., 2006. Systematic review of prevalence studies of autism spectrum disorders. Arch. Dis. Child 91, 8–15. Wills, C.D., 2014. DSM-5 and neurodevelopmental and other disorders of childhood and adolescence. J. Am. Acad. Psychiatry Law 42, 165–172. Wong, V., Hui, L.H., Lee, W.C., Leung, L.S., Ho, P.K., Lau, W.L., Fung, C.W., Chung, B., 2004. A modified screening tool for autism (Checklist for Autism in Toddlers [CHAT-23]) for Chinese children. Pediatrics 114, e166–e176. Zahorodny, W., Shenouda, J., Howell, S., Rosato, N.S., Peng, B., Mehta, U., 2014. Increasing autism prevalence in metropolitan New Jersey. Autism 18, 117–126.

Please cite this article as: Zhou, H., et al., Validity and reliability analysis of the Chinese parent version of the Autism Spectrum Rating Scale (6–18 years). Psychiatry Research (2015), http://dx.doi.org/10.1016/j.psychres.2015.09.003i