Psychometric properties of the children's sleep habits questionnaire in children with autism spectrum disorder

Sleep Medicine 20 (2016) 5–11

Contents lists available at ScienceDirect

Sleep Medicine j o u r n a l h o m e p a g e : w w w. e l s e v i e r. c o m / l o c a t e / s l e e p

Original Article

Psychometric properties of the children’s sleep habits questionnaire in children with autism spectrum disorder Cynthia R. Johnson a,*, Alexandra DeMand b, Luc Lecavalier c, Tristram Smith d, Michael Aman c, Emily Foldes b, Lawrence Scahill e a

Department of Clinical & Health Psychology, University of Florida, Gainesville, FL University of Pittsburgh, Pittsburgh, PA c Nisonger Center & Ohio State University, Columbus, OH d Department of Pediatrics, University of Rochester, Rochester, NY e Marcus Autism Center, Children’s Healthcare of Atlanta & Emory University, 1920 Briarcliff Road Atlanta, GA 30329-4010 b

A R T I C L E

I N F O

Article history: Received 24 July 2015 Received in revised form 9 December 2015 Accepted 10 December 2015 Available online 29 December 2015 Keywords: Autism spectrum disorder Autism Sleep disturbance Sleep disturbances Sleep questionnaires Children’s sleep habits questionnaire

A B S T R A C T

Background and purpose: Sleep disturbances in autism spectrum disorder (ASD) are very common. Psychometrically sound instruments are essential to assess these disturbances. Children’s Sleep Habit Questionnaire (CSHQ) is a widely used measure in ASD. The purpose of this study was to explore the psychometric properties of the CSHQ in a sample of children with ASD. Participants and methods: Parents/caregivers of 310 children (mean age: 4.7) with ASD completed the CSHQ at study enrollment. Correlations between intelligence quotient (IQ) scores and the original CSHQ scales were calculated. Item endorsement frequencies and percentages were also calculated. A principal component analysis (PCA) was performed, and internal consistency was assessed for the newly extracted components. Results: Correlations between IQ scores and CSHQ subscales and total scores ranged from .015 to .001 suggesting a weak, if any, association. Item endorsement frequencies were high for bedtime resistance items, but lower for parasomnia and sleep-disordered breathing items. A PCA suggested that a fivecomponent solution best fits the data. Internal consistency of the newly extracted five components ranged α = .87–.50. Conclusions: Item endorsement frequencies were highest for bedtime resistance items. A PCA suggested a five-component solution. Three of the five components (Sleep Routine Problems, Insufficient Sleep, and Sleep-onset Association Problems) were types of sleep disturbances commonly reported in ASD, but the other two components (Parasomnia/Sleep-disordered Breathing and Sleep Anxiety) were less clear. Internal consistencies ranged from mediocre to good. Further development of this measure for use in children with ASD is encouraged. © 2015 Elsevier B.V. All rights reserved.

1. Introduction The prevalence of sleep problems in the general pediatric population is estimated at 25% [1], but poorly regulated sleep patterns affect up to nearly 80% of children with autism spectrum disorder (ASD) independent of intellectual functioning [2–4]. The common sleep problems identified in this research include delayed sleep onset, night wakings after sleep onset, early morning wakings, decreased total sleep time, bedtime resistance, and sleep-onset association problems (eg, sleeping with a parent, sleeping in places other than the child’s own bed, and requiring idiosyncratic objects

* Corresponding author. Department of Clinical & Health Psychology, University of Florida, Gainsville, FL 32611. Tel.: +4128977435; fax: 3522736156. E-mail address: [email protected]fl.edu (C.R. Johnson). http://dx.doi.org/10.1016/j.sleep.2015.12.005 1389-9457/© 2015 Elsevier B.V. All rights reserved.

to initiate sleep onset). Sleep disturbances can interfere with cognition, attention, memory consolidation, and daytime behavioral adjustment in general [5–10]. In children with ASD, sleep disturbances may amplify already delayed social interactions, repetitive behaviors, affective problems, and inattention/hyperactivity [11–15]. Given the high prevalence of sleep disturbances in children with ASD, a low-cost, but psychometrically sound, instrument to assess sleep disturbances for intervention research and clinical practice is needed. In comparison to the well-established instruments measuring daytime behavioral problems in children with ASD, however, few psychometrically sound measures to assess sleep problems are available for pediatric populations [16,17]. One approach to instrument development in ASD is to adapt a measure used in the broader population for use in ASD [18–20]. One widely used pediatric sleep questionnaire is the Children’s Sleep Habits Questionnaire (CSHQ) [21]. The CSHQ was developed

6

C.R. Johnson et al./Sleep Medicine 20 (2016) 5–11

to screen for sleep disorders for 4–10-year-olds based on the pediatric International Classification of Sleep Disorders (ICSD) [21]. The 33-item scale consists of eight subscales: 1) Bedtime Resistance, 2) Sleep Onset Delay, 3) Sleep Duration, 4) Sleep Anxiety, 5) Night Wakings, 6) Parasomnias, 7) Sleep-disordered Breathing, and 8) Daytime Sleepiness. These subscales range in length from 1 to 8 items (Sleep-onset Delay to Daytime Sleepiness). The initial publication on this measure described some psychometric properties, including internal consistency, test–retest reliability, and discriminated validity between a community sample (n = 1099) and clinical sample (n = 154) [21]. Test–retest reliability in a subset of 60 participants in the community sample over two weeks ranged from .62 to .79. Internal consistency for the CSHQ was .68 for the community sample and .78 for clinical sample. However, there was considerable variability in alphas for the subscales ranging from .36 to .83. The total score and subscales differentiated between the community sleep sample and a clinic sample of problem sleepers, supporting discriminate validity of the CSHQ. Results of a receiver operator characteristics (ROC) analysis suggested that a total score of 41 was considered the optimal clinical cutoff for sleep problems. A later study showed that the CSHQ discriminated between good and poor sleepers in a sample of 194 preschool children down to two years of age [22]. A review of the available pediatric sleep questionnaires examined whether the CSHQ met 11 steps toward psychometric validity [16]. The criteria applied included 1) purpose of the tool; 2) research question at hand; 3) response format; 4) generate items; 5) pilot; 6) item analysis and non-response analyses; 7) structure; 8) reliability; 9) validity; 10) confirmatory analyses; and 11) standardization and development of norms. The review concluded that the CSHQ met steps, 1, 2, 3, 8, and 9. Despite identifying several strengths, the review suggested that further refinement of the CSHQ is warranted. Moreover, assignment of some items to subscales can be questioned. For example, “child is afraid of sleeping alone” and “child needs parent in the room to fall asleep” are in both Bedtime Resistance and Sleep Anxiety subscales. Logically, such similar items should be included in only one subscale. Some items are also not developmentally appropriate for young children, particularly those with developmental delays (eg, the question on bedwetting in the Parasomnia subscale and the question of waking once a night on the Night Waking subscale). The bedwetting item was earlier highlighted as problematic in young children where bedwetting is commonplace and not likely associated with parasomnia [22], while waking once a night is common in young children [23]. As noted, the number of items in each subscale is highly variable, ranging from one item (Sleep Onset) to eight (Daytime Sleepiness). This raises questions about the adequacy of symptom coverage across subscales. It also raises concerns about reliability as it is axiomatic that reliability declines as the number of component items diminishes. Several large-scale studies have examined the psychometric properties of non-English versions of the CSHQ. A factor analysis in a large sample of Chinese children (n = 20,457) did not replicate the eight-factor structure of the CSHQ [24]. These authors proposed a three-factor structure: 1) Bedtime Behavioral Problems, 2) Sleep Disturbance, and 3) Sleep Duration and Daytime Sleepiness. As only the abstract for this publication was available in English, examination of the factor loadings of items was not possible. In a Dutch sample of children (n = 2385), Waumans et al. [25] reported moderate to good test–retest reliability, interobserver reliability, and internal consistency. A confirmatory factor analysis again failed to confirm the original eight-factor instrument. These authors suggested a four-factor model, but three items loaded on two factors, and four items (“falls asleep in own bed,” “snores loudly,” “awakes once during night,” and “wakes by himself”) were removed. Unfortunately, the factors were not named in this publication, nor were the factor loadings presented. A German study of 298 nonclinical

and 45 clinically ascertained children evaluated the reliability and validity of a German version of the CSHQ [26]. For the total score, internal consistency was 0.68 and test–retest reliability was 0.76. Across subscales, internal consistency ranged from 0.23 to 0.70 (very poor, overall) and test–retest reliability from 0.46 to 0.81. However, they also conducted a principal component analysis (PCA), suggesting only one component with high loadings. Unfortunately, as the factor or component loadings and assignments were not available, it was not possible to evaluate or compare any of these studies. Taken together, nonetheless, these studies suggest that the CSHQ is a candidate for further development given the inconsistent psychometric findings. Despite somewhat limited psychometric development, the CSHQ has been used extensively in research to document prevalence of and to characterize sleep disorders in various pediatric populations [27,28];. Further, it has been used widely in the emerging literature of sleep disturbances in children with ASD including its use as an outcome measure [13,29–31]. Indeed, the CSHQ is currently being used by the 17 sites of the Autism Speaks Autism Treatment Network (AS-ATN) to assess bedtime behavior and sleep patterns in its registry of children and adolescents with ASD in North America [12,32]. Spruyt & Gozal [17] reported another pediatric sleep questionnaire, that is, the Sleep Disturbance Scale for Children [33], which met all the 11 criteria and then the Behavior Evaluation of Disorders Sleep Scale (BEDS) [34]. These two measures have also been used in ASD [15,35,36]. Given the limited psychometric examination of the CSHQ in ASD and widespread use with this population, there is a need for further psychometric development of the CSHQ in the field of ASD. This study has two goals: 1) to describe the CSHQ results across three different groups of young children with ASD (with one group identified as having clinically significant sleep disturbances) and 2) to explore the component structure of the CSHQ for this sample and to determine the internal consistency of the newly identified principal components of the CSHQ in children with ASD. 2. Methods 2.1. Sample A total of 310 children with ASD were enrolled from three studies. The 24-week, multisite randomized controlled trial of parent training (PT) versus parent education (PE) conducted at Emory University, Indiana University, Ohio State University, University of Pittsburgh, University of Rochester, and Yale University [37] included 177 children (57%; age: 3–7 years). In addition to ASD, eligible children had disruptive behavior as evidenced by a score of ≥15 on the irritability subscale of the Aberrant Behavior Checklist (ABC) [38] and a Clinical Global Impressions Severity (CGI-S) [39] score of ≥4 (at least moderate). Either participants were taking no medication or medication had to be stable for 6 weeks. Children with Rett’s disorder or childhood disintegrative disorder, any serious medical conditions or psychiatric disorder, developmental level of <18 months, and current or past enrollment in structured PT program were excluded. The diagnosis of ASD was based on DSM-IV criteria [40] and corroborated by the Autism Diagnostic Observation Schedule (ADOS) [41, 42]. The second study enrolled 34 children (11%; age: 2–6 years) with ASD who participated in an 8-week randomized trial of a PT program designed to address sleep disturbances versus parent education [43]. The diagnostic assessment was similar to the inclusion to the multisite trial described earlier. In addition, eligible subjects with ASD had at least one sleep problem (bedtime resistance, delayed sleep disorders, sleep-onset association problem, night wakings, and/or early morning wakings) at a frequency of at least five days per week. Exclusions were similar to the RUBI study but with the

C.R. Johnson et al./Sleep Medicine 20 (2016) 5–11

addition of sleep apnea, restless legs, or periodic limb disorder during sleep, or a circadian-based sleep disorder. Participants were randomized to either a structured PT program designed to address sleep disturbances or a parent education (PE) program. Finally, the participants included 99 children with ASD (32%; age: 2–10 years) at the University of Pittsburgh site of the Autism Speaks Autism Treatment Network (AS-ATN). The AS-ATN includes 17 North American centers funded by Autism Speaks and the Health Resources and Services Administration (HRSA) with the goal of establishing medical standards of care for individuals with ASD [44]. In the AS-ATN, the diagnosis of ASD was based on DSM-IV-TR criteria and supported by the ADOS. Although the AS-ATN enrolled children between 2 and 17 years, only children between 2 and 10 years were included in the current study to be consistent with previous studies [21,22]. Some of these data have been reported in a previous publication used for different analyses [45]. In all three studies, Institutional Review Boards approved the protocol at all study sites. 2.2. Study measures Demographics Although the demographic forms in each of these studies were not identical, a common core of variables was collected for this study. Autism Diagnostic Observation Schedule (ADOS) [41, 42] is an investigator-based assessment conducted in naturalistic social situations demanding specific social, communication, and restricted/ repetitive responses. Although administration follows a standard protocol, the situations themselves are unstructured or semistructured. Behaviors are scored in the areas of social communication, social relatedness, play and imagination, and repetitive behaviors. This measure was used to support clinical diagnosis. Developmental level All three studies used the Stanford–Binet Intelligence Scales: 5th Edition [46] or the Mullen’s Scale of Early Learning [47] to accommodate each child’s developmental level. The Mullen was used in children with limited language skills or with mental age <2–3 years. The abbreviated or full Stanford–Binet was used in verbal children with mental age >3–4 years. Children’s Sleep Habits Questionnaire (CSHQ) The CSHQ is a 33item (two items repeated for a total of 35 items scored), parentreport measure designed to screen for sleep problems in children aged 4–10 years [21]. Subsequent research provided data on preschoolaged children down to age 2 years [22]. Items are rated on a 1–3 scale, where 3 = usually, 5–7 nights per week; 2 = sometimes, 2–4 nights per week; and 1 = never/rarely, 0–1 nights per week. In addition to the frequency ratings, parents are asked to indicate “yes” or “no” to the question “is this a problem?” on each item. The summation of the frequency ratings results in the total score, with higher scores reflecting more sleep disturbances as described earlier. Again, the CSHQ has eight subscales: Bedtime Resistance, Sleep Onset Delay, Sleep Duration, Sleep Anxiety, Night Wakings, Parasomnias, Sleep Disordered Breathing, and Daytime Sleepiness. Psychometric properties from these earlier studies are provided in the introduction section and are not repeated here to avoid redundancy. 2.3. Data analyses Data analyses were conducted using PASW statistics AMOS, Version 22 [48]. Descriptive statistics were conducted to describe the samples. Pearson product moment correlations between intelligence quotient (IQ) scores and the original CSHQ subscale and total scores were determined. Item endorsement statistics were calculated to include mean, standard deviation, and frequency and percentage for each item rank. A PCA was performed with oblimin rotations. Model fit indices were selected including Chi-square (with a score of 1–3 indicating good fit), root mean square error of approximation (RMSEA; with a

7

score of <.05 indicating good fit), Tucker–Lewis Index (TLI >.95 is favorable), and comparative fit index (CFI >.90 indicating traditional fit) [49]. Internal consistency (Cronbach’s alpha) was assessed for each component and total score. 3. Results 3.1. Sample characteristics The sample included 310 children with ASD (age: 2–10 years) with 86% being male and 14% female. As the study was conducted prior to DSM-V, diagnoses were based on DSM-IV criteria. The diagnostic criteria for autistic disorder were met by 71% of the participants, while the remaining participants met criteria for autism spectrum disorder (ASD, pervasive development disorder, not otherwise specified [PDD-NOS], or Asperger’s disorder). Developmental functioning was also assessed with around half of the participants administered the Stanford–Binet Fifth Edition [46] and the remainder by the Mullen’s Scale of Early Learning [47]. Table 1 provides demographics and characterization information for the three groups. Table 2 provides the correlations between the IQ and the subscale scores and total score of the 33-item CSHQ. IQ scores were not strongly correlated with CSHQ scores; hence we did not stratify by or control for IQ.

Table 1 Description & Characteristics of Study Sample. RUBI*

Sleep**

ATN***

N = 177

N = 34

N = 99

Variable

Mean (SD)

Mean (SD)

Mean (SD)

Mean age (in years) Original CSHQ Total

4.7 (1.14) 53.25 (10.62) # (%)

3.56 (1.02) 60.21 (8.11) # (%)

5.27 (2.27) 52.34 (10.32) # (%)

Intellectual Level ≥70 <70 Gender Male Female Diagnosis Autistic Disorder ASD Race White African American Asian Pacific Islander Other/Mixed Ethnicity Hispanic Non-Hispanic

124 (69.3) 39 (21.8)

13 (59) 20 (38)

48 (48.5) 38 (38.4)

155 (87.6) 22 (12.4)

26 (76.5) 8 (23.5)

84 (84.8) 15 (15)

121 (68.4) 56 (31.6)

29 (85.3) 5 (14.7)

71 (71.7) 28 (28.3)

154 (87) 14 (7.9) 6 (3.4) 2 (1.1) 1 (<1)

32 (94) 2 (5.9) 0 0 0

87 (87.9) 11 (11) 1 (1) 0 0

26 (14.7) 151(85.3)

2 (5.8) 32 (94)

1 (1) 98 (88)

* 16 missing cognitive measures. ** 1 missing cognitive measure. *** 13 missing cognitive measures.

Table 2 Correlation between 33-item CSHQ subscales and IQ scores (N = 265).

Bedtime Resistance Sleep Onset Sleep Duration Sleep Anxiety Night Wakings Parasomnias Sleep-disordered Breathing Daytime Sleepiness Total CSHQ

r

R

−0.149 −0.043 −0.119 0.028 −0.11 −0.09 −0.001 0.032 −0.1

P = .015 P = 4.481 P = .053 P = .648 P = .075 P = .143 P = .983 P = .605 P = .106

8

C.R. Johnson et al./Sleep Medicine 20 (2016) 5–11

Table 3 Per Item means and frequency/% of item endorsement. Subscale Items Bedtime Resistance R

Child goes to bed at the same time at night Child falls asleep alone in own bed R Child falls asleep in parent’s or sibling’s bed Child needs parent in the room to fall asleep Child struggles at bedtime Sleep-onset Delay Child falls asleep within 20 minR Sleep Duration Child sleeps too little Child sleeps right amount R Child sleeps about the same amount each day R Sleep Anxiety Child needs parent in room to sleep Child is afraid of sleeping in dark Child afraid of sleep alone Child has trouble sleeping away Night Wakings Child moves to other bed Child awakes once during night Child awakes more than once Parasomnias Child wets the bed at night Child talks during sleep Child is restless and moves a lot during sleep Child sleepwalks during the night Child grinds teeth during sleep Child awakens during night screaming, sweating, and inconsolable Child awakens alarmed by a frightening dream Sleep-disordered Breathing Child snores loudly Child seem to stop breathing during sleep Child snorts/and or gasps during sleep Daytime Sleepiness Child wakes up by him/herself R Child wakes up in a negative mood Adults or siblings wake up child Child has difficulty getting out of bed in the morning 30. Child takes a long time to become alert in the morning Child seems tired

32. Watching T.V. 33. Riding in car R

Rarely

Sometimes

Usually

Mean (SD)

Freq/%

Freq/%

Freq/%

1.3 (.64) 1.74 (.9) 1.58 (.84) 1.73 (.88) 1.76 (.83)

225/72.3 178/57.2 201/64.6 173/55.6 153/49.2

55/17.7 36/11.6 38/12.2 48/15.4 80/25.7

23/7.4 96/30.9 71/22.8 89/28.6 77/24.8

1.88 (.83)

128/41.2

92/29.6

90/28.9

1.71 (.77) 1.67 (.79) 1.4 (.66)

150/48.2 164/52.7 213/68.5

100/32.2 83/26.7 68/21.9

60/19.3 63/20.3 29/9.3

1.73 (.88) 1.54 (.83) 1.58 (.82) 1.62 (.77)

173/55.6% 210/67.6 197/63.3 172/55.3

48/15.4 32/10.3 47/15.1 82/26.4

89/28.6 68/21.0 66/21.2 56/18

1.6 (.78) 1.9 (.78) 1.5 (.72)

191/61.4 115/37 206/66.2

64/20.6 118/37.9 63/20.3

55/17.7 77/24.8 41/13.2

1.81 (.92) 1.4 (.62) 2 (.83) 1.1 (.35) 1.3 (.57) 1.24 (.52) 1.24 (.47)

167/53.7 204/65.6 104/33.4 285/91.6 251/80.7 249/80.1 239/76.8

36/11.6 84/27 97/31.2 20/6.4 38/12.2 48/15.4 66/21.2

107/34.4 22/7.1 109/35 5/1.6 21/6.8 13/4.2 5/1.5

1.4 (.58) 1.05 (.24) 1.1 (.35)

215/69.1 297/95.5% 283/91

79/25.4 11/3.5 22/7.1

16/5.1 2/0.6 5/1.6

1.4 (.59) 1.6 (.66) 1.67 (.72) 1.43 (.64) 1.4 (.62) 1.66 (.66)

215/69.1 166/53.2 148/47.6 203/65.3 213/68.5 138/44.4 Not Sleepy 201/64.6 143/46

77/24.8 116/37.3 117/37.6 81/26 74/23.8 140/45 Very Sleepy 59/19 67/21.5

18/5.8 28/9 45/14.5 26/8.4 23/7.4 32/10.3 Falls Asleep 50/16.1% 100/32.2

1.5 (.76) 1.9 (.88)

Reversed Scored.

3.2. Item endorsement Table 3 provides descriptive statistics on the endorsement of individual CSHQ items. Endorsement frequencies were high for bedtime resistance items, but relatively low for parasomnia and sleep-disordered breathing items with a few exceptions such as bedwetting, being restless, and moving a lot. 3.3. Principal component analysis A PCA method with oblimin rotations was conducted. Six items were removed because they were not developmentally appropriate for all young children ([1] “child wakes once during the night), had low endorsement frequencies ([2] “child grinds teeth during night,” [3] “stops breathing,” and [4] “snorts”), or did not load >.30 on any of the components ([5] “child falls asleep riding in the car,” [6] “child wets bed at night”). “Child wets the bed at night” proved problematic in previous studies as many young children are not toilet trained [22,43]. With six items removed, examination of the scree plot suggested a five-component solution. Table 4 provides item loadings for this five-component solution. This solution had an RMSEA value of .054 (95% CI = .048–.061) indicating a reasonable fit [49]; including additional components yielded only slight

improvements. Moreover, the five-component model explained 51% of the total variance, had an acceptable Chi-square value of 1.91, and moderate fit values for CFI = .894 and TLI = .879 [49]. Therefore, the five-component solution was identified as optimal using the scree plot, factor loadings, and model fit indices. We labeled the components as 1) Sleep Routine Problems, 2) Insufficient Sleep, 3) Sleep-onset Association Problems, 4) Parasomnia/ Sleep-disordered Breathing, and 5) Sleep Anxiety. The first component, Sleep Routine Problems, had nine items and a mean factor loading of .59. It consisted of items from the original insomnia, bedtime resistance, and daytime sleepiness subscales. The second factor, Insufficient Sleep, had five items with a mean factor loading of .74, and described behaviors secondary to insufficient sleep. The third component, Sleep-onset Association Problems, had four items that capture both sleep-onset association problems and sleep anxiety and had a mean component loading of .73. The fourth component, Parasomnia/ Sleep-disordered Breathing, had a mean factor loading of .53 and with six items describing primarily behaviors associated with parasomnias and sleep-disordered breathing; it also included an item originally on the night waking subscale. The fifth factor, Sleep Anxiety, contained only three items with a mean factor of .47 and included two items from the original Parasomnia subscale and one from the original Sleep Anxiety subscale.

C.R. Johnson et al./Sleep Medicine 20 (2016) 5–11

9

Table 4 CSHQ Factor Loadings.

10. 9. 11. 6. 31. 1. 2. 25. 32. 29. 28. 26. 30. 27. 3. 5. 4. 8. 13. 7. 15. 14. 16. 18. 22. 26. 23.

Child sleeps the right amount Child sleeps too little Child sleeps about the same amount each day Child struggles at bedtime Child seems tired Child goes to bed at the same time at night Child falls asleep within 20 min after going to bed Child is awake more than once during the night Child falls asleep watching T.V. Child has difficulty getting out of bed in the morning Adults or siblings wake up child Child wakes up by him/herself Child takes a long time to become alert in the morning Child wakes up in a negative mood Child falls asleep alone in own bed Child needs parent in the room to fall asleep Child falls asleep in parent’s or sibling’s bed Child is afraid to sleep alone Child talks during sleep Child is afraid of sleeping in the dark Child sleepwalks during the night Child is restless and moves a lot during sleep Child moves to someone else’s bed during the night Child snores loudly Child awakens during night screaming, sweating, and inconsolable Child has trouble sleep away from home Child awakens alarmed by a frightening dream

3.4. Internal consistency For the newly identified five components, internal consistencies were as follows: Sleep Routine Problems = .79, Insufficient Sleep = .81, Sleep-onset Association Problems = .87, Parasomnias/ Sleep-disordered Breathing = .56 and Sleep Anxiety =.50. 4. Discussion The purpose of this study was to evaluate the psychometric properties of the CSHQ in 310 well-characterized children with ASD (age: 2–10 years). Results suggested weak, if any, association between IQ and the original CSHQ subscale and total scores. This observation is consistent with other prior studies [2–4]. Inspection of item endorsement frequencies in this sample of children with ASD indicated high endorsement of those in the original Bedtime Resistance subscales, but low endorsement of items in the Sleep-disordered Breathing and Parasomnia subscales. The two exceptions were “child wets bed” and “restless, moves a lot.” Given that many young children with ASD have not yet been fully toilet trained, particularly for night, the item was problematic and in fact eliminated in our five-component solution as it did not load on any component. The latter item may reflect the general restlessness and overactivity in children with ASD rather than a symptom of parasomnia. Using PCA, a five-component solution was obtained following removal of six items. The remaining 27 items fell into five components. These empirically derived item clusters differed substantially from the original subscales [21]. Clinically, the items comprising the Sleep Routine Problems component tapped regularity of sleep patterns. High scores on the fifth component Insufficient Sleep would indeed reflect inadequate sleep. The third component was labeled Sleep-onset Association Problems. Although these three components are not aligned with ICSD sleep taxonomy compared to the original subscale scores, they capture frequently observed sleep disturbances in children with ASD [30, 43, 50, 51]. For example, many children with ASD sleep less than suggested for their age and show signs of insufficient sleep [43, 51]. Moreover, they often have behaviors that are evoked by seemingly extraneous or idiosyncratic

Factor 1

Factor 2

Factor 3

Factor 4

Factor 5

.859 .847 .657 .523 .513 .500 .487 .395 .355 −.011 −.029 .041 −.013 .054 .001 −.012 −.010 −.020 −.012 .097 −.070 .215 .190 −.035 .142 .061 .063

−.116 −.148 .033 .092 .225 .162 .116 −.122 −.039 .837 .828 .795 .706 .515 .042 −.013 .041 .020 −.001 .033 .166 −.025 −.109 .001 .038 .004 .003

−.012 .046 .083 −.292 .217 −.146 −.161 −.117 −.001 −.048 −.019 .019 −.056 .044 −.903 −.847 −.843 −.741 .085 .007 −.003 −.146 −.317 −.095 −.092 −.063 .008

−.048 −.119 .092 −.021 .077 −.183 .012 .314 .111 .035 −.053 −.013 .045 .122 −.126 .000 −.047 .244 .738 .571 .479 .474 .400 .377 .142 .074 .373

−.111 −.142 −.038 .032 −.160 −.008 .228 −.358 .083 −.045 .107 .238 −.127 −.335 −.111 −.097 −.017 .070 −.028 .488 −.089 −.119 .266 −.121 −.636 −.462 −.404

stimuli [52] which is at the crux of sleep-onset association problems (e.g., sleeping with a parent, sleeping places other than a child’s own bed, and requiring idiosyncratic objects to initiate sleep onset). These three components may also be useful targets of intervention [43]. Four of five items in the Parasomnia/Sleep-disordered Breathing component were parasomnias and one was sleepdisordered breathing. The Sleep Anxiety component has only three items and may need additional items and coverage for this subscale to be useful. Overall, although the five-component solution had acceptable psychometric properties, our results suggest that the CSHQ may not be optimally designed to assess commonly occurring sleep problems in children with ASD. This study, similar to several other studies, diverged from the original subscale organization and item inclusion for the CSHQ. Notably, Schlarb et al. [26] aborted a PCA because of poor loading, while Waumans et al. [25] reported a four-factor model (but unnamed and factor loading were not included). Li et al. [24] reported a three-factor model of bedtime behavior problems, sleep disturbance, and sleep duration/daytime sleepiness. Comparisons to our findings are not possible without knowledge of the factor loadings. Internal consistency of the newly derived structure ranged from .50 to .87, that is, poor (.50) to strong (.87). This range is slightly higher than that reported by the instrument developers and others in non-ASD samples [21, 25, 26], but in line with what was earlier reported in a sample of children with ASD using the original 33 items [45]. The three components with the highest internal consistency are those that are most relevant to children with ASD and sleep disturbances (Sleep Routine Problems, Insufficient Sleep, and Sleeponset Association Problems). The divergence of our derived structure from the original subscale suggests that the original organization of subscales and scoring procedure for the CSHQ should be used with caution in children with ASD. However, the moderate-to-high internal consistency and relevance of the Sleep Routine Problems, Sleep Inadequacy, and Sleep Onset Association Problem components to children with ASD suggest that modifications could strengthen the CSHQ for children with ASD. Indeed, there is a pressing need for a reliable, valid, and relevant

10

C.R. Johnson et al./Sleep Medicine 20 (2016) 5–11

measure of sleep disturbances in children with ASD for screening and for tracking change with treatment [53]. As mentioned in the introduction, one approach to instrument development for use in children with ASD is to adapt and modify the existing measures similar to that done by several other research teams [18–20,54]. The CSHQ could be improved by adding more items relating to disruptive behavior, rigidity, and ritualistic behaviors at bedtime which may adversely influence a smooth transition to bedtime and wake to sleep transition. Second, items that query about sensory arousal at bedtime and specific sleep-onset association problems relevant to children with ASD would also be useful. For example, questions about whether a child requires certain bedsheets or bedclothes to sleep or other idiosyncratic items present at bedtime may enhance the scale. Third, additional items on delayed sleep onset, which is a common complaint from parents of children with ASD and often a target for intervention, also warrants consideration. The original measure has only one such item that may undermine its reliability. Items assessing a combination of sleep latency and sleep association issues may also be useful. For example, a child may fall asleep within 20 min, but only in the presence of an adult or in the parents’ bed. Conversely, the number of sleep-disordered breathing items may be less relevant in this population and thus warrant less attention. This is not to imply that sleep-disordered breathing should not be screened, but rather items be reconsidered, reworded, and organized in a different manner. The presence of only three items to tap sleep anxiety may be inadequate. Additional items may be useful and confirm the validity of this nascent component. Indeed, an association between anxiety and sleep problems in children with ASD has been reported [14,55,56]. The limited communication skills of many young children with ASD, however, pose challenges for measuring anxiety in this population [57–59]. The current study has several limitations. Children with ASD in this study were primarily Caucasian, which may limit the generalizability of the findings. Our sample was also relatively young with a mean age of <five years. Likewise, over 50% of the sample had the required elevated scores on a measure of disruptive behaviors in order to qualify for the RUBI study; thus, this sample was not a representative of all children with ASD. Moreover, although the sample of 310 children with ASD had a subject-to-item ratio of 9, a larger sample size to validate our structure may have been more robust. Finally, this was a cross-sectional analysis, and thus, test–retest reliability was not assessed. To summarize, the current paper adds to our knowledge about the psychometric properties of the CSHQ in a sample of children with ASD. The present study of a relatively large, well-characterized sample of children with ASD suggested that alternative scoring may be an improvement for assessing children with ASD. Internal consistency was mediocre to high for the components. Given the frequency and often severity of sleep disturbances in children with ASD, having a psychometrically sound measure is highly desirable. Alternative tools might also be considered such as the less utilized SDSC [33] or the BEDS [15]. Acknowledgments This work was funded by the National Institute of Mental Health by grants to the University of Pittsburgh (MH080965; principal investigator: Dr. Johnson), Yale/University/Emory University (MH081148 & MH082882; principal investigator: Dr. Scahill), Ohio State University (MH081105; principal investigator: Dr Lecavalier), Indiana University (MH081221; principal investigator: Dr. Swiezy), and the University of Rochester (MH080906; principal investigator: Dr. Smith). The project described in this publication also was supported by Autism Speaks, Autism Service, Education, Research and Training grant from the Pennsylvania Bureau of Autism

Services, Department of Public Welfare, and National Institute for Research Resources (UL1 RR024153-06 & ULTR000005) (University of Pittsburgh), MH079130 (principal investigator: Dr. Sukhodolsky); University of Rochester Clinical and Translational Scholar Award (CTSA) (UL1 TR000042) from the National Center for Advancing Translational Sciences of the National Institutes of Health (NIH); a CTSA (UL1 RR024139) and grant from the National Center for Research Resources (NCRR)(5KL2RR024138), a component of the NIH; and the NIH Roadmap for Medical Research. This work was supported in part by a Public Health Service grant (UL1 RR025008) from the CTSA program of then IH NCRR at Emory University School of Medicine and also supported by the Marcus Foundation. Note: Dr. Johnson is now at the University of Florida, Department of Clinical & Health Psychology

Conflict of interest The ICMJE Uniform Disclosure Form for Potential Conflicts of Interest associated with this article can be viewed by clicking on the following link: http://dx.doi.org/10.1016/j.sleep.2015.12.005.

References [1] Owens JA. Classification and epidemiology of childhood sleep disorders. Sleep Med Clin 2007;2(3):353–61. [2] Allik H, Larsson JO, Smedje H. Sleep patterns of school-age children with asperger syndrome or high-functioning autism. J Autism Dev Disord 2006;36(5):585–95. doi:10.1007/s10803-006-0099-9. [3] Krakowiak P, Goodlin-Jones B, Hertz-Picciotto I, et al. Sleep problems in children with autism spectrum disorders, developmental delays, and typical development: a population-based study. J Sleep Res 2008;17(2):197–206. [4] Polimeni MA, Richdale AL, Francis AJ. A survey of sleep problems in autism, asperger’s disorder and typically developing children. J Intellect Disabil Res 2005;49(Pt 4):260–8. doi:10.1111/j.1365-2788.2005.00642.x. [5] Gruber R, Wiebe S, Montecalvo L, et al. Impact of sleep restriction on neurobehavioral functioning of children with attention deficit hyperactivity disorder. Sleep 2011;34(3):315. [6] Owens JA. Neurocognitive and behavioral impact of sleep disordered breathing in children. Pediatr Pulmonol 2009;44(5):417–22. [7] Sadeh A. Consequences of sleep loss or sleep disruption in children. Sleep Med Clin 2007;2(3):513–20. [8] Sadeh AVI, Gruber R, Raviv A. Sleep, neurobehavioral functioning, and behavior problems in school-age children. Child Dev 2002;73(2):405–17. doi:10.1111/ 1467-8624.00414. [9] Steenari M-R, Vuontela V, Paavonen EJ, et al. Working memory and sleep in 6- to 13-year-old schoolchildren. J Am Acad Child Adolesc Psychiatry 2003;42(1):85–92. [10] Vriend JL, Davidson FD, Corkum PV, et al. Sleep quantity and quality in relation to daytime functioning in children. Child Health Care 2012;41(3):204–22. [11] Gabriels RL, Cuccaro ML, Hill DE, et al. Repetitive behaviors in autism: relationships with associated clinical features. Res Dev Disabil 2005;26(2):169– 81. doi:10.1016/j.ridd.2004.05.003. [12] Goldman SE, McGrew S, Johnson K, et al. Sleep is associated with problem behaviors in children and adolescents with autism spectrum disorders. Res Autism Spectr Disord 2011;5(3):1223–9. [13] Malow BA, Marzec ML, McGrew SG, et al. Characterizing sleep in children with autism spectrum disorders: a multidimensional approach. Sleep 2006;29(12): 1563–71. [14] Mayes SD, Calhoun SL. Variables related to sleep problems in children with autism. Res Autism Spectr Disord 2009;3(4):931–41. [15] Schreck KA, Mulick JA, Smith AF. Sleep problems as possible predictors of intensified symptoms of autism. Res Dev Disabil 2004;25(1):57–66. [16] Spruyt K, Gozal D. Development of pediatric sleep questionnaires as diagnostic or epidemiological tools: a brief review of dos and don’ts. Sleep Med Rev 2011a;15(1):7–17. [17] Spruyt K, Gozal D. Pediatric sleep questionnaire as diagnostic and epidemiological tools: a review of currently available instruments. Sleep Med Rev 2011b;15:19–32. doi:10.1016/j.smrv.2010.07.005. [18] Chowdhury M, Aman MG, Scahill L, et al. The home situations questionnairePDD version: factor structure and psychometric properties. J Intellect Disabil Res 2010;54(3):281–91. [19] Hallett V, Lecavalier L, Sukhodolsky D, et al. Exploring the manifestations of anxiety in children with autism spectrum disorders. J Autism Dev Disord 2013;43(10):2341–52. doi:10.1007/s10803-013-1775-1. [20] Lecavalier L, Aman M, Hammer D, et al. Factor analysis of the nisonger child behavior rating form in children with autism spectrum disorders. J Autism Dev Disord 2004;34(6):709–21. doi:10.1007/s10803-004-5291-1.

C.R. Johnson et al./Sleep Medicine 20 (2016) 5–11

[21] Owens JA, Spirito A, McGuinn M. The children’s sleep habits questionnaire (CSHQ): psychometric properties of a survey instrument for school-aged children. Sleep 2000;23(8):1043–51. [22] Goodlin-Jones BL, Sitnick SL, Tang K, et al. The children’s sleep habits questionnaire in toddlers and preschool children. J Dev Behav Pediatr 2008;29(2):82–8. [23] Mindell JA. Empirically supported treatments in pediatric psychology: bedtime refusal and night wakings in young children. J Pediatr Psychol 1999;24(6):465– 81. [24] Li SH, Jin XM, Shen XM, et al. Development and psychometric properties of the Chinese versuib of children’s sleep habits questionnaire. Zhonghua Er KE Za Zhi 2007;45:176–80. [25] Waumans RC, Terwee CB, Van den Berg G, et al. Sleep and sleep disturbance in children: reliability and validity of the Dutch version of the Child Sleep Habits Questionnaire. Sleep 2010;33(6):841–5. [26] Schlarb AA, Schwerdtle B, Hautzinger M. Validation and psychometric properties of the German version of the Children’s Sleep Habits Questionnaire (CSHQ-DE). Somnologie. 2010;14:260–6. [27] El-Sheikh M, Buckhalt JA, Mize J, et al. Marital conflict and disruption of children’s sleep. Child Dev 2006;77(1):31–43. doi:10.1111/j.14678624.2006.00854.x. [28] Liu X, Hubbard JA, Fabes RA, et al. Sleep disturbances and correlates of children with autism spectrum disorders. Child Psychiatry Hum Dev 2006;37(2):179–91. [29] Cortesi F, Giannotti F, Ivanenko A, et al. Sleep in children with autistic spectrum disorder. Sleep Med 2010;11(7):659–64. doi:10.1016/j.sleep.2010.01.010. [30] Giannotti F, Cortesi F, Cerquiglini A, et al. Sleep in children with autism with and without autistic regression. J Sleep Res 2011;20(2):338–47. doi:10.1111/ j.1365-2869.2010.00882.x. [31] Reed HE, McGrew SG, Artibee K, et al. Parent-based sleep education workshops in autism. J Child Neurol 2009;24(8):936–45. doi:10.1177/0883073808331348. [32] Sikora DM, Johnson K, Clemons T, et al. The relationship between sleep problems and daytime behavior in children of different ages with autism spectrum disorders. Pediatrics 2012;130(Suppl. 2):S83–90. [33] Bruni O, Ottaviano S, Guidetti V, et al. The sleep disturbance scale for children (SDSC) construct ion and validation of an instrument to evaluate sleep disturbances in childhood and adolescence. J Sleep Res 1996;5(4):251–61. [34] Schreck K, Mulick JA, Rojahn J. Development of the behaivoral evaluation of disorders of sleep scale. J Child Fami Studi 2003;12:349–59. [35] Miano S, Bruni O, Maurizio E, et al. Sleep in children with autism spectrum disorder: a questionnaire and poly somnographic study. Sleep Med 2007;9:64– 70. doi:10.1016/j.sleep.2007.01.014. [36] Taylor MA, Schreck KA, Mulick JA. Sleep disruption as a correlate to cognitive and adaptive behavior problems in autism spectrum disorders. Res Dev Disabil 2012;33(5):1408–17. doi:10.1016/j.ridd.2012.03.013. [37] Bearss K, Johnson CR, Smith T. Effect of parent training vs parent education on behavioral problems in children with autism spectrum disorder: a randomized clinical trial. J Am Med Assoc 2015;313(15):1524–33. [38] Aman MG, Singh NN, Stewart AW, et al. Psychometric characteristics of the Aberrant Behavior Checklist. Am J Ment Def 1985;89(5):492–502. [39] Guy W. ECDEU Assessment manual of psychopharmacology (NIMH Publication No. 76-338). Rockville, MD: National Institute of Mental Health US Department of Health, Education, and Welfare publication (ADM), Psychopharmacology Research Branch; 1976. [40] American Psychiatric Association. Diagnostic and statistical manual of mental disorders: DSM-IV. Amer Psychiatric Pub Inc; 1994.

11

[41] Lord C, Risi S, Lambrecht L, et al. The autism diagnostic observation schedule-generic: a standard measure of social and communication deficits associated with the spectrum of autism. J Autism Dev Disord 2000;30:205– 23. [42] Lord C, Rutter M, DiLavore P, et al. Autism diagnostic observation schedule-WPS edition. Los Angeles, CA: Western Psychological Services; 1999. [43] Johnson CR, Turner KS, Foldes E, et al. Behavioral parent training to address sleep disturbances in young children with autism spectrum disorder: a pilot trial. Sleep Med 2013;14(10):995–1004. . [44] Coury D, Jones NE, Klatka K, et al. Healthcare for children with autism: the autism treatment network. Curr Opin Pediatr 2009;21(6):828–32. doi:10.1097/ MOP.0b013e328331eaaa. [45] Johnson CR, Turner KS, Foldes EL, et al. Comparison of sleep questionnaires in the assessment of sleep disturbances in children with autism spectrum disorders. Sleep Med 2012;13(7):795–801. [46] Roid GH. Stanford-binet intelligence scales: examiner’s manual. 5th ed. Itasca, IL: Riverside; 2003. [47] Mullen E. Mullen scales of early learning (AGS Edition ed.). Circle Pines: American Guidance Service; 1995. [48] SPSS Inc. Predictive analytic software (PASW). Chicago, IL: SPSS Inc; 2010. [49] Browne MW, Cudeck R. Alternate ways of assessing model fit. In: Bollen KA, Long JS, editors. Testing structural equation models. Newbury, CA: Sage; 1993. p. 136–62. [50] Richdale AL, Schreck KA. Sleep problems in autism spectrum disorders: prevalence, nature, and possible biopsychosocial aetiologies. Sleep Med Rev 2009;13(6):403–11. [51] Wiggs L, Stores G. Sleep patterns and sleep disorders in children with autistic spectrum disorders: insights using parent report and actigraphy. Dev Med Child Neurol 2004;46(6):372–80. [52] Ploog BO. Stimulus overselectivity four decades later: a review of the literature and its implications for current research in autism spectrum disorder. J Autism Dev Disord 2010;40(11):1332–49. [53] Hodge D, Parnell A, Hoffman CD, et al. Methods for assessing sleep in children with autism spectrum disorders: a review. Res Autism Spectr Disord 2012;6:1337–44. [54] DeMand A, Johnson C, Foldes E. Psychometric properties of the brief autism mealtime behaviors inventory. J Autism Dev Disord 2015;1–7:doi:10.1007/ s10803-015-2435-4. [55] Johnson CR, DeMand A, Shui A. Relationships between anxiety and sleep and feeding in young children with ASD. J Dev Phys Disabil 2015;27(3):359–73. doi:10.1007/s10882-015-9419-3. [56] Mayes SD, Calhoun SL, Murray MJ, et al. Anxiety, depression, and irritability in children with autism relative to other neuropsychiatric disorders and typical development. Res Autism Spectr Disord 2011;5(1):474–85. [57] Kerns CM, Kendall PC. The presentation and classification of anxiety in autism spectrum disorder. Clin Psychol Sci Pr 2012;19(4):323–47. doi:10.1111/ cpsp.12009. [58] Lecavalier L, Wood JJ, Halladay AK, et al. Measuring anxiety as a treatment endpoint in youth with autism spectrum disorder. J Autism Dev Disord 2014;44(5):1128–43. doi:10.1007/s10803-013-1974-9. [59] Wood JJ, Gadow KD. Exploring the nature and function of anxiety in youth with autism spectrum disorders. Clin Psychol Sci Pr 2010;17(4):281–92. doi:10.1111/ j.1468-2850.2010.01220.x.