Development of the Parent-Rated Anxiety Scale for Youth With Autism Spectrum Disorder

N EW R E S E A R C H

Development of the Parent-Rated Anxiety Scale for Youth With Autism Spectrum Disorder Lawrence Scahill, MSN, PhD, Luc Lecavalier, PhD, Robert T. Schultz, PhD, Andrea Nichole Evans, BA, Brenna Maddox, PhD, Jill Pritchett, MA, John Herrington, PhD, Scott Gillespie, MS, Judith Miller, PhD, R. Toby Amoss, PhD, Michael G. Aman, PhD, Karen Bearss, PhD, Kenneth Gadow, PhD, Michael C. Edwards, PhD

Objective: Anxiety is common in youth with autism spectrum disorder (ASD). There is no accepted outcome measure for anxiety in this population. Method: Following a series of focus groups with parents of youth with ASD, we generated 72 items (scored 0
3). Parents of 990 youth with ASD (aged 5
17 years; 80.8% male) completed an online survey. Factor analysis and item response theory analyses reduced the content to a single factor with 25 items. Youth with at least mild anxiety (n ¼ 116; aged 5
17 years; 79.3% male) participated in a comprehensive clinical assessment to evaluate the validity and reliability of the 25-item Parent-Rated Anxiety Scale for ASD (PRAS-ASD). Results: In the online sample, the mean PRAS-ASD score was 29.04  14.9 (range, 0
75). The coefficient a was 0.93. The item response theory results indicated excellent reliability across a wide range of scores with low standard errors. In the clinical sample (n ¼ 116), the PRAS-ASD mean was 31.0  15.6 (range, 1
65). Pearson correlations with parent ratings of ASD symptom severity, repetitive behavior, and disruptive behavior ranged 0.33 to 0.66, supporting divergent validity of the PRAS-ASD. Pearson correlation with a parent-rated measure of anxiety used in the general pediatric population of 0.83 supported convergent validity. A total of 40 participants (32 boys, 8 girls; mean age, 11.9  3.4 years) returned at time 2 (mean, 12.2 days) and time 3 (mean, 24.2 days). Intraclass correlation showed test
retest reliabilities of 0.88 and 0.86 at time 2 and time 3, respectively. Conclusion: The 25-item PRAS-ASD is a reliable and valid scale for measuring anxiety in youth with ASD. Key words: autism, anxiety, outcome measurement J Am Acad Child Adolesc Psychiatry 2019;58(9):887–896.

utism spectrum disorder (ASD) is defined by early childhood onset of social communication impairment, repetitive behavior, and restricted interests.1 In clinically ascertained samples, an estimated 40% of youth with ASD meet criteria for at least one anxiety disorder.2-4 The identification of generalized anxiety, separation anxiety, and social anxiety disorders in youth with ASD implies that these disorders are independent of ASD and not different from anxiety disorders in the general pediatric population. It also suggests that youth with ASD are at elevated risk for anxiety disorders compared to children without ASD. For example, social impairment in a child with ASD may hinder age-appropriate social interactions, contribute to failure in the social milieu, and promote social anxiety. However, Kerns et al. have proposed that the manifestations of anxiety in youth with ASD may not fit neatly into current anxiety diagnostic categories. In this view, the confluence of ASD and anxiety symptoms

A

Journal of the American Academy of Child & Adolescent Psychiatry Volume 58 / Number 9 / September 2019

may mean that ASD and anxiety disorders are not completely independent.5 Whether or not ASD and anxiety disorders are completely independent, measuring anxiety in youth with ASD may require alternative approaches to the measurement of anxiety in the general pediatric population.6 Several studies have pursued a dimensional, rather than a categorical, approach to the measurement of anxiety in youth with ASD.7 A meta-analysis of 83 publications, nearly all of which used parent-rated scales, reported that youth with ASD had significantly higher levels of anxiety than youth in the general population and slightly higher anxiety than clinically referred children without ASD.7 In a prior study, we explored anxiety symptoms in 415 medication-free children with ASD (aged 4
17 years) using a 20-item, parent-rated, Child Adolescent Symptom Inventory (CASI), DSM-IV
referenced scale.8,9 Anxiety symptoms in this sample ranged from mild to severe.8

www.jaacap.org

887

SCAHILL et al.

However, parents rarely endorsed items that began with “worries” and “complains.” Failure to endorse languagedependent items was even more pronounced in nonverbal children and those with an IQ below 70. These findings provide additional evidence that measures of anxiety used in the general pediatric population may be less suitable for youth with ASD. This key point was also cited in the metaanalyses.7 The lack of an accepted, reliable, and valid outcome measure for anxiety in ASD has been cited as a major impediment for intervention research in youth with ASD.6 Guidelines from the US Food and Drug Administration and from the British Department of Health emphasize the importance of incorporating input from the patient population in the development of a new outcome measure.10,11 For children with developmental disabilities, primary caregivers are the appropriate proxies. We conducted a series of focus groups with 48 parents (85% mothers) of 45 children with ASD (aged 3
17 years; 71.1% boys) to explore the behavioral manifestations of anxiety.12 The focus group transcripts were systematically reviewed and coded to generate candidate items for a parent-reported measure for anxiety in youth with ASD. An expert panel, including the focus group reviewers, examined the identified text entries in the transcripts and generated 52 candidate items. This bank of 52 items and the 20 items from the original CASI Anxiety scale were compiled (72 items total) as a first draft of the parent-rated anxiety scale for youth with ASD (Flesch-Kincaid Grade level reading index, 7.1). Here we describe item reduction procedures and psychometric analyses that produced the final version of the new dimensional parent-rated anxiety scale. METHOD Study Design

The study was designed to develop and to test the reliability and validity of a parent-rated measure of anxiety in children with ASD. Here we report on the results of a large-scale online survey, followed by a detailed clinical assessment conducted at three sites (Emory University, Ohio State University, and Children’s Hospital of Philadelphia). Online Survey. To capture frequency and impairment, we

applied a 0 to 3 scale (0 ¼ not present; 1¼ present sometimes, not a real problem; 2 ¼ often present and a problem; 3 ¼ very frequent and a major problem) to the 72 items. This Draft 1 of the parent-rated anxiety scale for youth with ASD was placed on a secure website managed by Children’s Hospital of Philadelphia (CHOP). Parents were asked to consider the past 2 weeks. Information about the study was posted on the websites at each study site. In addition, we 888

www.jaacap.org

contracted with the Interactive Autism Network (IAN), which is a voluntary online registry for children diagnosed with ASD and their families. The high volume of the registry does not allow confirmation of an ASD diagnosis for all children. In a study of 118 youth on the IAN registry, Daniels et al. were able to confirm the ASD diagnoses in 98% of the sample.13 For the current study, parents on the IAN registry who previously indicated willingness to complete online research surveys received an e-mail notice about the study. Interested parents were directed to a Qualtrics survey on the secure website. The consent document was approved by the institutional review board at CHOP and was presented at the start of the online survey. A total of 1,565 parents electronically signed the consent document between April and September 2015. The survey introduction noted that the purpose of the study was to learn more about anxiety in children with ASD. The statement emphasized our interest in including children with and without anxiety. In addition to the 72-item anxiety measure (Draft 1 of the parent-rated anxiety scale), parents were also asked to complete a demographic form, the Aberrant Behavior Checklist, and the Social Communication Questionnaire (described below). Clinical Assessment. Following the completion of the on-

line survey, each site enrolled children and adolescents with ASD to participate in a detailed clinical assessment. A subsample of these participants returned to evaluate the test and retest of the parent-rated scale. The institutional review boards at each collaborating site approved this phase of the study. Participants were recruited through postings on websites at each institution, flyers to parent support groups in each region, and parents of children who received clinical care within each site. We also contacted parents who completed the online survey, agreed to be re-contacted, and lived within 2 hours of a study site. Initial screening was done via telephone. Children and adolescents between the ages of 5 and 17 years, with a community diagnosis of ASD and a history of at least mild anxiety via parent-report, were invited to participate in the clinical assessment. Prior to data collection, the purpose, procedures, and expectations of the study were described to parents and participants. Parents signed the consent document, and youth who were capable provided assent. The detailed assessment included standard measures to support the ASD diagnosis, commonly used parent ratings of behavior (Table 112,14-21) and the parentrated anxiety scale for youth with ASD. Participants and caregivers who agreed, returned for assessments at time 2 (10
14 days later) and time 3 (10
14 days after time 2) for assessment of test
retest reliability. Participants were Journal of the American Academy of Child & Adolescent Psychiatry Volume 58 / Number 9 / September 2019

PARENT-RATED ANXIETY SCALE FOR ASD

offered $100 for the detailed visit at time 1 and $50 for each follow-up visit. Measures Used in This Report

The measures used in the online survey and clinical assessment are listed in Table 1. Analytic Plan Online Survey. The data set of 990 youth with complete

information was randomly divided into a test sample and validation sample (N ¼ 495 for each). An expert panel (M.C.E., L.L., L.S.) examined item pair correlations. Highly correlated items were assessed for redundancy and marked for possible removal. Exploratory factor analyses on the randomly selected test sample (n ¼ 495) was conducted with the Comprehensive Exploratory Factor Analysis Program.22 An iterative approach was used to reduce the number of items and to drill down to understand the structure underlying retained items. Ordinary least squares estimation and polychoric correlations were used for categorical data at the item level.23 Oblique CF-Quartimax rotations were also

applied where appropriate. Parallel analysis, examination of scree plots, and clinical relevance were used to guide the number of factors to retain.24 Statistical findings were always considered against the backdrop of qualitative information from the focus groups. Following the exploratory factor analysis, confirmatory factor analysis was used to evaluate a single-factor model and to obtain measures of model fit. Confirmatory factor analysis was conducted with Mplus25 using polychoric correlations and robust weighted least squares.23 Model fit was evaluated with the commonly used indices: root mean square error of approximation, comparative fit index, Tucker
Lewis index, and standardized root mean square residual (SRMR).26-28 Guidelines to evaluate whether a given model provided a good approximation to the data included the following: root mean square error of approximation and SRMR, less than 0.1; comparative fit index and Tucker
Lewis index, greater than 0.9.29,30 The application of these analytic methods led to the removal of 31 items and resulted in a single factor consisting of 41 items (Draft 2 of the parent-rated anxiety scale). Additional analyses using item response theory (IRT) approaches,

TABLE 1 Measures Collected Online and in the Detailed Clinical Evaluation Measure Online Measures Demographic survey 40-Item Social Communication Questionnaire 72-Item Parent-Rated Anxiety ScaleLASD 58-Item Aberrant Behavior Checklist Clinical Assessment Measures Demographic survey Developmental and Medical History Autism Diagnostic Observation Schedule Stanford-Binet V Vineland II Anxiety Diagnostic Interview Schedule 41-Item SCARED 43-Item Repetitive Behavior SurveyLRevised 58-Item Aberrant Behavior Checklist ABC

Type of Measure

Purpose

Source

Parent completed Parent rated

Family demographics Support ASD diagnosis

Developed for the study Corsello et al., 200714

Parent rated

Draft 1 of anxiety rating scale

Bearss et al., 201612

Parent rated

Behavioral characterization of participants

Kaat et al., 201415

Parent rated Clinician interview

Family demographics Support ASD diagnosis

Direct assessment

Support ASD diagnosis

Developed for the study National Data Base for Autism Research Lord et al., 200016

Direct assessment Parent rated

IQ Adaptive functioning

Clinician interview Parent rated Parent rated

Assessment of anxiety disorders Convergent validity Divergent validity

Parent rated

Divergent validity

Roid, 200317 Sparrow, Cicchetti and Balla, 200518 Silverman et al., 200019 Birmaher et al. 199920 Bodfish et al., 200021 Kaat et al. 201415

Note: ASD ¼ autism spectrum disorder; SCARED ¼ Screen for Child Anxiety Related Disorders.

Journal of the American Academy of Child & Adolescent Psychiatry Volume 58 / Number 9 / September 2019

www.jaacap.org

889

SCAHILL et al.

conducted with flexMIRT,31 resulted in further reduction to a single factor with 25 items. Final analyses were conducted to assess the model on the validation sample (n ¼ 495) and then with the full sample (N ¼ 990) to obtain parameter estimates, standard errors, and summary measures such as internal consistency (Cronbach’s a) and IRT marginal reliability.32,33 Clinical Assessment. The distribution of the 25-item

Parent-Rated Anxiety Scale (PRAS-ASD) in the 116 youth who participated in the detailed clinical assessment at time 1 is shown in Figure S1, available online. The mean scores on this 25-item measure (scored 0
3) were compared across subgroups (eg, boys versus girls; children with average IQ versus those with intellectually disability). The evaluation of reliability included calculation of coefficient a in the time 1 sample, followed by test
retest in a subsample using intraclass correlation and Pearson correlation. To evaluate convergent and discriminant validity, we compared differences in Pearson r values for instruments designed to measure separate constructs (eg, subscales on the Aberrant Behavior Checklist and Repetitive Behavior Scale
Revised) and with the Screen for Child Anxiety Related Disorders (SCARED) using the Fisher R to Z transformation. The SCARED is a 41-item parent rating that has been shown to be reliable, valid, and sensitive to change in the general pediatric population.21,34 It has limited use in youth with ASD.35 RESULTS Online Survey

Of the 1,565 parents who consented to participate in the online survey, 363 parents provided no additional information, and 212 completed the demographic form. Complete data were available on 990 youth from 46 states (mean age, 11.4  3.7 years; range, 5
17 years). As shown in Table 2, 80.8% were male, and 63.5% (627 of 988; missing data, 2) were placed in special education. Of the 212 participants with demographic data only, the mean age of 10.7  3.6 years was slightly younger than that of the group with complete data. The percentages of male participants and youth in special educational placements among the nonparticipants were similar to the percentages shown in Table 2 for the full sample with complete data. There was no difference in the percentage of maternal informants (90.3% in full sample; 87.7% for the nonparticipants). The proportion of mothers with a college or graduate degree was 67.8% in the sample with complete data compared to 66% of mothers in the group with demographic data only. As shown in Table 2, the mean score on the SCQ was 17.2  5.8, and 67.3% (666 of 990) of participants were at or above the conventional threshold of 15.35 The mean and standard deviation of 14.2  10.4 on the Aberrant Behavior 890

www.jaacap.org

TABLE 2 Demographic and Clinical Characteristics of Participants in Online Survey (N ¼ 990) Male Youth Female Youth Diagnosis (n [ 990) Autism spectrum disorder Nonverbal (SCQ) Parent-Reported Child IQ (n [ 654) Above average Average Below average Not reported Race (n [ 989) White Black Asian Pacific Islander Mixed race Other Maternal education (n [ 988) Less than high school High school graduate or General Education Development certificate Some college or 2-year degree 4-Year college degree Graduate or professional school Vocational/technical School Placement (n [ 988) Regular class public or private school Regular class D special education Special education program Home school Other Aberrant Behavior Checklist, mean (SD) Irritability Social withdrawal Stereotypy Hyperactivity Inappropriate speech Social Communication Questionnaire, mean (SD)

n 800 190

% 80.8 19.2

990 82

100 8.3

219 188 247 336

33.5 28.8 37.8 33.9

862 59 13 0 39 16

87.2 6.0 1.3 0 3.9 1.6

5 62

0.5 6.3

195 360 312 54

19.7 36.4 31.6 5.5

222

22.5

334

33.8

293 49 90

29.7 5.0 9.1

14.2 11.8 5.7 18.6 4.3 17.2

(10.4) (8.4) (5.3) (11.7) (3.3) (5.8)

Note: GED ¼ General Educational Development; SCQ ¼ Social Communication Questionnaire.

Checklist (ABC) Irritability subscale suggests wide variation on disruptive behavior. The ABC Social Withdrawal subscale mean of 11.8  8.4 suggests relatively mild social disability.19 Journal of the American Academy of Child & Adolescent Psychiatry Volume 58 / Number 9 / September 2019

PARENT-RATED ANXIETY SCALE FOR ASD

FIGURE 1 Distribution of 25-Item Parent-Rated Anxiety Scale for Autism Spectrum Disorder (PRAS-ASD): Total Scores in 990 Online Participants

Note: Mean, 29.04  14.92; range, 0 to 75.

Exploratory Factor Analysis. In exploratory factor analysis,

eigenvalues provide information about the amount of variability that is explained by a proposed factor solution. The exploratory factor analysis results for the 72-item and 41item versions revealed one dominant eigenvalue, which was roughly four times larger than the next largest eigenvalue. Parallel analyses suggested between four and eight factors. Upon fitting these models, however, factors beyond the dominant first factor consisted of minor factors with two or three items. Confirmatory Factor Analysis. Following the first data

reduction step from 72 to 41 items, model fit indices were used to identify a subset of items that could be well represented by a single-factor confirmatory factor analysis model. We examined three models in the exploratory half, the validation half, and the combined full sample. The three models included a one-factor model, a onefactor model with two pairs of correlated residuals, and a one-factor model with four pairs of correlated residuals (see Tables S1 and S2, available online). The factor loadings from the one-factor model with four pairs of correlated residuals correlated at 0.96 with the factor loadings of the one-factor model without correlated residuals. Thus, the inclusion of correlated residuals did not have a substantive impact on the factor analytic results. Given these results, the single factor model provided a reasonable representation of the structure underlying the 25-item measure, and it was accepted as the final set of items for the PRAS-ASD. Figure 1 presents the

Journal of the American Academy of Child & Adolescent Psychiatry Volume 58 / Number 9 / September 2019

distribution of the parent-rated 25-item scale for the 990 online survey participants (mean, 29.04  14.92; range, 0
75). The coefficient a was 0.93. Item Response Theory. The Samejima graded response model was fitted to the final 25-item scale.36 This model is appropriate when analyzing ordered categorical item responses.37 Using common IRT fit indices, the model fit well (M2 ¼ 1,490.74, df ¼ 225, root mean square error of approximation ¼ 0.08). Item slopes showed variability, but overall strong links between the items and the construct and the severity parameters varied, indicating a good spread within and between items (see Table S3, available online, for IRT parameters). In IRT, reliability and the standard error of a score on the measure are conditional across the range of scores on the latent continuum. Figure 2 presents the standard error and reliability curves for the 25-item PRAS-ASD. For a wide range of scores (roughly 1.5 SDs below the mean to 2.5 SDs above), the scores have reliabilities over 0.9. Marginal reliability, an IRT-based single item summary of reliability, was 0.92. Clinical Assessment. Of the 129 children who were screened

in person for the clinical assessment, 121 met entry criteria and were enrolled. Five subjects had incomplete data and were excluded from further analysis. Complete data were obtained for 116 participants (Table 3). Scores on the PRAS-ASD ranged from 1 to 65, with a mean of 31.0  15.6 (Figure S1, available online, shows the distribution of PRASwww.jaacap.org

891

SCAHILL et al.

FIGURE 2 Item Reliability and Standard Error Curves for 25-Item Parent-Rated Anxiety Scale for Autism Spectrum Disorder (PRAS-ASD) (N ¼ 990) 1.00

0.70

0.90 0.60 0.80

0.50

0.70

Reliability

0.40 0.50 0.30

Standard Error

0.60

0.40

0.30

0.20

0.20 Reliability 0.10 Standard Error

0.10

0.00

0.00 -3

-2.5

-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

2.5

3

Anxiety

Note: Horizontal axis shows z scores, with higher z scores reflecting greater anxiety. Left vertical axis shows the reliability of the measure, and right vertical axis shows the standard error of measurement. Figure shows that the measure provides reliable information (0.9) from low scores (z score of
1.5) to high scores (z score of 2.5).

ASD). There was no difference in the mean values for boys and girls (31.6  (15.2) versus 29  17.3, respectively) or mean scores for participants with an IQ of 70 or above (31.1  15.6) compared to those with an IQ below 70 (30.9  15.9). Table 4 shows positive Pearson correlations of the 25item PRAS-ASD with parent ratings on disruptive behavior, social disability, and repetitive behavior ranging from 0.33 to 0.66. The PRAS-ASD was most strongly correlated with the SCARED (r ¼ 0.83), which is a parentrated anxiety measure validated in the general pediatric population. The linear association between the 25-item PRAS-ASD and SCARED was significantly higher than any other parent-rated measure correlation with the 25-item PRAS-ASD (all p < .01). A total of 40 individuals (32 boys and 8 girls) with a mean age of 11.9  3.4 years returned for time 2 after a mean of 12.2  2.9 days and 24.2  4.9 days later for time 3. Using intraclass correlation, the test
retest reliabilities were 0.88 (from T1 to T2), 0.93 (from T2 to T3), and 0.86 (from T1 to T3). Pearson correlations were nearly identical: 0.88 (from T1 to T2), 0.94 (from T2 to T3), and 0.86 (from T1 to T3). 892

www.jaacap.org

DISCUSSION This report describes the development and testing of a new, dimensional, parent-rated outcome measure for anxiety in youth with ASD. A work group empaneled by Autism Speaks concluded that the lack of an accepted measure for assessing the severity of anxiety in youth with ASD is an impediment to treatment development.6 We showed in a prior study that DSM-IV
referenced scale items that rely on verbal expression from youth with ASD are unlikely to be endorsed by parents, especially in children with language or cognitive delays.8 More recently, two studies in youth with ASD examined the reliability and validity of the parent-rated Spence Children’s Anxiety Scale. Although this measure is considered to be reliable and valid in the general pediatric population, both studies raised fundamental questions about its use in ASD.38,39 In keeping with the Food and Drug Administration guidance on Patient Reported Outcomes, we began the development of the current 25-item PRAS-ASD with a series of focus groups with parents to establish content validity.10 The six focus groups gathered information from parents on the behavioral manifestations of anxiety in youth Journal of the American Academy of Child & Adolescent Psychiatry Volume 58 / Number 9 / September 2019

PARENT-RATED ANXIETY SCALE FOR ASD

TABLE 3 Demographic and Clinical Characteristics of Youth With Autism Spectrum Disorder (ASD) in Clinical Assessment (n ¼ 116) Characteristic Age, y, mean (SD) Male Youth Female Youth SCQ Verbal (n [ 116) IQ (n [ 115) 70 <70 Race (n [ 116) White Black Asian Mixed race Maternal education (n [ 116) High school graduate or GED Some college or 2-year degree 4-Year college degree Graduate or professional school School Placement (n [ 116) Regular class public or private school Regular class D special education Special education program Home school Vineland, mean (SD) Socialization (n [ 111) Communication (n [ 112) Daily Living (n [ 112)

n 11.8 92 24 114

% (3.2) 79.3 20.7 98.3

87 28

75.6 24.4

78 22 3 13

67.2 19 2.6 11.2

9 34 45 28

7.8 29.3 38.8 24.1

65 25 14 12

56.0 21.6 12.1 10.3

69.1 76.2 76.9

(15.0) (14.6) (14.0)

Note: Data are number and percentage unless specified as mean (SD).

with ASD.12 Following the review of focus group material and consultation with an expert panel, we posted a 72-item draft of the scale (rated 0
3) online that was completed by parents of 990 youth with a community diagnosis of ASD. Iterative computations using factor analysis and item response analysis reduced the content to single factor of 25 items. Summed scores from the 25-item PRAS-ASD (rated 0
3) in the online sample (N ¼ 990) showed a normal distribution, a coefficient a of 0.93, and IRT marginal reliability of 0.92. Moreover, the IRT-based reliability and standard error curves showed that the scale provides a reliable estimate of anxiety severity across a wide range of scores (z scores from – 1.5 to 2.5) (Figure 2). The PRAS-ASD also showed impressive test
retest values from time 1 to time 2 (mean, 12.2 days later) and from time 1 to time 3 (24.2 days later). These observations indicate that the PRAS-ASD is not vulnerable to random fluctuation, which is a wellknown threat to detection of treatment effects in a clinical trial.40 Future research can focus on sensitivity to change Journal of the American Academy of Child & Adolescent Psychiatry Volume 58 / Number 9 / September 2019

TABLE 4 Mean, Standard Deviations, and Pearson Correlations for Parent Ratings of Behavior, Social Disability, and Anxiety With 25-Item Parent-Rated Anxiety Scale for ASD (PRAS-ASD)

25-Item Anxiety (n [ 116) SCARED (n [ 115) Aberrant Behavior Checklist (n [ 115) Irritability Social withdrawal Stereotypy Hyperactivity Inappropriate speech Social Communication Questionnaire (n [ 116) Repetitive Behavior Scale-R (n [ 116) Stereotyped behavior Self-injurious behavior Compulsive behavior Ritualistic behavior Sameness behavior Restricted behavior

Mean SD 31.0 15.6 23.2 14.6

Pearson r — 0.83

R to Z pa — <.001

12.1 9.9 11.4 8.3 6.0 5.2 16.8 11.3 4.5 3.5 20.1 6.6

0.59 0.45 0.47 0.39 0.47 0.33

<.001 <.001 <.001 <.001 <.001 <.001

5.2 3.4 5.4 6.4 10.0 4.2

0.44 0.36 0.50 0.53 0.66 0.45

<.001 <.001 <.001 <.001 .003 <.001

3.9 3.5 4.6 4.5 6.8 2.9

Note: aDifferences in Pearson correlations between PRAS-ASD and Screen for Child Anxiety Related Disorders (SCARED) versus other parent rating via Fisher R to Z transformation.

with treatment without concern about test-retest variability. There is preliminary evidence that the PRAS-ASD is sensitive to change following intervention.41 To evaluate validity of the 25-item PRAS-ASD, we conducted a comprehensive assessment of 116 youth with ASD (aged 5
17 years). Convergent validity of the PRASASD was supported by the correlation of 0.83 with the SCARED (a parent rating of anxiety validated in the general pediatric population). The correlation of the PRAS-ASD and the SCARED was significantly greater than the next highest correlations: 0.66 for Insistence on Sameness subscale of the Repetitive Behavior Scale–Revised (RBSR) and 0.59 for the Irritability subscale of Aberrant Behavior Checklist (ABC). These results suggest that the RBSR Sameness subscale and the ABC-Irritability subscale may be measuring related, albeit separate, constructs from the PRAS-ASD. Despite the strong correlation with the SCARED, a potential drawback of the SCARED is that nearly half of the 41 scale items rely on verbal expression from the child. By contrast, 7 of 25 items on the PRAS-ASD depend on verbal expression by the child. The low reliance on language in the PRAS-ASD is supported by the observation of no difference www.jaacap.org

893

SCAHILL et al.

in the mean values for participants with an IQ of 70 or above and those with an IQ of less than 70. In the metaanalysis of anxiety symptom severity, the investigators reported that higher IQ was associated with higher scores on parent-rated anxiety measures. These investigators noted that the measures used in these studies were developed for use the general pediatric population.7 As observed by Hallett et al.,8 it may be that reliance on language-dependent items resulted in lower parent-rated scores in youth with intellectual disability. There is solid consensus that impairing anxiety symptoms and anxiety disorders are common in youth with ASD.3,4,7,42 Whether the manifestations of anxiety are the same as those in youth in the general population is less clear.8,5,6,43,44 Assessment of anxiety in children with ASD may be hindered by language delays, cognitive delays, or both. Even children with ASD without language or cognitive delays may have limited ability to identify and to manage internal emotional states. Because of the limited ability to express worries or complaints about somatic symptoms anxiety, children and adolescents with ASD may simply act out their anxiety with emotional outbursts. In our focus groups, several parents asserted that at least some tantrums and overreactions to changes in the environment in their children with ASD were driven by anxiety.12 These parental inferences may partially explain the moderately positive correlations of the RBSR Sameness and ABCIrritability subscales with the PRAS-ASD in the clinically ascertained sample. Insistence on routines, overreaction to changes in the environment, and social avoidance may reflect a fundamental blending of anxiety and ASD.5 Alternatively, insistence on sameness, related vigilance, and need for reassurance may reflect uncertainty and excessive worry about events in the immediate future. Although the same informant in the clinically ascertained sample completed the RBSR Sameness and ABC-Irritability subscales and the PRAS-ASD, the separation of the PRASASD from these other parent ratings supports this alternative view. Anxiety as a mental state involves subjective feelings of worry and apprehension in response to perceived threat. Heightened anxiety is accompanied by physiological (increased heart rate), psychological (vigilance), and behavioral (avoidance, seeking safety) responses. These behavioral and physiological responses also accompany the mental state of fear in situations of imminent danger. Thus, anxiety disorders may reflect faulty threat detection in the presence of uncertainty.45,46 Interest in human responses to threat has prompted decades of research focused on cortical and limbic circuitry, with particular interest in the amygdala.47-49 Accumulated findings from animal models and human 894

www.jaacap.org

neuroimaging studies support a role for the amygdala in the pathophysiology of fear and anxiety, but many questions remain. A deeper understanding of interactions between cortical and subcortical contributions to anxiety disorders awaits further study.46 Future studies could also explore the biological markers of anxiety using new technologies to measure heart rate variability. The results of this study support a dimensional approach to the measurement of anxiety in youth with ASD. Because it is not disorder based, the total score reflects the severity of anxiety and ranges from little or no anxiety to severe anxiety. Systematic factor and item analysis identified a single factor with 25 items. The resulting 25-item scale (scored 0
3) demonstrated discriminant validity, excellent internal consistency, IRT reliability, and test
retest reliability. These findings should be considered in light of several limitations. The findings came from two samples. The first was an online sample of youth with a community diagnosis of ASD. Two-thirds of the mothers who completed the survey had a college or graduate degree, and 87%of the sample was of white ethnicity. The second sample, which was ascertained through one of three specialty services for children with ASD, also had a high percentage of mothers with a college or graduate degree. These samples of convenience may not be representative of all children with ASD. Readers who are interested in using the PRASD-ASD can contact the corresponding author. Accepted November 6, 2018. Drs. Scahill, Amoss, Mr. Gillespie, and Ms. Evans are with the Marcus Autism Center, Emory University School of Medicine, Atlanta, GA. Drs. Lecavalier, Aman, and Ms. Pritchett are with Ohio State University, Nisonger Center, Columbus, OH. Drs. Schultz, Maddox, Herrington, and Miller are with the Center for Autism Research, Children’s Hospital of Philadelphia and Perelman School of Medicine, University of Pennsylvania, Philadelphia. Dr. Bearss is with the University of Washington, Seattle. Dr. Gadow is with the State University of New York at Stony Brook, NY. Dr. Edwards is with Arizona State University, Tempe, AZ. This work was supported by a grant from the National Institute of Mental Health (NIMH; R01 MH099021) to Dr. Scahill and funding from the Marcus Foundation. Dr. Edwards, Mr. Gillespie, Dr. Lecavalier, and Dr. Scahill served as the statistical experts for this research. The authors thank consultants Matthew Goodwin, PhD, of Northeastern University, John Walkup, MD, of Northwestern University and the Lurie Children's Hospital, Stefan Hofmann, PhD, of Boston University, and Susan White, PhD, of University of Alabama at Tuscaloosa, for their expertise and input during this project. The authors also thank Jodi Salim, MPH, of the Marcus Autism Center, and Leila Bateman, BFA, of Children's Hospital of Philadelphia, for data management. Disclosure: Dr. Scahill has received grant or research support from NIMH, the US Department of Defense, and the Marcus Foundation. He has served on a Data Safety Monitoring Committee with Janssen. He is a co-author of the Children's YaleeBrown Obsessive Compulsive Scales (CYBOCS) and the Parent-Rated Anxiety Scale for youth with autism spectrum disorders (PRASASD). He has served as a consultant to Roche, Neurocrine Biosciences, Supernus Pharmaceuticals Inc., Shire, Yamo, and the Tourette Association of

Journal of the American Academy of Child & Adolescent Psychiatry Volume 58 / Number 9 / September 2019

PARENT-RATED ANXIETY SCALE FOR ASD

America. He has served as consulting editor of the Journal of Child and Adolescent Psychopharmacology and on the editorial board of the International Journal of Developmental Disabilities. He has received royalties from Oxford University Press, Guilford Press, and American Psychological Association. Dr. Lecavalier has received grant support from the Eunice Kennedy Shriver National Institute of Child Health and Human Development (NICHD). Dr. Schultz has received grant funding from the NIMH, the NICHD, the National Institute of Environmental Health Sciences (NIEHS), and the Simons Foundation. He has received consultant fees from Hoffman-La Roche, Aevi Genomic Medicine, and Yale University; has held stock options for consulting work from Akili Interactive Labs Inc; and he has filed a patent as a co-inventor on a method for measuring human behavior. None of these financial interests are directly related to the subject of this publication. Dr. Maddox has received grant support from the NIMH and the FAR Fund and royalties from Oxford University Press. Dr. Herrington has received grant support from the National Institutes of Health (NIH) and the National Science Foundation (NSF). He has received lecture fees and research funds from Shire and consultancy fees from Aevi Genomic Medicine. Dr. Miller has received grant support from Shire, Lumos Pharma, Inc., Janssen, and Aevi Genomic Medicine. She has received consultant fees from Shire, Lumos Pharma Inc., and Janssen and has provided

legal consultation or expert testimony for Division of Vaccine Injury Compensation of Saxton and Stump LLC and Akin Gump Strauss Hauer and Feld LLP. Dr. Aman has received research contracts, consulted with, served on advisory boards, or done investigator training for Hoffman-La Roche, MedAvanteProphase, Ovid Therapeutics, Yamo Pharmaceuticals, and Zynerba Pharmaceuticals. He has received royalties from Slosson Educational Publications. Dr. Gadow is shareholder in Checkmate Plus, publisher of the Child and Adolescent Symptom Inventory. He has received grant support from the NIMH and the U.S. Department of Defense. Dr. Edwards is part owner of flexMIRT. He has received grant support from the NIH and the NSF. Drs. Amoss and Bearss, Mss. Evans and Pritchett, and Mr. Gillespie report no biomedical financial interests or potential conflicts of interest. Correspondence to Lawrence Scahill, MSN, PhD, Marcus Center, Emory University School of Medicine, 1920 Briarcliff Road, Atlanta, GA 30329; e-mail: [email protected] 0890-8567/$36.00/ª2019 American Academy of Child and Adolescent Psychiatry https://doi.org/10.1016/j.jaac.2018.10.016

REFERENCES 1. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders. 5th ed. Washington, DC: American Psychiatric Press; 2013. 2. Simonoff E, Pickles A, Charman T, Chandler S, Loucas T, Baird G. Psychiatric disorders in children with autism spectrum disorders: prevalence, comorbidity, and associated factors in a population-derived sample. J Am Acad Child Adolesc Psychiatry. 2008;47: 921-929. 3. White SW, Oswald D, Ollendick T, Scahill L. Anxiety in children and adolescents with autism spectrum disorders. Clin Psychol Rev. 2009;29:216-229. 4. van Steensel FJ, B€ogels SM, Perrin S. Anxiety disorders in children and adolescents with autistic spectrum disorders: a meta-analysis. Clin Child Fam Psychol Rev. 2011;14:302. 5. Kerns CM, Kendall PC, Berry L, et al. Traditional and atypical presentations of anxiety in youth with autism spectrum disorder. J Autism Dev Disord. 2014;44:2851-2861. 6. Lecavalier L, Wood JJ, Halladay AK, et al. Measuring anxiety as a treatment endpoint in youth with autism spectrum disorder. J Autism Dev Disord. 2014;44:1128-1143. 7. van Steensel FJ, Heeman EJ. Anxiety levels in children with autism spectrum disorder: a meta-analysis. J Child Fam Stud. 2017;26:1753-1767. 8. Hallett V, Lecavalier L, Sukhodolsky DG, et al. Exploring the manifestations of anxiety in children with autism spectrum disorders. J Autism Dev Disord. 2013;43:2341-2352. 9. Gadow KD, Sprafkin JN. Child Symptom Inventory 4: Screening and Norms Manual. Stony Brook, New York: Checkmate Plus; 2002. 10. Food and Drug Administration. Guidance for industry: Patient-reported outcome measures: use in medical product development to support labeling claims. 2009. Available at: http://www.fda.gov/downloads/drugs/guidances/ucm193282.pdf. Accessed November 5, 2014. 11. Department of Health. Guidance on the routine collection of patient reported outcome measures (PROMs), For the NHS in England 2009/10. London: Department of Health; 2008. 12. Bearss K, Taylor CA, Aman MG, et al. The application of qualitative methods in instrument development for anxiety in children with autism spectrum disorder. Autism. 2016;20:663-672. 13. Daniels AM, Rosenberg RE, Anderson C, et al. Verification of parent-report of child autism spectrum disorder diagnosis to a Web-based autism registry. J Autism Dev Disord. 2012;42:257-265. 14. Corsello C, Hus V, Pickles A, et al. Between a ROC and a hard place: decision making and making decisions about using the SCQ. J Am Acad Child Psychiatry. 2007;48:932-940. 15. Kaat AJ, Lecavalier L, Aman MG. Validity of the Aberrant Behavior Checklist in children with autism spectrum disorders. J Autism Dev Disord. 2014;44:1103-1116. 16. Lord C, Risi S, Lambrecht L, et al. The Autism Diagnostic Observation Schedule
Generic: a standard measure of social and communication deficits associated with the spectrum of autism. J Autism Dev Disord. 2000;30:205-223. 17. Roid GH. Stanford-Binet Intelligence Scales. Fifth Edition. Rolling Meadows, IL: Riverside; 2003. 18. Sparrow SS, Ciccetti DV, Balla DA. Vineland II: Vineland Adaptive Behavior Scales. In: Survey Forms Manual. Second Edition. Circle Pines, MN: AGS Publishing; 2005. 19. Silverman WK, Saavedra LM, Pina AA. Test-retest reliability of anxiety symptoms and diagnoses with the Anxiety Disorders Interview Schedule for DSM-IV: Child and Parent Versions. J Am Acad Child Psychiatry. 2001;40:937-944. 20. Birmaher B, Brent DA, Chiappetta L, Bridge J, Monga S, Baugher M. Psychometric properties of the Screen for Child Anxiety Related Emotional Disorders scale (SCARED): a replication study. J Am Acad Child Psychiatry. 1999;38:1230-1236.

Journal of the American Academy of Child & Adolescent Psychiatry Volume 58 / Number 9 / September 2019

21. Bodfish JW, Symons FJ, Parker DE, Lewis MH. Varieties of repetitive behavior in autism: comparisons to mental retardation. J Autism Dev Disord. 2000;30:237-243. 22. Browne MW, Cudeck R, Tateneni K, Mels G. CEFA: comprehensive exploratory factor analysis, Version 2. 2004. Available at: http://faculty.psy.ohio-state.edu/browne/software. php. Accessed February 8, 2007. 23. Wirth RJ, Edwards MC. Item factor analysis: current approaches and future directions. Psychol Methods. 2007;12:58-79. 24. Horn J. A rationale and test for the number of factors in factor analysis. Psychometrika. 1965;30:179-185. 25. Muthen LK, Muthen BO. Mplus User’s Guide. Seventh Edition. Los Angeles, CA: Muthen & Muthen; 1998-2015. 26. Bentler PM. Comparative fit indexes in structural models. Psychol Bull. 2007;107: 238-246. 27. Steiger JH, Lind J. Statistically-based tests for the number of common factors. Paper presented at the Annual Spring Meeting of the Psychometric Society. Iowa City; 1980. 28. Tucker LR, Lewis CA. Reliability coefficient for maximum likelihood factor analysis. Psychometrika. 1973;38:1-10. 29. Browne MW, Cudeck R. Alternative ways of assessing model fit. In: Bollen KA, Lang JS, eds. Testing Structural Models. Newbury Park, CA: Sage Publications; 1993:136-162. 30. Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structural analysis: conventional criteria versus new alternatives. Struct Equat Model. 1999;6:1-55. 31. Cai L. flexMIRTR version 3.51: flexible multilevel multidimensional item analysis and test scoring [Computer software]. Chapel Hill, NC: Vector Psychometric Group; 2017. 32. Cappelleri JC, Lundy JJ, Hays RD. Overview of classical test theory and item response theory for quantitative assessment of items in developing patient-reported outcome measures. Clin Ther. 2014;36:648-662. 33. Mokkink LB, Terwee CB, Patrick DL, et al. The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study. Qual Life Res. 2010;19:539-549. 34. Cartwright-Hatton S, McNally D, Field AP, et al. A new parenting-based group intervention for young anxious children: results of a randomized controlled trial. J Am Acad Child Psychiatry. 2011;50:242-251. 35. Guy L, Souders M, Bradstreet L, DeLussey C, Herrington JD. Emotion regulation and respiratory sinus arrhythmia in autism spectrum disorder. J Autism Dev Disord. 2014;44: 2614-2620. 36. Samejima F. Estimation of Latent Ability Using a Response Pattern of Graded Scores. New York: Psychometric Society; 1969. 37. Edwards MC. An introduction to item response theory using the Need for Cognition Scale. Soc Personal Psychol Compass. 2009;3:507-529. 38. Glod M, Creswell C, Waite P, et al. Comparisons of the factor structure and measurement invariance of the spence children’s anxiety scale—parent version in children with autism spectrum disorder and typically developing anxious children. J Autism Dev Disord. 2017;47:3834-3846. 39. Jitlina K, Zumbo B, Mirenda P. Psychometric properties of the Spence Children’s Anxiety Scale: parent report in children with autism spectrum disorder. J Autism Dev Disord. 2017;47:3847-3856. 40. Kraemer HC. To increase power in randomized clinical trials without increasing sample size. Psychopharmacol Bull. 1991;27:217-224. 41. Strang JF, Anthony L, Pugliese CE. Improving the executive functioning of adolescents with asd through school-based intervention: the On Target Life Curriculum. Presented at

www.jaacap.org

895

SCAHILL et al.

the Annual Meeting of the International Society for Autism Research in Rotterdam, Netherlands, May 9–12, 2018. 42. Kerns CM, Kendall PC. The presentation and classification of anxiety in autism spectrum disorder. Clin Psychol (New York). 2012;19:323-346. 43. White SW, Lerner MD, McLeod BD, et al. Anxiety in youth with and without autism spectrum disorder: examination of factorial equivalence. Behav Ther. 2015; 46:40-53. 44. Capriola NN, Maddox BB, White SW. No offense intended: fear of negative evaluation in adolescents and adults with autism spectrum disorder. J Autism Dev Disord. 2017;47: 3803-3813.

896

www.jaacap.org

45. Woody S, Rachman S. Generalized anxiety disorder (GAD) as an unsuccessful search for safety. Clin Psychol Rev. 1994;14:743-753. 46. LeDoux JE, Pine DS. Using neuroscience to help understand fear and anxiety: a twosystem framework. Am J Psychiatry. 2016;173:1083-1093. 47. Phelps EA. Emotion and cognition: insights from studies of the human amygdala. Annu Rev Psychol. 2006;57:27-53. 48. Morrison FG, Ressler KJ. From the neurobiology of extinction to improved clinical treatments. Depress Anxiety. 2014;31:279-290. 49. Herrington JD, Maddox BB, Kerns CM, et al. Amygdala volume differences in autism spectrum disorder are related to anxiety. J Autism Dev Disord. 2017;47:3682-3691.

Journal of the American Academy of Child & Adolescent Psychiatry Volume 58 / Number 9 / September 2019

PARENT-RATED ANXIETY SCALE FOR ASD

TABLE S1 Fit Statistics for Factor Analysis in Test Sample, Validation Sample, and Full Sample Sample/Model Exploratory (n [ 445) 1-factor 1-factor (2 CRs) 1-factor (4 CRs) Confirmatory (n [ 445) 1-factor 1-factor (2 CRs) 1-factor (4 CRs) Full (N [ 990) 1-factor 1-factor (2 CRs) 1-factor (4 CRs)

c2

df

RMSEA

CFI

TLI

SRMR

1414.36 1063.43 938.24

275 273 271

0.09 0.08 0.07

0.88 0.92 0.93

0.87 0.91 0.93

0.08 0.07 0.07

1609.83 1224.20 1136.06

275 273 271

0.10 0.08 0.08

0.88 0.92 0.92

0.87 0.91 0.91

0.09 0.08 0.08

2971.39 2117.64 1882.05

275 273 271

0.10 0.08 0.08

0.88 0.91 0.92

0.86 0.90 0.91

0.08 0.07 0.07

Note: CFI ¼ comparative fit index; CR ¼ correlated residual; df ¼ degrees of freedom; RMSEA ¼ root mean square error of approximation; SRMR ¼ standardized root mean square residual; TLI ¼ Tucker
Lewis index.

TABLE S2 Factor Loadings From Final Model Item Item Item Item Item Item Item Item Item Item Item Item Item Item Item Item Item Item Item Item Item Item Item Item Item

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25

Factor Loadings 0.63 0.58 0.62 0.57 0.78 0.71 0.71 0.83 0.82 0.58 0.70 0.60 0.60 0.63 0.63 0.51 0.63 0.56 0.65 0.60 0.50 0.82 0.61 0.68 0.60

Note: Although not reported here, all standard errors were between 0.02 and 0.05. Correlated residuals were between Items 4 and 10 (0.58), 13 and 20 (0.061), 10 and 25 (0.40), and 17 and 24 (0.38).

Journal of the American Academy of Child & Adolescent Psychiatry Volume 58 / Number 9 / September 2019

www.jaacap.org

896.e1

SCAHILL et al.

TABLE S3 Item Response Theory (IRT) Parameter Estimates and Standard Errors (SE) Item Item Item Item Item Item Item Item Item Item Item Item Item Item Item Item Item Item Item Item Item Item Item Item Item

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25

a 1.40 1.27 1.31 1.21 2.17 1.73 1.70 2.65 2.63 1.31 1.68 1.26 1.30 1.37 1.42 0.94 1.45 1.17 1.42 1.32 0.99 2.58 1.28 1.66 1.32

a(SE) 0.10 0.10 0.10 0.09 0.13 0.11 0.11 0.15 0.15 0.10 0.11 0.12 0.11 0.11 0.09 0.08 0.10 0.11 0.10 0.11 0.08 0.15 0.1 0.11 0.11

b1 e0.51 e2.68 e0.18 e2.13 e0.56 e0.94 e0.73 e1.03 e0.89 0.05 e0.82 1.24 0.49 e0.27 e0.93 e1.83 e1.09 1.27 e1.03 0.12 e1.46 e0.70 0.42 e1.32 0.37

b1(SE) 0.07 0.2 0.07 0.17 0.06 0.07 0.07 0.06 0.06 0.07 0.07 0.11 0.08 0.07 0.09 0.17 0.09 0.12 0.09 0.08 0.13 0.06 0.07 0.09 0.07

b2 0.58 e0.7 0.88 e0.45 0.28 0.24 0.71 e0.02 0.25 1.39 0.51 2.37 1.45 0.81 0.18 e0.2 0.12 2.27 e0.05 0.93 0 0.14 1.55 e0.15 1.51

b2(SE) 0.07 0.09 0.09 0.09 0.05 0.06 0.07 0.05 0.05 0.11 0.06 0.18 0.12 0.08 0.07 0.09 0.07 0.19 0.07 0.10 0.08 0.05 0.12 0.06 0.13

b3 1.99 1.01 1.96 1.53 1.21 1.41 1.83 1 1.33 2.64 1.74 4.03 2.61 1.98 1.35 1.18 1.37 3.16 0.96 1.84 1.36 0.98 2.60 0.92 2.38

b3(SE) 0.13 0.10 0.14 0.13 0.08 0.10 0.12 0.06 0.08 0.20 0.11 0.34 0.21 0.14 0.10 0.12 0.10 0.28 0.09 0.15 0.13 0.06 0.20 0.08 0.18

Note: a ¼ IRT slope parameter, b ¼ IRT severity/threshold parameter.

FIGURE S1 Distribution of Total Scores on Parent-Rated Anxiety Scale for Autism Spectrum Disorder (PRAS-ASD) for Children (Age 5 to 17 Years) Who Participated in Clinical Evaluation (N ¼ 116)

Note: Scores on the PRAS-ASD ranged from 1 to 65; mean was 31.5  15.6.

896.e2

www.jaacap.org

Journal of the American Academy of Child & Adolescent Psychiatry Volume 58 / Number 9 / September 2019