Parent and child outcomes of JumpStart™, an education and training program for parents of children with autism spectrum disorder

Research in Autism Spectrum Disorders 56 (2018) 21–35

Contents lists available at ScienceDirect

Research in Autism Spectrum Disorders journal homepage: www.elsevier.com/locate/rasd

Parent and child outcomes of JumpStart™, an education and training program for parents of children with autism spectrum disorder

T

⁎

Nicole L. Matthews , Beatriz C. Orr, Brittani Harris, Rachel McIntosh, Daniel Openden, Christopher J. Smith Southwest Autism Research and Resource Center, 300 N 18th Street, Phoenix, AZ, 85006, United States

A R T IC LE I N F O

ABS TRA CT

Number of reviews completed is 2

Background: This pilot study examined the effectiveness of JumpStart™, a 4-week education program including a Behavioral Skills Training model for parents of children with autism spectrum disorder (ASD). JumpStart aims to teach parents to begin implementing evidence-based behavioral intervention while learning to navigate service systems in order to establish a comprehensive intervention program. Method: Change in parent self-efficacy, knowledge, stress, depressive symptoms, ability to implement intervention, and child responsivity were compared in treatment (n = 18) and waitlist control (n = 18) groups. Additionally, exploratory analyses examined 3-month follow-up data in a subset of participants (n = 12). Results: Findings indicated significantly larger increases in parent outcome expectations, parent competence, parent knowledge, fidelity of implementation, and child responsivity in the treatment group compared to the waitlist control group. Three-month follow-up data indicated maintenance of parent outcome expectations, knowledge, and child responsivity. Conclusions: Findings provide preliminary evidence of the effectiveness of JumpStart. Given the growing number of children with ASD and finite resources, short-term programs like JumpStart may be an effective method by which parents are able to begin intervening with their child’s development while they arrange for comprehensive services.

Keywords: Autism spectrum disorder Early intervention Parent training Parent education Pivotal Response Treatment Behavioral Skills Training

Arguably as important as the early identification of autism spectrum disorder (ASD) (Boyd, Odom, Humphreys, & Sam, 2010) is the ability of caregivers to access and provide effective services for their child. Although variable depending on geographic location, parents report a paucity of qualified specialists and long waitlists for services (Montes, Halterman, & Magyar, 2009; Pickard & Ingersoll, 2016). Thus, families may experience difficulty establishing evidence-based treatment programs in a timely manner. This is troubling from a public health perspective given that early intervention is related to optimal outcomes (Orinstein et al., 2014; Vismara & Rogers, 2010). The current pilot study examined the effectiveness of JumpStart™, a 4-week program that teaches parents of children with ASD to begin implementing evidence-based behavioral intervention while learning to navigate the service system.

⁎

Corresponding author. E-mail addresses: [email protected] (N.L. Matthews), [email protected] (B.C. Orr), [email protected] (B. Harris), [email protected] (R. McIntosh), [email protected] (D. Openden), [email protected] (C.J. Smith). https://doi.org/10.1016/j.rasd.2018.08.009 Received 23 August 2017; Received in revised form 23 August 2018; Accepted 26 August 2018 1750-9467/ © 2018 Elsevier Ltd. All rights reserved.

Research in Autism Spectrum Disorders 56 (2018) 21–35

N.L. Matthews et al.

1. Access to services for families of children with ASD Beginning with the diagnostic process and persisting throughout childhood as families seek treatment, children with ASD experience long waitlists for services (Austin et al., 2016; Brookman-Frazee, Baker-Ericzen, Stadnick, & Taylor, 2012; Pickard & Ingersoll, 2016). Further, qualitative and mixed-methods research studies have found that parents of children with ASD experience confusion and frustration with complicated service systems, report needing additional knowledge about available services, perceive a lack of service providers who are trained to work with individuals with ASD, and experience challenges financing appropriate services (Brookman-Frazee et al., 2012; Pickard & Ingersoll, 2016). Parent training in the implementation of evidence-based intervention, increased parent knowledge about available services, and providing parents assistance in navigating the service system have been identified as strategies that could address barriers to services experienced by families of children with ASD (Pickard & Ingersoll, 2016). JumpStart is an existing parent education and training program that has been deployed by a non-profit autism center for more than a decade to address these issues. 2. History of the JumpStart program The original JumpStart program was developed in 2002 by a parent of a child with ASD to address the lack of evidence-based treatment services available in Phoenix, Arizona. Prior to starting JumpStart, the parent brought behavioral interventionists from the University of California, Los Angeles to Phoenix so that she could learn to implement an in-home behavior therapy program (Lovaas, 1987). She partnered with a non-profit autism center to share her acquired knowledge with parents throughout the Phoenix metropolitan area. The original program was led by this parent, other parents of children with ASD, and special education practitioners. It included guided observation of Discrete Trial Training (see Smith, 2001) sessions and informal discussions of issues faced by parents concurrently enrolled in the program. In 2008, the program was restructured with supervision from a doctorallevel Board Certified Behavior Analyst (BCBA). The restructured model includes didactic lessons about specific topics (see Table 1) that were selected based on needs assessment surveys completed by parents who completed the program. Additionally, the model includes brief parent training in Applied Behavior Analysis (ABA), with a focus on Pivotal Response Treatment (PRT; see Mohammadzaheri, Koegel, Rezaee, & Rafiee, 2014), using Behavioral Skills Training (Parsons, Rollyson, & Reid, 2012). Specifically, parent training includes didactic lessons that describe parent target skills, guided observation of child-interventionist sessions in which target skills are modeled, and in-vivo coaching that allows the parent to practice target skills with feedback from a trained interventionist. JumpStart is implemented by Registered Behavior Technicians who are supervised by a Certified and Licensed Behavior Analyst. The non-profit center that administers JumpStart does not have the capacity to provide comprehensive intervention, or even comprehensive parent training, to the large volume of families seeking intervention services. The center defines comprehensive intervention as ongoing intervention designed and supervised by a BCBA that is tailored to target the child’s specific areas of impairment. The restructured JumpStart model provides parents with the knowledge necessary to better understand autism and implement intervention consistently while they obtain appropriate comprehensive intervention. As JumpStart is short-term, the waitlist is relatively short and 60–80 families are served annually. Since 2008, more than 700 families have completed the current model of JumpStart. 3. Parent training in PRT Mounting evidence indicates that early intensive interventions that include principles of ABA are an effective method of treatment for many individuals with ASD (Warren et al., 2011). PRT is a naturalistic extension of ABA with an emphasis on parent training (Mohammadzaheri et al., 2014). PRT includes motivational procedures that target four pivotal areas considered to be critical for children with ASD, including motivation, responsivity to multiple cues, self-initiations, and self-management. Parents are trained in the motivational procedures of PRT, which include shared control, opportunities to respond, interspersing tasks, contingent and natural reinforcement, and attempts. The motivational procedures are supported by extensive research when used together, and the effectiveness of each individual motivational procedure has also been documented (Koegel & Koegel, 2012). Shared control involves following the child’s lead (e.g., following the child’s activity initiations; allowing the child to choose or reject toys) and identifying the natural reinforcer (e.g., identifying and gaining control of preferred objects or activities that the child finds motivating). Opportunities to respond involves only providing language opportunities when the child is attending to the task, and providing clear instructions for language opportunities (i.e., instructions are short, simple, and related to the task). Interspersing tasks requires that the majority of language opportunities be related to the maintenance of acquired skills, and the minority of tasks be related to the acquisition of new skills. Additionally, it involves using a variety of prompting strategies, alternating words that are prompted, and/or offering a new activity. Contingent and natural reinforcement requires that the parent provide a contingent reinforcer that is related to the child’s verbalization (e.g., giving a ball if the child says “ball”), that the reinforcer be provided immediately after the child’s response, and that the parent does not reinforce inappropriate responses (e.g., disruptive behaviors; vocal stereotypy). Last, attempts requires that the parent provide contingent reinforcement following reasonable verbal attempts, even if the attempt was not the desired response (Koegel & Koegel, 2012). Some parents are able to implement PRT with fidelity after only 6 h of training; however, it is more commonly recommended that parents receive 25 h of training to meet fidelity of implementation (Coolican, Smith, & Bryson, 2010). Because JumpStart is a shortterm, low-intensity program, parents receive only 10 h of training, some of which is conducted in a small group format (i.e., 3 h of 22

Research in Autism Spectrum Disorders 56 (2018) 21–35

N.L. Matthews et al.

Table 1 JumpStart Content and Format. Session

Didactic Topic (75 min)

Guided Observation (75 min)

In-vivo coaching (75 min)

Orientation 1 (Week 1)

Collection of video probes; completion of questionnaires; overview of the program, and intake data collected from children to inform individual goals Autism Spectrum Disorders X What is autism? Core areas of development affected by autism with examples Autism as a spectrum disorder

2 (Week 1)

Applied Behavior Analysis (ABA) Definition; seven dimensions of ABA ABA-based interventions Basic principles

3 (Week 2)

Pivotal Response Treatment (PRT) Pivotal areas ABCs of PRT Pivotal response techniques

4 (Week 2)

Disruptive Behaviors What is behavior? Concerning behaviors; common reasons for disruptive behaviors Functional Behavior Assessment Intervention options

5 (Week 3)

Accessing Services – State Funding State agencies serving the autism population, types of services, eligibility, and steps to qualify Finding providers

6 (Week 3)

Accessing Services – Insurance Funding Legislation, coverage, and exemptions; self-insured plans; Affordable Care Act

7 (Week 4)

Individualized Education Plan (150 min) Process for obtaining special education services under IDEA Preparing for the IEP meeting; IEP components; least restrictive environment placements; strategies for ensuring the IEP is thorough and cohesive; communication with IEP team

8 (Week 4)

Toilet Training (Video probes collected at start of session) When to start training The Potty Party Scheduled toileting, pants checks, and positive practices for accidents

• • •

X

• • •

X

• • •

X

• • • • • • •

X

X

• •

X

• • •

Follow-up

Completion of questionnaires and discussion about ‘next steps’.

guided observation).1 The goal is not for parents to reach conventional levels of fidelity, but to begin practicing implementation of PRT while they establish a comprehensive intervention program that includes parent training. 4. Parent well-being and self-efficacy When compared to parents of typically developing children and children with other developmental disorders, parents of children with ASD report elevated levels of mental health issues, and reduced self-efficacy and quality of life (see Karst & Van Hecke, 2012 for a review). Because there is likely a bidirectional relationship between child functioning and parent characteristics, evaluations of ASD treatments should consider change in both child and parent characteristics (Karst & Van Hecke, 2012). Although JumpStart does not specifically target parent mental health and self-efficacy, it is possible that better understanding of autism, evidence-based treatments, and service systems may positively impact these parental characteristics. 5. Parent education and training programs Previous literature includes a handful of early parent education and training programs with varying lengths and foci. Whereas some programs target parents’ ability to interact with their child and/or provide autism-specific therapy through in-vivo training (Carter et al., 2011; Coolican et al., 2010; Dawson et al., 2010; Estes et al., 2013; Hardan et al., 2015; Ingersoll & Wainer, 2013; Solomon, Van 1 Additionally, the post-JumpStart video probe used to measure parent fidelity of implementation and child responsivity is collected after only 8.75 h of parent training due to scheduling restraints associated with the program.

23

Research in Autism Spectrum Disorders 56 (2018) 21–35

N.L. Matthews et al.

Egeren, Mahoney, Quon Huber, & Zimmerman, 2014; Wetherby et al., 2014), others combine parent training and didactic lessons about ASD, parenting, disruptive behaviors, and other related topics (Shields, 2001; Tonge, Brereton, Kiomall, Mackinnon, & Rinehart, 2014; Wang, 2008). Parents who complete these programs report improved mental well-being, efficacy, and short-term child outcomes (e.g., Estes et al., 2013; McConachie, Randle, Hammal, & Le Couteur, 2005; Whittingham, Sofronoff, Sheffield, & Sanders, 2009). Children of parents who complete in-vivo training demonstrate improved language, behavioral outcomes, and/or IQ, depending on the program (e.g., Coolican et al., 2010; Dawson et al., 2010; Hardan et al., 2015; Solomon et al., 2014). Some evidence exists for the effectiveness of short-term parent education and training models. EarlyBird and EarlyBird Plus (an extension for older children) include eight 3-h group sessions (weekly) with parents and two home visits, during which a professional works with the parent-child dyad on strategies learned during group sessions (Shields, 2001). The programs focus on improving parents’ understanding of autism, how to build positive social interaction and communication between the parent and child, and how to understand triggers and functions of children’s aberrant behaviors. Positive parent outcomes have been documented, including reductions of parent-reported stress, improved parent-child communication, improved parent perceptions of their child, and improved behavior management skills (Cutress & Muncer, 2014). Another parent training program implemented in Shenyang, China (Wang, 2008) involves group training (four 4-h sessions) and four 1–2 h home visits over a period of four weeks. The group training sessions provide an overview of ASD, principles of ABA, naturalistic teaching and functional assessment of aberrant behaviors and teaching replacement behaviors. Findings from a small randomized controlled trial indicated that parents in the treatment group demonstrated higher parent sensitivity, more appropriate responses, more effective engagement, more enjoyment, and more warmth during interactions with their children compared to the control group. Studies of these programs did not document changes in child communication through standardized assessments or coded video probes. Like JumpStart, both EarlyBird and the parent-training program implemented in China are short-term, involve a didactic component, and include direct coaching or training to assist parents in implementing intervention. In contrast, these programs conduct the direct coaching at home, whereas in JumpStart, children accompany their parent(s) to the autism center. While parents complete the didactic lessons, children are in a classroom with behavioral interventionists. Coaching sessions occur at the autism center, begin the first week of JumpStart, and occur on the same schedule for every parent-child dyad in the program. Providing parent coaching at the autism center may serve as a disadvantage as it is not the child’s natural environment. However, this format allows for the autism center to see more families. The provision of in-home services would increase time demands on staff and fewer families would be served. A collateral benefit of this model is that children receive structured intervention in a classroom setting, thus exposing them to a pre-school environment. Didactic instruction regarding the processes for obtaining state-funded services, insurance-funded services, and special education services are a unique attribute of JumpStart relative to other short-term programs in the literature. JumpStart aims to equip parents with the knowledge necessary to efficiently navigate the service system and to establish and maintain appropriate, evidence-based treatment for their child. Reported in Table 1, three separate didactic lessons provide step-by-step instructions for accessing state funding, insurance funding, and special education services. The lessons teach parents how to identify appropriate service providers once funding is obtained, how to communicate with service providers and educators, and strategies for appealing service denials. Last, parents are provided with resources to pursue if they face obstacles when obtaining services. Theoretically, teaching parents to advocate for services should positively impact the family during and after the completion of the short-term program. Many of the parent and education programs that have been documented in the literature have yet to be implemented and tested in community-based settings. Whereas improvements demonstrated in empirical studies and randomized controlled trials are promising, they do not necessarily generalize to realistic service settings that introduce variables such as availability of trained interventionists, payers, and an ever-growing population in need of services. The current study examined families seeking services in a non-research setting. Families were recruited after enrolling in an existing fee-for-service program. Thus, findings should more realistically reflect outcomes that can be expected in a typical service setting. 6. The current study JumpStart represents a parent training and education program developed and implemented in a community-based setting that has withstood the test of time. Despite being established for more than fifteen years, the current study is the first to systematically study parent and child outcomes of families who complete JumpStart. In families who completed JumpStart and a waitlist control (WLC) group, this pilot study compared change in parent self-efficacy, knowledge, stress, depressive symptoms, and fidelity of implementation, and change in child responsivity. It was predicted that families in the treatment group would demonstrate larger increases in parent self-efficacy, knowledge, fidelity of implementation, and child responsivity relative to the WLC group, as well as larger decreases in parent stress and depressive symptoms. Three-month follow-up data from a subset of families that returned to the autism center were examined to explore whether change was maintained. Last, exploratory analyses examined potential associations among baseline child characteristics and treatment gains. 7. Method 7.1. Participants Participants were 36 children (35 males) with an independent professional DSM-IV or DSM-5 ASD diagnosis (n = 30) or at-risk for ASD classification (n = 6). At-risk classifications are provided by some diagnostic practitioners in Arizona when early signs of ASD 24

Research in Autism Spectrum Disorders 56 (2018) 21–35

N.L. Matthews et al.

Table 2 Participant Demographics. Variable

Treatment

WLC

Excluded

(n = 18)

(n = 18)

(n = 5)

M(SD); range

M(SD); range

M(SD); range

40.83 (11.30); 21–59 100.00

40.72 (11.61); 18–61 94.44

45.60 (16.21); 32–72 60.00 40.00 20.00 0.00 40.00

Child Age (months) Gender (percent male) Race (percent) Caucasian Hispanic Asian African American Annual Household Income (percent) < $25,000 $25,000–$49,999 $50,000–74,999 $75,000–$99,999 ≥$100,000 Vineland ABC Vineland Communication Vineland Daily Living Skills Vineland Socialization SRS Total T-score

77.77 5.55 11.11 5.55

66.67 22.22 11.11 0.00

11.10 5.60 11.10 22.20 50.00 67.83 66.67 72.11 68.33 75.00

0.00 11.10 16.70 22.20 50.00 65.67 64.83 66.44 65.78 76.35

Age (years) Percent mothers

34.61 (5.61); 26–47 83.33

(11.86); 51–91 (17.80); 42–102 (13.24); 53–98 (10.69); 47–89 (9.10); 58–90

(8.60); 48–84 (13.82); 40–89 (10.40); 46–82 (8.34); 55–90 (9.98); 58–90

64.00 65.80 66.80 66.40 70.40

(5.15); (8.17); (5.45); (8.53); (7.89);

62–73 57–76 62–75 57–79 62–83

Parent 36.77 (7.05); 24–50 88.90

34.52 (2.48); 31–36 80.00

Note: WLC = Waitlist control group. ABC = Adaptive Behavior Composite. SRS = Social Responsiveness Scale, Second Edition. There were no significant group differences in child age, Vineland scores, SRS scores, or parent age.

are present, but criteria for an official DSM diagnosis are not yet satisfied. Like an ASD diagnosis, this classification qualifies the child to receive state funding for early intervention services if additional eligibility criteria are met. JumpStart enrolls families of children 6 years of age and younger; thus, the age range was relatively wide (M age = 40.78 months, SD = 11.29, range: 18–61 months). Descriptive statistics for child and parent demographics and characteristics are reported in Table 2. Vineland Adaptive Behavior Scales scores (Vineland-II; Sparrow, Cicchetti, & Balla, 2005) and Social Responsiveness Scale scores (SRS-2; Constantino & Gruber, 2012) indicated a wide range of adaptive functioning and autism severity within the sample. Highest level of maternal education and annual household income were utilized to measure socioeconomic status. The majority of mothers had a college (25.0%), graduate (25.0%), or trade school (5.6%) degree; 16.7% completed some college and 16.7% reported their highest degree as a high school diploma. Four families did not report on maternal education. Reported by study group in Table 2, the majority of families reported incomes consistent with middle to upper-middle class. However, approximately 16% of the treatment group and 11% of the WLC group reported household incomes lower than the median household income for the state of Arizona. One parent of each child was identified as the primary participating parent (i.e., attended sessions consistently, participated in the video probes, and completed child functioning measures; n = 2 fathers). For three families, both parents attended sessions consistently. In these cases, the mother served as the primary participating parent in all roles except video probes. Recruitment and enrollment procedures are depicted in Fig. 1. Participants were recruited from the waitlist over a 15-month period, which corresponds to 17 cohorts, or 102 eligible families. Of the eligible families, 41 were consented and enrolled in the study, and 36 were included in the current analyses. Five families were excluded because of attrition prior to the second time point (n = 3; two treatment group and one WLC group) or the lack of a matched participant (n = 2; WLC group). The remaining 61 eligible families either declined to participate in the study, enrolled in JumpStart too late to be included in the study2, or enrolled in JumpStart toward the end of the study recruitment period and did not meet the matching criteria described below. Because this study was designed to evaluate an existing, fee-for-service program, randomization to study groups was not feasible. Instead, participants were assigned to the WLC group (n = 18) if there were at least 6 weeks remaining until the start of their session upon enrollment, or to the Treatment group (n = 18) if less than 6 weeks remained until the start of their session. The waitlist is generally 1- to 2-months (although this fluctuates in both directions), but many families choose to wait longer for a variety of reasons (e.g., ability to take time off work). During the recruitment period, all families on the waitlist were contacted by a researcher and informed of their eligibility to participate in the study. Recruitment became more specific toward the end of the recruitment period in order to match participants between groups. Participants were matched on age and diagnostic classification; as expected, there were no significant between

2 When families withdrawal from the roster prior to the start of the program, JumpStart staff make every effort to fill empty spots with families from the waitlist. There were instances during the recruitment period for this study in which families were moved to an earlier cohort with as little as one day’s notice; thus, it was not feasible to contact them about the study prior to the start of their JumpStart session.

25

Research in Autism Spectrum Disorders 56 (2018) 21–35

N.L. Matthews et al.

Fig. 1. Recruitment and Study Group Assignment.

group differences in age or diagnostic classification, nor were there differences in initial Vineland-II (Sparrow et al., 2005) adaptive functioning composite scores or Social Responsiveness Scale (Constantino & Gruber, 2012) Total T-scores (all ps > .05; see Table 2). Reported in Table 3, groups did not differ significantly on time 1 scores for primary outcome variables. Study procedures were approved by the Western Institutional Review Board and informed consent was obtained from at least one parent of each child. 7.2. Procedure 7.2.1. JumpStart JumpStart is an education program that follows a Behavioral Skills Training model for parents of children six years of age and under with a recent ASD diagnosis or “at-risk” classification. Each JumpStart cohort includes six children and at least one parent/ caregiver of each child. Whereas multiple caregivers are welcome to attend, the program requires that one parent/caregiver commit to attending every session to ensure consistency in parent training. JumpStart cohorts are enrolled on a first-come, first-serve basis, and no efforts are made to form groups based on age, ability-level or any other variable. Any child with an ASD diagnosis or “at-risk” classification is eligible for JumpStart; thus, the program serves children that represent all levels of functioning. One week prior to the start of JumpStart, families attend a 1.5 h orientation, during which the program format is explained to parents and clinicians collect intake/baseline data on children using direct observation. These data are used to develop individualized goals. Although JumpStart is a fee-for-service program, the cost is subsidized through various grants and scholarships, which makes the program accessible to families from a wide range of socioeconomic backgrounds. The 4-week program meets twice weekly for 2.5 h and consists of three main components: (1) didactic lessons (11.25 h total); (2) guided observation of interventionist-child sessions (3.75 h total), and (3) in-vivo parent coaching (5 h total). At each session, parents participate in a group-format, 75-min didactic lesson on various topics (see Table 1) taught by one of the JumpStart clinicians. The clinician gives a standardized presentation using PowerPoint slides. Lessons that focus on teaching parents specific ABA skills (e.g., natural reinforcement; following the child’s lead; shared control; functional behavior assessment) include brief video examples of the implementation of each skill. During the didactic lesson, children participate in a classroom setting with a 3:1 child-to-interventionist ratio. Within the classroom, interventionists target group goals identified as necessary to participate in a preschool classroom (e.g., transitions, circle time, appropriate social initiations and responses, participation in art activities) and individualized goals 26

Research in Autism Spectrum Disorders 56 (2018) 21–35

N.L. Matthews et al.

Table 3 Descriptive Statistics for Primary Variables and Time 1 to Time 2 Change Scores. Outcome Variables

Parent Outcome Expectations (EIPSES) Parent Competence (EIPSES) Parent Knowledge (JumpStart Test) Parenting Stress (PSI) Parent Depressive Symptoms (CESD) Parent Fidelity of Implementation Child Responsivity

Treatment (n = 18)

WLC (n = 18)

T1 M(SD); Range

T2 M(SD); Range

T1-T2 Change Score M(SD); Range

T1 M(SD); Range

T2 M(SD); Range

T1-T2 Change Score M(SD); Range

Cohen’s d

5.14 (0.86); 3.10–6.30

5.88 (0.52); 5.20–6.80

0.74 (0.53); 0.10–2.40

5.24 (0.66); 3.90–6.30

5.51 (0.77); 4.20–7.00

0.28 (0.74); −0.90–1.90

0.72**

5.37 (1.02); 3.50–7.00 9.87 (2.68); 4.50–14.00 86.94 (20.40); 54.00–136.00 10.06 (9.09); 0.00–31.00 28.39 (10.12); 0.00–44.00 16.89 (17.42); 0.00–48.00

6.34 (0.51); 5.25–7.00 15.79 (1.75); 11.00–18.50 81.22 (15.96); 57.00–117.00 5.72 (4.11); 0.00–14.00 46.06 (13.69); 23.00–64.00 36.61 (23.79); 0.00–74.00

0.97 (0.77); −0.25–2.75 5.75 (2.79); 2.00–12.50 5.72 (14.72); −37.00–32.00 4.33 (6.99); −4.00–20.00 17.67 (10.56); −8.00–32.00 19.72 (19.29); −9.00–54.00

5.25 (0.80); 3.25–6.25 8.50 (4.75); 0.00–17.50 91.94 (19.71); 49.00–132.00 11.53 (6.61); 4.00–27.00 31.78 (11.12); 16.00–62.00 17.23 (24.89); 0.00–83.00

5.57 (0.54); 4.50–6.5 10.43 (4.42); 0.50–18.00 95.06 (18.19) 54.00–118.00 10.83 (4.61); 4.00–19.00 30.33 (11.85); 4.00–64.00 13.95 (21.04); 0.00–59.00

0.32 (0.87); −0.50–3.25 0.42 (2.80); −7.50–6.50 −3.12 (14.62); −23.00–28.00 0.29 (7.29); −10.00–16.00 −1.44 (13.08); −22.00–32.00 −3.28 (11.97); −31.60–22.00

1.05** 1.91*** 0.60 0.57 1.61*** 1.43***

Note: WLC = Waitlist control group. T1 = Time 1. T2 = Time 2. EIPSES = Early Intervention Parenting Self-Efficacy Scale. PSI = Parenting Stress Index. CESD = Center for Epidemiological Studies Depression Scale. Descriptive statistics for change scores reflect group mean replacement for missing values. A Bonferroni corrected alpha (.007) was used to account for multiple comparisons. **p < .007. ***p < .001.

determined at the orientation through an informal assessment of social communication, pre-academic skills, play skills, transitions, and challenging behaviors conducted by the interventionists. The first session of each week includes 75 min of guided observation, during which a BCBA-level clinician explains live 1:1 intervention sessions delivered via a closed circuit television. Bachelors- or Masters-level interventionists implement brief intervention sessions (12–15 min) with each child while the BCBA-level clinician explains the principles PRT utilized to the observing parents. During guided observation, interventionists are using the motivational procedures of PRT (Koegel & Koegel, 2012) described in the Introduction to target individualized goals. Although goals vary by child, all children have a social communication goal. Examples of other goals include play skills, transitions, following instructions, turn-taking, pre-academic skills, and perspective taking. The second session of each week includes 75 min of in-vivo parent coaching in PRT. The guided observation sessions and invivo coaching sessions always focus on the implementation of ABA to address child-specific goals. Thus, there is not direct alignment between the topics addressed in the didactic lessons and the parent coaching sessions (e.g., didactic lessons on accessing services do not directly correspond to the focus of parent coaching sessions). Parents are encouraged to practice the intervention skills they learn consistently in their natural environments. Generally, parents are not given specific homework assignments. As needed, some parents are given the option of collecting data to aid in determining the function of challenging behaviors. 7.2.2. Data collection Approximately 1-week prior to their study visit, orientation, or follow-up meeting (depending on study group and time point), parents were mailed a packet that included all questionnaires described below (with the exception of the JumpStart test) with instructions for completion. Parents completed questionnaires prior to their orientation or follow-up meeting to avoid using time devoted to JumpStart content for data collection. The same procedure was followed for families in the WLC group at their first study visit. The JumpStart test was always completed at the autism center in the presence of a member of the research team (i.e., at a study visit, orientation, or follow-up) to prevent participants from referencing other sources. 7.2.3. Time 1 At orientation, parents in the treatment group submitted/completed the questionnaires described below. For each dyad, one parent was videotaped interacting with his/her child during a 10-min probe, which was later coded as described below. Parents also completed the Vineland-II (Sparrow et al., 2005) and the SRS-2 (Constantino & Gruber, 2012) within one week of their orientation. Participants in the WLC group completed the same measures, probe, Vineland-II, and SRS-2 during a separate study visit approximately 6 weeks prior to their orientation. All parents reported on self and child demographics, including age, gender, race/ethnicity, socioeconomic status, and diagnostic information. 7.2.4. Time 2 One week after the last session (treatment group) or at their orientation (WLC group), families completed the same measures administered at Time 1. WLC families completed all measures prior to the delivery of any JumpStart content in order to prevent contamination. Families in the WLC group also completed these assessments one week after their last session. 27

Research in Autism Spectrum Disorders 56 (2018) 21–35

N.L. Matthews et al.

7.2.5. Three-month follow-up Twelve weeks after their last session, parents were invited to attend a follow-up session as a part of the program. The primary purpose was for staff to check in with parents regarding progress toward child goals and obtaining services. Secondarily, parents were asked to complete the same measures administered at Time 1 and Time 2. Families were also asked to capture a 10-min home video of the primary participating parent interacting with the child, which was later coded for parent fidelity of implementation of PRT and child responsivity. Families received a gift card for $30 and a small toy in thanks for their participation. 7.3. Measures 7.3.1. Vineland-II Survey Interview Form (Sparrow et al., 2005) The Vineland-II Survey Interview Form is a parent/caregiver interview about the child participant’s current adaptive functioning. It is commonly used in the field of autism research and was normed on a nationally representative sample. The interview takes 20–60 min to complete, and yields standard scores for an adaptive behavior composite (ABC) and four domains of functioning: communication, daily living skills, socialization, and motor skills. Standard scores have a mean of 100 and standard deviation of 15. Reported in the Vineland-II manual, the survey interview form has acceptable psychometrics (Sparrow et al., 2005). 7.3.2. SRS-2 pre-school or school-age form (Constantino & Gruber, 2012) The SRS-2 is a 65-item parent/caregiver report questionnaire regarding autism-related behaviors demonstrated by the child participant. Respondents read each item and respond using a 4-point scale ranging from “1-Not true” to “4-Almost always true”. The questionnaire yields an SRS-2 Total T-score with a mean of 50 and standard deviation of 10. Total T-scores of 59 or lower are considered within normal limits, whereas scores of 60 or higher reflect clinically significant impairments consistent with ASD. Psychometrics of the SRS-2 are well-established and are summarized in the SRS-2 manual (Constantino & Gruber, 2012). 7.3.3. The Early Intervention Parenting Self-Efficacy Scale (EIPSES; Guimond, Wilcox, & Lamorey, 2008) The EIPSES is a 16-item questionnaire regarding the self-efficacy of parents involved in early intervention for their child. Respondents indicate agreement with each statement on a 7-point Likert scale ranging from ‘strongly disagree’ to ‘strongly agree.’ Item scores were averaged to create scores on two factors that were developed by the authors of the EIPSES: Parent outcome expectations and parent competence. Parent outcome expectations represents the extent to which the respondent believes that his/ her child’s outcomes are due to environmental influences, such as intervention and community support. Importantly, this scale refers to environmental influences in general, not JumpStart specifically. Higher scores indicate higher perceived influence of the environment on positive child outcomes. Parent competence represents the respondent’s belief about his/her personal ability to produce positive changes in his/her child’s development. Higher scores indicate higher perceived parent competence. The authors of the EIPSES reported good internal consistency in the validation sample (Cronbach’s α = 0.80); additional psychometric information is reported by Guimond et al. (2008). In the current sample, internal consistency was acceptable (Cronbach’s α = 0.77). 7.3.4. JumpStart test A 20-item multiple choice and short answer test was administered to parents at each time point. The test was developed to assess parent understanding of topics taught during JumpStart. Example items include “What is the correct ratio for maintenance tasks and acquisition tasks when implementing PRT?”, “What is the difference between Respite and Habilitation services,” and “Who prepares the Individual Education Plan?” Each correct response receives one point. Possible scores range from 0 to 20, with higher scores indicating better understanding of topics taught during JumpStart. Internal consistency of the JumpStart test was good (Cronbach’s α = 0.88). 7.3.5. Parenting Stress Index – short form, third edition (PSI-3; Abidin, 1995) The PSI-3 short form is a 36-item, parent-report measure of stress experienced within the parent-child dyad. The respondent indicates agreement with each item on a five-point scale ranging from ‘strongly agree’ to ‘strongly disagree’. Item responses were summed to create PSI Total Stress Scale scores ranging from 36 to 180, with higher scores indicating higher levels of parenting stress. The PSI Total Stress scale has excellent internal consistency (Cronbach’s α = 0.95) and test-retest reliability ranging from 0.65 to 0.96 (Abidin, 1995). Internal consistency of the PSI in the current sample was excellent (Cronbach’s α = 0.93). 7.3.6. Center for Epidemiological Studies Depression Scale (CES-D; Radloff, 1977) The CES-D is a 20-item, self-report measure of depressive symptoms. Respondents indicate how often they experienced item statements in the past week on a four-point scale ranging from ‘rarely or none of the time (less than 1 day)’ (0) to ‘Most or all of the time (5–7 days)’ (3). After reverse scoring, item scores are summed to create a total score ranging from 0 to 60, with higher scores indicating higher levels of depressive symptoms. Research examining the sensitivity and specificity of the CESD suggest cutoff scores of 16 or 21 to screen for depression in the general population (Shean & Baldwin, 2008). Radloff (1977) reported that the CES-D has good to excellent internal consistency (Cronbach’s α = 0.84–0.90) and moderate test-retest reliability (r = 0.51–0.67). Internal consistency was good in the current sample (Cronbach’s α = 0.88). 28

Research in Autism Spectrum Disorders 56 (2018) 21–35

N.L. Matthews et al.

7.3.7. 10-min parent-child probe Parent-child dyads were videotaped interacting naturally for ten minutes. All videos except those from the 3-month follow-up were collected by JumpStart staff at the autism center. Parents were not given an introduction to PRT by JumpStart staff prior to the first time point. Dyads were set up in a room with access to a variety of age-appropriate, interactive toys. Parents were told to interact with their children naturally while eliciting as much communication as possible. Three-month follow-up videos were collected by parents in the home. Parents received written instructions for setting up a play area and interacting with their child in the video. Written instructions were identical to verbal instructions given by JumpStart staff at the autism center for previous video probes. Videos were coded by trained clinicians and/or research assistants for parent fidelity of implementation and child responsivity to communication opportunities provided by the parent. Coders were blinded to study group and time point; 33% of videos were coded by a second coder to calculate interobserver agreement (Interobserver agreement for fidelity: M = 86%, range 73–100%; Interobserver agreement for responsivity: M = 84%, range 74–100%). 7.3.7.1. Fidelity of implementation. The motivational procedures of PRT (i.e., shared control, opportunities to respond, interspersing tasks, contingent and natural reinforcement, and attempts) were coded as “+” or “−” for each minute of the 10-min probe. To receive a “+” code, the parent had to demonstrate correct implementation of the respective procedure throughout the entire 1-min interval. If the parent did not provide language opportunities during an entire interval, the parent did not demonstrate PRT components/strategies and was coded “−” for that interval. Percentages of correct implementation for each skill were averaged to create an overall fidelity of implementation percentage score. 7.3.7.2. Child responsivity. Child responsivity was operationally defined as appropriate verbal responses to parent-provided communication opportunities. Appropriate verbal responses are child dependent and thus differed depending on the child’s specific goals. For example, an appropriate response for minimally verbal children may be defined as an open mouth sound not including crying, screeches or whining; eye contact, or a gesture. In contrast, an appropriate response for a verbally fluent child may be initiation of contextually appropriate and non-stereotyped three-to-four word sentences. The number of appropriate responses to parent-provided communication opportunities was divided by the total number of parent-provided opportunities to create the overall child responsivity percentage score. 7.4. Data analysis Data were missing for up to 11% of cases for each time point of the JumpStart test because the respective parent was absent or the test was mistakenly not administered, and for 3% of all other variables due to incomplete questionnaires. Missing data were managed using group mean substitution for the first and third set of analyses described below (Tabachnick & Fidell, 1996). Mean substitution was used rather than multiple imputation because of the small sample size. Research on the performance of multiple imputation in small samples is limited, and assumptions of multiple imputation (e.g., multivariate normality) are often not met in small samples (McNeish, 2017). Given considerable attrition at the 3-month follow-up, only participants with data at all three time points were included in the second set of analyses. Reported in Table 4, there were no significant differences between participants included in and excluded from the 3-month follow-up analyses in baseline functioning or time 1 to time 2 change scores. However, inspection of group means suggests potential differences that may not have been detected due to inadequate statistical power. For interpretative purposes, Cohen’s d effect sizes are also reported. The majority of effect sizes were small or negligible. Of note, effect sizes indicated medium sized differences in time 1 to time 2 change scores on the JumpStart Test and in child responsivity favoring participants included in the 3-month analyses relative to participants who were excluded. There was also a large difference in time 1 to time 2 parenting stress change scores favoring participants included in the 3-month analyses. For each measure, three sets of analyses were conducted. Primary analyses compared time 1 to time 2 change scores between the treatment and WLC groups. Data collected at the orientation (time 1) and the 1-week follow-up (time 2) were examined in the treatment group, whereas data from the first study visit (time 1) and the orientation (time 2) were examined in the WLC group. Positive change scores were calculated for each variable. Specifically, change scores were calculated for parent self-efficacy variables, parent knowledge (i.e., the JumpStart Test), parent fidelity of implementation, and child responsivity by subtracting time 1 scores from time 2 scores. In contrast, change scores for parenting stress and parent depression were calculated by subtracting time 2 scores from time 1 scores. Separate independent samples t-tests were used to examine the effect of study group on change scores for each outcome variable. Mann-Whitney U tests were used when data violated assumptions of normality. A corrected alpha of p = .007 (0.05/7) was used to account for multiple comparisons. The second set of analyses were exploratory and examined scores at times 1, 2, and 3 in treatment group participants who completed the third time point. The waitlist for JumpStart was not long enough to allow for the collection of 3-month follow-up data without treatment in the WLC group. Attrition at time 3 resulted in a relatively small sample size (n = 12). Some outcome variables also had non-normal distributions; thus, non-parametric Friedman tests were used. A corrected alpha of p = .007 (0.05/7) was used for Friedman tests. Bonferroni correction applied by the statistical software was used for follow-up pairwise comparisons. The last set of analyses were also exploratory, and examined associations between child age, baseline Vineland-II scores, baseline SRS-2 scores, and time 1 to time 2 change scores for the primary outcome variables among participants in the treatment group. Due to the small sample size and non-normal distributions of some variables, non-parametric Spearman’s rank tests were used. Small sample sizes combined with corrections for multiple comparisons increased the likelihood for type II error. Thus, effect sizes are reported and interpreted in tandem with tests of statistical significance. Specifically, Cohen’s d statistics are reported and 29

Research in Autism Spectrum Disorders 56 (2018) 21–35

N.L. Matthews et al.

Table 4 Descriptive Statistics for Participants Included and Excluded at Time 3. Variable

Included (n = 12) M(SD)

Excluded (n = 6) M(SD)

p

Cohen’s d

Child Age (months) Child Gender (percent male) Child Race (percent)

40.50 (11.86) 100.00

41.50 (11.15) 100.00

.89

0.09

Caucasian Hispanic Asian African American

75.00 8.30 8.30 8.30

83.30 0.00 16.70 0.00

< $25,000 $25,000–$49,999 $50,000–74,999 $75,000–$99,999 ≥$100,000

8.30% 8.30% 8.30% 25.00% 50.00% 69.67 (9.47) 68.42 (15.02) 74.17 (12.99) 69.92 (8.70) 74.00 (9.61)

16.70% 0.00% 16.70% 16.70% 50.00% 64.17 (16.02) 63.17 (23.64) 68.00 (13.91) 65.17 (14.29) 77.00 (8.44)

.21 .49 .29 .34 .68

0.42 0.27 0.46 0.40 0.33

0.73 (0.62) 0.96 (0.83) 6.27 (3.10) 10.33 (10.76) 4.42 (6.58) 18.83 (11.46) 24.33 (19.80)

0.74 (0.28) 1.00 (0.73) 4.71 (1.83) -3.50 (18.12) 4.17 (8.40) 15.33 (8.98) 10.50 (15.80)

1.00 .89 .39 .10 .62 .39 .15

0.02 0.05 0.61 0.93 0.03 0.34 0.77

Annual Household Income (percent)

Vineland ABC Vineland Communication Vineland Daily Living Skills Vineland Socialization SRS Total T-score T1-T2 Positive Change Scores Parent Outcome Expectations (EIPSES) Parent Competence (EIPSES) Parent Knowledge (JumpStart Test) Parenting Stress (PSI) Parent Depressive Symptoms (CESD) Fidelity of Implementation Child Responsivity

Note: WLC = Waitlist control group. ABC = Adaptive Behavior Composite. SRS = Social Responsiveness Scale, Second Edition. T1 = Time 1. T2 = Time 2. EIPSES = Early Intervention Parenting Self-Efficacy Scale. PSI = Parenting Stress Index. CESD = Center for Epidemiological Studies Depression Scale.

interpreted using conventional criteria of small effect d = 0.20; medium effect d = 0.50, and large effect d = 0.80. 8. Results 8.1. Pre-post comparison of treatment and WLC groups Descriptive statistics for the primary analyses are reported in Table 3. The treatment group demonstrated a significantly larger increase in parent outcome expectations from time 1 to time 2 relative to the WLC group; the effect size was medium (z = 2.66, p = .007, d = 0.72). The treatment group also demonstrated a significantly larger increase in parent competence from time 1 to time 2 relative to the WLC group; the effect size was large (z = 2.85, p = .004, d = 1.05). The treatment group demonstrated significantly greater improvements in JumpStart test performance from time 1 to time 2 relative to the WLC group; the effect size was large (z = 4.52, p < .001, d = 1.91). The treatment and WLC groups did not differ significantly in total stress change scores from time 1 to time 2; however, there was a medium effect favoring the treatment group (t(34) = −1.81, p = .08, d = 0.60). The treatment and WLC also did not differ significantly in depressive symptom change scores from time 1 to time 2; however, there was a medium effect favoring the treatment group (z = 1.87, p = .06, d = 0.57). The treatment group demonstrated larger gains in fidelity of implementation from time 1 to time 2 relative to the WLC group; the effect size was large (t(34) = −4.82, p < .001, d = 1.61). The treatment group demonstrated larger gains in child responsivity from time 1 to time 2 relative to the WLC group; the effect size was large (z = 3.69, p < .001, d = 1.43). 8.2. Three-month follow-up data Descriptive statistics for the exploratory analyses are reported in Table 5. There were significant differences in the distributions of parent outcome expectation scores at times 1, 2, and 3 (Friedman’s Q(2) = 11.79, p = .003). Pairwise comparisons indicated significantly higher scores at time 2 relative to time 1 (z = −3.16, p = .005, d = 1.45) and time 3 relative to time 1 (z = −2.65, p = .02, d = 0.58); effect sizes were large and medium, respectively. There was not a significant difference between time 2 and time 3 scores; the effect size indicated a small decrease (z = 0.51, p = 1.00, d = 0.21). There were not significant differences in the distributions of parent competence scores at times 1, 2, and 3 (Friedman’s Q (2) = 8.23, p = .02). However, Cohen’s d statistics suggested a large increase from time 1 to time 2 (d = 1.28), a medium increase from time 1 to time 3 (d = 0.66), and a small decrease from time 2 to time 3 (d = 0.38). 30

Research in Autism Spectrum Disorders 56 (2018) 21–35

N.L. Matthews et al.

Table 5 Descriptive Statistics for Primary Variables at times 1, 2, and 3 (n = 12).

Parent Outcome Expectations (EIPSES)** Parent Competence (EIPSES) Parent Knowledge (JumpStart Test; n = 9) *** Parenting Stress Index (PSI) Parent Depressive Symptoms (CESD) Fidelity of Implementation (n = 10) Child Responsivity (n = 10)***

Time 1 M(SD)

Time 2 M(SD)

Time 3 M(SD)

T1-T2 Cohen’s d

T2-T3 Cohen’s d

T1-T3 Cohen’s d

5.11 (0.88) 5.52 (0.82) 9.61 (2.98) 91.08 (20.19) 10.25 (7.65) 32.20 (7.27) 10.90 (11.39)

5.84 (0.48) 6.48 (0.38) 16.22 (1.18) 80.75 (17.68) 5.83 (3.83) 52.80 (11.08) 37.50 (17.57)

5.63 (0.95) 6.19 (0.85) 14.72 (2.17) 81.67 (17.62) 8.92 (6.60) 41.70 (12.82) 43.70 (14.46)

1.45** 1.28 2.04*** 0.99 0.57 1.81 1.35

0.21 0.38 0.68 0.04 0.44 0.73 0.53

0.58* 0.66 2.11* 0.55 0.17 0.61 1.88***

Note: T1 = Time 1. T2 = Time 2. T3 = Time 3. EIPSES = Early Intervention Parenting Self-Efficacy Scale. PSI = Parenting Stress Index. CESD = Center for Epidemiological Studies Depression Scale. A Bonferroni corrected alpha (.007) was used to account for multiple comparisons. Bonferroni correction applied by the statistical software was used for follow-up pairwise comparisons. *p < .05. **p < .007. ***p ≤ .001.

There were significant differences in the distributions of JumpStart test scores at times 1, 2, and 3 (Friedman’s Q(2) = 14.89, p = .001; n = 9). Pairwise comparisons indicated significantly higher scores at time 2 relative to time 1 (z = −3.77, p < .001, d = 2.04) and time 3 relative to time 1 (z = −2.59, p = .03, d = 2.11); effect sizes were large. There was not a significant difference between time 2 and time 3 scores; the effect size indicated a medium decrease (z = 1.18, p = .72, d = 0.68). There were not significant differences in the distributions of parent stress (Friedman’s Q(2) = 6.17, p = .05) or depressive symptoms (Friedman’s Q(2) = 5.78, p = .06) at times 1, 2, and 3. For parent stress, Cohen’s d statistics indicated a large decrease from time 1 to time 2 (d = 0.99), a medium decrease from time 1 to time 3 (d = 0.55), and a negligible increase from time 2 to time 3 (d = 0.04). For depressive symptoms, there was a medium decrease from time 1 to time 2 (d = 0.57), a negligible decrease from time 1 to time 3 (d = 0.17), and a small increase from time 2 to time 3 (d = 0.44). There were not significant differences in the distributions of fidelity of implementation scores at times 1, 2, and 3 (Friedman’s Q (2) = 8.60, p = .01; n = 10). However, Cohen’s d statistics indicated a large increase from time 1 to time 2 (d = 1.81), a medium increase from time 1 to time 3 (d = 0.61), and a medium decrease from time 2 to time 3 (d = 0.73). There were significant differences in the distributions of child responsivity scores at times 1, 2, and 3 (Friedman’s Q(2) = 14.60, p = .001; n = 10). Pairwise comparisons indicated significantly higher child responsivity at time 3 relative to time 1; the effect size was large (z = −3.80, p < .001, d = 1.88). There were not significant differences in child responsivity between time 1 and time 2 (z = −2.24, p = .08, d = 1.35) or between time 2 and time 3 (z = −1.57, p = .35, d = 0.53). However, the effect sizes were large and medium, respectively. 8.3. Correlations among child age, baseline functioning, and treatment response There was a negative correlation between baseline Vineland-II daily living skills scores and change in parent outcome expectations such that parents of children with higher baseline daily living skills scores (i.e., more independent functioning) demonstrated smaller gains in parent outcome expectations from pre-to-post JumpStart (rs = −0.52, p = .03). There was also a negative correlation between baseline Vineland-II adaptive behavior composite scores and change in parent outcome expectations (rs = −0.46, p = .05). There were no other significant correlations among child age, baseline functioning, and treatment response variables. 9. Discussion This is the first study to systematically examine parent and child outcomes of JumpStart, a short-term parent education program using a Behavioral Skills Training model that has been completed by more than 700 families over the last decade. Findings indicate statistically significant gains in parent outcome expectations, parent competence, knowledge of topics covered during JumpStart, fidelity of implementation, and child responsivity. Although not statistically significant, medium effect sizes suggested potential decreases in parenting stress and depressive symptoms. These preliminary results suggest improved outcomes for families who complete JumpStart. JumpStart is primarily a parent education and training program, thus it is unsurprising that changes in parent outcomes were observed in the current study. Two facets of parent self-efficacy were examined, including parent outcome expectations (i.e., the belief that the environment can contribute to positive changes in children’s development) and parent competence (i.e., belief that parents’ own abilities can contribute to positive changes). The treatment group demonstrated significantly larger gains than the WLC group in both of these areas from orientation to the 1-week follow-up. Parents in the treatment group also demonstrated larger gains in understanding of concepts covered during JumpStart, including knowledge regarding autism, behavior analytic treatment approaches, state-funded services, insurance-funded services, and special education services, compared to the WLC group. Future research should examine whether this knowledge is meaningfully applied by parents in the form of successfully obtaining appropriate services and funding. Comparisons of parenting stress and depressive symptoms between the treatment and WLC groups were not statistically significant; however, both comparisons were of medium magnitude and descriptive statistics suggested that the treatment group 31

Research in Autism Spectrum Disorders 56 (2018) 21–35

N.L. Matthews et al.

reported greater decreases in parenting stress and depressive symptoms. Additional research with adequate statistical power to detect medium and small effects will clarify whether participating in JumpStart is associated with meaningful improvements in parent wellbeing. Approximately half of the treatment group had parenting stress scores that were above normal limits (> 80th percentile) at baseline and follow-up, which suggests that a considerable proportion of parents have consistently elevated stress levels that may not be adequately addressed through participation in a short-term program like JumpStart. Further, it is possible that participation in this type of educational program could be associated with maintenance of existing stress or increased stress due to the volume of information presented to parents regarding the processes for obtaining services. Future research should examine this possibility, and whether parenting stress eventually declines once comprehensive services are established. Notably, group means for depressive symptoms at baseline and follow-up were well below the proposed CESD cutoff scores of 16 and 21 that have been used to screen for depression in the general population (Shean & Baldwin, 2008). Given the lack of elevated depressive symptoms at baseline, it is somewhat intuitive that reductions in depressive symptoms in the treatment group were only of medium magnitude. JumpStart was not designed to directly target parent mental health; parenting stress and depressive symptoms were examined in the current study to determine if the program has collateral effects in these areas. JumpStart currently provides parents with a resource directory at the end of the program that includes local parent support groups and mental health providers. Given heightened parenting stress observed in the current study and greater frequency of depression among parents of children with ASD relative to parents of children without ASD documented by previous studies (Cohrs & Leslie, 2017), it could be beneficial for JumpStart and other short-term programs to screen for elevated parenting stress and depressive symptoms and to refer affected parents for additional services. JumpStart aims to provide parents with the basic skills necessary to implement behavior analytic intervention at home while waiting to begin a more comprehensive intervention program. Current findings provide support for the achievement of this goal, as families in the treatment group demonstrated significantly greater gains in parent fidelity of implementation and child responsivity compared to the WLC after very brief training in PRT. It is important to note the considerable variability in fidelity of implementation upon JumpStart completion (i.e., 23–80%). It is possible that parents toward the higher end of the distribution might need less support in the future which could reduce strain on already limited community resources. Future research should compare the trajectories of PRT implementation and service consumption among families who exit JumpStart and other parent training programs with varying levels of fidelity of implementation. It is also important to consider the families toward the lower end of the distribution, and the potential for unintended deleterious effects of poor fidelity of implementation. Two of the 18 children in the treatment group showed negative change in responsivity (3% and 9%). The child of the parent who showed a 3% decrease was toward the lower end of the distribution for fidelity of implementation (24%). In contrast, the parent of the child with a 9% decrease had a fidelity of implementation score (54%) that was above the group mean (48%). Additionally, exploratory post hoc analyses indicated no significant association between post-JumpStart parent fidelity of implementation and child responsivity, and many of the children of parents at the lower end of the distribution for fidelity of implementation demonstrated gains in responsivity. For example, the child of the parent with the lowest fidelity of implementation score post-JumpStart (23%) demonstrated a 16% gain in responsivity. Thus, in the current sample, there is no clear indication that low parent fidelity of implementation was associated with deleterious effects on child responsivity. Multiple factors may have contributed to the lack of association between parent fidelity of implementation and child responsivity, including variability in children’s propensity to respond to treatment (Sherer & Schreibman, 2005) and the presence of additional interventions. The exploratory examination of 3-month follow-up data in a small subsample indicated maintenance of some gains and potential loss of others. Parent outcome expectations continued to be higher than orientation scores at the 3-month follow-up and were not statistically different from 1-week follow-up scores, indicating maintenance upon program exit. Distributions of parent competence scores did not differ significantly across the three time points, but effect sizes and descriptive statistics suggest that parent competence scores follow a similar pattern to parent outcome expectation scores, just to a smaller magnitude. It is likely that the current analyses were underpowered to detect these differences. Performance on the JumpStart test continued to be significantly better at the 3-month follow-up than at orientation, and scores did not differ significantly at the 1-week and 3-month follow-ups, suggesting that parent understanding of the concepts taught during JumpStart was maintained. Although differences in fidelity of implementation were not statistically significant across the three time points, descriptive statistics and medium to large effect sizes suggest that parent fidelity of implementation was lower at the 3-month follow-up than at the end of JumpStart, but still higher than at JumpStart orientation. Thus, parents appear to lose some skills, but are still better at implementing PRT than before participating in JumpStart. Gains in child responsivity were maintained at the 3-month follow-up, which suggests that these gains are enduring. It is also possible that parents’ remaining skills continued to support children’s development, which may have contributed to the observed maintenance of child responsivity. Of note, the WLC group was not assessed at the 3-month follow-up, so the possibility that child maturation from time 2 to time 3 (as opposed to effects of JumpStart) contributed to the observed maintenance of child responsivity cannot be ruled out. Despite maintenance of gains in child responsivity, examination of group means suggests that the rate of improvement slowed upon program exit. This could be due to many factors, including the removal of the JumpStart classroom environment and child interaction with trained interventionists, and the possibility that parents reduced or stopped their implementation of PRT after intervention exit. It is also possible that some children initially showed large gains upon the introduction of intervention during JumpStart, and that growth simply slowed over time. Importantly, group means and effect sizes indicated that gains at the 1-week follow-up reverted at the 3-month follow-up to varying degrees for all variables except child responsivity. There was considerable attrition at the 3-month follow-up, and there is some evidence that families who remained in the study demonstrated greater treatment response compared to the families who were lost to attrition. Thus, it is possible that findings from these exploratory analyses would not generalize to families lost to attrition. 32

Research in Autism Spectrum Disorders 56 (2018) 21–35

N.L. Matthews et al.

Future research should focus on efforts to retain as many participants as possible to increase validity of 3-month follow-up findings. Parent training components of other short-term parent education and training programs are conducted in the child’s home (Cutress & Muncer, 2014; Wang, 2008). The parent-training component of JumpStart is completed at an autism-center at the expense of the natural environment in order to serve more families. Despite reduction in the naturalistic aspect of PRT, parents in the current study demonstrated significant improvements in fidelity of implementation and children demonstrated increased responsivity that was maintained at a 3-month follow-up. This may reflect one of the benefits of parent training. Specifically, despite completing training in a non-naturalistic setting, parents may be implementing their new skills in the home and community settings. An exploratory aim of the current study was to examine associations among child age, baseline functioning, and treatment gains in order to better understand associations between child characteristics and JumpStart outcomes. The only statistically significant correlations were observed between children’s baseline adaptive functioning (overall and daily living skills) and changes in parent outcome expectations from pre-to-post JumpStart. Thus, parents of children who demonstrated greater independence in adaptive functioning skills at baseline demonstrated smaller gains in parent outcome expectations, or the belief that the environment can contribute to positive changes in children’s development. A potential explanation for this finding is that parents of children who were less independent at baseline perceived greater changes in their children during the program relative to parents of children who were more independent at baseline. An important question that remains unanswered regards the potential sources of change observed in the current study. For example, is increased parent self-efficacy due to knowledge gained from participating in the program, improvements in child functioning, or both? Alternatively, is the implementation of PRT the sole source of improved child responsivity, or could improved parent well-being also be a contributing factor? Future studies with larger sample sizes are necessary to examine these questions. JumpStart provides scholarships to improve access to the program for low income families; however, additional efforts are necessary to directly target families of low socioeconomic status and other underserved populations. Although not examined in the current study, the non-profit autism center offers JumpStart in Spanish to monolingual Spanish-speaking families one-to-two times annually depending on the volume of families seeking to enroll. Additionally, a pilot version of JumpStart online has been developed for families who may not be able to access the traditional program for a variety of reasons (e.g., work; distance; transportation; child care for the focal child’s siblings). Structural barriers to services like work and transportation have been identified as one mechanism underlying the association between low socioeconomic status and reduced access to services (Pickard & Ingersoll, 2016). Thus, JumpStart online may be a viable option for lower socioeconomic status families. Future research will examine the effectiveness of this mode of delivery. JumpStart online may eventually be made available to families outside of the state of Arizona, which will require modification to the didactic lesson on state-funded services. A similar modification would be necessary for practitioners aiming to implement the in-person JumpStart program in other states.

9.1. Limitations Many of the limitations were associated with the preliminary nature of this study, which was designed around families participating in an existing, community-implemented program. Rather than being specifically recruited to participate in a study, families who had already enrolled in JumpStart were approached and asked to complete additional procedures to generate data for the study. Measures were chosen with feasibility in mind to minimize time associated with participation and to encourage families to participate. Randomization of participants was not feasible; however, groups were matched on age and did not differ at enrollment on diagnostic label, autism symptoms, or adaptive functioning. There was a wide child age range, but this was reflective of the population that participates in JumpStart. Many of the outcome variables were measured using parent-report, and parents were not blind to study group, which may have biased their reporting. Future research may include a comparative efficacy trial of JumpStart and another intervention to address this issue. The service setting for this study was well-accustomed to providing JumpStart because it was the same setting in which JumpStart was developed. This could limit generalizability of the current findings. However, staff turnover for clinicians who implement JumpStart is relatively frequent due to the entry-level nature of the positions and clinicians moving between JumpStart and other clinical programs within the same autism center. Child intervention targets were selected using an assessment that was developed specifically for JumpStart. It was designed to allow for a brief but thorough assessment of children’s abilities. The format of JumpStart requires that this assessment be completed for six children in a classroom setting within a 90-min period, which precludes the use of standardized measures that require more time and/or interviews with parents and caregivers to inform the selection of intervention targets. Attrition at the 3-month follow-up resulted in a substantial reduction in sample size, and likely, generalizability of the data. It is possible that families who did not contribute data at this time point differed from families who did in meaningful ways. Data for this time point were collected at a scheduled 3-month follow-up that was a component of JumpStart. Despite families expressing interest in a 3-month check-in with clinicians, attendance rates were consistently low, and this component of the program has subsequently been removed. Future research may include randomized controlled trials that will include more stringent inclusion criteria such as the ability to complete in-person direct observation assessments and ability to complete follow-up visits. However, this may also reduce generalizability of findings. Additionally, future research may include alternative options for the 3-month follow-up, such as home visits and/or online administration of questionnaires. Last, additional research is necessary to compare whether families who complete JumpStart demonstrate increased service uptake compared to families who do not complete JumpStart.

33

Research in Autism Spectrum Disorders 56 (2018) 21–35

N.L. Matthews et al.

9.2. Implications Findings provide preliminary evidence for the effectiveness of JumpStart in improving parent self-efficacy, knowledge about autism and the service system, fidelity of implementation, and child responsivity. Additionally, findings add to the small existing literature on short-term programs for families of children with ASD. Given the growing number of children with ASD and finite resources, short-term programs like JumpStart may be an effective method by which parents are able to begin intervening with their child’s development while they arrange for more comprehensive services. Conflicts of interest All authors are employed by the non-profit autism center that developed and implements the JumpStart program. Acknowledgements We thank Autism Speaks for financial support of this project. We gratefully acknowledge the contributions made by Southwest Autism Research Center research and clinical staff to data collection and coding, and the JumpStart clinical team for their implementation of the program. Most importantly, we thank the families who participated in this study. References Abidin, R. R. (1995). Parenting stress index (third edition). Odessa, FL: Psychological Assessment Resources. Austin, J., Manning-Courtney, P., Johnson, M. L., Weber, R., Johnson, H., Murray, D., et al. (2016). Improving access to care at autism treatment centers: A system analysis approach. Pediatrics, 137(S2), S149–S157. https://doi.org/10.1542/peds.2015-2851M. Boyd, B. A., Odom, S. L., Humphreys, B. P., & Sam, A. M. (2010). Infants and toddlers with autism spectrum disorder: Early identification and early intervention. Journal of Early Intervention, 31(2), 75–98. https://doi.org/10.1177/105381511036290. Brookman-Frazee, L., Baker-Ericzen, M., Stadnick, N., & Taylor, R. (2012). Parent perspectives on community mental health services for children with autism spectrum disorders. Journal of Child and Family Studies, 21, 533–544. https://doi.org/10.1007/s10826-011-9506-8. Carter, A. S., Messinger, D. S., Stone, W. L., Celimli, S., Nahmias, A. S., & Yoder, P. (2011). A randomized controlled trial of Hanen’s ‘More Than Words’ in toddlers with early autism symptoms. The Journal of Child Psychology and Psychiatry, 52, 741–752. https://doi.org/10.1111/j.1469-7610.2011.02395.x. Cohrs, A. C., & Leslie, D. L. (2017). Depression in parents of children diagnosed with autism spectrum disorder: A claims-based analysis. Journal of Autism and Developmental Disorders, 47, 1416–1422. https://doi.org/10.1007/s10803-017-3063-y. Constantino, J. N., & Gruber, C. P. (2012). Social responsiveness scale (Second Edition (SRS-2)). Torrance, CA: Western Psychological Services. Coolican, J., Smith, I. M., & Bryson, S. E. (2010). Brief parent training in pivotal response treatment for preschoolers with autism. Journal of Child Psychology and Psychiatry, 51(12), 1321–1330. https://doi.org/10.1111/j.1469-7610.2010.02326.x. Cutress, A. L., & Muncer, S. J. (2014). Parents’ views of the National Autistic Society’s EarlyBird Plus Programme. Autism, 18(6), 651–657. https://doi.org/10.1177/ 1362361313495718. Dawson, G., Rogers, S., Munson, J., Smith, M., Winter, J., Greenson, J., et al. (2010). Randomized, controlled trial of an intervention for toddlers with autism: The Early Start Denver Model. Pediatrics, 125, e17–e23. https://doi.org/10.1542/peds.2009-0958. Estes, A., Olson, E., Sullivan, K., Greenson, J., Winter, J., Dawson, G., et al. (2013). Parenting-related stress and psychological distress in mothers of toddlers with autism spectrum disorders. Brain & Development, 35(2), 133–138. https://doi.org/10.1016/j.braindev.2012.10.004. Guimond, A. B., Wilcox, M. J., & Lamorey, S. G. (2008). The early intervention parenting self-efficacy scale (EIPSES) scale construction and initial psychometric evidence. Journal of Early Intervention, 30(4), 295–320. Hardan, A. Y., Gengoux, G. W., Berquist, K. L., Libove, R. A., Ardel, C. M., Phillips, J., et al. (2015). A randomized controlled trial of pivotal response treatment group for parents of children with autism. The Journal of Child Psychology and Psychiatry, 56, 884–892. https://doi.org/10.1111/jcpp.12354. Ingersoll, B. R., & Wainer, A. L. (2013). Pilot study of a school-based parent training program for preschoolers with ASD. Autism, 17, 434–448. https://doi.org/10. 1177/1362361311427155. Karst, J. S., & Van Hecke, A. V. (2012). Parent and family impact of autism spectrum disorders: A review and proposed model for intervention evaluation. Clinical Child and Family Psychology Review, 15(3), 247–277. https://doi.org/10.1007/s10567-012-0119-6. Koegel, R. L., & Koegel, L. K. (2012). The PRT pocket guide. Pivotal response treatment for autism spectrum disorders. Baltimore, Maryland: Brookes Publishing. Lovaas, O. I. (1987). Behavioral treatment and normal educational and intellectual functioning in young autistic children. Journal of Consulting and Clinical Psychology, 55, 3–9. McConachie, H., Randle, V., Hammal, D., & Le Couteur, A. (2005). A controlled trial of a training course for parents of children with suspected autism spectrum disorder. The Journal of Pediatrics, 147(3), 335–340. https://doi.org/10.1016/j.jpeds.2005.03.056. McNeish, D. (2017). Missing data methods for arbitrary missingness with small samples. Journal of Applied Statistics, 44, 24–39. https://doi.org/10.1080/02664763. 2016.1158246. Mohammadzaheri, F., Koegel, L. K., Rezaee, M., & Rafiee, S. M. (2014). A randomized clinical trial comparison between pivotal response treatment (PRT) and structured applied behavior analysis (ABA) intervention for children with autism. Journal of Autism and Developmental Disorders, 44, 2769–2777. https://doi.org/ 10.1007/s10803-014-2137-3. Montes, G., Halterman, J. S., & Magyar, C. I. (2009). Access to and satisfaction with school and community health services for US children with ASD. Pediatrics, 124, S407–S413. https://doi.org/10.1542/peds.2009-1255L. Orinstein, A. J., Helt, M., Troyb, E., Tyson, K. E., Barton, M. L., Eigsti, I., et al. (2014). Intervention for optimal outcome in children with a history of autism. Journal of Developmental and Behavioral Pediatrics, 35, 247–256. https://doi.org/10.1097/DBP.0000000000000037. Parsons, M. B., Rollyson, J. H., & Reid, D. H. (2012). Evidence-based staff training: A guide for practitioners. Behavior Analysis in Practice, 5(2), 2–11. Pickard, K. E., & Ingersoll, B. R. (2016). Quality versus quantity: The role of socioeconomic status on parent-reported service knowledge, service use, unmet service needs, and barriers to service use. Autism, 20, 106–115. https://doi.org/10.1177/1362361315569745. Radloff, L. S. (1977). The CES-D scale: A self-report depression scale for research in the general population. Applied Psychological Measurement, 1, 385–401. Shean, G., & Baldwin, G. (2008). Sensitivity and specificity of depression questionnaires in a college-age sample. The Journal of Genetic Psychology, 169, 281–288. Sherer, M. R., & Schreibman, L. (2005). Individual behavioral profiles and predictors of treatment effectiveness for children with autism. Journal of Consulting and Clinical Psychology, 73, 525–538. https://doi.org/10.1037/0022-006X.73.3.525. Shields, J. (2001). The NAS EarlyBird Programme – Partnership with parents in early intervention. Autism, 5(1), 49–56. https://doi.org/10.1177/ 1362361301005001005. Smith, T. (2001). Discrete trial training in the treatment of autism. Focus on Autism and Other Developmental Disabilities, 16, 86–92. https://doi.org/10.1177/ 108835760101600204.

34

Research in Autism Spectrum Disorders 56 (2018) 21–35

N.L. Matthews et al.

Solomon, R., Van Egeren, L. A., Mahoney, G., Quon Huber, M. S., & Zimmerman, P. (2014). PLAY Project home consultation intervention program for young children with autism spectrum disorders: A randomized controlled trial. Journal of Developmental and Behavioral Pediatrics, 35, 475–485. https://doi.org/10.1097/DBP. 0000000000000096. Sparrow, S. S., Cicchetti, D. V., & Balla, D. A. (2005). Vineland adaptive behavior scales, second edition Vineland II: Survey forms manual. Bloomington, MN: Pearson Assessments. Tabachnick, B. G., & Fidell, L. S. (1996). Using multivariate statistics (third edition). New York, NY: HarperCollins College Publishers. Tonge, B., Brereton, A., Kiomall, M., Mackinnon, A., & Rinehart, N. J. (2014). A randomised group comparison controlled trial of ‘preschoolers with autism’: A parent education and skills training intervention for young children with autistic disorder. Autism, 8(2), 166–177. https://doi.org/10.1177/1362361312458186. Vismara, L. A., & Rogers, S. J. (2010). Behavioral treatments in autism spectrum disorder: What do we know? Annual Review of Clinical Psychology, 6, 447–468. Wang, P. S. (2008). Effects of parent training program on the interactive skills of parents of children with autism in China. Journal of Policy and Practice in Intellectual Disabilities, 5(2), 96–104. https://doi.org/10.1111/j.1741-1130.2008.00154.x. Warren, Z., McPheeters, M. L., Sathe, N., Foss-Feig, J. H., Glasser, A., & Veenstra-VanderWeele, J. (2011). A systematic review of early intensive intervention for autism spectrum disorders. Pediatrics, 127, e1303–e1311. Wetherby, A. M., Guthrie, W., Woods, J., Schatschneider, C., Holland, R., Morgan, L., et al. (2014). Parent-implemented social intervention for toddlers with autism: An RCT. Pediatrics, 134, 1084–1093. https://doi.org/10.1542/peds.2014-0757. Whittingham, K., Sofronoff, K., Sheffield, J., & Sanders, M. R. (2009). Behavioural family intervention with parents of children with ASD: What do they find useful in the parenting program Stepping Stones Triple P? Research in Autism Spectrum Disorders, 3(3), 702–713. https://doi.org/10.1016/j.rasd.2009.01.009.

35