Research in Autism Spectrum Disorders 4 (2010) 367–376
Contents lists available at ScienceDirect
Research in Autism Spectrum Disorders Journal homepage: http://ees.elsevier.com/RASD/default.asp
Comparison of methods for varying item presentation during noncontingent reinforcement Rocio Rosales a,*, April Worsdell b,1, Maranda Trahan b a b
Youngstown State University, United States Southern Illinois University, United States
A R T I C L E I N F O
A B S T R A C T
Article history: Received 2 October 2009 Accepted 3 October 2009
The purpose of the present investigation was to evaluate the relative effectiveness of three item presentation methods during noncontingent reinforcement (NCR). Four individuals with developmental disabilities and problem behavior maintained by automatic reinforcement were recruited for the study. Single-item stimulus preference assessments were conducted to identify a variety of preferred items associated with low levels of problem behavior. During the treatment analysis, a multielement design embedded within a reversal design was utilized, and the following NCR conditions were compared: (a) presenting preferred items one at a time, (b) presenting multiple items simultaneously, and (c) rotating single items after a predetermined time period. Results indicate that all three NCR presentations methods were effective in decreasing problem behavior, but the single-item condition resulted in the lowest levels of problem behavior for three of the four participants. Published by Elsevier Ltd.
Keywords: Noncontingent reinforcement Item presentation Stimulus preference assessment
1. Introduction Noncontingent reinforcement (NCR) is a frequently used intervention for reducing problem behavior in persons with disabilities, and the successful application of NCR has been demonstrated across both social and nonsocial functions (Carr et al., 2000). In most applications of NCR as treatment for socially maintained problem behavior, the reinforcer responsible for behavioral maintenance is withheld following occurrences of problem behavior, and that same maintaining reinforcer is delivered on a time-based schedule that is response independent. However, when NCR is used to treat problem behavior maintained by nonsocial (i.e., automatic) reinforcement, it may be difficult or impossible to withhold access to the maintaining reinforcer because sensory stimulation is delivered as a direct result of engaging in the behavior (Lindberg, Iwata, Roscoe, Worsdell, & Hanley, 2003). As a result, applications of NCR for automatically maintained problem behavior typically involve continuous access to alternative stimuli, and the degree of success of NCR often depends on the extent to which those stimuli either compete with or are more highly preferred than the sensory reinforcers produced via problem behavior. The selection of alternative stimuli for use during NCR has been the subject of much empirical investigation. A number of studies have shown that high-preference stimuli identified via preference assessments can reduce the occurrence of automatically maintained problem behavior (Ahearn, Clark, DeBar, & Florentino, 2005; Shore, Iwata, Khang, & Smith, 1997).
* Corresponding author at: Department of Psychology, Youngstown State University, One University Plaza, Youngstown, OH 44555-0001, United States. Tel.: +1 330 941 1620; fax: +1 330 941 2304. E-mail address:
[email protected] (R. Rosales). 1 Now affiliated with Coyne & Associates. 1750-9467/$ – see front matter . Published by Elsevier Ltd. doi:10.1016/j.rasd.2009.10.004
368
R. Rosales et al. / Research in Autism Spectrum Disorders 4 (2010) 367–376
By contrast, other studies have focused on the degree to which the alternative stimuli used during NCR match the hypothesized sensory consequence for problem behavior (Favell, McGimsey, & Schell, 1982; Fisher, Lindauer, Alterson, & Thompson, 1998; Piazza, Adelinis, Hanley, Goh, & Delia, 2000). Although both methods of item selection have produced successful results during short-term applications of NCR (i.e., 10 min), little is known about the degree to which alternative stimuli will adequately compete with automatically maintained problem behavior over longer periods of time. It is possible that fluctuations in preference or satiation to the alternative stimuli could result in increases in problem behavior over time. To mitigate these limitations, researchers have evaluated methods for varying the presentation of alternative stimuli during NCR. Lindberg et al. (2003) compared 120-min NCR conditions during which either one or four preferred stimuli were continuously available. Results demonstrated that the presentation of one NCR item resulted in low levels of self-injurious behavior for one of three participants. NCR with four preferred items was successful for another participant, and neither NCR condition was effective for a third participant. In another investigation, DeLeon, Anders, Rodriguez-Catter, and Neidert (2000) evaluated two methods for varying item presentation during NCR. In one condition, two equally preferred toys were rotated once every 10 min during a 30-min NCR session. This condition was compared to one in which the participant had access to both equally preferred toys throughout the duration of the NCR session. Results of this study showed that both NCR item presentation methods resulted in equally low levels of SIB, although item engagement was higher when the items were presented simultaneously rather than rotated. Despite the positive results presented by DeLeon et al. (2000), limitations of the study precluded conclusive remarks about the effectiveness of one treatment condition over another. Specifically, the experimental design consisted of an ABACAD design, in which baseline was followed by a single-item condition (e.g., B phase), after which a combination of single- and rotate- (e.g., C phase) or rotate- and multiple-item conditions (e.g., D phase) were compared following baseline conditions. In addition, only one participant was recruited, leaving the need for extension and replication of this procedure. The purpose of the present investigation was to replicate and extend the findings demonstrated by DeLeon et al. (2000) by evaluating the relative effectiveness of three item presentation methods during 30-min NCR sessions, and rectifying the limitations of the previous study. This included comparing three conditions similar to those employed in DeLeon et al. (2000) and included: (a) presenting preferred items one at a time, (b) presenting multiple items simultaneously, and (c) rotating single items after a predetermined time period. The three treatment conditions were randomly rotated in a multielement design embedded within a reversal design to directly compare their relative efficacy. 2. Methods Four individuals who displayed problem behavior maintained by automatic reinforcement were recruited from a community residential home or a local elementary school to participate in this study. Individuals were selected based on the recommendation of staff members or teachers due to the observed occurrence of high frequencies of non-functional or repetitive problem behavior. David was a 50-year-old male diagnosed with severe mental retardation (IQ < 47), seizure disorder, and autism. David’s problem behavior consisted of head hitting, which was defined as one or both hand(s) or fist(s) making forceful contact with the side or front of his head. Rose was a 39-year-old female diagnosed with profound mental retardation (IQ = 11) and PDD-NOS. Rose engaged in saliva play, which involved ejecting visible liquid from her mouth onto her chin, accompanied by side-to-side movement of her hand against the chin, or insertion of at least one finger past the plane of the lips. Kyle was a 9-year-old male diagnosed with autism, and he exhibited thumb sucking, which was defined as insertion of the right or left thumb past the plane of the lips for at least two consecutive seconds. Karen was a 45-year-old female diagnosed with autism and severe mental retardation (IQ < 45). She engaged in face rubbing, which involved moving her hand back and forth against her face, with or without visible mucus present. Sessions were conducted 2–4 times per day, 3–4 days per week, in a secluded area of the residential facility or school. Session areas were furnished with tables and chairs, and leisure items were provided during the treatment conditions as described below. Sessions during phase 1 (functional analysis) were 10 min in length, and sessions during phase 3 (treatment comparison) were 30 min in length. 2.1. Response measurement and interobserver agreement Data were recorded by trained observers utilizing a handheld device (palm One Zire 31) designed for data collection purposes. The primary dependent variable during the study was the occurrence of problem behavior, and was measured as either frequency (i.e., head hitting) or percentage of 10-s intervals in which the behavior occurred (i.e., saliva play, thumb sucking, and face rubbing). During the preference assessment (Phase 2) and the treatment comparison phase (Phase 3), data were also collected on item engagement, which was defined as touching or physically manipulating a leisure item with at least one hand for a period of at least 3 s. Problem behavior data were converted to responses per minute or percentage of 10s intervals in which the behavior occurred. Item engagement data were expressed as duration (Phase 2) or as the percentage of 10-s intervals with item manipulation (Phase 3). A second trained observer simultaneously collected data on all target behaviors during an average of 29% (range, 23–45%) of all sessions. When percentage of interval data were collected on problem behavior, IOA was calculated by dividing the total number of agreement intervals by the total number of session intervals and multiplying by 100%. In the case of frequency data, IOA for problem behavior was calculated on an interval-by-interval basis by dividing the lower number of
R. Rosales et al. / Research in Autism Spectrum Disorders 4 (2010) 367–376
369
responses by the higher number of responses and multiplying the average across intervals by 100% (David only). IOA for item engagement was calculated by dividing the total number of agreement intervals by the total number of intervals in a session and multiplying by 100%. Across all phases of the study, mean percentage agreement for problem behavior was 96.8% (range, 80–100%) for David; 96.9% (range, 77–100%) for Rose; 98.5% (range, 90–100%) for Kyle; and 97.8% (range, 88–100%) for Karen. Mean IOA for item engagement was 96% (range, 82–100%) for David; 99% (range, 95–100%) for Rose; 95% (range, 93– 100%) for Kyle; and 98% (range, 94–100%) for Karen. 2.2. Experimental design A multielement design was employed during the functional analysis (Phase 1), and the four assessment conditions were randomly selected for order at the beginning of every day. During the treatment comparison (Phase 3), a multielement design embedded in a reversal design was used to compare levels of problem behavior and item engagement during baseline and treatment phases. A block of three NCR conditions (i.e., single, rotate and multiple) were presented in a random order during treatment. 2.2.1. Phase 1: functional analysis A functional analysis was conducted to determine the variables maintaining each participant’s problem behavior (Iwata, Dorsey, Slifer, Bauman, & Richman, 1982/1994). Four assessment conditions were alternated to determine the variables maintaining each participant’s problem behavior. In the attention test condition, participants had continuous noncontingent access to arbitrarily selected leisure items and a therapist provided attention in the form of brief reprimands or statements of disapproval (e.g., ‘‘Don’t do that!’’) contingent upon occurrence of the target behavior, while all other behaviors were ignored. In the demand condition, learning trials were presented to participants on a fixed-time 30-s schedule (FT-30 s) using a three-prompt sequence (verbal instruction, model prompt and physical guidance), and were provided with a 30-s ‘‘break’’ from these tasks contingent upon each occurrence of the target behavior. During the alone condition, participants were placed in a room void of leisure or work materials and all behavior, including the target behavior, were ignored. During this condition, a therapist was present in the room, but did not interact with the participants at any time. Finally, in the control condition, participants were provided with leisure materials continuously, and a therapist briefly interacted with them on a FT-30 s of reinforcement independent of all behavior. 2.2.2. Phase 2: preference assessment Prior to the treatment comparison, a single-item duration preference assessment (DeLeon, Iwata, Conners, & Wallace, 1999) was conducted with all participants. Fifteen to twenty items identified by staff familiar with the participants were included in the preference assessment. Prior to the assessment, the therapist familiarized the participant with each item by presenting it briefly (i.e., for 10 s) and modeling appropriate manipulation. At the beginning of each trial, the therapist handed one item to the participant and had no further interaction except to retrieve an item if it was dropped on the floor, at which point it was placed within arm’s reach. During each trial, an observer recorded duration of item engagement, as well as percentage of intervals or frequency of the target behavior. Each item was presented three times in random order for a total of 2 min each. High-preference items were defined as those items with a mean level of item engagement above 83 s, and a mean percentage of intervals with problem behavior below 9% (or below 0.25 rpm; David only). The five most preferred items for each participant were selected for inclusion in the treatment comparison. 2.2.3. Phase 3: treatment comparison 2.2.3.1. Baseline. This phase was identical to the alone condition of the functional analysis, with the exception that data were collected during 30 min sessions. Sessions were conducted in a room void of any leisure or work materials, and a therapist was always present, but no programmed consequences were in effect for the target or other behaviors emitted by the participant throughout these sessions. 2.2.3.2. NCR single-item. The participant had continuous access to one of the 5 high-preference items for the duration of the 30-min session. A different high-preference item was presented during each single-item session, and the order of item selection was randomized across sessions with one exception: if the same item was selected twice consecutively in the single-item condition, then that item was removed from the selection array and a different item was selected at random from the remaining stimuli for use during that session. A therapist was present during all sessions but did not interact with the participant except to retrieve items dropped on the floor. Problem behavior resulted in no social reinforcement. 2.2.3.3. NCR rotate. The participant was given access to one high-preference item at a time for an equivalent time period over the course of the 30-min session. Specifically, each of the five items was presented singly for a period of 6 min. No programmed social interaction was arranged during this condition; however, the therapist approached the participant to remove the current item and place a different high-preference item in front of the participant before the time period elapsed (e.g., at 5 min and 58 s). The therapist avoided interaction with the participant during this time by looking down (e.g., no eye contact), remaining silent during the exchange, and moving away from the participant immediately after the item was
370
R. Rosales et al. / Research in Autism Spectrum Disorders 4 (2010) 367–376
replaced. The order of item presentation was randomized for this condition in the same manner as the NCR single-item condition. No programmed social contingencies were in effect for the occurrence of problem behavior. 2.2.3.4. NCR multiple. Throughout the 30-min session, the participant had simultaneous and continuous access to all five high-preference leisure items identified in the preference assessment. A therapist was present but did not attempt to interrupt item engagement in any way, and no programmed social consequences were arranged for problem behavior.
Fig. 1. Percentage of intervals or rate of problem behavior during the functional analysis.
R. Rosales et al. / Research in Autism Spectrum Disorders 4 (2010) 367–376
371
3. Results 3.1. Phase 1 Results of the functional analysis revealed that each participant’s problem behavior was maintained by automatic reinforcement (see Fig. 1). For both David and Kyle, problem behavior occurred almost exclusively in the alone condition (M = 1.23 rpm for David; M = 30.56% for Kyle). In Rose’s case, saliva play occurred at equally variable rates across all
Fig. 2. Preference assessment results. Solid bars represent mean item engagement, and striped bars represent mean percentage of intervals or rate of problem behavior. Items with an asterisk (*) indicate those selected for use during treatment comparison.
372
R. Rosales et al. / Research in Autism Spectrum Disorders 4 (2010) 367–376
Fig. 3. Percentage of intervals or rate of problem behavior during NCR treatment comparison.
R. Rosales et al. / Research in Autism Spectrum Disorders 4 (2010) 367–376
Fig. 4. Percentage of intervals with item engagement during NCR treatment comparison.
373
374
R. Rosales et al. / Research in Autism Spectrum Disorders 4 (2010) 367–376
conditions of the functional analysis (M = 15.33% during control, 17.32% during alone, 13.66% during demand, and 15.66% during attention). Karen’s face rubbing occurred during all assessment conditions, with the highest rates observed during the alone condition (M = 59%). 3.2. Phase 2 Preference assessment results for each participant are depicted in Fig. 2. The top five items associated with the highest engagement and the lowest problem behavior were selected for inclusion in the treatment comparison. David’s highpreference items included a coin sorter, a bingo game, beads and string, Perfection1, and Lite Brite1. Although two additional items met the definition of high preference (i.e., shape sorter and Chinese checkers), these items were not included based on additional anecdotal observations indicating that engagement with these items was limited. Items included in the treatment analysis for Kyle included beads and string, coin sorter, Lite Brite1, building blocks, and single-insert puzzles. Items selected for Rose were single-insert puzzles, coin sorter, bingo game, Connect 41, and Perfection1. In Karen’s case, items included in NCR sessions were glue and paper, a puzzle book, paint and paper, a Wiggle pen1, and Brain Teaser1 peg games. 3.3. Phase 3 Fig. 3 depicts the levels of problem behavior exhibited during baseline and the three NCR conditions. As seen in the top panel of the figure, David engaged in high rates of head hitting during baseline (i.e., M = 4.32 rpm). Due to the severity of this target behavior, NCR conditions were implemented almost immediately, upon which an immediate decrease on the occurrence of the target behavior was observed. Despite two sessions with high rates of head hitting (e.g., sessions 9 and 12), overall results indicated that NCR was effective in reducing the target behavior. Specifically, the single-item condition was deemed the most effective in decreasing head hitting (M = 0.84 rpm), while results for multiple- and rotate-item conditions were also effective, but resulted in higher overall rates (M = 1.04 and 1.12 rpm, respectively). The second panel depicts results for Rose. During baseline, Rose’s saliva play was observed during an average of 18.93% of all intervals. Upon introduction of the NCR conditions, her levels of saliva play decreased significantly. Data presented in the second panel of Fig. 3 indicate that the single-item condition was also the most effective in decreasing the selected target behavior (M = 7.63%) for Rose. In addition, results for the multiple- and rotate-item conditions were also effective in reducing levels of saliva play below baseline levels (M = 11.71 and 11.20% of intervals, respectively). Results for Kyle are depicted in the third panel of Fig. 3. As can be seen in this figure, Kyle engaged in thumb sucking at high levels during baseline sessions (M = 36.90%), and this decreased significantly upon the introduction of the NCR conditions. In the multiple-item condition, Kyle only engaged in thumb sucking during an average of 3.73% of intervals. This was the lowest level observed of the target behavior for Kyle. Therefore, unlike Rose and David, the multiple-item condition was deemed most effective for Kyle. In addition, similar to the other participants in the study, reduced levels of the target behavior were observed in the rotate- and single-item conditions as well (M = 7.87 and 13.23% of intervals, respectively). Finally, the fourth panel depicts results of the NCR conditions for Karen. Her baseline levels were also significantly higher (M = 54.21% of intervals), than those seen during the NCR conditions. Similar to Kyle, the multiple-item condition also resulted in the lowest occurrence of the target behavior for Karen (M = 2.17% of intervals). This was in comparison to an average of 5.50% of intervals in the rotate-condition, and 19.94% of intervals in the single-item conditions. Fig. 4 depicts item engagement for all participants. These results indicate that for all participants, item engagement was the highest in the multiple-item condition (M = 92.66, 63.93, 79.7 and 96.66% for David, Rose, Kyle, and Karen, respectively). These results are most clearly depicted in David and Kyle’s results (1st and 3rd panel of Fig. 4). Although Rose’s item engagement data appear more variable in the first phase of NCR, it is most apparent that the multiple-item condition resulted in higher levels of item engagement in the second phase of NCR for Rose (2nd panel). 4. Discussion Collectively, results of this study demonstrated that all NCR presentation methods were effective in decreasing problem behavior relative to baseline, which lends further support for the use of NCR as an intervention for automatically maintained problem behavior. For two of the participants in this study (Karen and Kyle), results of the treatment comparison showed superior effects of a multiple-item condition to reduce levels of problem behavior for extended periods of time (i.e., 30 min). For the two other participants in the study (David and Rose), results of the treatment comparison showed superior effects of a single-item condition to reduce levels of the selected target behavior. In addition, all participants demonstrated significantly lower levels of the selected target behavior during all conditions (e.g., single-, rotate-, or multiple-) when compared to baseline sessions, despite which of these was deemed the most effective for each individual. These results differ from those of DeLeon et al. (2000), which demonstrated equal effectiveness of rotate- and multiple-conditions and high levels of problem behavior during the single-item condition. For Karen and Kyle, the multiple-item condition may have been more effective in reducing levels of the target behavior because of the opportunity to engage with and alternate amongst several stimuli during only this condition. Therefore, satiation could be mitigated freely by the participants as opposed to waiting for a therapist to arbitrarily rotate items (i.e., during the rotate-condition), or having access to only one item (i.e., during the single-condition) throughout the 30 min
R. Rosales et al. / Research in Autism Spectrum Disorders 4 (2010) 367–376
375
session. In addition, participants had the choice to consume multiple items at once during this condition. In fact, anecdotal reports indicate that this often occurred for both Karen and Kyle. That is, participants would play with more than one item at a time, or alternate rapidly amongst items available for manipulation. For David and Rose, results are contrary to previous work comparing the effectiveness of NCR conditions. For both of these participants, access to a single highly preferred item resulted in lower levels of the selected target behavior when compared to access to multiple items during rotate- and multiple-item conditions. These results are somewhat surprising given results previously reported by DeLeon et al. (2000). However, these results partially replicated the findings of Lindberg et al. (2003) which reported a single-item condition as being effective for at least one of the participants in the study. As previously mentioned, the single-item condition was effective in reducing the rate or percentage of intervals in which the target behavior occurred for all participants relative to baseline. It is interesting that these two participants did not demonstrate satiation to a single-item, even after repeated presentations during this NCR condition. It is possible that the items selected for these two participants were more highly preferred than those items selected for the other two participants. However, given results of the stimulus preference assessment, this possibility may not be likely because all items were demonstrated to be almost equally preferred. Alternatively, David and Rose may have been more likely to manipulate one item presented at a time because of the potential aversive qualities of having to choose between five highly preferred items, or the possibility that participants viewed these items as ‘‘work’’ tasks rather than leisure activities. Item manipulation results were less surprising across all participants in the study. As previously started, the multipleitem condition resulted in the highest levels of engagement for all participants. These results may have been expected, given that more items were made available to participants during this condition. If participants satiated on one item, they had four other items readily available during the 30-min session. However, despite the fact that the multiple-item condition resulted in highest engagement across all participants, it did not result in the lowest levels of the target problem behavior for 2 of the 4 participants (e.g., David and Rose). Given these results, it is possible that for David and Rose, the multiple-item condition evoked problem behavior due to the potential aversive qualities of choosing between highly preferred items, or the potential similarities of ‘‘work’’ tasks presented at their residential facility and the items used as leisure items in the study. Some limitations of the current study should be noted. First, although we evaluated the effects of NCR during more extended training sessions, no generalization probes were conducted in the participants’ natural setting. This was not possible for at least one of the participants in the study (Rose) because she was transferred to a different residential facility shortly after data collection was completed. However, for the other three participants, it would have been interesting to note whether differences were observed in the effectiveness of NCR when implemented daily in their home setting. Natural environment probes such as these should be conducted in future research. Related to this issue is the fact that all treatment sessions were conducted in an analogue setting. In their review of future research, Carr et al. (2000) suggested the need to evaluate the effects of NCR in naturalistic settings. Although this would result in a less-controlled study, perhaps the use of generalization probes in the participants’ natural setting would be a step in the right direction. Finally, the lack of follow-up data leaves the question of how long the suppressive effects of NCR would maintain for each individual participant. The longer treatment sessions are an improvement from previous research in this area, and the potential for satiation to items after prolonged exposure should always be accounted for in practical settings via frequent stimulus preference assessments. Despite these limitations, the present study extends current research on the effectiveness of NCR. Given the mixed results presented, the need for future studies seems warranted. For example, a comparison of the three conditions over longer period of time may provide a more clear distinction amongst the three conditions presented. Although the findings of this study were mixed, they may present valuable practical utility for potential generalization settings. For example, if multiple items are not available for use in treatment facilities due to lack of resources or the rotation of items is not feasible due to shortages or unavailability of staff, presenting individuals with a single item to engage with up to 30-min may help decrease levels of problem behavior maintained by automatic reinforcement. The implementation of NCR as a treatment intervention for automatically maintained problem behavior also presents a straightforward method for intervening upon problem behavior, which may be easy for staff in residential settings to implement. Acknowledgements We thank the staff at Choate Mental Health and Developmental Center in Anna, IL and School for the Hearing Impaired in Marion, IL for allowing us to recruit participants and collect data at their facility. We also thank Nicole Marlow, Amanda Ryan, Shannon Freeman, Seth Clark, Jessica Trahan, and Maya Shales for assistance with reliability data. References Ahearn, W. H., Clark, K. M., DeBar, R., & Florentino, C. (2005). On the role of preference in response competition. Journal of Applied Behavior Analysis, 38, 247–250. Carr, J. E., Coriaty, S., Wilder, D. A., Gaunt, B. T., Dozier, C. L., Britton, L. N., et al. (2000). A review of ‘‘noncontingent’’ reinforcement as treatment for the aberrant behavior of individuals with developmental disabilities. Research in Developmental Disabilities, 21, 377–391. DeLeon, I. G., Anders, B. M., Rodriguez-Catter, V., & Neidert, P. L. (2000). The effects of noncontingent access to single- versus multiple-stimulus sets on selfinjurious behavior. Journal of Applied Behavior Analysis, 33, 623–626. DeLeon, I. G., Iwata, B. A., Conners, J., & Wallace, M. D. (1999). Examination of ambiguous stimulus preferences with duration-based measures. Journal of Applied Behavior Analysis, 32, 111–114. Favell, J. E., McGimsey, J. F., & Schell, R. M. (1982). Treatment of self-injury by providing alternate sensory activities. Analysis & Intervention in Developmental Disabilities, 2, 83–104.
376
R. Rosales et al. / Research in Autism Spectrum Disorders 4 (2010) 367–376
Fisher, W. W., Lindauer, S. E., Alterson, C. J., & Thompson, R. H. (1998). Assessment and treatment of destructive behavior maintained by stereotypic object manipulation. Journal of Applied Behavior Analysis, 31, 513–527. Iwata, B. A., Dorsey, M. F., Slifer, K. J., Bauman, K. E., & Richman, G. S. (1994). Toward a functional analysis of self-injury. Journal of Applied Behavior Analysis, 27, 197– 209 (Reprinted from Analysis and Intervention in Developmental Disabilities, 2, 3–20, 1982). Lindberg, J. S., Iwata, B. A., Roscoe, E. M., Worsdell, A. S., & Hanley, G. P. (2003). Treatment efficacy of noncontingent reinforcement during brief and extended application. Journal of Applied Behavior Analysis, 36, 1–19. Piazza, C. C., Adelinis, J. D., Hanley, G. P., Goh, H. L., & Delia, M. D. (2000). An evaluation of the effects of matched stimuli on behaviors maintained by automatic reinforcement. Journal of Applied Behavior Analysis, 33, 13–27. Shore, B. A., Iwata, B. A., Khang, S. W., & Smith, R. G. (1997). An analysis of reinforce substitutability using object manipulation and self-injury as competing responses. Journal of Applied Behavior Analysis, 30, 21–41.