Operational specificity: Implications for field-based replications

OPERATIONAL SPECIFICITY: IMPLICATIONS FIELD-BASED REPLICATIONS KRISTINE State of Hawaii,

RENEE

State Implementation

FOR

DERER’ Grant,

Honolulu.

Hawaii

Summary-This paper discusses problems with field-based replications of behavioral interventions. Major issues identified include operational specificity, mislabelling of procedures, relative contributions of components in multi-faceted treatment packages, and impediments to systematic replication. The paper concludes with a call for increased operational specificity in the reporting of both baseline and intervention procedures on the part of researchers and clinicians responsible for dissemination.

In an issue of The Behavior Analyst (1981), Lute et al. discussed the importance of operational specificity for precise communication and comparison of response-cost interventions. Their survey revealed considerable disagreement concerning the specific operations involved in response-cost which variously included contingent reinforcement loss, contingent effort and response requirement. Lute and his associates concluded that, in the absence of operational specificity, the utility of response-cost as a descriptor was questionable. The demand for operational specificity should not be limited to response-cost alone. Equally pressing is the need for precise definitions and accurate descriptions of other procedures commonly found in the behavioral intervention literature. A recent extensive review of the literature in the area of behavioral interventions with severely handicapped children found considerable disparity among categorical intervention labels when the specific operations involved were compared (Derer and Hanashiro, 1982). In some cases, the actual

procedures employed had nothing in common with the label being used as a referrent. For example, contingent physical restraint has been variously referred to as time-out (hlurphy, Ruprecht and Nunes, 1979), ignoring (Lucero, Frieman, Spoering and Fehrenbacher, 1976), and overcorrection (Savie and Dickie, 1979). Ignoring and extinction have been used interchangeably to label time-out procedures (Myers, 1975), and contingent faradic shock has been labeled reinforcement (Luckey, Watson and Musick, 1968). The purpose of this paper is to identify some labelling confusions and suggest problems which result from the widespread misuse and/or confusion of labels. This paper does not intend to evaluate the ethical merit of the procedures identified nor does it claim an exhaustive review of available literature. Rather, studies were selected because they displayed one or more characteristics that contribute to problems in procedural definitions. These problems will be examined as they relate to replication efforts with an emphasis placed

*This work was completed while the author was affiliated with the University of Hawati and has supported in part by Grant No. GO@790-160 from the U.S. Department of Education, Special Education Programs, Division of Innovation and Development. awarded to the University of Hawaii Departments of Psychology and Special Education. The author wishes to express appreciation to Drs. Ian Evans, Joel Fischer, Linda )lcCormick and Luanna Voeltz for their helpful re\ie\v of this manuscript. Requests for reprints should be addressed Avenue. Honolulu, Hawaii 96815.

to Kristine

Renee Derer.

9

Hauaii

State

Implementation

Grant,

3430 Leahi

I II

KRISTINE

RENEE

on the more complicated, multifaceted intervention techniques that recently have begun appearing in the literature.

DEFINING

INTER&NTIO?GS

Time out \Iurphy and his associates (1979) reported success in decreasing the face slapping behavior of a profoundly retarded 17-year-old boy through the use of a “time-out” procedure which consisted of the boy’s being strapped (arms, legs and waist) to a chair. In the final phase of the study, the authors added a blindfold to the procedure. Repp and Deitz (1974) used a brief physical restraint procedure, to reduce the aggressive labeled time-out, ‘behavior of a I?-year-old, severely retarded boy. Ausman, 3all and Alexander (1974), in an attempt to reduce pica behavior, reported success through the use of a procedure in vvhich the child vvas placed in a “time-out” helmet. The helmet was constructed of light-gauge sheet metal with a ventilator on top and a plexiglass window in front. It was pop-riveted to a jacket that vvas laced up the back with a locking belt. In another study, Perline and Levinsky (1968) combined a procedure labeled time-out with a response-cost token economy. Time-out consisted of buckling the child to a restraint post for 5-15 min contingent upon aggressive responding. Perline and Levinsky do, however, include the term restraint when discussing their procedure. The components of the time-out procedures just cited are at variance with the definition offered by Ferster and Skinner (1957) in which time-out was described as contingent removal from the reinforcing situation, removal of stimulus materials and/or persons previously correlated with reinforcement, or presentation of a stimulus previously correlated with no reinforcement. The central principle behind timeout is that al/ reinforcement for all behavior (not just the target response) is removed, but no behaviors are prevented from occurring. The prevention of the occurrence of a response

DERER

through physical holding or mechanical means (e.g., splints. restraining jacket) is a restraint procedure and should be properly identified in the jtudy. Extinction Extinction is another label that has been indiscriminately used in the intervention literature. During extinction, a behavior is allowed to occur in the absence of reinforce(Skinner, ment for that behavior 1953). treatments incorporating extinction During reinforcement is still available procedures, for non-target behaviors. This reinforcement availability serves to differentiate extinction (the non-revvarding of a particular behavior of an organism) from time-out (the non-rewarding of all behavior of an organism). The differences in the two procedures can be demonstrated using the following example: A teacher is conducting a musical instrument activity with five children. One child begins to kick. If the teacher ignores the kicking and continues to make reinforcement available for playing the instruments, he or she is using estinction. If the teacher takes away the instrument and turns the child around so that the child’s back is facing the group while the group continues the activity, he or she is using timeout. Lucero and his associates (1976) designated a procedure as ignoring (i.e., extinction) in which the therapist physically held the child’s arms for 15 set contingent upon head slapping. Contingent removal of food and contingent withdrawal of the socially reinforcing therapist were also referred to as ignoring. Lovaas and Simmons (1969) labeled a 90-min period of total social isolation as extinction. IMyers (1975) referred to a procedure involving time-out from social reinforcement as extinction. Although extinction may have been a portion of the treatment for each of the studies outlined, all three studies included procedures that utilize restraint and/or time-out contingent upon the emission of a behavior. Therefore, the categorical label applied to the treatment-extinction-is inap-

OPERATIONAL

propriate and misleading, and readers would have been more accurately informed had the authors used instead the label time-out or restraint. Overcorrection Overcorrection has been the victim of similar definitional discrepancies. Savie and Dickie (1979) described a positive practice overcorrection procedure in which the child was removed to a corner of the room where her hands were taped, palms down, to the top of a desk for 3 min. Marholin and Townsend (1978) used an overcorrection procedure consisting of a verbal instruction to stop engaging in the behavior, removal of the child from the ongoing activity, and 3 min of forced hands down: that is, the therapist held the child’s hands immobile on a table in front of the child. Marholin and Townsend referred to their procedure as restraint in the Method section, but the balance of the article interchanged the terms restraint and overcorrection. The intent of positive practice overcorrection is to teach a positive behavior while punishing a negative behavior (Foxx and Azrin, 1973). I would seriously question the inherent value for a severely handicapped child in learning to keep hands palms down on a desk. The procedures outlined by Savie and Dickie and by Marholin and Townsend do not refIect the intent of overcorrection but more closely resemble restraint with perhaps a time-out component evident in the removal from social reinforcement. MULTIFACETED

INTERVENTIONS

It might be argued that a few mislabeled procedures are not sufficient cause for concern, that close scrutiny of the articles in question would prevent any misinterpretation by other clinicians. Such an argument might be valid were the problem confined to those studies utilizing a single intervention technique. However, the growing literature on interventions with severe excess behaviors has shown an increasing tendency toward the use of multifaceted interventions. With studies employing

SPECIFICITY

11

multicomponent treatment packages, specificity of treatment procedures becomes especially salient. Studies incorporating verbal punishment with a second procedure designed to decrease a behavior are perhaps the most common. Derer and Hanashiro (1982) identified 49 out of a potential 133 studies employing behavioral interventions with severely handicapped children in which a verbal punishment component was an integral part of the treatment package. In one study (Doleys and Wells, 1978), a positive practice overcorrection procedure was used to decrease the head weaving response of an 8-year-old retarded boy. After initiating the overcorrection procedure, the authors added a loud reprimand component. In reporting their data, Doleys and Wells did not differentiate between performance under overcorrection and performance under overcorrection plus verbal punishment. The omission suggests that either the overcorrection procedure used was not successful in bringing the behavior under control or that the verbal procedure was added in an attempt to develop control over the behavior using a less aversive technique. In either case, the failure to report differential performance prevents the determination of the relative contribution of each procedure. The failure to report outcome is easy to identify; the number of studies failing to report the use of a verbal procedure accompanying a second punishment procedure is impossible to ascertain. Although some researchers may not consider the verbal portion a significant aspect of the treatment package, a careful review of the child’s learning history may reveal that negative verbal comments do serve as a conditioned aversive stimulus and may account for some measure of program success. Other studies have emphasized a particular component of a treatment package to the virtual exclusion of other contributing components. Chiang, Iwata and Dorsey (1979), in an attempt to decrease the disruptive behavior on a school bus of a lo-year-old retarded boy, stated in their discussion that “the combined

12

KRISTI.UE

RESEE

social and token reinforcement procedure was effective in eliminating the student’s disruptive bus riding behavior” (p. 105). The procedure section labeled the technique “token reinforcement”, and a similar reference was made in the title of the study. Closer examination, hovvev-et-, revealed that the study also included verbal reprimands for disruptive behavior, loss of tokens for disruptive behavior (responsecost), and social reinforcement contingent upon lapse time (DRO). Obviously, a clinician faced vvith a similar problem who implements a token program without the added components may find the procedure either ineffective or not cost efficient vvith respect to treatment duration. In a study designed to investigate the use of DRO (differential reinforcement of other behaviors), Repp and Deitz (1974) reported success in decreasing the excess behaviors of four severely handicapped children. DRO was variously accompanied by restraint, responsecost and verbal punishment. Unfortunately, their design did not allow for a comparison of the various components, and no conclusions could be drawn regarding the relative worth of each component. In another study employing DRO, O’Neill et a/. (1979) emphasized the success achieved using the DRO procedure to reduce rumination. The study attempted to compare DRO with a variety of punishment procedures including contingent application of lemon juice time-out, and slapping. The design used in the study most closely resembles a successive treatments design, with DRO at times appearing alone and at other times paired with one of the aversive procedures. The successive treatments design does not allow for the comparison of different treatments nor for the analysis of contributing elements (Browning and Stover, 1971). Researchers attempting to address issues or relative contribution or differential effectiveness should employ designs that allow such comparisons. (See, for example, strip/construction designs in Jayaratne and Levy, 1979; multiple schedule, multi-element and simultaneous treatments designs in Kratoch\vill, 1978.)

DERER

I~IPLIC.1TIONS

FOR REPLIC.ATION

V’oeltz and Evans (1982) have suggssted that interventions described in the literature are rarely. perfectly replicable. They attribute this lack of replicability to the brevity of treatment descriptions resulting in (1) the failure adequately to describe the essential treatment components, (2) the failure to identify all of the relevant treatment components, or (3) the failure to specify the conditions under which the treatment vvould be expected to succeed. Voeltz and Evans asserted that vvithout a concern for treatment or “educational” integrity-whether the intervention occurred as specified in the plan-replication efforts may not be successful. Billingsley, White and Munson (1980) have also emphasized the need to include data on procedural reliability in the dissemination of behavioral research. Yet, in a survey of research articles appearing in the Journal of‘ Applied Behavior .4nalysis during the period 196881980, Peterson, Homer and Wonderlich (198 1) found that a majority of articles failed to report reliability of treatment implementation and that appros. 3Oqo of the articles failed to prov.ide operational definitions of the treatment components. The logical result of inadequate treatment descriptions and the absence of data to support the accuracy of implementing the stated treatment is a series of irrelevant replication efforts vvhich fail to assess the power of a given intervention to effect change. Ln their discussion of dissemination of research data, Jayaratne and Levy (1979) emphasized the importance and necessity of providing specific and accurate information on the intervention procedures responsible for the behavior change. Others have recommended the documentation of veridicality of the independent v,ariable through procedures comparable to those used to assess observer agreement (Billingslsy er al., 1980; Peterson et al., 1981; Voeltz and Evans, 1982). Such integrity in reporting intervention procedures vrould make available information on

OPER.ATIONAL

the quality (i.e., accuracy), quantity (i.e., precise number/duration of treatments), and exact operations deemed responsible for the observed behavior change. For the individual clinician, this more rigorous intervention description would assist in the evaluation of a given technique for potential use with a particular client. Furthermore, the field of applied behavior analysis would benefit by the development of a cumulative body of knowledge based on reliably replicated treatment procedures. Once the validity of specific treatment techniques has been established, potential guidelines for generalized application could be developed. As Hersen and Barlow (1976) stated, the purpose of systematic replication is to determine the parameters beyond which a treatment will not be efficient and/or effective. Analysis of these parameters requires a carefully delineated and adequately documented description of the treatment being employed. Such descriptions, however, should not be limited to the treatment in isolation but should report the prevailing conditions prior to treatment (i.e., baseline). The ecology of a classroom or family setting is not the highly controlled, sterile environment of a laboratory. An intervention program implemented in a classroom setting will invariably result in changes to an already existing and active environment (e.g., skill programs, reinforcement schedules, behavior programs, adult behavior, physical setting, etc.). A 5-min DRO interval instituted following a long baseline in which no reinforcement is available could be expected to produce results vastly different from a similar DRO schedule following a baseline in which reinforcement is available on a continuous basis. It is imperative, if replications are to have meaningful application in natural settings, that the prevailing conditions under which baseline information is gathered be adequately conveyed in intervention studies. Where treatment descriptions are concerned, more effort needs to be employed in detailing the precise operations involved. With an increasing tendency toward the use of multifaceted treatment

13

SPECIFICITY

the possibility of inadvertently packages, deleting a seemingly irrelevant component may present a real problem for both researchers and clinicians. Such a deletion would be particularly salient Lvhen the component is a major contributor to the speed or magnitude of change. Potential problems in field-based replications would be diminished if researchers in this area were to determine the therapeutic contribution of each component of the treatment. REFERESCES Ausman J., Ball T. S. and Alexander D. (1971) Behavior therapy of pica with a profoundly retarded adolescent. Menr. Relard. 12, 16-18. Billingsley F., VVhite 0. R. and Munson R. (1980) Procedural reliability: A rationale and an example. Behavl Assess. 2. 229-241. Browning R. >I. and Stover D. 0. (1971) Behovror.Modification in Child Treoimen!: Approach. Aldine-Atherton.

An Experlmenral

and Clinical

Chicago. Chiang S. J., lwata B. A. and Dorsey 51. F. (1979) Elimination of disruptive bus riding behavior via token reinforcement on a “distance-based” schedule. Educarion

Derer

and

Treatmen

of Children

2, 101-109.

K. R. and

Hanashiro R. (1982) Behoviorai Intervenlion Index. University of Hawaii Behavioral Systems Intervention Project (Technical Report No. 9). Honolulu. Doleys D. Xl. and Wells K. C. (1978) Situation eeneralitv of over-corrective functional movement training. Psycho/. Rep. 43. 759-762.

Ferster

C.

B. and Skinner B. F. (1957) Schedules of Appleton-Century-Crofts, New York. Fovv R. \I. and Azrin N. H. (1973) The elimination of autistic self-stimulatory behaviors by overcorrection. Reinforcemenl.

J. appl.

Hersen

Behav.

hl. and

Anal.

Barlow

mental Change.

Designs:

Behavior

Analyst.

6, I-14.

D. H. (1976) Single Case Experi-

Straregies

for

Studying

Behavror

Pergamon Press, New York. Jayaratne S. and Levy R. L. (1979) Empirical Clinical Practice. Columbia University Press, New York. Kratochwill T. R. (1978) Single Subject Research Srraregles for Evaluating Change. Academic Press, Neu York. Lovaas 0. I. and Simmons J. Q. (1969) Manipulation of self-destruction in three retarded children. J. uppl. BehaL,. rlnal. 2, 143-157. Lute S. C., Christian W. P., Lipsker L. and Hall R. 1’. (1981) Response cost: A case for specificity. The 1, 75-80.

Lucero IV. J.. Frieman J.. Spoering K. and Fehrenbacher J. (1976) Comparison of three procedures in reducing self-injurious behavior. Am. J. ment. Defic. 80, 548-554. Luckey R. E.. VVatson C. >l. and Xlusick J. K. (1968) Aversive conditioning as a means of inhibiting vomiting and rumination. .-lm. J. ment. Defic. 73, 139-142. hlarholin D. and Townsend N. Xl. (1978) An experimental analysis of side effects and response maintenance of a

II

KRISTINE

RENEE

modified o\ercorrection procedure: The case of the persistent twiddler. Behav. Ther. 9, 383-390. Alurphy R. J.. Ruprecht X4. and Nunes D. L. (1979) Elimination of self-injurious behavior in a profoundly retarded adolescent using intermittent time-out. restraint, and blindfold procedures. A4ESPH Rev. 1, 334-315. \lyers D. ‘A’. (1975) E.xinction. DRO, and response cost procedures for elimmating self-injurious behavior: .A case study. Behav. Res. Ther. 13. 189-191. O’Neill P. &l., White J. L., King C. R. and Carek D. J. (19i9) Controlling childhood rumination through differential reinforcement of other behavior. Behav. .Lfod. 3, 355-372. Perline I. H. and Levinsky D. (1968) Controlling maladaptibe classroom behavior in the sr\erely retarded. .-lm. J. ment. De/k. 73, 74-78. Peterson L.. Homer A. L. and \6’onderlich S. A. (1981)

DERER

.Llerhodoloy_v in applied behavior analysis II: Independent variable rellabilrry. Paper presented at the .Assoc~ation for BshaLior Analysis. Seventh Annual Convsn[Ion. hlilwaukee. LVisconsln. Repp A. C. and Deitr S. &I. (19i-1) Reducing aggressibe and self-injurious behawor of lnstitutionallzed retarded children through reinforcement of other behaviors. J. appl. Behav. Anal. 7. 313-325. Savie P. and Dickie R. F. (1979) Overcorrection of topographically dissimilar autistic behaviors. Educarlon and Treatment of Children, 2, 177-184. Skinner B. J. (1953) Science and Human Behavior. The Free Press, New York. Voeltz L. Xl. and Evans I. hl. (1982) Educational validity: Procrdures to evaluate outcomes in programs for severe]) handicapped learners. Journal of ihe Association for rhe Severely Handicapped.