Linear responses

Behavioural Processes 44 (1998) 19 – 43

Linear responses Larry Hawkes, Charles P. Shimp * Psychology Department, Uni6ersity of Utah, 390 S. 1530 E. Rm 502, Salt Lake City, UT 84112 -0251, USA Received 27 August 1997; received in revised form 5 May 1998; accepted 15 May 1998

Abstract Four experiments established complex choice responses a few seconds in duration. A response was reinforced if it sufficiently approximated a ‘target’ response. The two target responses in each experiment were linear in the sense that they involved either constant rates, or constant rates of change, in component key pecking by pigeons. For example, in Experiment 4, one target response consisted of the linearly increasing pattern of 0, 1, 2, and 3 pecks per s in four successive seconds, and the other response consisted of 3, 2, 1, and 0 pecks per s. Contingencies were ‘tolerant’ in the sense that they permitted variability across different reinforced exemplars of a response. Responses approximated target responses, at least crudely in terms of overall cumulative records, and sometimes even quantitatively in terms of within-trial, local sequential organization. In this sense, the ‘contents’ of at least some choice responses, like their ‘envelopes’, can be shaped. That is, patterning within their boundaries, in addition to their relative and absolute durations, can be shaped. Some responses may have emerged from variability inherent in component pecking occurring at a constant probability, thereby demonstrating a few cases where the relation between molecular and molar analyses fully legitimizes a molar analysis. © 1998 Published by Elsevier Science B.V. All rights reserved. Keywords: Molar and molecular analyses

1. Introduction Complex behavior of nonhuman animals challenges both method and theory because of difficulties in establishing and in conceptualizing sequential organization (Lashley, 1951; Anger, 1956; Jenkins, 1970; Shimp, 1976a,b; Hulse, 1978; * Corresponding author. Tel.: + 1 801 5818483; fax: +1 801 5815841; e-mail: [email protected]

Marr, 1979; Thompson and Zeiler, 1986; Iversen, 1991; Terrace, 1991, 1993; Shimp et al., 1994; Terrace et al., 1996). That is, sequential organization poses two interrelated challenges. First, to determine which arbitrary sequential patterns can or cannot be established by the direct shaping effects of reinforcement requires complex empirical methods. Second, in addition to this methodological challenge, there is the conceptual challenge to articulate the link between simple and complex

0376-6357/98/$ - see front matter © 1998 Published by Elsevier Science B.V. All rights reserved. PII S0376-6357(98)00029-1

20

L. Hawkes, C.P. Shimp / Beha6ioural Processes 44 (1998) 19–43

behaviors: what is the process by which a complex reinforced response is constructed from its component behaviors? The present experiments addressed both challenges, methodological and conceptual, posed by the sequential organization of complex responses. We developed a method for the study of complex choice responses. Our method hinged on the generic or categorical nature of a reinforced response, which permits the shaping of some of its properties that vary from exemplar to exemplar, such as topography (Smith, 1974; Pear et al., 1982; Pear, 1985; Palya, 1992; Stokes and Balsam, 1991), duration (Anger, 1956; Shimp, 1968; Staddon, 1968; Shimp, 1975), force (Notterman and Mintz, 1965), and response location (Herrnstein, 1961a). It is often convenient simply to ignore difficult problems that result from the categorical nature of a response. Much of the time in the study of, say, preference, ‘a response’ is conveniently defined in terms of a simple binary switch, and the categorical nature of the response is thereby obscured. The methodological and conceptual tractability thus gained has presumably contributed to progress in the analysis of preference (for example, see Bush and Mosteller (1955) p. 330, and Estes (1959) p. 399). This simplification has not gone uncriticized, however, because by analogy with other sciences, one might expect conceptual ambiguities and other problems to occur in the development of behavioral science if its basic response units were misinterpreted. Various proposals therefore have been made on how to deal with the problem of the nature of the behavioral unit, or ‘response’. One approach, deriving from an ecological perspective, recommends that progress in understanding behavior would be facilitated by the development of a formal system emphasizing particularistic details of specific responses (Jacobs et al., 1988). A second approach has been to turn to rich descriptions of specific features (or the ‘fine grain’) of behavior (Smith, 1974; Pear, 1985; Palya, 1992). A third approach has been to make temporal features of reinforced behaviors an explicit part of experimental tasks, in order to examine how various principles and empirical phenomena obtained with simple responses do or

do not generalize to more complex, temporally extended, responses (Shimp, 1968; Staddon, 1968; Shimp, 1975, 1976a,b). We define this approach to involve the explicit reinforcement of sequential organization, rather than the setting up of a task such as a Fixed Interval schedule and seeing what organization, such as a Fixed Interval ‘scallop’ emerges (Schneider, 1969; Zeiler, 1977). While this latter, indirect approach has proven utility, it seems appropriate also to explore a more direct approach, which was the present experiment’s purpose (Hawkes, 1977)). We made reinforcement contingent on two relatively complex ‘target’ responses having specified sequential organization, and we then examined the relation between the structure of a target response and that of the responses the task maintained. It is a relatively simple matter to arrange a contingency based on response force or response location. A contingency inevitably becomes more complex, however, when it involves behavioral organization over time. We simplified the problem and asked, ‘‘How can one take a temporal block of key-pecking behavior and decide whether that block, taken as a whole, sufficiently approximates a target response to qualify for reinforcement?’’ One way to answer this question is to use a contingency that may be described as ‘tolerant’, in the sense that it arranges reinforcement for a complex target and includes a criterion that permits reinforcement of behaviors similar to, but not identical to, the target. Such a contingency can maintain behavior even though the prototype is so complex that the sequential organization of many responses does not perfectly conform to it. Hawkes and Shimp (1975) used this latter approach and adapted a chi-square statistic (Hays, 1988; Hogg and Craig, 1970) to define a criterion disparity between a complex response and a target response, such that if the disparity was less than, or greater than, the criterion disparity, a complex response was, or was not, reinforced, respectively. There was no ‘defining feature’ of reinforced responses, since reinforcement did not depend on any particular feature or combination of features of a response. (It is worth noting in passing that the reinforced response was, therefore, not only a ‘generic’ concept, as explained by Skinner (1935),

L. Hawkes, C.P. Shimp / Beha6ioural Processes 44 (1998) 19–43

but was also a ‘fuzzy’ concept, as in Wittgenstein (1953) and Herrnstein and Loveland (1964).) Reinforcement depended instead on the overall extent to which a response sufficiently conformed to the target or prototype response. Target responses were selected to be special and tractable cases of the general category of any type of increasing or decreasing rates of key pecking within discrete blocks of time a few seconds in duration. In particular, target responses consisted of key pecking at rates that changed at constant rates, and which we therefore call ‘linear responses’. Hawkes and Shimp (1975) determined that such a contingency could indeed establish complex linear responses: over successive trials birds produced enough complex responses that adequately approximated the target response so that responding was maintained. The present experiment’s two goals corresponded to the two challenges posed by the sequential organization of complex responses. The first goal was to develop a method by which complex responses could be established in a choice setting. We therefore immediately confronted the question of which complex responses we should try to establish. The range of possible complex responses seems virtually limitless and therefore quite daunting, so we chose to begin by simplifying the problem. We chose to use very simple cases of linear responses. This choice in favor of simplicity had the additional advantage that we already knew from Hawkes and Shimp (1975) that at least one special case of the general category of linear responses was actually workable, at least in a non-choice setting. We therefore adapted the approach of Hawkes and Shimp (1975) described above, where complex responses were required to satisfy an overall, or ‘holistic’, criterion in order to be reinforced, and added a second target response on a random half of the trials, so that an organism had to ‘choose’ which complex response to emit on any particular trial. (A reader should note that the meaning of ‘choice’ here is different from the norm. Since responses were temporally extended here, there was no immediately observable behavior, such as a peck to a left or right key, corresponding to a choice.)

21

The second goal was to make progress on the theoretical problem of how to account for the overall structure that emerges from the direct reinforcement of behavioral patterns. In general, this is a forbidding challenge, but progress may be possible on appropriately simplified special cases. In one such case, a ‘least-frequent’ contingency reinforced behavior (either inter-response times, see Blough (1966), or sequences of choices, see Shimp (1967)) that had occurred least often in the recent past. In these cases, overall structure appeared so simply related to component behaviors, pecks in Blough (1966) and choices in Shimp (1967), that overall structure may have been little more than the variability inherent in simple random processes, involving essentially constant probabilities of component behaviors, operating over time. This hope, in fact, may be part of the rationale for a distinction between ‘molar’ and ‘molecular’ ‘levels of analysis’ of behavior. Molar behavioral analyses can safely ignore, or average over, events at the molecular level, provided that molecular events occur with fixed and independent probabilities. The most common example of this approach is to compute a mean rate of some response, say of key pecking, lever pressing, or button pressing, on the grounds that local or ‘fine grain’ events such as inter-response times, inter-changeover times, runs of choices, and so on, merely reflect the variability deriving from a constant response probability operating over time (Herrnstein, 1961b; Skinner, 1966). Alternatively, however, if complex or overall performance does not emerge simply from random variability in constant probabilities of component behaviors, then the justification for ignoring local temporal organization would be more problematic. Viewed in this light, it is important to know when local behaviors reflect an overall constant response probability, in order to understand the relation between local and overall behaviors (Donahoe et al., 1997; see also, Machado, 1997). Thus, our second goal was to determine if any complex responses emerge simply from random variability determined by a constant probability of component behaviors. Such a link between simple and complex behaviors would: (1) conform

22

L. Hawkes, C.P. Shimp / Beha6ioural Processes 44 (1998) 19–43

to the gratifyingly simple case well known as the Bernoulli-trials stochastic process (Feller, 1950), of which coin tossing is the most familiar example; and (2) provide support for a molar analysis.

2. General method

2.1. Animals Six male white carneaux (Columba li6ia) pigeons were maintained at 80 – 90% of their freefeeding weights, with supplemental grain provided as needed in the home cages after each session. Birds were housed individually in standard pigeon cages with free access to water and grit in the colony room. Experimental sessions occurred at approximately the same time 5 – 7 days a week. The animals had a variety of experimental histories: none was experimentally naive.

2.2. Apparatus The tasks used the left key in each of six standard two- or three-key Lehigh Valley Electronics pigeon chambers. The chambers were interfaced to a Digital Equipment Corporation PDP-8e computer which arranged all experimental contingencies and recorded the data on magnetic tape for subsequent analysis. White noise helped to mask extraneous noises from outside the chambers.

2.3. Experimental procedure The contingencies were complex in detail, but were simple in general outline. A reinforcer was delivered after the occurrence of an appropriate pattern of key pecking lasting just a few seconds. A reinforcer was not contingent upon the occurrence or non-occurrence of any single key peck at any particular moment. Rather, the computer’s decision to deliver a reinforcer was based on the degree to which an entire temporal pattern of key pecks, or complex response, conformed to a required pattern, or ‘target’ response. The computer waited until the entire complex response was completed to decide whether to deliver a reinforcer,

and then did so only if the temporal pattern of key pecking throughout that period conformed sufficiently closely to the target response. The degree to which any complex response conformed to the target response was measured by an index of overall goodness-of-fit. The details of this method are described next.

2.3.1. Trial presentation Complex responses were observed during brief trials, the duration of which varied over experiments. During a trial, the left key was lit white and the house light was on. If a trial ended with the 2.5-s reinforcer, a 2.5-s blackout preceded the subsequent illumination of the key at the beginning of the next trial. Otherwise, the blackout was 5.0 s, so that a constant 5.0-s interval separated the end of each trial from the beginning of the next. Both the keylight and the houselight were turned off for 0.1 s after every key peck. Pecks during this blackout were not counted. All sessions consisted of 300 trials. 2.3.2. Choice contingency After each reinforcement, the computer randomly selected one of two target responses to be reinforced next. The selected target response on a trial remained in effect until reinforcement was collected. This arrangement, a type of correction procedure, ensured that the obtained relative frequency of reinforcement for a target response differed from the programmed probability of 0.5 only by binomial probabilities (see also Shimp, 1966; Stubbs and Pliskoff, 1969). 2.3.3. Target responses Target responses may be conveniently summarized by functions which specify the target number of key pecks per 1-s subinterval. In different experiments, the target responses consisted of increasing, constant, or decreasing key-pecking rates. Whenever the target function’s key pecking rate changed over time, it did so at a constant rate, so that the complex responses were in this sense linear. An important qualification about our terms needs to be made explicit. The rate of key-pecking

L. Hawkes, C.P. Shimp / Beha6ioural Processes 44 (1998) 19–43

in target responses changed only in terms of 1-s bins, so our term ‘rate’ corresponds only loosely to a rate defined over continuous time, especially since there were only a few intervals in a target response.

2.3.4. Goodness-of-fit Reinforcement on each trial was contingent on a measure of the overall extent to which a complex response a bird produced conformed to a target response. The overall extent to which a complex response on a given trial deviated from the target response on that trial was measured by a sum of squared deviations. After each 1-s interval, the computer calculated the difference between the number of key pecks required by the target response and the number of pecks actually observed in that interval. The computer then squared this difference. At the end of a trial, the computer obtained the sum of the squared deviations, one squared deviation for each interval, to obtain a measure indicating how closely a bird’s overall performance on that trial conformed to the target response. This measure can be expressed symbolically as n

D= % ( fi −oi )2 i=1

where oi is the observed frequency of key pecks in the ith 1-s interval, fi is the corresponding required frequency from the target response (described below for each experiment) and n is the duration of a target response in seconds. The larger the value of D on a trial, the poorer the match between obtained response and target response on that trial. A value of D equal to zero on a trial defined a perfect match between obtained and target responses on that trial. The criterion was an integer, C ]0, such that a reinforcer was presented at the end of a trial if, and only if, D 5 C on that trial: if D \C, no reinforcer was presented on that trial. A criterion C\ 0 enabled the reinforcement of complex responses not perfectly conforming to the target response. It was discovered early in pretraining that different target responses required different criteria, so throughout all four experiments,

23

each target had its own criterion. The criterion for each target was set at the beginning of each experiment to a value sufficiently large so that initially virtually any complex response was reinforced. The criterion was then reduced over days as it was judged that a bird’s responses more closely approximated the target response. The experimenter made this daily judgment based on the number of reinforcers the birds earned. Initially, criteria were lowered following a day on which more than 30 reinforcements were obtained and raised following a day on which fewer than 15 reinforcements were obtained. Ultimately, the experimenter attempted to lower a target’s criterion to a point below which the obtained reinforcements per hour were insufficient to maintain responding. That is to say, the experimenter attempted to set a criterion at a value one greater than the value at which one or more subjects would stop pecking. Decisions about the terminal values of a criterion were therefore not entirely standardized. We expect that different methods of setting criteria might produce different results; indeed, complex responses may in general depend on all the parameters of the contingencies in terms of which they are defined (see Section 7). In any case, an experiment usually was not terminated until all birds seemed to be at their terminal value or values. Terminal values of C are shown separately below for each target in each experiment, as are the number of reinforcements received over the last two sessions.

2.4. Computer simulation methodology As noted above, one possible relation between complex responses and component key pecks is simply that the former emerge from random variability in constant-probability pecking, and we used a computer-simulation technique to investigate this possibility. We generated simulated sessions consisting of sequences of trials arranged as described above. The methodology for these simulated sessions was as follows. On each trial, the computer generated ‘key pecks’ according to a Bernoulli-trials process, specifically,

24

L. Hawkes, C.P. Shimp / Beha6ioural Processes 44 (1998) 19–43

with a constant probability, P, of producing a key peck every 0.25-s. Thus, the computer can be thought of as tossing a coin every 0.25 s, with probability of a ‘peck’ corresponding to the probability of heads, equal to P. The value of P was determined in the following way. When both target responses required the same total number of key pecks (Experiments 3 and 4), P was set to the value that would produce an average number of key pecks per trial equal to the number required by these target responses. These values of P were 0.250 and 0.375 for Experiments 3 and 4, respectively. Our computational resources were limited, and we sought only to find initial, ‘first-order’ approximations, so for the experiments which required different numbers of key pecks for the two target responses (Experiments 1 and 2), P was set to a value which would produce an average number of key pecks per trial equal to the average number of pecks actually obtained in these experiments by an arbitrarily selected one of the six birds in the choice contingency. The data of bird 6 were used in the simulation of Experiment 1 with P= 0.444, and the data of bird 4 were used in the simulation of Experiment 2 with P= 0.294. For each experiment, the simulated behavior of three identical ‘stat birds’ was obtained for 600 experimental trials.

2.5. Methods of data analysis 2.5.1. Sorting technique Over trials, an animal produced a distribution of complex responses and this distribution had to be categorized into response alternatives. We solved this problem with a data analysis procedure that categorized pigeons’ complex responses by the same goodness-of-fit equation that was involved in the original on-line determination of whether or not obtained and target responses were sufficiently similar to produce reinforcement. Specifically, the goodness-of-fit equation was used to determine which of the two target responses for a given experiment was more closely approximated by each one of the 600 complex responses from the last 600 trials of the experiment. That is, for each response, a

separate sum of squared deviations was obtained for each of the two target responses. A complex response was said to approximate the target response that yielded the lower value of D. Responses which were shown to approximate equally well both target responses (i.e. which yielded exactly equal values of D) were sorted into a third, ‘indeterminate’ class.

2.5.2. O6erall responses Cumulative records averaged over the last 2 days of an experiment were calculated for each of the two response categories. The total number of key pecks within each 0.25-s interval of a category was divided by the total number of responses belonging to that category. This relative frequency of key pecks per 0.25-s interval was then plotted as a function of interval duration. Recall that the contingency generated two different types of trials: choice trials and correction trials. Cumulative curves were plotted both for choice trials alone and for combined choice trials and correction trials. For those birds with a sufficient number of choice trials to produce smooth cumulative curves, the choice trial curves and the combined curves did not appear sufficiently different to justify constructing different curves for each. For this reason, and to provide a larger sample of behavior upon which a more representative average cumulative record could be based, the cumulative curves were averaged over both choice and correction trials. We use the term ‘overall performance’ to refer to these average cumulative curves. As in other tasks, they do not necessarily accurately represent local behavior exhibited within individual trials (Schneider, 1969). Indeed, one of our goals here was to determine the relation between overall performances and more local, within-trial performances, which are defined next. 2.5.3. Within-response structure In each experiment, the two target responses had different numbers of pecks in the first second as well as different numbers of pecks in the last second. To the extent to which complex responses approximated target responses, different component behaviors observed in the first

L. Hawkes, C.P. Shimp / Beha6ioural Processes 44 (1998) 19–43

second should have predicted different component behaviors in the last second. (The particulars depend on the specific experiment, as described below.) The extent to which the structure of responses conformed to target responses was therefore examined in the following way. The mean number of pecks in the last second of a response was computed separately for those responses which had 0, 1, 2 or 3 pecks in the first second. That is to say, a bird’s behavior was partitioned into separate categories on the basis of the number of pecks in the first second of a response. The mean number of pecks obtained in the last second of all responses in a given category was then computed for each of the separate categories. If a response conformed to a target, the mean number of pecks obtained in the last second of the response should have been different for the different categories, in a manner that depended, as will be described below, on the target responses in a particular experiment. The mean number of pecks in the last second of a trial are reported only for those categories which had frequencies of occurrence greater than 30.

2.5.4. Stat bird data The stat bird data were analyzed in the same way as were the data for real birds. 2.5.5. Order of experiments For the sake of conceptual continuity and expository convenience, the experiments are described in an order different from that in which they were conducted. Experiments 1, 2, 3, 4, as described below were conducted in the order 2, 3, 4 and 1.

3. Experiment 1 Since so little is known about complex responses, to say nothing of choice between complex responses, it seemed advisable to examine the basic situation in which complex responses have only constant rates of component key pecking. Experiment 1 in this way simplified the ‘complex’ linear target responses about as far as was possible to do.

25

3.1. Method 3.1.1. Animals and apparatus The pigeons and apparatus were as described above. 3.1.2. Procedure The two equally likely target responses were f(t)= 1, and f(t)= 3, and were 4 s in duration, so that the target number of pecks per second was 1, 1, 1, 1 for the first, second, third and fourth s of one complex response, and 3, 3, 3, 3 for the other response. Terminal values of C are shown in Table 1. 3.2. Results 3.2.1. O6erall responses Fig. 1 shows the two average cumulative curves for each bird. (As in all four experiments, patterns that fell in the indeterminate response category are not represented. The relative frequencies of such patterns in Experiment 1 were low, with the average equal to 0.113.) Visual inspection shows that the two sets of empirical points for each bird qualitatively resembled the straight lines representing the corresponding target responses. At the same time, the birds’ responses clearly differed systematically from the target responses; lowerrate and higher-rate responses were too similar, and fell in between the two target responses. This quantitative difference nevertheless does not obscure the overall qualitative resemblance between observed and target responses. Fig. 2 shows the average cumulative records from the stat birds. The two curves for each of the stat birds qualitatively resembled both the corresponding target responses and the curves produced by the real birds. The correspondence between the simulation curves and the target responses appears roughly equivalent to that between real curves and target curves, although there is less variability across the stat-bird curves than across the real-bird curves. Otherwise, in terms of average cumulative records, stat birds’ complex responses conformed to the contingency about as well as did those of the real birds: the Bernoulli-trials process generated over trials aver-

14 14 14 20 20 20

1

2

3

4

5

6

3 5 3 6 3 7 3 5 2 6 3 10

Terminal criterion (C)

87

82

94

150

125

172

Total number of obtained reinforcers over last 2 days

Note: Only values based on at least 30 responses are shown.

Number of sessions

Bird number

Table 1 Experiment 1

1.81

2.01

2.09

1.48

2.22

2.28

1.47

1.97

1.90

1.25

1.90

2.12

—

1.92

1.84

1.26

2.14

2.12

—

1.84

1.60

—

2.13

2.38

3

1.84

1.72

2.05

0

2

0

1

Stat birds

Real birds

1.65

1.67

1.83

1

1.93

1.87

1.77

2

1.72

1.68

1.76

3

Average number of pecks in the last second given 0, 1, 2, 3 pecks in the first second

26 L. Hawkes, C.P. Shimp / Beha6ioural Processes 44 (1998) 19–43

L. Hawkes, C.P. Shimp / Beha6ioural Processes 44 (1998) 19–43

27

Fig. 1. Mean cumulative number of pecks per 0.25-s interval, for the complex responses approximating the two target responses, averaged over the last 600 trials of Experiment 1. The horizontal dashed lines represent the cumulative number of pecks in the two target responses, which are themselves represented by solid straight lines. Circles and triangles represent the complex responses approximating the target responses f(t)= 1, t= 1,4, and f(t)= 3, t= 1,4, respectively.

age cumulative records that resembled real ones. Fig. 2 shows also that the total output of key pecks per response corresponded roughly to that of the target response and, interestingly, differed from it in ways similar to that of real birds, in the sense that the low-rate curves exceeded the lowrate target total and the high-rate curves failed to reach the high-rate target total.

3.2.2. Within-response structure Let us now examine the conditional distributions, which reflect the extent to which behavior in the first second of a complex response predicted behavior in the last second of that same response. If responding had closely approximated target responses, there should have been a direct, positive relation between the number of pecks in the first and last seconds. For example, if there were one peck in the first second, there should have been one peck in the last second; if there had been

three pecks in the first second, there should have been three pecks in the last second. Let us now examine the conditional distributions, which reflect the extent to which behavior in the first second of a complex response predicted behavior in the last second of that same response. Table 1 shows that no bird’s performance satisfied this relation. Furthermore, inspection of Table 1 suggests a crude similarity between the real conditional distributions and those of the stat birds. In general, there is no compelling evidence that within-trial response structure conformed to that of target responses. The similarity between real and simulated complex responses is indeed sufficiently great to suggest that the link between complex responses and their component key pecks in Experiment 1 may be summarized largely by a simple Bernoulli-trials process, with complex responses emerging from the variability inherent in pecking with a probability constant over time.

28

L. Hawkes, C.P. Shimp / Beha6ioural Processes 44 (1998) 19–43

Fig. 2. Mean cumulative number of simulated pecks for the three stat birds per 0.25-s interval, for the complex responses approximating the two target responses of Experiment 1, averaged over 600 trials. The horizontal dashed lines represent the cumulative number of pecks in the two target responses, which are represented by solid straight lines. Circles and triangles represent the complex responses approximating the target responses f(t)= 1, t= 1,4, and f(t)= 3, t= 1,4, respectively.

3.3. Discussion Complex responses were effectively shaped in terms of overall performances in the form of average cumulative records. Shaping was less effective, however, by the more diagnostic criterion of within-response structure and, in fact, the stat data’s similarity to the real data encourages the tentative conclusion that random pecking can generate overall performances similar to those of real birds. When birds are required to choose between two complex target responses consisting of constant rates of either one or three pecks per second, they appear to ‘solve’ the problem chiefly by pecking at an intermediate, constant rate. The resulting chance variability over trials generates approximations sufficiently close to the two target responses to generate reinforcement and to sustain pecking. In this case, the theoretical problem of explaining the relation between overall, molar, performances and local, molecular performances seems to have a relatively easy answer; overall performance largely emerged from random variation in component performance. The contingency in Experiment 1 therefore defines a situation in which a molar analysis in terms of mean rate of pecking would likely be superior to a molecular

analysis, since the latter seems to consist chiefly of random noise. The kind of argument advanced by Skinner and others on behalf of the advantages of mean rate of responding as a basic datum may apply with little or no qualification to the contingency in Experiment 1.

4. Experiment 2 Experiment 2 examined another pair of complex responses that were nearly as simple as they could be and still retain some temporal extension and temporal patterning. Indeed, in one way, targets in Experiment 2 were even simpler than in Experiment 1. Now, one target consisted of not pecking at all, of pecking, that is, at a zero rate, while the other target consisted, as it did in Experiment 1, of a constant non-zero rate of key pecking. There might be a qualitative difference between zero pecks per second, and a constant non-zero rate of pecking, because not pecking might involve a different posture, standing in a different location in the chamber, and so on, in comparison to pecking at a constant rate. Thus, while the task was similar to that of Experiment 1 in the formal sense that each target had a

L. Hawkes, C.P. Shimp / Beha6ioural Processes 44 (1998) 19–43

29

Fig. 3. Mean cumulative number of pecks per 0.25-s interval, for the complex responses approximating the two target responses, averaged over the last 600 trials of Experiment 2. The horizontal dashed line represents the cumulative number of pecks in the target response with the higher rate, which is represented by the solid straight line. The corresponding horizontal line for the lower-rate response is obscured by the horizontal axis. Circles and triangles represent complex responses approximating the target responses f(t)= 0, t= 1,4, and f(t)= 2, t= 1,4, respectively.

constant number of pecks per second throughout the target, a qualitative difference between the tasks in the two experiments might be expected; the difference between pecking at one peck per second and pecking at zero pecks per second might be much greater than the numerical difference might suggest. Many behavioral tasks, including many discrete-trials discrimination tasks, involve just this difference between pecking, or not pecking, within some short period of time, over which mean rates of pecking are calculated. It was therefore important to examine the relation between component and overall performances in this common special case involving tasks with zero and non-zero pecks per second.

4.1. Method 4.1.1. Animals and apparatus Animals and apparatus were as described above.

4.1.2. Procedure The two equally likely target responses were f(t)= 0, and f(t)= 2, and were again 4 s in duration so that the target number of pecks was 0, 0, 0, 0 for the first, second, third and fourth second of one target, and was 2, 2, 2, 2 for the other. Table 2 shows terminal values of C. 4.2. Results 4.2.1. O6erall responses Fig. 3 shows the two average cumulative records for each bird. (Patterns that fell in the indeterminate response category are excluded. The average relative frequency of this category was 0.009.) One curve within each pair closely resembles the target response with a zero rate of pecking and the other curve closely resembles the target rate with two pecks per second. Fig. 3 shows also that the total output of key pecks per response for each bird approximately equalled the

18 18 18 14 14 14

1

2

3

4

5

6

0 1 0 1 0 1 0 1 0 2 0 1

Terminal criterion (C)

154

160

76

150

60

119

Total number of obtained reinforcers over last 2 days

Note: Only values based on at least 30 responses are shown.

Number of sessions

Bird number

Table 2 Experiment 2

0.40

0.41

0.21

0.18

0.17

0.25

1.58

1.95

1.93

1.51

2.52

2.19

1.63

2.13

1.89

1.50

2.62

2.58

—

2.06

1.75

—

2.70

2.39

3

1.15

1.03

1.26

0

2

0

1

Stat birds

Real birds

1.10

1.26

1.19

1

1.12

1.30

1.19

2

1.31

1.24

1.12

3

Average number of pecks in the last second given 0, 1, 2 or 3 pecks in the first second

30 L. Hawkes, C.P. Shimp / Beha6ioural Processes 44 (1998) 19–43

L. Hawkes, C.P. Shimp / Beha6ioural Processes 44 (1998) 19–43

31

Fig. 4. Mean cumulative number of simulated pecks for the three stat birds per 0.25-s interval, for the complex responses approximating the two target responses in Experiment 2, averaged over 600 trials. The horizontal dashed line represents the cumulative number of pecks in the target response with the higher rate, which is represented by the solid straight line. The corresponding horizontal line for the lower-rate response is obscured by the horizontal axis. Circles and triangles represent complex responses approximating the target responses f(t)= 0, t= 1,4, and f(t) =2, t =1,4, respectively.

number, either 0 or 8, required by the corresponding target response. Each of the two curves, therefore, corresponded at least qualitatively and in some ways quantitatively to its corresponding target response. Each panel of Fig. 4 shows the average cumulative record for a stat bird. While the two curves shown for each stat bird are reasonably linear, the low-rate curve is too high (too far above the target value of 0), and the high-rate curve is too low. It will be recalled that stat birds in Experiment 1 behaved similarly and thereby approximated behavior of real birds. Here, however, the stat-bird curves in Fig. 4 do not diverge from each other as much as do those of real birds in Fig. 3, and the average cumulative records from the real birds more closely approximated the target responses than did those from the stat birds. In this experiment, a Bernoulli-trials process with a P value intermediate between the two target responses was unable to generate sufficient behavioral variability over trials, so that a Bernoulli-trials process was unlikely to have been the sole link between complex responses and component behaviors.

4.2.2. Within-response structure If complex responses had closely conformed to the target responses, there should have been

a direct, positive relation between the number of pecks in the first and last seconds: if a bird started the lower-rate response, by not pecking in the first second, it should have finished the same response by not pecking in the last second, so that there would have been low rates at both the beginning and ending. Similarly for the beginning and ending of the higher-rate response. Table 2 shows that given that there were zero responses in the first second, all six birds pecked an average, rounded off to the nearest whole number, of zero responses in the last second of a response. Similarly, given that there were one or more pecks in the first second, the average number of pecks in the last second, again rounded to the nearest whole number, was, for different birds, either 2 or 3. Therefore, the sequential structure of responses in Experiment 2 at least crudely approximated that of the target responses. The stat birds did not display this sequential organization, indicating that a local coin-tossing process was unable to conform to the contingency. Thus, the real birds’ performances approximated the within-trial structure of target responses notably better than did that of the stat birds, and in this case, overall performance did not emerge purely from random variability inherent in pecking with a constant probability.

32

L. Hawkes, C.P. Shimp / Beha6ioural Processes 44 (1998) 19–43

4.3. Discussion The shaping procedure in Experiment 2 established new complex responses, and these responses furthermore resembled target responses by both overall and within-response criteria. The Bernoulli process that approximated performances in Experiment 1 was unable, for Experiment 2, to generate even appropriate average cumulative records, let alone appropriate withintrial sequential structure of complex responses. The relation between overall performance and component performance was therefore more complex than in Experiment 1, and overall performance did not emerge simply from variability inherent in a constant-probability component performance. Thus, even a very small numerical difference between target responses in Experiments 1 and 2, that between the two different low-rate target responses of either zero or one pecks per second, can change the relation between local and overall performance. When in Experiment 1 there was only a quantitative difference between the two target responses, the link between the two analyses could be explained in terms of random variability. When there was a qualitative difference, specifically between pecking at a constant rate and not pecking at all, the link between local and overall performances could not be explained in terms of random variability, so that molar performance did not emerge purely from random variability in molecular performance. The relation between molar and molecular performances in Experiment 2 was therefore more complex, and the averaging of local performances in Experiment 2, required to calculate mean rate of pecking, would obscure what appears to have been a causal contribution of molecular structure to molar performance. Experiment 2, unlike Experiment 1, therefore does not fully support the idea of mean response rate as a fundamental datum.

5. Experiment 3 Experiments 1 and 2 examined the highly special case of complex linear responses involving a

rate of change in the component behavior equal not only to a constant but to the special constant of 0. Experiment 3 examined a more general linear case where the rate of change of the component behavior was still constant but was non-zero. The task was similar to that examined in a singleresponse setting by Hawkes and Shimp (1975), but simplified in the sense that response duration was shortened and the number of component key-pecking rates was reduced. These simplifications generated target responses that were relatively likely to emerge from random variability inherent in pecking at a constant probability. It was expected, therefore, that this arrangement might define a situation suitable for molar analysis, that is, one in which overall performance might emerge from random variability in molecular performance, as described by the Bernoulli-trials process.

5.1. Method 5.1.1. Animals and apparatus Animals and apparatus were as described above. 5.1.2. Procedure The two equally likely target responses were f(t)= 2t− 2, and f(t)=4− 2t, and each was 2 s in duration, so that the target number of pecks per second was 0 and 2 for the first and second second of one response, and was 2 and 0, for the other. Table 3 shows the terminal values of C. 5.2. Results 5.2.1. O6erall responses Fig. 5 shows the average cumulative records. (The relative frequency of omitted patterns that fell in the indeterminate response category was higher than in the other experiments: the worst case was 0.326 for one bird. It was 0.170 for two others, and much less for the remaining two.) Of the two curves for each bird in Fig. 5, one is concave downward, and the other is concave upward, corresponding to the two target responses involving increasing and decreasing key-pecking rates. Fig. 5 shows also that the total output of

30 30 30 34 34 34

1

2

3

4

5

6

0 1 0 1 0 1 0 0 0 1 0 1

Terminal criterion (C)

81

86

62

107

92

78

Total number of obtained reinforcers over last two days

Note: Only values based on at least 30 responses are shown.

Number of sessions

Bird number

Table 3 Experiment 3

1.56

1.52

1.73

2.17

1.65

1.78

0.94

1.15

0.99

1.15

0.68

0.97

0.84

1.33

0.57

0.74

—

1.09

—

—

—

—

—

—

3

1.48

1.41

1.34

0

2

0

1

Stat birds

Real birds

1.01

0.99

1.10

1

0.98

1.03

1.00

2

—

—

—

3

Average number of pecks in the last second given 0, 1, 2, or 3 pecks in the first second

L. Hawkes, C.P. Shimp / Beha6ioural Processes 44 (1998) 19–43 33

34

L. Hawkes, C.P. Shimp / Beha6ioural Processes 44 (1998) 19–43

Fig. 5. Mean cumulative number of pecks per 0.25-s interval, for the complex responses approximating the two target responses in Experiment 3, averaged over the last 600 trials. The horizontal dashed line represents the cumulative number of pecks for both target responses. Circles and triangles represent the complex responses approximating the target responses f(t)= 2t − 2, t= 1,2, and f(t) = 4 − 2t, t= 1,2, respectively. The two target responses are approximated by the dashed and solid functions.

key pecks per response roughly corresponded to the number required by the corresponding target response. These curves qualitatively corresponded, therefore, to the target responses. Fig. 6 shows the average cumulative records for the stat birds. Fig. 6 also shows that the two curves for each of the stat birds resemble those of the real birds, with one concave downward and the other concave upward. The curves for the stat birds in this sense qualitatively corresponded to the target responses, and further support the expectation that the contingency was easy for a Bernoulli-trials process to satisfy.

5.2.2. Within-response structure If within-trial structure of complex responses had closely resembled that of target responses, the number of pecks in the first and last seconds would have been inversely related. That is, responses beginning with lower rates of key pecking should have ended with higher rates, and vice versa. Table 3 shows that indeed, for each bird, the more pecks there were in the first second, the fewer there were in the last second. Table 3 shows also that the stat birds produced a qualitatively similar, although quantitatively

smaller, inverse relation. It is important to consider the implications of this small but noticeable within-response structure produced by the Bernoulli-trials process. Of course, on the average, one would not expect to find any such structure. Why, one must therefore ask, was some structure apparent in the stat bird performances? There are two possible reasons. First, it might simply be due to sampling variation. Second, it might be produced by the sorting procedure. The present data cannot discriminate between these two possibilities, or determine the extent to which either or both contributed to the observed within-response structure in stat bird performances. It is, therefore, important to qualify in this sense our finding shown in Table 3 that Experiment 3 produced within-response structure resembling that of the target responses. Indeed, the within-response results can be seen to have been qualitatively but not entirely quantitatively consistent with a Bernoulli-trials process and, except for this one quantitative exception, this contingency’s effects could be described by random variability deriving from constant component pecking.

L. Hawkes, C.P. Shimp / Beha6ioural Processes 44 (1998) 19–43

35

Fig. 6. Mean cumulative number of simulated pecks for the three stat birds per 0.25-s interval, for the complex responses approximating the two target responses in Experiment 3, averaged over 600 trials. The horizontal dashed line represents the cumulative number of pecks for both target responses. Circles and triangles represent the complex responses approximating f(t)= 2t −2, t= 1,2, and f(t) =4− 2t, t =1,2, respectively. The two target responses are approximated by the dashed and solid functions.

5.3. Discussion

6. Experiment 4

Average cumulative records and within-trial response structure both suggest the procedure shaped complex responses having some of the properties of the complex linear target responses. Random variability in component key pecking accounted for much, but not quite all, of the temporal organization of complex responses, so that the link between local and overall performances was not quite as simple as that described by a Bernoulli process; overall performance seemed to emerge largely, but not solely, from random variability in local performances. Recall that targets in this experiment were short in duration (2 s) and had few components (just two). The Bernoulli process, therefore, was able to account for some of the within-trial structure, since it was relatively easy for a process with a constant probability of pecking to generate patterns of pecking that roughly approximated the target responses. More generally, this result warns us that the relation between overall (molar) and local (molecular) performances may depend on the experimental parameter of trial duration, since complex responses may spontaneously emerge when trial durations provide an opportunity to respond or not to respond for just a few seconds.

We have seen so far that birds can learn several different types of complex linear responses, even in the present non-cued choice setting. In none of these cases, however, did the target rate of component key pecking change in a quantitatively graded way over even a few seconds. Experiment 4 examined the more general case, corresponding to the original task explored in a simpler, nonchoice setting by Hawkes and Shimp (1975), who under different conditions arranged target responses with graded increasing or decreasing component pecking. In Experiment 4, we arranged a contingency involving target choices differing both in terms of graded changes in component rates and in terms of qualitatively different behaviors, pecking at zero and non-zero rates.

6.1. Method 6.1.1. Animals and apparatus Animals and apparatus were as described above. 6.1.2. Procedure The two equally likely target responses were f(t)= t−1, and f(t)= 4− t, and each target was 4 s in duration, so that the target number of pecks per second was 0, 1, 2, 3, for the 1st, 2nd, 3rd and

120 73 122 81 80 84

1

2

3

4

5

6

1 6 2 5 1 3 2 5 2 6 1 5

Terminal criterion (C)

46

46

65

77

82

54

Total number of obtained reinforcers over last two days

Note: Only values based on at lease 30 responses are shown.

Number of sessions

Bird number

Table 4 Experiment 4

1.46

1.98

2.25

2.83

1.86

1.97

0.66

1.92

2.19

1.05

0.89

1.86

—

2.01

2.16

0.56

0.58

1.66

—

2.17

2.14

—

—

1 56

3

1.43

1.60

1.35

0

2

0

1

Stat birds

Real birds

1.55

1.52

1.56

1

1.55

1.41

1.58

2

1.55

1.39

1.47

3

Average number of pecks in the last second given 0, 1, 2, or 3 pecks in the first second

36 L. Hawkes, C.P. Shimp / Beha6ioural Processes 44 (1998) 19–43

L. Hawkes, C.P. Shimp / Beha6ioural Processes 44 (1998) 19–43

37

Fig. 7. Mean cumulative number of pecks per 0.25-s interval, for the two complex responses approximating the two target responses, averaged over the last 600 trials of Experiment 4. The horizontal dashed line represents the cumulative number of pecks in both target responses, which are represented by solid lines. Circles and triangles represent the complex responses approximating the target responses f(t)= t− 1, t= 1,4, and f(t)= 4−t, t= 1,4, respectively.

4th s of one response, and 3, 2, 1, 0 for the other response. Table 4 shows the terminal values of C.

6.2. Results 6.2.1. O6erall responses Fig. 7 shows the two average cumulative records for each bird. (Patterns that fell in the indeterminate category are not presented. The relative frequency of such patterns was never more than 0.065.) Fig. 7 shows that each of the two curves at least crudely resembled the corresponding target curve, in the sense that for each bird there is a concave upwards and a concave downwards curve. Fig. 7 shows that there was some variability across birds in the mean total numbers of key pecks per response. The overall mean number of pecks, 6.4, nevertheless closely resembled that required by the contingency, 6.0. Fig. 8 shows the stat data corresponding to Fig. 7. The stat birds, like the real birds, produced concave-upwards and concave-downwards curves. Recall that stat birds ‘pecked’ on the average at a

constant rate throughout the interval. Fig. 8 therefore demonstrates that pecking with a constant probability can produce, over 4-s trials, average complex responses involving accelerating or decelerating rates of key pecking; random pecking in this task can produce overall performances resembling complex target responses, and can do so sufficiently often so that a tolerant contingency can sustain responding. Furthermore, given the variability in behavior across real birds, it is not obvious that an observer would be able to distinguish behavior of a single stat bird from that of a real bird, in the sense that the stat data seem to fall within the variability of the real birds. However, there is less variability across stat birds than across real birds. Of course, the stat bird variability could be increased if the P value in the Bernoulli-trials process were estimated separately for each bird. (It will be recalled that the perfect match between observed total number of pecks for obtained and target responses shown in Fig. 8 was forced for the stat birds by the experimenters’ selection of a value for P in the Bernoulli-trials process.)

38

L. Hawkes, C.P. Shimp / Beha6ioural Processes 44 (1998) 19–43

Fig. 8. Mean cumulative number of simulated pecks for the three stat birds per 0.25-s interval, for the two complex responses approximating the two target responses in Experiment 4, averaged over 600 trials. The horizontal dashed line represents the cumulative number of pecks in both target responses, which are represented by solid lines. Circles and triangles represent the complex responses approximating the target responses f(t)= t −1, t=1,4, and f(t) =4 − t, t =1,4, respectively.

6.2.2. Within-response structure To the extent to which complex responses resembled target responses, there should have been an inverse relation between the number of key pecks in the first and last second of a complex response. (Responses beginning with lower rates should have ended with higher rates, and vice versa.) The conditional distributions shown in Table 4 show that this inverse relation was obtained in a strict monotonic form for birds 1, 2, 3, 4 and 6, but not for bird 5. With the exception of bird 5, the birds tended to produce, on average, responses having the target sequential structure, namely, inversely correlated beginning and ending key-pecking rates. Table 4 suggests that the Bernoulli process generally did not produce the same within-trial response structure as did the real birds: while one of the three stat birds (stat bird 2) generated some appropriate structure, it was smaller in magnitude than that of the real birds, and there was no overall effect across stat birds. This simple random process can, however, approximate the behavior of bird 5, which generated no within-trial structure. Thus, different birds may generate complex responses in different ways in this task. While most birds may generate within-trial response structure approximating target responses, some may peck at a constant probability, which over trials, by chance, produces exemplars of target responses. The relation between molar and molec-

ular performances therefore may be different for different animals in the same task. Given this reality, the tradition within behavior analysis of analyzing behavior of individual organisms seems exceedingly appropriate.

6.3. Discussion Did Experiment 4 establish linear responses that resembled target responses? The answer, by the criterion of average cumulative records, is, yes. How were these overall performances related, however, to their component behaviors? Did these average cumulative records emerge merely from random variability resulting from a constant probability of the component behaviors? Generally, no. A random process according to which there were neither increasing nor decreasing rates of key pecking produced overall average cumulative records that qualitatively resembled those of real birds, but that random process could not, for most birds, produce the correct within-trial response structures. Thus, even by the more rigorous criterion of within-response structure, the task may be said to have established new linear responses. Experiment 4 generalized to a choice setting the contingency in Hawkes and Shimp (1975), and succeeded for all but one bird in establishing both overall cumulative records and within-trial se-

L. Hawkes, C.P. Shimp / Beha6ioural Processes 44 (1998) 19–43

quential organization that approximated complex target responses. These data, along with the data from Experiment 2, therefore define an extraordinarily difficult theoretical question for future research: what is the link between overall performance and local performance when the former does not emerge from chance variation in the latter?

7. General discussion Research on response sequences has struggled with the relation between parts and wholes, or components and patterns (Marr, 1979; Terrace, 1991; Terrace et al., 1996). That relation was the focus of our experiments. We initiated exploratory research on the nature of that relation in the context of ‘linear responses’, which we define to be complex responses having constant rates or, at most, constant rates of change, in component key pecking. We suggest linear responses, because of their relative simplicity, define a reasonable starting place for research on the relation between parts and wholes of complex responses. Experiments 1 – 4 in fact provide tentative answers to two basic questions. First, is it possible to establish complex linear choice responses consisting of temporal patterns of key pecking? Second, can the relation between a complex response and its component key pecks ever be identified? The answer to the first question is a carefully qualified, yes. We have identified tasks in which birds learn linear choice responses. This answer, however, is qualified by the answer to the second question. A stochastic process randomly deciding at each moment whether or not to respond, and doing so with the same probability, moment after moment, might seem to be too simple to be able to produce complex responses, or to realistically portray the temporal organization of behavior (Shimp, 1975). Despite this simplicity, we have identified contexts where such a process can sometimes produce complex linear responses that resemble those of real birds. In these tasks, what appear to be complex responses are merely emergent properties of random component pecking. In

39

other tasks, however, complex responses were shaped even by the higher standard of within-response structure. These answers clearly reveal how little is known about complex responses and their relations to component behavior. Presumably one reason why so little is known is the extreme methodological complexity of the problem, and on the certainty that results will generally depend on a host of methodological details. In the present case, for example, we expect that the sorting technique, the terminal value of the criterion, C, the decision to set C at a value just above that necessary to sustain pecking, the duration of the complex target response, the number of intervals within target responses, the use of linear rather than non-linear targets, and so on, all affect our results. The present experiments evaluated all these procedural features in combination, and much more research will be required to identify their separate contributions. Given the fragile nature of understanding of these issues, it presumably is premature to try to study quantitative relations involving choice between complex responses. We hope the present experiments will serve to remind researchers of the magnitude of the task remaining.

7.1. The ‘content’ and ‘en6elope’ of a complex response The present results suggest that, in at least some cases, what we will describe as a response’s ‘content’ can be added to the list of its shapable characteristics, including force, relative and absolute duration, spatial location, and so on (Gilbert, 1958; Herrnstein, 1961a; Notterman and Mintz, 1965; Shimp, 1976a,b; Pear, 1985). The ‘content’ of a complex response refers to its ‘internal structure’ (Machado, 1997) and may be defined by contrast with its ‘envelope’. By the envelope of a complex response, we refer to the relative and absolute durations of that response. The envelope of a response thus refers to the response’s temporal boundaries, and it has long been known that such boundaries can be shaped and that the envelopes of two reinforced responses affect preference between them (Shimp, 1968; Staddon, 1968;

40

L. Hawkes, C.P. Shimp / Beha6ioural Processes 44 (1998) 19–43

Shimp, 1969; Hawkes and Shimp, 1974; Shimp et al., 1994). The ‘content’ of a complex response, on the other hand, may be defined as the sequential structure between its boundaries, that is, between its ‘beginning’ and ‘ending’ (Shimp, 1983; Shimp et al., 1989, 1994). The present paper demonstrates that, in at least some cases, the ‘content’ of a reinforced choice response, like its envelope, can be shaped and maintained. The fact that complex responses can be established and maintained in a choice setting opens up for future research the question of whether content, like envelope, affects preference, and if so, how.

7.2. Implications for the relation between molar and molecular analyses The relation between molar and molecular analyses is multi-faceted (Herrnstein, 1961b; Shimp, 1966, 1975, 1990; Herrnstein, 1970; Hinson and Staddon, 1983a,b; Hineline et al., 1987; Shull, 1995; Donahoe et al., 1997), but presumably many researchers would agree that one of the issues is the nature of the appropriate unit of analysis. A question has been the conditions under which behavior should be regarded in the aggregate, or in the small, with specific emphasis on the relation between overall performance and its constituent parts. Molar analyses have more often than not focused on overall behavioral output (total number of some response alternative, or total time allocated to some alternative) over some amount of time, without explicit consideration of how that behavior derives from, or is related to, what is sometimes referred to as ‘fine grain detail’ (Commons et al., 1982) or local organization (Shimp, 1975, 1976b). Molecular analyses, on the other hand, have often focused on the existence of lawful relations involving local organization, and sometimes that lawfulness has seemed to threaten the primacy of certain molar phenomena (Shimp, 1966, 1969, 1992; Hinson and Staddon, 1983a,b; Silberberg and Ziriax, 1985; Hineline et al., 1987). The present results suggest a less polarized and more ecumenical position. On the one hand,

a molar position is supported by the discovery that in some cases (Experiment 1 and bird 5 in Experiment 4) local organization, albeit admittedly over only a few seconds of a linear response, can be interpreted to some degree as random noise, with organization emerging in part because pecking with a constant probability generates appropriate variability (see also Blough, 1966). Local structure in these cases is the sort of thing one would want to average over, in order better to see, magnify, and clarify, causal relations at the molar level. Mean response rate in such contexts therefore may be not only a legitimate dependent variable, but a variable to be preferred to some others, just as Skinner proposed. On the other hand, in other cases (Experiment 2, and all birds except bird 5 in Experiment 4), a molecular position is supported by the discovery that local performance, again within a linear response only a few seconds in duration, cannot be ignored. We apparently face a situation where neither radical position, neither molecular nor molar, can capture the full range of results, as there are examples in the present data where each is applicable. Apparently, an analysis is needed that takes the context into account and is either molar, molecular, or both (as in the present experiments), as the situation demands (see also Machado, 1997).

7.3. Complex responses and beha6ioral 6ariability The present approach to behavioral variability is but one of many. Neuringer (1991) reinforced only responses that differed from those that had occurred over a previous time window. The contingency, like the present one, was ‘tolerant’ in the sense that it accepted different responses for reinforcement at different times, and thereby ‘tolerated’ behavioral variability. Other dynamic contingencies also reinforce a behavior when it satisfies some shifting behavioral criterion defined over a moving window of time (Blough, 1966; Shimp, 1967; Platt, 1973; Weiss, 1973; Page and Neuringer, 1985; Machado, 1992, 1994, 1997; Galbicka et al., 1993). While these dy-

L. Hawkes, C.P. Shimp / Beha6ioural Processes 44 (1998) 19–43

namic contingencies are highly useful, especially in the study of behavioral variability and dynamics, the constantly shifting nature of the target responses, and the actual encouragement of variability the contingencies sometimes involve, may make the results difficult to interpret from the perspective of the definition of a response category. In all these cases, however, behavioral variability has been discovered to be shapable. This discovery is tantalizingly similar to what we see here, namely, that variability across exemplars of complex responses can sometimes be brought under explicit control of a reinforcement contingency. It is to be hoped that future research will lead to a detailed and well-articulated conceptual analysis to unite these different approaches to behavioral variability.

Acknowledgements This research was supported in part by grants from NIMH and NSF. This paper is based on data in a dissertation submitted by the first author to the Department of Psychology, University of Utah, in partial fulfilment of the requirements for the Ph.D., which was awarded in 1977. The present manuscript was prepared by the second author, who would like gratefully to acknowledge Thane Fremouw and Walter Herbranson for their constructive comments, on both substance and style, on an earlier draft of this paper.

References Anger, D., 1956. The dependence of interresponse times upon the relative reinforcement of different interresponse times. J. Exp. Psychol. 52, 145–161. Blough, D.S., 1966. The reinforcement of least-frequent interresponse times. J. Exp. Anal. Behav. 9, 581–591. Bush, R.R., Mosteller, F.A., 1955. Stochastic Models for Learning. Wiley, New York. Commons, M.L., Herrnstein, R.J., Rachlin, H. (Eds.), 1982. Quantitative Analyses of Behavior, vol. 2. Bollinger, Cambridge, MA. Donahoe, J.W., Palmer, D.C., Burgos, J.E., 1997. The S–R issue: its status in behavior analysis and in Donahoe and Palmer’s Learning and Complex Beha6ior. J. Exp. Anal. Behav. 67, 193 – 211.

41

Estes, W.K., 1959. The statistical approach to learning theory. In: Koch, S. (Ed.), Psychology: A Study of a Science, vol. 2. McGraw-Hill, New York, pp. 380 – 491. Feller, W., 1950. An Introduction to Probability. Wiley, New York. Gilbert, T.F., 1958. Fundamental dimensional properties of the operant. Psychol. Rev. 65, 272 – 282. Galbicka, G., Kautz, M.A., Jagers, T., 1993. Response acquisition under targeted percentile schedules: A continuing quandary for molar models of operant behavior. J. Exp. Analy. Behav. 60, 171 – 184. Hawkes, L., 1977. Reinforcement of behavioral patterns: Preference and the internal structures of the behavioral alternatives. Doctoral Dissertation, University of Utah, June 1977. Hawkes, L., Shimp, C.P., 1974. Choice between response rates. J. Exp. Anal. Behav. 21, 109 – 115. Hawkes, L., Shimp, C.P., 1975. Reinforcement of behavioral patterns: Shaping a scallop. J. Exp. Anal. Behav. 23, 3 – 16. Hays, W.L., 1988. Statistics, 4th ed. Harcourt Brace, New York. Herrnstein, R.J., 1961a. Stereotypy and intermittent reinforcement. Science 133, 2067 – 2069. Herrnstein, R.J., 1961b. Relative and absolute strength of response as a function of frequency of reinforcement. J. Exp. Anal. Behav. 4, 267 – 273. Herrnstein, R.J., Loveland, D.H., 1964. Complex visual concepts in the pigeon. Science 46, 549 – 551. Herrnstein, R.J., 1970. On the law of effect. J. Exp. Anal. Behav. 3, 243 – 266. Hineline, P.N., Silberberg, A., Ziriax, J.M., Timberlake, W., Vaughan, W., 1987. Commentary prompted by Vaughan’s reply to Silberberg and Ziriax 1987. J. Exp. Anal. Behav. 48, 341 – 346. Hinson, J.M., Staddon, J.E.R., 1983a. Hill-climbing by pigeons. J. Exp. Anal. Behav. 39, 25 – 47. Hinson, J.M., Staddon, J.E.R., 1983b. Matching, maximizing, and hill-climbing. J. Exp. Anal. Behav. 40, 321 – 331. Hogg, R.V., Craig, A.T., 1970. Introduction to Mathematical Statistics. Macmillan, London. Hulse, S.H., 1978. Cognitive structure and serial pattern learning by animals. In: Hulse, S.H., Fowler, H., Honig, W.K. (Eds.), Cognitive Processes in Animal Behavior. Erlbaum, Hillsdale, NJ, pp. 311 – 340. Iversen, I.H., 1991. Methods of analyzing behavior patterns. In: Iversen, I.H., Lattal, K.A. (Eds.), Experimental Analysis of Behavior, Part 2. Elsevier, Amsterdam, pp. 193 – 241. Jacobs, W.J., Blackburn, J.R., Buttrick, M., Harpur, T.J., Kennedy, D., Mana, M.J., MacDonald, M.A., McPherson, L.M., Paul, D., Pfaus, J.G., 1988. Observat. Psychobiol. 16, 3 – 19. Jenkins, H.M., 1970. Sequential organization in schedules of reinforcement. In: Schoenfeld, W.N. (Ed.), The Theory of Reinforcement Schedules. Appleton-Century-Crofts, New York, pp. 163 – 181.

42

L. Hawkes, C.P. Shimp / Beha6ioural Processes 44 (1998) 19–43

Lashley, K.S., 1951. The problem of serial order in behavior. In: Jeffress, L.H. (Ed.), Cerebral Mechanisms in Behavior. Wiley, New York, pp. 112–146. Machado, A., 1992. Behavior variability and frequency-dependent selection. J. Exp. Anal. Behav. 58, 241–263. Machado, A., 1994. Polymorphic response patterns under frequency-dependent selection. Anim. Learn. Behav. 22, 53– 71. Machado, A., 1997. Increasing the variability of response sequences in pigeons by adjusting the frequency of switching between two keys. J. Exp. Anal. Behav. 68, 1–25. Marr, M.J., 1979. Second-order schedules and the generation of unitary response sequences. In: Zeiler, M.D., Harzem, P. (Eds.), Reinforcement and The Organization of Behavior. Wiley, New York, pp. 223–260. Neuringer, A., 1991. Operant variability and repetition as functions of interresponse time. J. Exp. Psychol.: Anim. Behav. Proc. 17, 3 – 12. Notterman, J.M., Mintz, D.E., 1965. Dynamics of Behavior. Wiley, New York. Page, S., Neuringer, A., 1985. Variability is an operant. J. Exp. Psychol.: Anim. Behav. Proc. 11, 429–452. Palya, W.L., 1992. Dynamics in the fine structure of schedule-controlled behavior. J. Exp. Anal. Behav. 57, 267– 287. Pear, J.J., 1985. Spatiotemporal patterns of behavior produced by variable-interval schedules of reinforcement. J. Exp. Anal. Behav. 44, 217–231. Pear, J.J., Rector, B.L., Legris, J.A., 1982. Toward analyzing the continuity of behavior. In: Commons, M.L., Herrnstein, R.J., Rachlin, H. (Eds.), Quantitative Analyses of Behavior, Ballinger, Cambridge, MA, pp. 3– 24. Platt, J.R., 1973. Percentile reinforcement: Paradigms for experimental analysis of response shaping. In: Bower, G.H. (Ed.), Psychology of Learning and Motivation: Advances in Research and Theory, vol. 7. Academic Press, New York, pp. 271 – 296. Schneider, B.A., 1969. A two-state analysis of fixed-interval responding in the pigeon. J. Exp. Anal. Behav. 12, 677– 687. Shimp, C.P., 1966. Probabilistically reinforced choice behavior in pigeons. J. Exp. Anal. Behav. 9, 443–455. Shimp, C.P., 1967. Reinforcement of least-frequent sequences of choices. J. Exp. Anal. Behav. 10, 57–65. Shimp, C.P., 1968. Magnitude and frequency of reinforcement and frequencies of interresponse times. J. Exp. Anal. Behav. 11, 525 –535. Shimp, C.P., 1969. Concurrent reinforcement of two interresponse times: The relative frequency of an interresponse time equals its relative harmonic length. J. Exp. Anal. Behav. 12, 403 – 411. Shimp, C.P., 1975. Perspectives on the behavioral unit: Choice behavior in animals. In: Estes, W.K. (Ed.), Hand-

.

book of Learning and Cognitive Processes, vol. 2. Lawrence Erlbaum, Hillsdale, NJ, pp. 225 – 268. Shimp, C.P., 1976a. Short-term memory in the pigeon: Relative recency. J. Exp. Anal. Behav. 25, 55 – 61. Shimp, C.P., 1976b. Organization in memory and behavior. J. Exp. Anal. Behav. 26, 113 – 130. Shimp, C.P., 1983. The local organization of behavior: Dissociations between a pigeon’s behavior and self-reports of that behavior. J. Exp. Anal. Behav. 39, 61 – 68. Shimp, C.P., 1990. Theory evaluation can be unintentional self portraiture: a reply to Williams. J. Exp. Psychol.: Anim. Behav. Proc. 16, 217 – 221. Shimp, C.P., 1992. Computational behavior dynamics: An alternative description of Nevin (1969). J. Exp. Anal. Behav. 57, 289 – 299. Shimp, C.P., Sabulsky, S.L., Childers, L.J., 1989. Preference for starting and finishing behavior patterns. J. Exp. Anal. Behav. 52, 341 – 352. Shimp, C.P., Fremouw, T., Ingebritsen, L.M., Long, K.A., 1994. Molar function depends on molecular structure of behavior. J. Exp. Psychol.: Anim. Behav. Proc. 20, 96 – 107. Shull, R.L., 1995. Interpreting cognitive phenomena: review of Donahoe and Palmer’s Learning and Complex Beha6ior. J. Exp. Anal. Behav. 63, 347 – 358. Silberberg, A., Ziriax, J.M., 1985. Molecular maximizing characterizes choice on Vaughan’s (1981) procedure. J. Exp. Anal. Behav. 43, 83 – 96. Skinner, B.F., 1935. The generic nature of the concepts of stimulus and response. J. Gen. Psychol. 12, 40 – 65. Skinner, B.F., 1966. Operant behavior. In: Honig, W.K. (Ed.), Operant Behavior: Areas of Research and Application. Appleton-Century-Crofts, New York. Smith, R.F., 1974. Topography of the food-reinforced key peck and the source of 30-millisecond interresponse times. J. Exp. Anal. Behav. 21, 541 – 551. Staddon, J.E.R., 1968. Spaced responding and choice: A preliminary analysis. J. Exp. Anal. Behav. 21, 669 – 682. Stokes, P.D., Balsam, P.D., 1991. Effects of reinforcing preselected approximations on the topography of the rat’s bar press. J. Exp. Anal. Behav. 55, 213 – 231. Stubbs, D.A., Pliskoff, S.S., 1969. Concurrent responding with fixed relative rate of reinforcement. J. Exp. Anal. Behav. 12, 887 – 895. Terrace, H.S., 1991. Chunking during serial learning by a pigeon: I. Basic evidence. J. Exp. Psychol.: Anim. Behav. Proc. 17, 94 – 106. Terrace, H.S., 1993. The phylogeny and ontogeny of serial memory: list learning by pigeons and monkeys. Psychol. Sci. 4, 162 – 169. Terrace, H.S., Shaofu, C., Jaswal, V., 1996. Recall of threeitem sequences by pigeons. Anim. Learn. Behav. 24, 193 – 205. Thompson, T., Zeiler, M.D., 1986. Analysis and Integration of Behavioral Units. Erlbaum, Hillsdale, NJ.

L. Hawkes, C.P. Shimp / Beha6ioural Processes 44 (1998) 19–43 Weiss, B., 1973. Digital computers and the microanalysis of behavior. In: Weiss, B. (Ed.), Digital Computers in The Behavioral Laboratory, Appleton-Century-Crofts, New York, pp. 99 – 140. Wittgenstein, L., 1953. Philosophical Investigations. MacMil-

.

43

lan, New York. Zeiler, M., 1977. Schedules of reinforcement: The controlling variables. In: Honig, W.K., Staddon, J.E.R. (Eds.), Handbook of Operant Behavior, Prentice-Hall, Englewood Cliffs, NJ, pp. 201 – 232.

.