- Email: [email protected]
The role of induction in operant schedule performance
G Model
ARTICLE IN PRESS
BEPROC 2990 1–8
Behavioural Processes xxx (2015) xxx–xxx
Contents lists available at ScienceDirect
Behavioural Processes journal homepage: www.elsevier.com/locate/behavproc
1
Q1
The role of induction in operant schedule performance
2
Q2
William M. Baum a,b,∗
3
Q3
a
4
b
University of California, Davis, USA University of New Hampshire, USA
5
6 18
a r t i c l e
i n f o
a b s t r a c t
7 8 9
Article history: Available online xxx
10 11 12 13 14 15 16 17
19 20 21 22 23 24 25 26 27 28 29
30
31 32 33 34
35
Keywords: Competition Induction Matching law Reinforcement Variable-interval schedule Variable-ratio schedule
Baum and Davison (2014b) showed that Baum’s (2012) recasting of reinforcement as induction may be quantified by assuming that induction follows a power function of reinforcer rate. This power-function induction is readily integrated with theory based on the matching law. Herrnstein (1970) originally assumed background activities (BO ) and their associated reinforcers ro to be constant, but ro should vary with BO . Further, power-function induction implies that BO should vary with reinforcer rate. Baum (1993) reported performance on a wide range of variable-ratio (VR) and variable-interval (VI) schedules. Pigeon’s VR peck rate followed an inverted U-shaped relation, but VI peck rate separated into three ranges of food rate: low-to-moderate, moderate-to-high, and extremely high. As food rate increases, the concave downward relation in the low range reaches an inflection point and gives way to a concave upward relation in the higher range. At the extremes of food rate, VI peck rate decreases. A model based on competition between induced pecking and BO accounted for VI peck rate in the moderate to extreme range of food rates. Further research will account for all three ranges, either by integrating power-function induction with matching theory or with a model based on competition between induced activities. © 2015 Published by Elsevier B.V.
A previous paper (Baum, 2012) recasts the various processes of reinforcement and punishment, stimulus control, classical conditioning, adjunctive behavior, and instinctive behavior as just one process, induction, as defined by Segal (1972). A subsequent paper (Baum and Davison, 2014b) began developing a quantitative model of induction that accounts for operant performance on variableinterval (VI) schedules and concurrent VI VI schedules. The present paper takes a further step toward quantifying induction and integrating it with the matching law (Herrnstein, 1961). Herrnstein (1961) originally presented the matching law as a relation between two behavioral alternatives in the form: B1 r1 = B1 + B2 r1 + r2
∗ Correspondence address: 611 Mason #504, San Francisco, CA 94108, USA. Tel.: +1 415 345 0050. E-mail address: [email protected]
B=
Kr r + rO
(3)
36 37 38 39 40 41
42
(1)
where, B1 and B2 are response rates or times spent at Alternatives 1 and 2, and r1 and r2 are reinforcer rates at Alternatives 1 and 2. Herrnstein (1970) subsequently proposed generalizing the matching law to any number of alternatives n in the form: Bi r = i ˙B ˙r
where, Bi represents response rate or time spent at any one of n alternatives, B represents the total of responding or time spent at the n alternatives, ri represents reinforcer rate at any of the n alternatives, and r represents the total or the n reinforcer rates. Using Eq. (2), Herrnstein (1970) derived an equation for responding at just one recorded alternative:
(2)
where, K replaces B on the assumption that other, unmeasured, activities occur and that, with BO representing those other activities, and r = r + ro , with ro representing reinforcers due to BO . Herrnstein (1970) fitted Eq. (3) to several data sets from Catania and Reynolds (1968). Subsequently de Villiers (1977) fitted it to additional data sets, and it has generally proven successful in describing performance across variable-interval (VI) schedules. One feature of the fits to Eq. (3) seems incorrect, however: ro is assumed to be constant as r varies. Baum (1981) and Davison (1993, 2004) pointed out that this assumed constancy is inconsistent with our general understanding of reinforcement contingencies, because BO must vary as B varies, and ro should vary with BO . By definition, a contingency creates a dependence of reinforcer rate on response
http://dx.doi.org/10.1016/j.beproc.2015.01.006 0376-6357/© 2015 Published by Elsevier B.V.
Please cite this article in press as: Baum, W.M., The role of induction in operant schedule performance. Behav. Process. (2015), http://dx.doi.org/10.1016/j.beproc.2015.01.006
43 44 45 46 47 48 49 50 51 52 53 54 55
G Model BEPROC 2990 1–8
W.M. Baum / Behavioural Processes xxx (2015) xxx–xxx
2 56 57
58
59 60
61
ARTICLE IN PRESS
rate; such a dependence is called a feedback function. For example, the feedback function for a variable-ratio (VR) schedule is given by: B r= V
(4)
where, V is the average number of responses required per reinforcer, and the feedback function for a VI schedule is approximately: 1 r= t + Ba
(5)
64
where, t is the average interval and a is a constant that represents a tendency to bursts at low reinforcer rates (Baum, 1992). Thus, a feedback function should exist between ro and BO :
65
ro = f (BO )
62 63
(6)
69
At least two questions arise: (1) what is BO ? and (2) what is the feedback function f relating ro to BO ? Baum (1981) and Davison (1993) suggested that the function should have characteristics of a ratio schedule (Eq. (4))—that is,
70
rO =
66 67 68
71 72 73 74 75 76 77 78 79 80 81 82 83
BO V
(7)
The basis for this conjecture was that BO activities ought to produce ro directly, with no time-limiting factor such as would characterize an interval schedule (Eq. (5)). Davison (2004) found evidence that BO is not one activity but a conglomerate, but Baum and Davison (2014b) found that treating BO as one activity allowed calculating variation in ro and discovering that Eq. (7) is the feedback function relating ro to BO , at least at high reinforcer (food) rates. An earlier paper (Baum, 2012), relying on the process of induction outlined by Segal (1972), suggested that phylogenetically important events induce activities denoted adjunctive or interim or terminal (Staddon, 1977). Following this reasoning, BO would be induced by a reinforcer such as food, and BO would depend on the
food rate r. Baum and Davison (2014b) found that BO varied with r and, through this variation and Eq. (7), that ro varied with r. Thus, at least part of the other activities represented in Eq. (6) as BO and implicit in Herrnstein’s hyperbola (Eq. (3)) is induced by the food (r). To be accurate, Baum and Davison (2014b) proposed that Eq. (3) should be modified to include activities unrelated to the food rate r—what Staddon (1977) called “facultative” activities. They represented these activities as BN and the reinforcers associated with BN as rN : B=
Kr r + r O + rN
(8)
with the understanding that K equals B + BO + BN and that some function g relates BO to r: BO = g(r)
(9)
Eq. (9) results in ro depending on r indirectly: rO =
g(r) V
BO = co r
85 86 87 88 89 90 91 92
93
94 95 96 97
(10)
To fit data from two large data sets (Baum and Davison, 2014a; Soto, McDowell, & Dallery, 2005), Baum and Davison (2014b) assumed that induction follows a power function: so
84
(11)
where, the exponent so may be thought of as the sensitivity of BO to r, and the coefficient co accounts for reconciling of units. Baum (1993) A data set that allows testing these proposals further for both VI and VR schedules was gathered in an experiment reported in 1993. The procedure was a two-component multiple schedule in which a VR component alternated with a VI component separated by substantial time-outs in between. The intervals generated in the VR component were played back in the VI component to roughly
Fig. 1. Pecks per minute versus food per minute from Baum (1993). Data are from a multiple VR VI schedule in which the VI component was yoked to the VR component.
Please cite this article in press as: Baum, W.M., The role of induction in operant schedule performance. Behav. Process. (2015), http://dx.doi.org/10.1016/j.beproc.2015.01.006
98
99 100 101 102 103 104 105 106 107 108 109 110 111
G Model BEPROC 2990 1–8
ARTICLE IN PRESS W.M. Baum / Behavioural Processes xxx (2015) xxx–xxx
112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145
equate the intervals and food rates in the two. The VR schedule varied across conditions, and each condition continued in daily sessions until both performances appeared stable, whereupon a new condition began. The VR was varied over as large a range as possible, from a large VR that produced a moderate food rate just sufficient to maintain responding to FR 1, the endpoint of ratio schedules. Since ratio schedules exhibit “ratio strain” when they are relatively large, no low food rates were possible, but food rate could increase up to the extreme at the high end with FR 1. Fig. 1 shows peck rate as a function of food rate for the four pigeons in the experiment. Most conditions were presented twice, and Fig. 1 shows the average peck rate across presentations. Performance on the VR schedules was simply a bitonic or upside-down U-shape, and peck rates were generally higher for the VR than the yoked VI. Performance on the VI schedules was more complex. Most notably, whereas the VR curve was concave downwards, the VI curve was concave upwards in the mid-range of food rates, which differed from pigeon to pigeon. Indeed, at least three of the pigeon’s curves show a clear inflection point: at about 11 fpm for B258; at about 5 fpm for B261 and B122; although less pronounced, at about 6 fpm for B348. These inflection points suggest that peck rate was relatively flat in the low range, but because food rates lower than about 0.4 per minute, equivalent to a VI 150s, could not be maintained—except for B261, which maintained pecking at 0.13 fpm, equivalent to a VI 460s. Previous research supports the existence of an inflection point, because a negatively accelerated pattern of responding in the lower range of food rates is well documented (e.g., Catania & Reynolds, 1968; de Villiers, 1977), and the shift from negative acceleration in the lower range to positive acceleration in the higher range requires an inflection point. All four pigeon’s peck rates decreased at the highest food rates. Since the VI performances included a relatively flat peck rate in the low range of food rates, an inflection point followed by an up-turn in peck rate, and a down-turn at the highest food rates, the
3
VI curves might be composed of three regions: low to moderate food rates, moderate to high food rates, and high to extreme food rates. Fig. 2 illustrates this possibility. In the lower, flatter range, the broken line in each graph represents the hyperbola in Eq. (3) fitted to the peck rates. Viewed this way, one sees clearly the up-turn in the mid-range of food rates and the down-turn at the extreme of food rate. Any model of these peck rates should account for both the up-turn and the down-turn. 1. An induction model
147 148 149 150 151 152 153
154
I modeled these data as follows. Paralleling the reasoning of Q4 Baum and Davison (2014b), I made the following assumptions: BO varies with r, because BO is induced by food according to Eq. (11), as assumed by Baum and Davison (2014b) with some empirical support (Staddon, 1977); Pecking also is induced by food, according to: B = cr s
146
(12)
When B and BO compete, BO replaces B: If cr s + co r so > K, then B = K − BO ; that is, the activities exist in a hierarchy, such that B and BO replace BN , and BO replaces B (BN
Fig. 2. Pecks per minute versus food per minute in the VI component of Baum (1993). The broken line represents the fit of Herrnstein’s hyperbola (Eq. (3)) to the low range of food rates. The inflection point between low food rates and high food rates may be seen at the intersection of the broken curve with the solid line.
Please cite this article in press as: Baum, W.M., The role of induction in operant schedule performance. Behav. Process. (2015), http://dx.doi.org/10.1016/j.beproc.2015.01.006
155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176
G Model BEPROC 2990 1–8
ARTICLE IN PRESS W.M. Baum / Behavioural Processes xxx (2015) xxx–xxx
4
Fig. 3. Pecks per minute versus food per minute in the VI component of Baum (1993) fitted with the induction model assuming power-function induction (Eqs. (11) and (12)).
177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209
Bi Bj
=
ci r si s cj r j
, for all i = / j of n operant activities. The power function
has some empirical support too (Staddon, 1977). The model was fitted to the peck rates using Solver in Microsoft Excel, varying c, s, co , and so , after setting K to a rate slightly above the maximum peck rate and placing a minimum of 0.00001 on co . Fig. 3 shows the results of fitting to the VI peck rates. The peck rates are fitted well, with one flaw—for all four pigeons, the inflection point demarcated in Fig. 2 falls below the curve. Because the data set includes no extremely low food rates, the model takes no special account of peck rates in that range. Replacing the hyperbola (Eq. (3); broken lines in Fig. 2) would require food rates in the lower range and additional assumptions about B, BO , and BN in that range. Fig. 4 shows the results of fitting the same model, with the same assumptions, to the VR peck rates. These fits are close and show no systematic deviation from the data. As might be expected from the simple bitonic shape, these peck rates were relatively easy to fit with Excel Solver, varying c, s, co , so , and K. They raise no problem about low food or peck rates. An immediate question to ask is whether the model’s parameters that best fitted the peck rates remained the same for the VI rates and the VR rates. Fig. 5 makes the comparison. It shows each VI parameter plotted against the corresponding VR parameter for each pigeon. The estimates of K (diamonds) were about the same, lying on the major diagonal (equality). The estimates of c and s (squares and triangles) also were about the same, indicating that the power function for the inducing of pecking was about the same for VI and VR. Notable differences occurred for co and so , however, indicating that the power function for the inducing of BO differed for the VI and VR. In fitting the model, the estimates of co , in particular, tended to be extremely small, and necessitated the lower limit of 0.00001 for co . All four estimates of co were far lower for the VI than the VR. In contrast, all four estimates of so were higher for the VI than for the VR. The reason why induction of BO would differ for
VI and VR schedules remains to be discovered, but probably stems from the more abrupt down-turn in the VI peck rates at the higher food rates. The combination of a small coefficient co with a large exponent so caused BO to be tiny at lower food rates but to ramp up quickly at higher food rates, thus allowing the model to fit that sharp downturn in VI peck rates. The large coefficient c paired with an exponent s less than 1.0 modelled a more gradual increase in peck rate, starting at moderately high levels at moderate food rates to begin with. To try to model these peck rates with optimality or matching theory, one needs to make additional assumptions. Although the present model requires no assumption about ro , optimality and matching require a feedback function for ro , possibly Eq. (7), which has some empirical support (Baum and Davison, 2014b). One must also assume a feedback function for rN , which would probably differ from direct proportionality, because BN represents the activities of the organism when it has nothing better to do—that is, background activity as originally conceived (Herrnstein, 1970). A possibility suggested by Baum and Davison (2014b) is: rN =
1 1 VN
+
A BN
(13)
which resembles a VI feedback function with A accounting for bursts in BN and VN the asymptote as BN becomes large. With these assumed feedback functions, the VR peck rates were readily fitted by both optimality (maximizing r + ro + rN ) and matching (Eq. (8)). Neither optimality nor matching was able to capture the up-turn in VI peck rates in the mid-range of food rates. Fig. 6 compares the results of applying Eq. (8) (matching) with the results of applying the present model to the peck rates averaged across pigeons (means; medians were almost identical to the means). Because of the large number of parameters, the curves were fitted by eye, holding most parameters constant and using Microsoft
Please cite this article in press as: Baum, W.M., The role of induction in operant schedule performance. Behav. Process. (2015), http://dx.doi.org/10.1016/j.beproc.2015.01.006
210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228
229
230 231 232 233 234 235 236 237 238 239 240
G Model BEPROC 2990 1–8
ARTICLE IN PRESS W.M. Baum / Behavioural Processes xxx (2015) xxx–xxx
5
Fig. 4. Pecks per minute versus food per minute in the VR component of Baum (1993) fitted with the induction model assuming power-function induction (Eqs. (11) and (12)).
255
Excel’s Solver to adjust one or two at a time. The top right graph shows the fit of the induction model to the VR peck rates; it resembles those in Fig. 4. The top left graph shows the induction model fitted to the mean VI peck rates. The fit resembles those in Fig. 3 in that it is close but leaves the inflection point below the curve. The lower right graph shows the results of applying Eq. (8) (matching) to the VR peck rates. The fit is just as good as to the induction model. The lower left graph shows the results of applying Eq. (8) to the VI peck rates. It illustrates the failure of Eq. (8) to account for the mid-range up-turn; it is only able to produce a bitonic function. Neither matching (Eq. (8)) nor optimality (not shown) was able to account for the up-turn. Killeen (1994) “mathematical principles of reinforcement,” with an equation similar to Eq. (3), likewise can account for the down-turn at high food rates, but not the up-turn in the mid-range.
256
2. Further development
241 242 243 244 245 246 247 248 249 250 251 252 253 254
257 258 259 260 261 262 263 264 265 266 267 268 269 270 271
As Fig. 2 indicates, the region of low-to-moderate food rates missing from the VI peck rates also must be included in a full account of VI performance. That range of food rates has often been described by the hyperbola in Eq. (3) (Herrnstein, 1970). VI peck rates in that range conform to a concave-downward pattern, only the tail end of which is shown in Fig. 2. Although the present induction model fits the VI peck rates well (Fig. 3), its failure to accommodate the inflection point marking the transition from the concave-downward pattern to the concave-upward pattern requires further development. The up-turn in the mid-range of food rates may result from a shift in topography of key pecking as the rich VI schedules come more and more to resemble ratio schedules. The resemblance may be understood as resulting from the change in shape of the VI feedback function. Though negatively accelerated, the VI feedback
function becomes closer and closer to linear as the asymptotic food rate increases. The changed topography of pecking would resemble more and more the topography characteristic of pecking on VR schedules reported by Palya (1992) as “flicking” at the key, called “swiping” at the key by Baum and Davison (2014b), that results in multiple operations of the key for a single backand-forth movement of the pigeon’s head. Thus, the up-turn in peck rates might represent a mix of two topographies—say, multioperation (flicking or swiping) and single-operation (pecking), with
Fig. 5. Comparison of fitting parameters for VI peck rates (Fig. 3) with those for VR peck rates (Fig. 4).
Please cite this article in press as: Baum, W.M., The role of induction in operant schedule performance. Behav. Process. (2015), http://dx.doi.org/10.1016/j.beproc.2015.01.006
272 273 274 275 276 277 278 279 280
G Model
ARTICLE IN PRESS
BEPROC 2990 1–8
W.M. Baum / Behavioural Processes xxx (2015) xxx–xxx
6
Fig. 6. Comparison of the induction model with matching theory applied to mean peck rates from Baum (1993). Top: the induction model fitted to VI peck rates (left) and to VR peck rates (right). Bottom: matching theory fitted to the VI peck rates (left) and to VR peck rates (right). Matching theory cannot accommodate the concave-upward range of peck rates.
281 282 283 284 285 286 287 288 289 290 291 292
293
294
295
multi-operation increasingly replacing single-operation as food rate increases from moderate to high. This change in topography might also explain the upturn in peck rates at high food rates that Baum and Davison (2014b) attributed to pecking B replacing BO . If so, one might assume that BO always replaces pecking. This possibility remains to be explored. For matching theory, the shift in topography would imply two different activities with two different tempos (Baum and Rachlin, 1969; Gilbert, 1958). The single-operation activity, BS , would have different units and a different asymptotic rate, KS , from the multioperation activity, BM , which would have the asymptotic rate KM . Eq. (8) would be replaced by: BM + uBS r = BM + uBS + BO + BN r + rO + rN where, u =
KM KS
(14)
as a correction for the different units and
KM = BM + uBS + BO + BN
.
302
One way optimality theory could approach the performance would be to suppose a four-dimensional space with performance minimizing distance from a bliss point (e.g., Rachlin and Burkhard, 1978; Staddon, 1983). For the present line of theory based on induction, the solution is much simpler. We need only assume that BM is induced according to a power function:
303
BM = cM r SM
296 297 298 299 300 301
(15)
305
and that in the hierarchy of activities, BM replaces BS , represented as:
306
BN < BS < BM < BO ,
304
implying that BO replaces BM at the extreme food rates. Fig. 7 illustrates the way the model would accommodate the three regions of VI peck rates. The squares show the mean peck rates across the four pigeons. In the low-to-moderate range of food rate, BS replaces BN (BN < BS ; lower solid line). In the moderate-to-high range, BM replaces BS (BS < BM ; broken line). In the extreme rates, BO replaces BM (BM < BO ; upper solid line). How would the induction model treat the low-to-moderate range of food rate that is missing from the present data set? Assuming power-function induction (Eqs. (11) and (12)), we may conclude
Fig. 7. The three ranges of VI peck rates. In the low-to-moderate range of food rates (solid curve), pecking (BS ) replaces true background activities (BN ), in the moderateto-high range (broken line), “flicking” or “swiping” the key (BM ) replaces BS , and in the extremely high range (solid line), other food-induced activities (BO ) replace BM .
Please cite this article in press as: Baum, W.M., The role of induction in operant schedule performance. Behav. Process. (2015), http://dx.doi.org/10.1016/j.beproc.2015.01.006
307 308 309 310 311 312 313 314 315 316
G Model
ARTICLE IN PRESS
BEPROC 2990 1–8
W.M. Baum / Behavioural Processes xxx (2015) xxx–xxx
7
Fig. 8. Comparison of Herrnstein’s hyperbola (Eq. (3)) with two induction-based models fitted to data from Catania and Reynolds (1968). The “Partition” model applies a matching-like relation to all food-induced activities (Eq. (16)). The “Difference” model assumes simply assumes that other food-induced activities interfere with pecking (Eq. (17)).
317 318 319 320 321 322 323 324 325 326 327 328 329 330
331
332 333 334 335 336 337
that, in that range, BS and BO both replace BN , but not independently, because if they replaced BN independently, the function relating BS to r would be a power function and would appear in logarithmic coordinates as a straight line. The relation between B and r in that range, however, is concave downward in logarithmic coordinates (Baum, 1981; Baum and Davison, 2014b). Thus, BS and BO must compete, even as they replace BN , and we require a model of that competition. At least two possible models of the competition arise. One approach would follow Baum and Davison’s reliance on the matching relation as expressed in Fig. 3, but with an important difference: K would be replaced with a variable B’ that is a power function of r, as in Eq. (12), but B’ would represent, not measured peck rate, but induced activity, which would result in measured pecking according to: B=
B, r r + rO
=
cr s+1 r + cO r sO
(16)
recalling that ro depends on r (Eqs. (10) and (11); the parameter V is absorbed into co ). Eq. (16) represents a model in which the total activity induced B’ is partitioned between B and BO according to the proportion of the reinforcer rates r and ro . A second possible model would assume simply that BO replaces pecking in the low-to-moderate range as well as the higher ranges
of r. Measured pecking B would equal the difference between induced pecking B’ and induced other activities BO : s
B = cr − cO r
sO
(17)
Fig. 8 shows the results of fitting Herrnstein’s hyperbola (Eq. (3)), Eq. (16) (“Partition”), and Eq. (17) (“Difference”) to the VI peck rates reported by Catania and Reynolds (1968) and used by Herrnstein (1970; his Fig. 8) to validate the hyperbola. All three equations fit the peck rates (diamonds) well and about equally well. Eqs. (16) and (17), though based on different assumptions, are practically indistinguishable. Further research will be required to decide between them. 3. Conclusions Extending Segal’s (1972) concept of induction to operant activities and quantifying it by assuming power-function induction affords quantitative accounts of operant performance that are plausible and simple (Baum, 2012; Baum and Davison, 2014b). Although VR performance conforms to a simple bitonic inverted U-shape (Figs. 1, 4, and 6), that is relatively easy to account for with matching, optimality, and induction, VI performance presents a more complicated picture. VI performance may be divided into
Please cite this article in press as: Baum, W.M., The role of induction in operant schedule performance. Behav. Process. (2015), http://dx.doi.org/10.1016/j.beproc.2015.01.006
338 339 340 341 342 343 344 345 346 347 348
349
350 351 352 353 354 355 356 357
G Model BEPROC 2990 1–8
ARTICLE IN PRESS W.M. Baum / Behavioural Processes xxx (2015) xxx–xxx
8
382
three ranges of food rate: low-to-moderate, moderate-to-high, and extremely high (Figs. 1, 2, and 7). In the low-to-moderate range, peck rates display a concave downward relation to food rate that has often been fitted with the hyperbola in Eq. (3) in the past. An inflection point occurs in the moderate range as the VI peck rates turn from the concave downward relation to a concave upward relation in the moderate-to-high range (Figs. 1, 2, and 7). Matching theory and optimality do not accommodate this shift (Fig. 6), at least not without many more assumptions. Finally, in the extremely high range, VI peck rate falls as r increases to the maximum (Figs. 1, 2, and 7). A model based on power-function induction fitted VI peck rates from the moderate to the extreme food rates (Figs. 3 and 6), but it failed to pick out the inflection point because the low-to-moderate range of food rate was largely missing from the data (Baum, 1993). A full account based on induction appears to be possible, because at least two models of competition between pecking and other activities are possible, as shown in Fig. 8. Further research with a still broader range of food rates should allow the induction model to be applied to the full range of VI peck rates. Further research might also explore the generality of power-function induction, hierarchical replacement, and topography shifts in other situations and other species. Although evidence thus far suggests that the model applies to pigeon’s pecking and rat’s lever pressing (Baum and Davison, 2014b), only more tests with wide ranges of food rates and other species will tell.
383
References
358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381
384 385 386 387 388 389 390 391 392
Baum, W.M., 1974. On two types of deviation from the matching law: bias and undermatching. J. Exp. Anal. Behav. 22, 231–242. Baum, W.M., 1979. Matching, undermatching: and overmatching in studies of choice. J. Exp. Anal. Behav. 32, 269–281. Baum, W.M., 1981. Optimization and the matching law as accounts of instrumental behavior. J. Exp. Anal. Behav. 36, 387–403. Baum, W.M., 1992. In search of the feedback function for variable-interval schedules. J. Exp. Anal. Behav. 57, 365–375. Baum, W.M., 1993. Performances on ratio and interval schedules of reinforcement: data and theory. J. Exp. Anal. Behav. 59, 245–264.
Baum, W.M., 2012. Rethinking reinforcement: allocation, induction: and contingency. J. Exp. Anal. Behav. 97, 101–124. Baum, W.M., Davison, M., 2014a. Choice with frequently-changing food rates and food ratios. J. Exp. Anal. Behav. 101, 246–274. Baum, W.M., Davison, M., 2014b. Background activities, induction, and behavioral allocation in operant performance. J. Exp. Anal. Behav. 102, 213–230. Baum, W.M., Rachlin, H.C., 1969. Choice as time allocation. J. Exp. Anal. Behav. 12, 861–874. Breland, K., Breland, M., 1961. The misbehavior of organisms. Am. Psychol. 16, 681–684. Catania, A.C., Reynolds, G.S., 1968. A quantitative analysis of the responding maintained by interval schedules of reinforcement. J. Exp. Anal. Behav. 11, 327–383. Davison, M., 1993. On the dynamics of behavior allocation between simultaneously and successively available reinforcer sources. Behav. Proces. 29, 49–64. Davison, M., 2004. Interresponse times and the structure of choice. Behav. Proces. 66, 173–187. Davison, M., McCarthy, D., 1988. The Matching Law: A Research Review. Erlbaum Associates, Hillsdale, NJ. de Villiers, P., 1977. Choice in concurrent schedules and a quantitative formulation of the law of effect. In: Honig, W.K., Staddon, J.E.R. (Eds.), Handbook of Operant Behavior. Prentice-Hall, Englewood Cliffs, NJ, pp. 233–287. Gilbert, T.F., 1958. Fundamental dimensional properties of the operant. Psychol. Rev. 65, 272–285. Herrnstein, R.J., 1961. Relative and absolute strength of response as a function of frequency of reinforcement. J. Exp. Anal. Behav. 4, 267–272. Herrnstein, R.J., 1970. On the law of effect. J. Exp. Anal. Behav. 13, 243–266. Killeen, P.R., 1994. Mathematical principles of reinforcement. Behav. Brain Sci. 17, 105–171. Lima, S.L., Dill, L.M., 1990. Behavioral decisions made under the risk of predation: a review and prospectus. Can. J. Zool. 68, 619–640. Palya, W.L., 1992. Dynamics in the fine structure of schedule-controlled behavior. J. Exp. Anal. Behav. 57, 267–287. Rachlin, H., Burkhard, B., 1978. The temporal triangle: response substitution in instrumental conditioning. Psychol. Rev. 85, 22–47. Segal, E.F., 1972. Induction and the provenance of operants. In: Gilbert, R.M., Millenson, J.R. (Eds.), Reinforcement: Behavioral Analyses. Academic, New York, pp. 1–34. Soto, P.L., McDowell, J.J., Dallery, J., 2005. Effects of adding a second reinforcement alternative: Implications for Herrnstein’s interpretation of re . J. Exp. Anal. Behav. 84, 185–225. Staddon, J.E.R., 1977. Schedule-induced behavior. In: Honig, W.K., Staddon, J.E.R. (Eds.), Handbook of Operant Behavior. Prentice-Hall, Englewood Cliffs, NJ, pp. 125–152. Staddon, J.E.R., 1983. Adaptive Behavior and Learning. Cambridge University Press, Cambridge.
Please cite this article in press as: Baum, W.M., The role of induction in operant schedule performance. Behav. Process. (2015), http://dx.doi.org/10.1016/j.beproc.2015.01.006
393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438
ARTICLE IN PRESS
BEPROC 2990 1–8
Behavioural Processes xxx (2015) xxx–xxx
Contents lists available at ScienceDirect
Behavioural Processes journal homepage: www.elsevier.com/locate/behavproc
1
Q1
The role of induction in operant schedule performance
2
Q2
William M. Baum a,b,∗
3
Q3
a
4
b
University of California, Davis, USA University of New Hampshire, USA
5
6 18
a r t i c l e
i n f o
a b s t r a c t
7 8 9
Article history: Available online xxx
10 11 12 13 14 15 16 17
19 20 21 22 23 24 25 26 27 28 29
30
31 32 33 34
35
Keywords: Competition Induction Matching law Reinforcement Variable-interval schedule Variable-ratio schedule
Baum and Davison (2014b) showed that Baum’s (2012) recasting of reinforcement as induction may be quantified by assuming that induction follows a power function of reinforcer rate. This power-function induction is readily integrated with theory based on the matching law. Herrnstein (1970) originally assumed background activities (BO ) and their associated reinforcers ro to be constant, but ro should vary with BO . Further, power-function induction implies that BO should vary with reinforcer rate. Baum (1993) reported performance on a wide range of variable-ratio (VR) and variable-interval (VI) schedules. Pigeon’s VR peck rate followed an inverted U-shaped relation, but VI peck rate separated into three ranges of food rate: low-to-moderate, moderate-to-high, and extremely high. As food rate increases, the concave downward relation in the low range reaches an inflection point and gives way to a concave upward relation in the higher range. At the extremes of food rate, VI peck rate decreases. A model based on competition between induced pecking and BO accounted for VI peck rate in the moderate to extreme range of food rates. Further research will account for all three ranges, either by integrating power-function induction with matching theory or with a model based on competition between induced activities. © 2015 Published by Elsevier B.V.
A previous paper (Baum, 2012) recasts the various processes of reinforcement and punishment, stimulus control, classical conditioning, adjunctive behavior, and instinctive behavior as just one process, induction, as defined by Segal (1972). A subsequent paper (Baum and Davison, 2014b) began developing a quantitative model of induction that accounts for operant performance on variableinterval (VI) schedules and concurrent VI VI schedules. The present paper takes a further step toward quantifying induction and integrating it with the matching law (Herrnstein, 1961). Herrnstein (1961) originally presented the matching law as a relation between two behavioral alternatives in the form: B1 r1 = B1 + B2 r1 + r2
∗ Correspondence address: 611 Mason #504, San Francisco, CA 94108, USA. Tel.: +1 415 345 0050. E-mail address: [email protected]
B=
Kr r + rO
(3)
36 37 38 39 40 41
42
(1)
where, B1 and B2 are response rates or times spent at Alternatives 1 and 2, and r1 and r2 are reinforcer rates at Alternatives 1 and 2. Herrnstein (1970) subsequently proposed generalizing the matching law to any number of alternatives n in the form: Bi r = i ˙B ˙r
where, Bi represents response rate or time spent at any one of n alternatives, B represents the total of responding or time spent at the n alternatives, ri represents reinforcer rate at any of the n alternatives, and r represents the total or the n reinforcer rates. Using Eq. (2), Herrnstein (1970) derived an equation for responding at just one recorded alternative:
(2)
where, K replaces B on the assumption that other, unmeasured, activities occur and that, with BO representing those other activities, and r = r + ro , with ro representing reinforcers due to BO . Herrnstein (1970) fitted Eq. (3) to several data sets from Catania and Reynolds (1968). Subsequently de Villiers (1977) fitted it to additional data sets, and it has generally proven successful in describing performance across variable-interval (VI) schedules. One feature of the fits to Eq. (3) seems incorrect, however: ro is assumed to be constant as r varies. Baum (1981) and Davison (1993, 2004) pointed out that this assumed constancy is inconsistent with our general understanding of reinforcement contingencies, because BO must vary as B varies, and ro should vary with BO . By definition, a contingency creates a dependence of reinforcer rate on response
http://dx.doi.org/10.1016/j.beproc.2015.01.006 0376-6357/© 2015 Published by Elsevier B.V.
Please cite this article in press as: Baum, W.M., The role of induction in operant schedule performance. Behav. Process. (2015), http://dx.doi.org/10.1016/j.beproc.2015.01.006
43 44 45 46 47 48 49 50 51 52 53 54 55
G Model BEPROC 2990 1–8
W.M. Baum / Behavioural Processes xxx (2015) xxx–xxx
2 56 57
58
59 60
61
ARTICLE IN PRESS
rate; such a dependence is called a feedback function. For example, the feedback function for a variable-ratio (VR) schedule is given by: B r= V
(4)
where, V is the average number of responses required per reinforcer, and the feedback function for a VI schedule is approximately: 1 r= t + Ba
(5)
64
where, t is the average interval and a is a constant that represents a tendency to bursts at low reinforcer rates (Baum, 1992). Thus, a feedback function should exist between ro and BO :
65
ro = f (BO )
62 63
(6)
69
At least two questions arise: (1) what is BO ? and (2) what is the feedback function f relating ro to BO ? Baum (1981) and Davison (1993) suggested that the function should have characteristics of a ratio schedule (Eq. (4))—that is,
70
rO =
66 67 68
71 72 73 74 75 76 77 78 79 80 81 82 83
BO V
(7)
The basis for this conjecture was that BO activities ought to produce ro directly, with no time-limiting factor such as would characterize an interval schedule (Eq. (5)). Davison (2004) found evidence that BO is not one activity but a conglomerate, but Baum and Davison (2014b) found that treating BO as one activity allowed calculating variation in ro and discovering that Eq. (7) is the feedback function relating ro to BO , at least at high reinforcer (food) rates. An earlier paper (Baum, 2012), relying on the process of induction outlined by Segal (1972), suggested that phylogenetically important events induce activities denoted adjunctive or interim or terminal (Staddon, 1977). Following this reasoning, BO would be induced by a reinforcer such as food, and BO would depend on the
food rate r. Baum and Davison (2014b) found that BO varied with r and, through this variation and Eq. (7), that ro varied with r. Thus, at least part of the other activities represented in Eq. (6) as BO and implicit in Herrnstein’s hyperbola (Eq. (3)) is induced by the food (r). To be accurate, Baum and Davison (2014b) proposed that Eq. (3) should be modified to include activities unrelated to the food rate r—what Staddon (1977) called “facultative” activities. They represented these activities as BN and the reinforcers associated with BN as rN : B=
Kr r + r O + rN
(8)
with the understanding that K equals B + BO + BN and that some function g relates BO to r: BO = g(r)
(9)
Eq. (9) results in ro depending on r indirectly: rO =
g(r) V
BO = co r
85 86 87 88 89 90 91 92
93
94 95 96 97
(10)
To fit data from two large data sets (Baum and Davison, 2014a; Soto, McDowell, & Dallery, 2005), Baum and Davison (2014b) assumed that induction follows a power function: so
84
(11)
where, the exponent so may be thought of as the sensitivity of BO to r, and the coefficient co accounts for reconciling of units. Baum (1993) A data set that allows testing these proposals further for both VI and VR schedules was gathered in an experiment reported in 1993. The procedure was a two-component multiple schedule in which a VR component alternated with a VI component separated by substantial time-outs in between. The intervals generated in the VR component were played back in the VI component to roughly
Fig. 1. Pecks per minute versus food per minute from Baum (1993). Data are from a multiple VR VI schedule in which the VI component was yoked to the VR component.
Please cite this article in press as: Baum, W.M., The role of induction in operant schedule performance. Behav. Process. (2015), http://dx.doi.org/10.1016/j.beproc.2015.01.006
98
99 100 101 102 103 104 105 106 107 108 109 110 111
G Model BEPROC 2990 1–8
ARTICLE IN PRESS W.M. Baum / Behavioural Processes xxx (2015) xxx–xxx
112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145
equate the intervals and food rates in the two. The VR schedule varied across conditions, and each condition continued in daily sessions until both performances appeared stable, whereupon a new condition began. The VR was varied over as large a range as possible, from a large VR that produced a moderate food rate just sufficient to maintain responding to FR 1, the endpoint of ratio schedules. Since ratio schedules exhibit “ratio strain” when they are relatively large, no low food rates were possible, but food rate could increase up to the extreme at the high end with FR 1. Fig. 1 shows peck rate as a function of food rate for the four pigeons in the experiment. Most conditions were presented twice, and Fig. 1 shows the average peck rate across presentations. Performance on the VR schedules was simply a bitonic or upside-down U-shape, and peck rates were generally higher for the VR than the yoked VI. Performance on the VI schedules was more complex. Most notably, whereas the VR curve was concave downwards, the VI curve was concave upwards in the mid-range of food rates, which differed from pigeon to pigeon. Indeed, at least three of the pigeon’s curves show a clear inflection point: at about 11 fpm for B258; at about 5 fpm for B261 and B122; although less pronounced, at about 6 fpm for B348. These inflection points suggest that peck rate was relatively flat in the low range, but because food rates lower than about 0.4 per minute, equivalent to a VI 150s, could not be maintained—except for B261, which maintained pecking at 0.13 fpm, equivalent to a VI 460s. Previous research supports the existence of an inflection point, because a negatively accelerated pattern of responding in the lower range of food rates is well documented (e.g., Catania & Reynolds, 1968; de Villiers, 1977), and the shift from negative acceleration in the lower range to positive acceleration in the higher range requires an inflection point. All four pigeon’s peck rates decreased at the highest food rates. Since the VI performances included a relatively flat peck rate in the low range of food rates, an inflection point followed by an up-turn in peck rate, and a down-turn at the highest food rates, the
3
VI curves might be composed of three regions: low to moderate food rates, moderate to high food rates, and high to extreme food rates. Fig. 2 illustrates this possibility. In the lower, flatter range, the broken line in each graph represents the hyperbola in Eq. (3) fitted to the peck rates. Viewed this way, one sees clearly the up-turn in the mid-range of food rates and the down-turn at the extreme of food rate. Any model of these peck rates should account for both the up-turn and the down-turn. 1. An induction model
147 148 149 150 151 152 153
154
I modeled these data as follows. Paralleling the reasoning of Q4 Baum and Davison (2014b), I made the following assumptions: BO varies with r, because BO is induced by food according to Eq. (11), as assumed by Baum and Davison (2014b) with some empirical support (Staddon, 1977); Pecking also is induced by food, according to: B = cr s
146
(12)
When B and BO compete, BO replaces B: If cr s + co r so > K, then B = K − BO ; that is, the activities exist in a hierarchy, such that B and BO replace BN , and BO replaces B (BN
Fig. 2. Pecks per minute versus food per minute in the VI component of Baum (1993). The broken line represents the fit of Herrnstein’s hyperbola (Eq. (3)) to the low range of food rates. The inflection point between low food rates and high food rates may be seen at the intersection of the broken curve with the solid line.
Please cite this article in press as: Baum, W.M., The role of induction in operant schedule performance. Behav. Process. (2015), http://dx.doi.org/10.1016/j.beproc.2015.01.006
155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176
G Model BEPROC 2990 1–8
ARTICLE IN PRESS W.M. Baum / Behavioural Processes xxx (2015) xxx–xxx
4
Fig. 3. Pecks per minute versus food per minute in the VI component of Baum (1993) fitted with the induction model assuming power-function induction (Eqs. (11) and (12)).
177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209
Bi Bj
=
ci r si s cj r j
, for all i = / j of n operant activities. The power function
has some empirical support too (Staddon, 1977). The model was fitted to the peck rates using Solver in Microsoft Excel, varying c, s, co , and so , after setting K to a rate slightly above the maximum peck rate and placing a minimum of 0.00001 on co . Fig. 3 shows the results of fitting to the VI peck rates. The peck rates are fitted well, with one flaw—for all four pigeons, the inflection point demarcated in Fig. 2 falls below the curve. Because the data set includes no extremely low food rates, the model takes no special account of peck rates in that range. Replacing the hyperbola (Eq. (3); broken lines in Fig. 2) would require food rates in the lower range and additional assumptions about B, BO , and BN in that range. Fig. 4 shows the results of fitting the same model, with the same assumptions, to the VR peck rates. These fits are close and show no systematic deviation from the data. As might be expected from the simple bitonic shape, these peck rates were relatively easy to fit with Excel Solver, varying c, s, co , so , and K. They raise no problem about low food or peck rates. An immediate question to ask is whether the model’s parameters that best fitted the peck rates remained the same for the VI rates and the VR rates. Fig. 5 makes the comparison. It shows each VI parameter plotted against the corresponding VR parameter for each pigeon. The estimates of K (diamonds) were about the same, lying on the major diagonal (equality). The estimates of c and s (squares and triangles) also were about the same, indicating that the power function for the inducing of pecking was about the same for VI and VR. Notable differences occurred for co and so , however, indicating that the power function for the inducing of BO differed for the VI and VR. In fitting the model, the estimates of co , in particular, tended to be extremely small, and necessitated the lower limit of 0.00001 for co . All four estimates of co were far lower for the VI than the VR. In contrast, all four estimates of so were higher for the VI than for the VR. The reason why induction of BO would differ for
VI and VR schedules remains to be discovered, but probably stems from the more abrupt down-turn in the VI peck rates at the higher food rates. The combination of a small coefficient co with a large exponent so caused BO to be tiny at lower food rates but to ramp up quickly at higher food rates, thus allowing the model to fit that sharp downturn in VI peck rates. The large coefficient c paired with an exponent s less than 1.0 modelled a more gradual increase in peck rate, starting at moderately high levels at moderate food rates to begin with. To try to model these peck rates with optimality or matching theory, one needs to make additional assumptions. Although the present model requires no assumption about ro , optimality and matching require a feedback function for ro , possibly Eq. (7), which has some empirical support (Baum and Davison, 2014b). One must also assume a feedback function for rN , which would probably differ from direct proportionality, because BN represents the activities of the organism when it has nothing better to do—that is, background activity as originally conceived (Herrnstein, 1970). A possibility suggested by Baum and Davison (2014b) is: rN =
1 1 VN
+
A BN
(13)
which resembles a VI feedback function with A accounting for bursts in BN and VN the asymptote as BN becomes large. With these assumed feedback functions, the VR peck rates were readily fitted by both optimality (maximizing r + ro + rN ) and matching (Eq. (8)). Neither optimality nor matching was able to capture the up-turn in VI peck rates in the mid-range of food rates. Fig. 6 compares the results of applying Eq. (8) (matching) with the results of applying the present model to the peck rates averaged across pigeons (means; medians were almost identical to the means). Because of the large number of parameters, the curves were fitted by eye, holding most parameters constant and using Microsoft
Please cite this article in press as: Baum, W.M., The role of induction in operant schedule performance. Behav. Process. (2015), http://dx.doi.org/10.1016/j.beproc.2015.01.006
210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228
229
230 231 232 233 234 235 236 237 238 239 240
G Model BEPROC 2990 1–8
ARTICLE IN PRESS W.M. Baum / Behavioural Processes xxx (2015) xxx–xxx
5
Fig. 4. Pecks per minute versus food per minute in the VR component of Baum (1993) fitted with the induction model assuming power-function induction (Eqs. (11) and (12)).
255
Excel’s Solver to adjust one or two at a time. The top right graph shows the fit of the induction model to the VR peck rates; it resembles those in Fig. 4. The top left graph shows the induction model fitted to the mean VI peck rates. The fit resembles those in Fig. 3 in that it is close but leaves the inflection point below the curve. The lower right graph shows the results of applying Eq. (8) (matching) to the VR peck rates. The fit is just as good as to the induction model. The lower left graph shows the results of applying Eq. (8) to the VI peck rates. It illustrates the failure of Eq. (8) to account for the mid-range up-turn; it is only able to produce a bitonic function. Neither matching (Eq. (8)) nor optimality (not shown) was able to account for the up-turn. Killeen (1994) “mathematical principles of reinforcement,” with an equation similar to Eq. (3), likewise can account for the down-turn at high food rates, but not the up-turn in the mid-range.
256
2. Further development
241 242 243 244 245 246 247 248 249 250 251 252 253 254
257 258 259 260 261 262 263 264 265 266 267 268 269 270 271
As Fig. 2 indicates, the region of low-to-moderate food rates missing from the VI peck rates also must be included in a full account of VI performance. That range of food rates has often been described by the hyperbola in Eq. (3) (Herrnstein, 1970). VI peck rates in that range conform to a concave-downward pattern, only the tail end of which is shown in Fig. 2. Although the present induction model fits the VI peck rates well (Fig. 3), its failure to accommodate the inflection point marking the transition from the concave-downward pattern to the concave-upward pattern requires further development. The up-turn in the mid-range of food rates may result from a shift in topography of key pecking as the rich VI schedules come more and more to resemble ratio schedules. The resemblance may be understood as resulting from the change in shape of the VI feedback function. Though negatively accelerated, the VI feedback
function becomes closer and closer to linear as the asymptotic food rate increases. The changed topography of pecking would resemble more and more the topography characteristic of pecking on VR schedules reported by Palya (1992) as “flicking” at the key, called “swiping” at the key by Baum and Davison (2014b), that results in multiple operations of the key for a single backand-forth movement of the pigeon’s head. Thus, the up-turn in peck rates might represent a mix of two topographies—say, multioperation (flicking or swiping) and single-operation (pecking), with
Fig. 5. Comparison of fitting parameters for VI peck rates (Fig. 3) with those for VR peck rates (Fig. 4).
Please cite this article in press as: Baum, W.M., The role of induction in operant schedule performance. Behav. Process. (2015), http://dx.doi.org/10.1016/j.beproc.2015.01.006
272 273 274 275 276 277 278 279 280
G Model
ARTICLE IN PRESS
BEPROC 2990 1–8
W.M. Baum / Behavioural Processes xxx (2015) xxx–xxx
6
Fig. 6. Comparison of the induction model with matching theory applied to mean peck rates from Baum (1993). Top: the induction model fitted to VI peck rates (left) and to VR peck rates (right). Bottom: matching theory fitted to the VI peck rates (left) and to VR peck rates (right). Matching theory cannot accommodate the concave-upward range of peck rates.
281 282 283 284 285 286 287 288 289 290 291 292
293
294
295
multi-operation increasingly replacing single-operation as food rate increases from moderate to high. This change in topography might also explain the upturn in peck rates at high food rates that Baum and Davison (2014b) attributed to pecking B replacing BO . If so, one might assume that BO always replaces pecking. This possibility remains to be explored. For matching theory, the shift in topography would imply two different activities with two different tempos (Baum and Rachlin, 1969; Gilbert, 1958). The single-operation activity, BS , would have different units and a different asymptotic rate, KS , from the multioperation activity, BM , which would have the asymptotic rate KM . Eq. (8) would be replaced by: BM + uBS r = BM + uBS + BO + BN r + rO + rN where, u =
KM KS
(14)
as a correction for the different units and
KM = BM + uBS + BO + BN
.
302
One way optimality theory could approach the performance would be to suppose a four-dimensional space with performance minimizing distance from a bliss point (e.g., Rachlin and Burkhard, 1978; Staddon, 1983). For the present line of theory based on induction, the solution is much simpler. We need only assume that BM is induced according to a power function:
303
BM = cM r SM
296 297 298 299 300 301
(15)
305
and that in the hierarchy of activities, BM replaces BS , represented as:
306
BN < BS < BM < BO ,
304
implying that BO replaces BM at the extreme food rates. Fig. 7 illustrates the way the model would accommodate the three regions of VI peck rates. The squares show the mean peck rates across the four pigeons. In the low-to-moderate range of food rate, BS replaces BN (BN < BS ; lower solid line). In the moderate-to-high range, BM replaces BS (BS < BM ; broken line). In the extreme rates, BO replaces BM (BM < BO ; upper solid line). How would the induction model treat the low-to-moderate range of food rate that is missing from the present data set? Assuming power-function induction (Eqs. (11) and (12)), we may conclude
Fig. 7. The three ranges of VI peck rates. In the low-to-moderate range of food rates (solid curve), pecking (BS ) replaces true background activities (BN ), in the moderateto-high range (broken line), “flicking” or “swiping” the key (BM ) replaces BS , and in the extremely high range (solid line), other food-induced activities (BO ) replace BM .
Please cite this article in press as: Baum, W.M., The role of induction in operant schedule performance. Behav. Process. (2015), http://dx.doi.org/10.1016/j.beproc.2015.01.006
307 308 309 310 311 312 313 314 315 316
G Model
ARTICLE IN PRESS
BEPROC 2990 1–8
W.M. Baum / Behavioural Processes xxx (2015) xxx–xxx
7
Fig. 8. Comparison of Herrnstein’s hyperbola (Eq. (3)) with two induction-based models fitted to data from Catania and Reynolds (1968). The “Partition” model applies a matching-like relation to all food-induced activities (Eq. (16)). The “Difference” model assumes simply assumes that other food-induced activities interfere with pecking (Eq. (17)).
317 318 319 320 321 322 323 324 325 326 327 328 329 330
331
332 333 334 335 336 337
that, in that range, BS and BO both replace BN , but not independently, because if they replaced BN independently, the function relating BS to r would be a power function and would appear in logarithmic coordinates as a straight line. The relation between B and r in that range, however, is concave downward in logarithmic coordinates (Baum, 1981; Baum and Davison, 2014b). Thus, BS and BO must compete, even as they replace BN , and we require a model of that competition. At least two possible models of the competition arise. One approach would follow Baum and Davison’s reliance on the matching relation as expressed in Fig. 3, but with an important difference: K would be replaced with a variable B’ that is a power function of r, as in Eq. (12), but B’ would represent, not measured peck rate, but induced activity, which would result in measured pecking according to: B=
B, r r + rO
=
cr s+1 r + cO r sO
(16)
recalling that ro depends on r (Eqs. (10) and (11); the parameter V is absorbed into co ). Eq. (16) represents a model in which the total activity induced B’ is partitioned between B and BO according to the proportion of the reinforcer rates r and ro . A second possible model would assume simply that BO replaces pecking in the low-to-moderate range as well as the higher ranges
of r. Measured pecking B would equal the difference between induced pecking B’ and induced other activities BO : s
B = cr − cO r
sO
(17)
Fig. 8 shows the results of fitting Herrnstein’s hyperbola (Eq. (3)), Eq. (16) (“Partition”), and Eq. (17) (“Difference”) to the VI peck rates reported by Catania and Reynolds (1968) and used by Herrnstein (1970; his Fig. 8) to validate the hyperbola. All three equations fit the peck rates (diamonds) well and about equally well. Eqs. (16) and (17), though based on different assumptions, are practically indistinguishable. Further research will be required to decide between them. 3. Conclusions Extending Segal’s (1972) concept of induction to operant activities and quantifying it by assuming power-function induction affords quantitative accounts of operant performance that are plausible and simple (Baum, 2012; Baum and Davison, 2014b). Although VR performance conforms to a simple bitonic inverted U-shape (Figs. 1, 4, and 6), that is relatively easy to account for with matching, optimality, and induction, VI performance presents a more complicated picture. VI performance may be divided into
Please cite this article in press as: Baum, W.M., The role of induction in operant schedule performance. Behav. Process. (2015), http://dx.doi.org/10.1016/j.beproc.2015.01.006
338 339 340 341 342 343 344 345 346 347 348
349
350 351 352 353 354 355 356 357
G Model BEPROC 2990 1–8
ARTICLE IN PRESS W.M. Baum / Behavioural Processes xxx (2015) xxx–xxx
8
382
three ranges of food rate: low-to-moderate, moderate-to-high, and extremely high (Figs. 1, 2, and 7). In the low-to-moderate range, peck rates display a concave downward relation to food rate that has often been fitted with the hyperbola in Eq. (3) in the past. An inflection point occurs in the moderate range as the VI peck rates turn from the concave downward relation to a concave upward relation in the moderate-to-high range (Figs. 1, 2, and 7). Matching theory and optimality do not accommodate this shift (Fig. 6), at least not without many more assumptions. Finally, in the extremely high range, VI peck rate falls as r increases to the maximum (Figs. 1, 2, and 7). A model based on power-function induction fitted VI peck rates from the moderate to the extreme food rates (Figs. 3 and 6), but it failed to pick out the inflection point because the low-to-moderate range of food rate was largely missing from the data (Baum, 1993). A full account based on induction appears to be possible, because at least two models of competition between pecking and other activities are possible, as shown in Fig. 8. Further research with a still broader range of food rates should allow the induction model to be applied to the full range of VI peck rates. Further research might also explore the generality of power-function induction, hierarchical replacement, and topography shifts in other situations and other species. Although evidence thus far suggests that the model applies to pigeon’s pecking and rat’s lever pressing (Baum and Davison, 2014b), only more tests with wide ranges of food rates and other species will tell.
383
References
358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381
384 385 386 387 388 389 390 391 392
Baum, W.M., 1974. On two types of deviation from the matching law: bias and undermatching. J. Exp. Anal. Behav. 22, 231–242. Baum, W.M., 1979. Matching, undermatching: and overmatching in studies of choice. J. Exp. Anal. Behav. 32, 269–281. Baum, W.M., 1981. Optimization and the matching law as accounts of instrumental behavior. J. Exp. Anal. Behav. 36, 387–403. Baum, W.M., 1992. In search of the feedback function for variable-interval schedules. J. Exp. Anal. Behav. 57, 365–375. Baum, W.M., 1993. Performances on ratio and interval schedules of reinforcement: data and theory. J. Exp. Anal. Behav. 59, 245–264.
Baum, W.M., 2012. Rethinking reinforcement: allocation, induction: and contingency. J. Exp. Anal. Behav. 97, 101–124. Baum, W.M., Davison, M., 2014a. Choice with frequently-changing food rates and food ratios. J. Exp. Anal. Behav. 101, 246–274. Baum, W.M., Davison, M., 2014b. Background activities, induction, and behavioral allocation in operant performance. J. Exp. Anal. Behav. 102, 213–230. Baum, W.M., Rachlin, H.C., 1969. Choice as time allocation. J. Exp. Anal. Behav. 12, 861–874. Breland, K., Breland, M., 1961. The misbehavior of organisms. Am. Psychol. 16, 681–684. Catania, A.C., Reynolds, G.S., 1968. A quantitative analysis of the responding maintained by interval schedules of reinforcement. J. Exp. Anal. Behav. 11, 327–383. Davison, M., 1993. On the dynamics of behavior allocation between simultaneously and successively available reinforcer sources. Behav. Proces. 29, 49–64. Davison, M., 2004. Interresponse times and the structure of choice. Behav. Proces. 66, 173–187. Davison, M., McCarthy, D., 1988. The Matching Law: A Research Review. Erlbaum Associates, Hillsdale, NJ. de Villiers, P., 1977. Choice in concurrent schedules and a quantitative formulation of the law of effect. In: Honig, W.K., Staddon, J.E.R. (Eds.), Handbook of Operant Behavior. Prentice-Hall, Englewood Cliffs, NJ, pp. 233–287. Gilbert, T.F., 1958. Fundamental dimensional properties of the operant. Psychol. Rev. 65, 272–285. Herrnstein, R.J., 1961. Relative and absolute strength of response as a function of frequency of reinforcement. J. Exp. Anal. Behav. 4, 267–272. Herrnstein, R.J., 1970. On the law of effect. J. Exp. Anal. Behav. 13, 243–266. Killeen, P.R., 1994. Mathematical principles of reinforcement. Behav. Brain Sci. 17, 105–171. Lima, S.L., Dill, L.M., 1990. Behavioral decisions made under the risk of predation: a review and prospectus. Can. J. Zool. 68, 619–640. Palya, W.L., 1992. Dynamics in the fine structure of schedule-controlled behavior. J. Exp. Anal. Behav. 57, 267–287. Rachlin, H., Burkhard, B., 1978. The temporal triangle: response substitution in instrumental conditioning. Psychol. Rev. 85, 22–47. Segal, E.F., 1972. Induction and the provenance of operants. In: Gilbert, R.M., Millenson, J.R. (Eds.), Reinforcement: Behavioral Analyses. Academic, New York, pp. 1–34. Soto, P.L., McDowell, J.J., Dallery, J., 2005. Effects of adding a second reinforcement alternative: Implications for Herrnstein’s interpretation of re . J. Exp. Anal. Behav. 84, 185–225. Staddon, J.E.R., 1977. Schedule-induced behavior. In: Honig, W.K., Staddon, J.E.R. (Eds.), Handbook of Operant Behavior. Prentice-Hall, Englewood Cliffs, NJ, pp. 125–152. Staddon, J.E.R., 1983. Adaptive Behavior and Learning. Cambridge University Press, Cambridge.
Please cite this article in press as: Baum, W.M., The role of induction in operant schedule performance. Behav. Process. (2015), http://dx.doi.org/10.1016/j.beproc.2015.01.006
393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438