- Email: [email protected]
Heuristics or general learning processes?
Behavioural Processes 69 (2005) 155–157
Commentary
Heuristics or general learning processes? Ben A. Williams ∗ Department of Psychology, University of California, San Diego, CA 92093 0109, USA
Abstract The concept of heuristics implies that the rules governing choice behavior may vary with ecological constraints. Behavior analysis, in contrast, seeks general principles that transcend specific situations. To the extent that search is successful, the concept of heuristics is unlikely to play a significant role in the analysis of animal behavior. © 2005 Elsevier B.V. All rights reserved. Keywords: Delay discounting; Heuristics; Matching law; Optimality theory
The distinction between human decision rules based on simple heuristics versus those putatively more rational is paralleled in the study of operant behavior by the distinction between optimality theories versus those that are based on more molecular principles. Beginning in the 1970s a number of prominent behavior theorists (Baum, 1981; Rachlin et al., 1981; Staddon and Motheral, 1978) argued that optimality models provided a good account of the behavior generated by a variety of schedules of reinforcement, most often choice procedures. But critical tests ultimately showed these accounts to be fundamentally flawed, both at the level of incorrect theoretical predictions (e.g., concurrent VI VR schedules: Heyman and Herrnstein, 1986) and in terms of direct tests of core assumptions (e.g., demonstrations that schedule feedback functions have little direct control of behavior: Vaughan and Miller, 1984; Ettinger et al., 1987). As with “rational” human ∗
Tel.: +1 858 5343938; fax: +1 858 5347190. E-mail address: [email protected].
0376-6357/$ – see front matter © 2005 Elsevier B.V. All rights reserved. doi:10.1016/j.beproc.2005.02.018
decision rules that attempt to take into account all available information, the failure of optimality theories for animal behavior seems to lie with the cognitive limitations of our subjects, specifically with respect to their lack of sensitivity to delayed reinforcement contingencies (Williams, 1992). The problem of impulsiveness is fundamental to understand these limitations in operant procedures, and as the authors note, it also seems problematic for behavioral biologists as well. After the shortcomings of optimality theories are recognized, the parallel between human decisionmaking and animal choice behavior seems not easily extended. Whereas the concept of heuristics implies qualitative changes in choice principles as a function of the ecological situation, the goal of behavior analysis has been to provide general principles that are trans-situational in nature. The matching law is one such general principle. Although there continue to be disputes about the basic underlying mechanisms by which matching is achieved, its essence is a continuous quantitative function between choice behavior and
156
B.A. Williams / Behavioural Processes 69 (2005) 155–157
the relative value of the reinforcers contingent on the different choice alternatives, with value aggregated according to a formula that combines the different sources and attributes of reinforcement. Such an approach stands in substantial contrast to heuristics such as “take the best”, or any other simple “rule of thumb”. Whether truly trans-situational principles of reinforcement actually do apply to animal behavior remains a matter of dispute, as there are numerous examples of learning and behavior that seem to require their own idiosyncratic principles (e.g., bird-song acquisition), which often can be understood in terms of the ecological adaptations for the specific species. No doubt there are numerous exceptions to trans-situational principles that constrain their general applicability, and it is possible that these exceptions may indeed prove amenable to the type of heuristic analysis applied to human decision-making. While some mechanisms of behavior may be limited to specific ecological domains, such specificity is not demanded by evolutionary principles. It is perhaps no coincidence that the animals most often used to study general principles of learning; rats and pigeons, are among the most ubiquitous of creatures, rivaling humans in the diversity of ecological niches in which they have adapted and survived. That a single species can survive in numerous quite different environments argues strongly for general mechanisms of adaptation, perhaps the most important of which are general learning processes. Behavioral principles of enormous generality do exist. One of the most notable examples is the hyperbolic form of delay-of-reinforcement gradients (Mazur, 1987), which forms the foundation of the analysis of self control, including why impulsiveness occurs and the conditions under which commitment procedures can counteract the detrimental effects of impulsiveness (Rachlin and Green, 1972). Not only does the hyperbolic discount function apply to delayed reinforcement procedures using a variety of different animals, it also seems to apply equally well to delay discounting of hypothetical future rewards by humans (Green and Myerson, 2004). Moreover, it provides important insights to significant social problems, such as gambling and other forms of addictive behavior (Madden et al., 1997). It is the search for such general principles that provides the foundation of behavioral analysis as a scientific discipline.
From an evolutionary perspective, general principles such as the hyperbolic discount function or the matching law reflect an intermediate level of adaptation, between the unrealistic assumption that animals are capable of optimization in their decisions, and the more domain-specific principles such as those involved in bird-song learning. General learning principles evolved in order to provide flexibility for adaptation across different ecological contexts while at the same time economizing on the neural resources necessary to cope with diverse environments. The effects of Pavlovian and operant conditioning are pervasive because they presumably meet the satisficing criterion for a range of different situations. However, because they may be sub-optimal adaptations to any given situation, it is not surprising that animals occupying different but stable ecological niches might evolve additional situation-specific behavioral mechanisms that provide an improvement in adaptive function. But neither these specific adaptations nor the general learning principles are true analogues to the heuristics of human choice behavior. Such heuristics can be applied to a range of different situations but cannot be used generally, which necessitates an additional set of rules regulating which heuristic is appropriate to which situations. For behavior analysis, this is an unnecessary intermediate level of analysis that can be set aside by defining the fundamental general learning processes regulating behavior.
References Baum, W.H., 1981. Optimization and the matching law as accounts of instrumental behavior. J. Exp. Anal. Behav. 53, 409–422. Ettinger, R.H., Reid, A.K., Staddon, J.E.R., 1987. J. Exp. Psychol. Anim. Behav. Process. 13, 366–375. Green, L., Myerson, J., 2004. A discounting framework for choice with delayed and probabilistic rewards. Psychol. Bull. 130, 769–792. Heyman, G.M., Herrnstein, R.J., 1986. More on concurrent intervalratio schedules: a replication and review. J. Exp. Anal. Behav. 46, 331–351. Madden, G.J., Petry, N.M., Badger, G.J., Bickel, W.K., 1997. Impulsive and self-control choices in opioid-dependent patients and non-drug-using control participants: drug and monetary rewards. Exp. Clin. Psychopharmcol. 5, 256–262. Mazur, J.E., 1987. An adjusting procedure for studying delayed reinforcement. In: Commons, M.L., Mazur, J.E., Nevin, J.A., Rachlin, H. (Eds.), Quantitative Analyses of Behavior: The Effect of Delay and Intervening Events on Reinforcement Value, vol. 5. Erlbaum, Hillsdale, NJ, pp. 55–73.
B.A. Williams / Behavioural Processes 69 (2005) 155–157 Rachlin, H., Battalio, R., Kagel, J., Green, L., 1981. Maximization theory in behavioral psychology. Behav. Brain Sci. 4, 371–388. Rachlin, H.C., Green, L., 1972. Commitment, choice, and selfcontrol. J. Exp. Anal. Behav. 17, 15–22. Staddon, J.E.R., Motheral, S., 1978. On the matching and maximizing in operant choice experiments. Psychol. Rev. 85, 436–444.
157
Vaughan, W., Miller, H.L., 1984. Optimization versus responsestrength accounts of behavior. J. Exp. Anal. Behav. 42, 337– 348. Williams, B.A., 1992. Dissociation of theories of choice by temporal spacing of choice opportunities. J. Exp. Psychol. Anim. Behav. Process. 18, 287–297.
Commentary
Heuristics or general learning processes? Ben A. Williams ∗ Department of Psychology, University of California, San Diego, CA 92093 0109, USA
Abstract The concept of heuristics implies that the rules governing choice behavior may vary with ecological constraints. Behavior analysis, in contrast, seeks general principles that transcend specific situations. To the extent that search is successful, the concept of heuristics is unlikely to play a significant role in the analysis of animal behavior. © 2005 Elsevier B.V. All rights reserved. Keywords: Delay discounting; Heuristics; Matching law; Optimality theory
The distinction between human decision rules based on simple heuristics versus those putatively more rational is paralleled in the study of operant behavior by the distinction between optimality theories versus those that are based on more molecular principles. Beginning in the 1970s a number of prominent behavior theorists (Baum, 1981; Rachlin et al., 1981; Staddon and Motheral, 1978) argued that optimality models provided a good account of the behavior generated by a variety of schedules of reinforcement, most often choice procedures. But critical tests ultimately showed these accounts to be fundamentally flawed, both at the level of incorrect theoretical predictions (e.g., concurrent VI VR schedules: Heyman and Herrnstein, 1986) and in terms of direct tests of core assumptions (e.g., demonstrations that schedule feedback functions have little direct control of behavior: Vaughan and Miller, 1984; Ettinger et al., 1987). As with “rational” human ∗
Tel.: +1 858 5343938; fax: +1 858 5347190. E-mail address: [email protected].
0376-6357/$ – see front matter © 2005 Elsevier B.V. All rights reserved. doi:10.1016/j.beproc.2005.02.018
decision rules that attempt to take into account all available information, the failure of optimality theories for animal behavior seems to lie with the cognitive limitations of our subjects, specifically with respect to their lack of sensitivity to delayed reinforcement contingencies (Williams, 1992). The problem of impulsiveness is fundamental to understand these limitations in operant procedures, and as the authors note, it also seems problematic for behavioral biologists as well. After the shortcomings of optimality theories are recognized, the parallel between human decisionmaking and animal choice behavior seems not easily extended. Whereas the concept of heuristics implies qualitative changes in choice principles as a function of the ecological situation, the goal of behavior analysis has been to provide general principles that are trans-situational in nature. The matching law is one such general principle. Although there continue to be disputes about the basic underlying mechanisms by which matching is achieved, its essence is a continuous quantitative function between choice behavior and
156
B.A. Williams / Behavioural Processes 69 (2005) 155–157
the relative value of the reinforcers contingent on the different choice alternatives, with value aggregated according to a formula that combines the different sources and attributes of reinforcement. Such an approach stands in substantial contrast to heuristics such as “take the best”, or any other simple “rule of thumb”. Whether truly trans-situational principles of reinforcement actually do apply to animal behavior remains a matter of dispute, as there are numerous examples of learning and behavior that seem to require their own idiosyncratic principles (e.g., bird-song acquisition), which often can be understood in terms of the ecological adaptations for the specific species. No doubt there are numerous exceptions to trans-situational principles that constrain their general applicability, and it is possible that these exceptions may indeed prove amenable to the type of heuristic analysis applied to human decision-making. While some mechanisms of behavior may be limited to specific ecological domains, such specificity is not demanded by evolutionary principles. It is perhaps no coincidence that the animals most often used to study general principles of learning; rats and pigeons, are among the most ubiquitous of creatures, rivaling humans in the diversity of ecological niches in which they have adapted and survived. That a single species can survive in numerous quite different environments argues strongly for general mechanisms of adaptation, perhaps the most important of which are general learning processes. Behavioral principles of enormous generality do exist. One of the most notable examples is the hyperbolic form of delay-of-reinforcement gradients (Mazur, 1987), which forms the foundation of the analysis of self control, including why impulsiveness occurs and the conditions under which commitment procedures can counteract the detrimental effects of impulsiveness (Rachlin and Green, 1972). Not only does the hyperbolic discount function apply to delayed reinforcement procedures using a variety of different animals, it also seems to apply equally well to delay discounting of hypothetical future rewards by humans (Green and Myerson, 2004). Moreover, it provides important insights to significant social problems, such as gambling and other forms of addictive behavior (Madden et al., 1997). It is the search for such general principles that provides the foundation of behavioral analysis as a scientific discipline.
From an evolutionary perspective, general principles such as the hyperbolic discount function or the matching law reflect an intermediate level of adaptation, between the unrealistic assumption that animals are capable of optimization in their decisions, and the more domain-specific principles such as those involved in bird-song learning. General learning principles evolved in order to provide flexibility for adaptation across different ecological contexts while at the same time economizing on the neural resources necessary to cope with diverse environments. The effects of Pavlovian and operant conditioning are pervasive because they presumably meet the satisficing criterion for a range of different situations. However, because they may be sub-optimal adaptations to any given situation, it is not surprising that animals occupying different but stable ecological niches might evolve additional situation-specific behavioral mechanisms that provide an improvement in adaptive function. But neither these specific adaptations nor the general learning principles are true analogues to the heuristics of human choice behavior. Such heuristics can be applied to a range of different situations but cannot be used generally, which necessitates an additional set of rules regulating which heuristic is appropriate to which situations. For behavior analysis, this is an unnecessary intermediate level of analysis that can be set aside by defining the fundamental general learning processes regulating behavior.
References Baum, W.H., 1981. Optimization and the matching law as accounts of instrumental behavior. J. Exp. Anal. Behav. 53, 409–422. Ettinger, R.H., Reid, A.K., Staddon, J.E.R., 1987. J. Exp. Psychol. Anim. Behav. Process. 13, 366–375. Green, L., Myerson, J., 2004. A discounting framework for choice with delayed and probabilistic rewards. Psychol. Bull. 130, 769–792. Heyman, G.M., Herrnstein, R.J., 1986. More on concurrent intervalratio schedules: a replication and review. J. Exp. Anal. Behav. 46, 331–351. Madden, G.J., Petry, N.M., Badger, G.J., Bickel, W.K., 1997. Impulsive and self-control choices in opioid-dependent patients and non-drug-using control participants: drug and monetary rewards. Exp. Clin. Psychopharmcol. 5, 256–262. Mazur, J.E., 1987. An adjusting procedure for studying delayed reinforcement. In: Commons, M.L., Mazur, J.E., Nevin, J.A., Rachlin, H. (Eds.), Quantitative Analyses of Behavior: The Effect of Delay and Intervening Events on Reinforcement Value, vol. 5. Erlbaum, Hillsdale, NJ, pp. 55–73.
B.A. Williams / Behavioural Processes 69 (2005) 155–157 Rachlin, H., Battalio, R., Kagel, J., Green, L., 1981. Maximization theory in behavioral psychology. Behav. Brain Sci. 4, 371–388. Rachlin, H.C., Green, L., 1972. Commitment, choice, and selfcontrol. J. Exp. Anal. Behav. 17, 15–22. Staddon, J.E.R., Motheral, S., 1978. On the matching and maximizing in operant choice experiments. Psychol. Rev. 85, 436–444.
157
Vaughan, W., Miller, H.L., 1984. Optimization versus responsestrength accounts of behavior. J. Exp. Anal. Behav. 42, 337– 348. Williams, B.A., 1992. Dissociation of theories of choice by temporal spacing of choice opportunities. J. Exp. Psychol. Anim. Behav. Process. 18, 287–297.