- Email: [email protected]
Heuristics and general principles of learning
Behavioural Processes 69 (2005) 137–138
Commentary
Heuristics and general principles of learning James E. Mazur ∗ Psychology Department, Southern Connecticut State University, New Haven, CT 06515, USA
Abstract This research on decision-making heuristics is similar to research on animal learning in at least two ways. First, optimality modeling has not proven to be very useful for either research area. Second, both of these research areas seek to find general principles (or heuristics) that are applicable to different species in different settings. However, the basic principles of classical and operant conditioning seem to be more uniform across species and situations, whereas decision-making heuristics can vary for different species and different situations, even for tasks with very similar characteristics. © 2005 Published by Elsevier B.V. Keywords: Heuristics; Optimality modeling; Decision-making
Hutchinson and Gigerenzer (2005) describe many wonderful examples of how both humans and animals use simple heuristics when making decisions and choices. As someone working in the animal learning tradition, I was eager to make comparisons between the authors’ theoretical approach and the approach taken by most researchers who study animal learning, typically by using the paradigms of classical and operant conditioning. There are at least two general similarities between these two approaches. First, both the authors and most researchers in animal learning make little use of optimality modeling, for somewhat different reasons. Hutchinson and Gigerenzer are quite clear about why they do not rely on optimality modeling: they are interested in the processes by which individuals make decisions, and they have clear evidence that both humans and nonhumans usually do ∗
Tel.: +1 203 392 6876; fax: +1 203 392 6805. E-mail address: [email protected].
0376-6357/$ – see front matter © 2005 Published by Elsevier B.V. doi:10.1016/j.beproc.2005.02.013
not make decisions by calculating the optimal solution to a problem. In the field of animal learning, there are some exceptions (e.g., Hackenberg and Hineline, 1992; Rachlin et al., 1976), but the great majority of published studies on classical and operant conditioning make no use of optimality modeling. Why should this be so? In my opinion, the answer is simple: optimality modeling is of little or no use when attempting to uncover the general principles of learning and behavior. To advocates of optimization theory who might not believe this, I offer this challenge: find someone who is skilled in using optimality modeling to make predictions about human or nonhuman behavior, but who has no knowledge of the basic principles of classical and operant conditioning (if such a person exists). Ask this theorist to use optimality modeling to make specific, unambiguous predictions about some of the most basic characteristics of classical or operant conditioning. Could the theorist use optimality modeling to predict the basic phenomena of classical conditioning,
138
J.E. Mazur / Behavioural Processes 69 (2005) 137–138
such as extinction, spontaneous recovery, conditioned inhibition, blocking, or the effects of different temporal arrangements such as long-delay conditioning, trace conditioning, or backward conditioning? In operant conditioning, could the theorist predict the stopand-go pattern of responding on a fixed-ratio schedule, the accelerating “scalloped” responding on a fixedinterval schedule, the pattern of inter-response times on a differential-reinforcement-of-low-rates schedule, or the partial-reinforcement extinction effect? I think it is implausible to suggest that anyone could use optimality modeling to predict even these basic phenomena, let alone more complex findings such as supernormal conditioning, occasion-setting, or the response patterns on concurrent-chains schedules. (However, I have no doubt that a clever theorist could use optimality modeling to derive many of these phenomena after the fact, once provided with the empirical findings.) In short, optimality modeling is not very helpful in the analysis of basic principles of learning, just as it is not very helpful in understanding the heuristics that humans or nonhumans actually use in a decision-making task. A second general characteristic that the authors’ theoretical approach shares with the animal learning tradition is the interest in finding general principles that apply to different tasks and are used by different species. For example, Hutchinson and Gigerenzer argue that the Gaze Heuristic is used by humans, bats, bees, and insects to help them track and capture a moving target. They present evidence that the Take the Best heuristic is used by people deciding which of two cities is bigger and by honeybees selecting flowers. Similarly, most of the phenomena of classical and operant conditioning (those mentioned above and many others) have been observed in many different species and many different learning situations both inside and outside the laboratory. There is, however, one puzzling difference between the heuristics studied by Hutchinson and Gigerenzer and the general principles studied by researchers in animal learning. Many of the phenomena of classical and operant conditioning are indeed very general, in the sense that it is often difficult to find exceptions. If an operant response is extinguished, some time is allowed to pass, and then the individual is reintroduced to the
situation where the response previously occurred, we are very likely to see some spontaneous recovery, no matter what operant response, what reinforcer, or what species is being tested. As another example, no organisms have been found that do not sometimes exhibit “impulsiveness” by choosing a small, more immediate reinforcer over a larger, more delayed reinforcer. This is presumably because all species are adversely affected by delayed reinforcement. In contrast, the heuristics studied by Hutchinson and Gigerenzer seem to be general (used by a number of species, in a variety of different situations) but not universal, because a range of different heuristics may be used for problems with a very similar formal structure. For instance, whereas the Take the Best heuristic is frequently used in choice situations where the alternatives differ along several dimensions, the authors report cases where other decision strategies (e.g., using a weighted average, or following a decision tree) are used instead. Indeed, an important part of their research program involves examining each specific case in enough detail to predict which heuristic is likely to be used, and why. The reason for this difference between the authors’ heuristics and principles of learning is not obvious, at least not to me. Perhaps it arises because the choice situations to which their heuristics apply are more complex, involving multiple cues and multiple dimensions, and multiple sources of uncertainty. Perhaps as choice situations become more complex, the number of heuristics (both the number theoretically possible and the number actually used by some species somewhere) increases as well. Whatever the reason, this certainly makes the analysis of decision-making heuristics a challenging task. References Hackenberg, T.D., Hineline, P.N., 1992. Choice in situations of time-based diminishing returns: immediate versus delayed consequences of action. J. Exp. Anal. Behav. 57, 67–80. Hutchinson, J.M.C., Gigerenzer, G., 2005. Simple heuristics and rules of thumb; where psychologists and behavioural biologists might meet. Behav. Proc. 69, 97–124. Rachlin, H., Green, L., Kagel, J.H., Battalio, R.C., 1976. Economic demand theory and psychological studies of choice. In: Bower, G.H. (Ed.), The Psychology of Learning and Motivation, vol. 10, pp. 129–154.
Commentary
Heuristics and general principles of learning James E. Mazur ∗ Psychology Department, Southern Connecticut State University, New Haven, CT 06515, USA
Abstract This research on decision-making heuristics is similar to research on animal learning in at least two ways. First, optimality modeling has not proven to be very useful for either research area. Second, both of these research areas seek to find general principles (or heuristics) that are applicable to different species in different settings. However, the basic principles of classical and operant conditioning seem to be more uniform across species and situations, whereas decision-making heuristics can vary for different species and different situations, even for tasks with very similar characteristics. © 2005 Published by Elsevier B.V. Keywords: Heuristics; Optimality modeling; Decision-making
Hutchinson and Gigerenzer (2005) describe many wonderful examples of how both humans and animals use simple heuristics when making decisions and choices. As someone working in the animal learning tradition, I was eager to make comparisons between the authors’ theoretical approach and the approach taken by most researchers who study animal learning, typically by using the paradigms of classical and operant conditioning. There are at least two general similarities between these two approaches. First, both the authors and most researchers in animal learning make little use of optimality modeling, for somewhat different reasons. Hutchinson and Gigerenzer are quite clear about why they do not rely on optimality modeling: they are interested in the processes by which individuals make decisions, and they have clear evidence that both humans and nonhumans usually do ∗
Tel.: +1 203 392 6876; fax: +1 203 392 6805. E-mail address: [email protected].
0376-6357/$ – see front matter © 2005 Published by Elsevier B.V. doi:10.1016/j.beproc.2005.02.013
not make decisions by calculating the optimal solution to a problem. In the field of animal learning, there are some exceptions (e.g., Hackenberg and Hineline, 1992; Rachlin et al., 1976), but the great majority of published studies on classical and operant conditioning make no use of optimality modeling. Why should this be so? In my opinion, the answer is simple: optimality modeling is of little or no use when attempting to uncover the general principles of learning and behavior. To advocates of optimization theory who might not believe this, I offer this challenge: find someone who is skilled in using optimality modeling to make predictions about human or nonhuman behavior, but who has no knowledge of the basic principles of classical and operant conditioning (if such a person exists). Ask this theorist to use optimality modeling to make specific, unambiguous predictions about some of the most basic characteristics of classical or operant conditioning. Could the theorist use optimality modeling to predict the basic phenomena of classical conditioning,
138
J.E. Mazur / Behavioural Processes 69 (2005) 137–138
such as extinction, spontaneous recovery, conditioned inhibition, blocking, or the effects of different temporal arrangements such as long-delay conditioning, trace conditioning, or backward conditioning? In operant conditioning, could the theorist predict the stopand-go pattern of responding on a fixed-ratio schedule, the accelerating “scalloped” responding on a fixedinterval schedule, the pattern of inter-response times on a differential-reinforcement-of-low-rates schedule, or the partial-reinforcement extinction effect? I think it is implausible to suggest that anyone could use optimality modeling to predict even these basic phenomena, let alone more complex findings such as supernormal conditioning, occasion-setting, or the response patterns on concurrent-chains schedules. (However, I have no doubt that a clever theorist could use optimality modeling to derive many of these phenomena after the fact, once provided with the empirical findings.) In short, optimality modeling is not very helpful in the analysis of basic principles of learning, just as it is not very helpful in understanding the heuristics that humans or nonhumans actually use in a decision-making task. A second general characteristic that the authors’ theoretical approach shares with the animal learning tradition is the interest in finding general principles that apply to different tasks and are used by different species. For example, Hutchinson and Gigerenzer argue that the Gaze Heuristic is used by humans, bats, bees, and insects to help them track and capture a moving target. They present evidence that the Take the Best heuristic is used by people deciding which of two cities is bigger and by honeybees selecting flowers. Similarly, most of the phenomena of classical and operant conditioning (those mentioned above and many others) have been observed in many different species and many different learning situations both inside and outside the laboratory. There is, however, one puzzling difference between the heuristics studied by Hutchinson and Gigerenzer and the general principles studied by researchers in animal learning. Many of the phenomena of classical and operant conditioning are indeed very general, in the sense that it is often difficult to find exceptions. If an operant response is extinguished, some time is allowed to pass, and then the individual is reintroduced to the
situation where the response previously occurred, we are very likely to see some spontaneous recovery, no matter what operant response, what reinforcer, or what species is being tested. As another example, no organisms have been found that do not sometimes exhibit “impulsiveness” by choosing a small, more immediate reinforcer over a larger, more delayed reinforcer. This is presumably because all species are adversely affected by delayed reinforcement. In contrast, the heuristics studied by Hutchinson and Gigerenzer seem to be general (used by a number of species, in a variety of different situations) but not universal, because a range of different heuristics may be used for problems with a very similar formal structure. For instance, whereas the Take the Best heuristic is frequently used in choice situations where the alternatives differ along several dimensions, the authors report cases where other decision strategies (e.g., using a weighted average, or following a decision tree) are used instead. Indeed, an important part of their research program involves examining each specific case in enough detail to predict which heuristic is likely to be used, and why. The reason for this difference between the authors’ heuristics and principles of learning is not obvious, at least not to me. Perhaps it arises because the choice situations to which their heuristics apply are more complex, involving multiple cues and multiple dimensions, and multiple sources of uncertainty. Perhaps as choice situations become more complex, the number of heuristics (both the number theoretically possible and the number actually used by some species somewhere) increases as well. Whatever the reason, this certainly makes the analysis of decision-making heuristics a challenging task. References Hackenberg, T.D., Hineline, P.N., 1992. Choice in situations of time-based diminishing returns: immediate versus delayed consequences of action. J. Exp. Anal. Behav. 57, 67–80. Hutchinson, J.M.C., Gigerenzer, G., 2005. Simple heuristics and rules of thumb; where psychologists and behavioural biologists might meet. Behav. Proc. 69, 97–124. Rachlin, H., Green, L., Kagel, J.H., Battalio, R.C., 1976. Economic demand theory and psychological studies of choice. In: Bower, G.H. (Ed.), The Psychology of Learning and Motivation, vol. 10, pp. 129–154.