Comments by John D. Sterman

dOURNALOF

ELSEVIER

Journal of EconomicB haviorand Organization Vol. 29 (1996) 251- 256

EconomicBehav~ &O r g a n ~

Comments by John D. Sterrnan

1. The p i n g l e / d a y paper This paper is a good example of exactly the type of empirical work I am calling tbr. Day's seven modes of economizing are tested through clever manipulations of decision cost and context, causing some dramatic shifts in the decision process people employ, as well as their performance, The results strongly suggest the role of various decision heuristics in the process of selecting which mode of choice to u:~e. One of the striking results they report is the attraction of subject choices to the prescribed choice of the authority even when the prescribed choice is poor. Subjects tended to search in the neighborhood of the prescribed choice, leading them to choices well short af optimal. This behavior is strongly reminiscent of "anchoring and adjustment', a heuristic widely studied in psychology in which people judge an unknown quantity by anchoring on a known reference point and adjusting for other factors that are less well known, less salient, or less persuasive. Studies of anchoring and adjustment in psychology have illuminated many of the conditions that cause people to adjust more or less relative to the anchor. The in:eresting issues for the present paper relate to the details of the choice process: What causes people to choose the size of the neighborhood in which they will search? How large is tire step size people employ in gradient search or other hill ciimb-style trial aM error modes? Are these parameters themselves subject to choice, learning, or evolution? As I argued in the case of John's pa[~:r, the existing literature in psychology would help illuminate some of these issues. For example, Hogarth et al. (1991) studied decision making performance where they manipulate both incentives and task difficulty. They find a significant interaction between incentives and difficulty such that higher incentives can reduce performance when the environment is 'exacting', that is, when the cost of error (here, the misuse cost) is high. In Hogarth's study, the benefits of learning from experimentation are outweighed L,y the costs of error. The fact that subjects nevertheless ex~rimented too much suggest how bounded the rationality of the process of choosing how to decide can be. 0167-2681/96/$15.00 ~9 1996Elsevier Science B.V. All rights reserved SSDI 0167- 268 1(95)00060- 7

252

Comments

This example suggests the importance of process data in experiments of this type. Process data are qualitative as well as quantitative data characterizing the thought processes and intermediate steps in choice. For example, verbal protocols, eye movement data, notebooks, and information access data can all illuminate the processes subjects use as they work problems such as those posed in these experiments. As a generalization, experimental economics does not adequately draw on these tools, available essentially 'off the shelf'. Our own experimental work at MIT as well as much other work in decision making suggests these methods can be powerful sources of constraint on plausible models of the de¢.Jion process. Process data should be particularly helpful in discriminating among the seven modes of economizing behavior and understanding the conditions that might cause people to switch from one mode to another. Such data will be increasingly necessary as the research expands to consider tasks of greater complexity. Here~ for example, trial and error and imitation were quite effective strategies, because there was excellent (accurate, deterministic, and immediate) feedback on the effectiveness of each decision observed by the ~ubjects. However, in tile real world, outcome fee,'lback is noisy, incomplete, biased, distorted, delayed, and confounded with many highly correlated variables. Research shows clearly that people cannot learn from outcome feedback in that environment. (B~ehmer, 1980). These realistic conditions produce 'superstitious learning' and may account for the persistence of beliefs in astrology, or the fact that former Boston Red S,ox batting champion Wade Boggs ate nothing but chicken for years on game days because he once happened to have a good day at the plate after a dinner of lemon chicken. The next step in these provocative experiments should be the addition of realistic complexity - both exogenous and endogenous - to the experimental task.

References Brehmer. B., 1980. In one word: Not from experience.Acta Psychologica45. 233-41. Hogarth. R., B. Gibbs, C. McKenzieand M. Marquis, 1991.Learningfrom feedback:Exactingnessand incentives,Journal of ExperimentalPsychology:Learning,Memory,and Cognition 17(4).734-9.;2.

2. The c y e r t / k u m a r paper The paper discusses an important issue: the scope of economizing behavior covers not just efficient allocation of resources within an established and usually assumed institutional structure, but the choice (or evolution) of that institutional structure itself. Of course, this has long been the concern of economics, in the institutionalist paradigm, the new institutional~sm, and the work of people like Williamson.

Comments

253

What is new here is the argument that in a world of uncertainty the optimal structure of a firm may change dynamically even when the environment is stable, solely as a consequence of the changing value of new learning. Uncertainty (about anything, but in this case, about consumer preferences over possible product offerings) means experimentation to elicit new information from the market is valuable. But if preferences are stable (or their distribution is stable) then the value of this learning declines over time. Thus organizational forms that favor knowledge elicitation (diversity of product lines) and swift and accurate transmission of that information to the owner/managers (integration, e.g.) are efficient when ignorance is high. As the firm learns, however, the value of further search falls, and it becomes efficient to specialize and focus on adaptation to reduce production costs. The idea that the optimal organization structure of the firm changes as a function of the dynamics of learning is important. Changing organizational form can be endogenous and does not depend on change in the environment, as most theories (not just in economics but also in organizational theory) assume. However, the idea that the optimal form changes and the conclusion that actual forms change in these ways and for these reasons are not the same. The paper assumes a high degree of rationality on the part of the managers (they are described as designing and carrying out an optimal sequence of experiments). The specific decision rules by which a firm would discover that the optimal form of organization had changed are not specified here, that is, there is no discussion of the procedural rationality of the firms. It is interesting to speculate as to the empirical consequences of the theory under more realistic assumptions that at least some of the managers are boundedly rational managers and are not aware of the changes in the optimal structure of their firms. In this case we should observe high rates of business failure at the time in the life cycle of the firms when the optimal structure shifts as a result of the diminishing value of learning. Firms that (somehow) shift to the proper forms, and new entrants designed for efficiency but possessing the knowledge of the established firms will drive the others out of business. No data are offered to support the model, and it is hard to imagine how the other assumptions (such as stationarity of the environment) can possibly be controlled for. It is interesting to note that an important school of thought in organizational theory, the population ecology school, argues that the bounds on the rationality of firms are so severe that attempts at organizational change to respond to changing environmental (or internal) conditions are essentially random with respect to survival value. In this model, which is supported by a good deal of evidence, ti~ough also has many critics, the best choice for a firm is not to change. Change is costly, and under most circumstances, costly change with random effects on competitiveness is not a good choice. Institutional theories of change suggest that firms that change do so by imitating the practices and forms of the "best' firms in thei; industry, where best usually means the highest performing firms. This offers

254

Comments

yet another model for organizational learning. The next step in this research should be to design tests tha,: would discriminate between these models of learning. Statistical and field studies will be necessary, since the decision processes (inertial vs. imitative vs. deliberate design based on ecoiiomizing principles) matter as well as the outcomes in terms of firm performance and structure.

3. The conlisk paper Perfect rationality assumes agents possess a costless, limitless ability to calculate. But economics is the science of scarcity, and calculating ability is scarce. Thus Conlisk states "it is curious for economists to neglect a cost' - the cost of deciding (p.2). I agree with the sentiment, and as a rhetorical device to motivate the profession to attend to bomMed rationality it is, we can only hope, effective. In reality, however, economists routinely ignore many of the most important costs that characterize important problems. Ignoring costs is hardly curious at all; rather it is the normal procedure. Mainstream theory (along with the practical world of the market) long ignored the costs of resources other than labor and capital (land was once considered, but had little role in the neoclassical world). Certainly nonrenewable resources, pollution, biodiversity, a stable climate, and a host of other resources supplied by the natural environment have played little if any role in economics. Despite accelerating work, these issues remain 'externalities' in most models and have no impact on prices in most markets. We have a long way to go before economics incorporates the important costs facing real decision makers and real societies. Rhetorical devices aside, the paper p-esents an explicit method for incorporating decision cost into formal models of economizing behavior. The paper summarizes we!] the flaw in the application of the traditional optimizing paradigm to the problem of deliberation cost: to decide optimally how much effort to exert in deciding leads to an infinite regress of ever less tractable optimizations. At some point, economists and real people alike must act without recourse to optimization. Thus all theories embody behavioral assumptions that are not grounded in rational behavior. The paper provides two good examples to show how first order decision cost (deciding how much effort to exert on the primary problem of, say, maximizing firm profit) can be incorporated into formal models. Both lead to models in which behavior is responsive to the perceived costs and benefits of decision effort along with traditional factors such as prices, technology, etc. Agents use rules of thumb to deter'mine how much search (or decision effort) to carry out. The rules of thumb lead, under appropriate assumptions of environmental stability and convexity of technology, to the unboundedly rational equilibrium, an aesthetically attractive

Comments

255

property. But as decision cost rises, the performance of the system relative to t l ~ equilibrium deteriorates, and behavior changes. In particular, boundedly rational decision behavior not only lowers average performance but can introduce random variation into an otherwise deterministic environment. John argues that this approach allows economists to incorporate bounded rationality into formal models without the need to become cognitive scientists. I agree that we do not need to become cognitive scientists (though there is no reason not to do so). However, we are not relieved of the need to draw on the results of cognitive science in specifying such models. John must assume behavioral rules of thumb the agents use to decide how much effort to expend. The rules he chooses are plausible - exponential smoothing of past outcomes, hill climbing, etc. But they are only a few of many plausible rules, and the paper is conspicuous for the lack of data on the actual heuristics people use in these settings. These are precisely the conditions that cause many economists to criticize non-optimizing models as ad hoc. In fact, as presented here, the models are ad hoc. It is no help that the traditional (and unmentioned) assumption of unbounded rationality is just as ad hoc and clearly less realistic than the behavioral rules used here. As Herb Simon has often said, "you can't beat something with nothing'. The something does exist. Psychologists have shown experimentally that people do try to adjust their decision strategies and effort in response to the effort/payoff tradeoffs they perceive. And of course this is the topic of the paper by Day and Pingle. The key is ~perceived tradeoffs', since we cannot optimally choose due to the infinite regress. The work of Bettman, Payne and Johnson is key here (see their new book, The Adaptive Decision Maker). Unfortunately, the bounds on rationality are severe enough that humans often consistently err in making judgements of effort and accuracy, Further, the errors are often not mitigated by training, experience, monetary incentives, or the presence of well-functioning markets. To give just one example, one of the simp[e behavioral rules John assumes, single exponential smoothing, outperforms the consensus forecast of inflation one year ahead over the past forty years (the Livingston panel forecasts and the ASA/NBER forecasts). To model inflat.ion expectations accurately, then, it would appear one must use a behavioral rule even less rational than the simplest form of adaptive expectations. The soothing vision that behavioral decision making can be accommodated in economics without the need for economists to study psychology is illusory. There is no avoiding psychology in economic models. If we try to avoid it, we only succeed in substituting an implicit psychology that is almost certain to be inferior. A vast body of knowledge of behavioral decision making is now available to ground our models. There are methods to test these models experimentally. We don't have to become cognitive scientists, but we do have to draw on the existing knowledge cognitive scientists, organizational observers, ethnographers, and anthropologists have compiled for us. The important ideas in John's paper will have real impact, I believe, only when explicit use of field study, experiment, and

256

Comments

existing knowledge of decision making behavior are taught as part of the necessary education of every economist. John D. Sterman

Massachusetts hlstimte of Technology. Sloan School of Management, Cambridge, MA, USA