Clark L. Hull's Behaviorism

5 CLARK

L. HULL'S

BEHAVIORISM

MICHAEL

E. RASHOTTE

ABRAM AMSEL University of Texas at Austin

Florida State University

Introduction Brief Biographical Sketch I. Theoretical Style A. Influence of Machines B. Plan for Developing a Behavior Theory C. Perspective on Hull's Attempt to Make a Formal Behavior Theory D. Physiology and Hull—Spence Theory II. S —R Analysis of Cognitive Processes A. Knowledge, Foresight, Directing Ideas, and Purpose B. Gestalt Phenomena C. Habit—Family Hierarchies (Networks of Associations) D. Kenneth Spence's Analysis of Transposition E. Application to Psychotherapy and Symbolic Processes III. Conceptualizations of Reinforcement and Reward: Evolution of Hull's 1943 Theory A. "Reinforcement" in Hull's 1943 Theory B. Difficulties Recognized by Hull in 1943

Handbook of Behaviorism

119

Copyright © 1999 by Academic Press. All rights of reproduction in any form reserved.

1 20

RASHOTTE AND AMSEL

C. The Elliott– Crespi–Zeaman Experiment D. Spence's Contributions to Reward–Incentive Theory E. Amsel's Frustration Theory: A Conditioning Model of Effects of Reward and Nonreward Concluding Comments

INTRODUCTION

For a period of 23 years, beginning about 1929, Clark Leonard Hull (1884-1952) devoted his academic efforts toward developing a sophisticated version of behaviorism that was fully committed to the language and orientations of stimulus–response (S –R) psychology. The success and influence of his work are a matter of record (e.g., Amsel & Rashotte, 1984; Logan, 1959). As behaviorism enters a new second century, and as cognitive science and neuroscience have come to be dominant influences in psychology, it is timely to reflect on the forces that motivated Hull's attempt, the breadth and character of his effort, and some points of its success and failure. It is our opinion that Hull would have warmly embraced many of the present-day developments, and that a reading of his work, and the critical responses it attracted, can provide useful historical perspective on some modem-day efforts in behavioral, cognitive, and neural science. In the three main sections of this chapter, we illustrate some important aspects of Hull's theoretical work and its influence. We begin with an overview of the style of his theoretical approach. Next, we review selected theoretical analyses developed by Hull and his associates to conceptualize cognitive phenomena within the S –R framework analyses that greatly expanded the scope of S –R behaviorism and have had a lasting effect in twentieth-century psychology. Finally, we illustrate how Hull's theoretical approach responded to the pressures of empirical findings in one focused area the conceptualization of reinforcement and reward that have been, and remain, enormously influential in behavior theory. We begin with a biographical sketch of Clark Hull.

BRIEF BIOGRAPHICAL SKETCH

Clark Leonard Hull was born on a farm near Akron, New York, on May 24, 1884, and died in New Haven, Connecticut, on May 10, 1952, where he was Sterling Professor of Psychology at Yale University. When he was 3 or 4 years of age, his family moved to an unimproved farm in West Saginaw, Michigan, where he spent his early years under "pioneer condi-

5

CLARK L. HULL'S BEHAVIORISM

121

tions" (Hull, 1952b, p. 143). He was educated in local elementary and high schools, at Alma College and the University of Michigan (bachelor's degree, 1913), and at the University of Wisconsin at Madison, where he did his graduate work in experimental psychology (master's degree, 1915; doctoral degree, 1918). Hull was 34 years old when he earned his doctorate, and his career as a scientist continued for an additional 34 years, right up to the time of his death. His scientific standing was recognized by election to membership in a number of societies, including the American Academy of Arts and Sciences (1935) and the National Academy of Sciences (1936). He was awarded the Warren Medal of the Society of Experimental Psychologists (1945), and he served as president of the American Psychological Association in 1935 –1936. Hull's accomplishments were made against a background of generally poor health that began at the time of his graduation from high school when a bout of typhoid fever left him with a poor memory. A few years later poliomyelitis left him with a permanently paralyzed leg. In his later years, he was plagued with a heart condition. His first academic appointment was at the University of Wisconsin, where he stayed for about a decade after receiving the Ph.D. and rose through the ranks to professor in 1925. During those years he worked on a variety of projects including the effects of tobacco smoking on mental and motor efficiency, the improvement of vocational guidance through improved aptitude testing, and the scientific basis of hypnosis (see Hull, 1952b). In 1929, at the age of 45 years, Hull accepted the position of research professor of psychology at the Institute of Psychology (later the Institute of Human Relations) at Yale University, where he remained for the rest of his career. He had no formal teaching assignment at Yale, but engaged in graduate instruction through a weekly seminar that attracted graduate students and personnel of the institute for discussion of a variety of issues in behavior theory. Hull is, perhaps, best known for Principles of Behavior: An Introduction to Behavior Theory (1943b), the first volume of a planned three-volume effort. In Principles, he described a set of formal axioms constituting a logical system from which hypotheses about mammalian behavior could be deduced. This was to be the first of a planned three-volume effort. The second volume, A Behavior System: An Introduction to Behavior Theory Concerning the Individual Organism (Hull, 1952a), appeared shortly after his death. It was intended to illustrate the ability of the system to generate fruitful hypotheses and deductions for the behavior of animals in nonsocial settings. The third volume, which was never written, was to have applied his system to some elementary phenomena of mammalian social behavior. In addition to these three volumes, Hull's published works include a large number of experimental papers and 21 Psychological Review articles that exerted great influence in psychology at the time of their publication and continue to be influential even today (Amsel & Rashotte, 1984; Spence, 1952a). Some of the key influences arising from Hull's contributions to behavior theory are discussed in the remainder of this chapter.

1 22

RASHOTTE AND AMSEL

I. THEORETICAL STYLE A. INFLUENCE OF MACHINES Among the forces that motivated Hull's work in behavior theory, his enduring interest in machine technology should be especially noted. Since Descartes, much theorizing in psychology has been influenced by machine metaphors. The hydraulic system of Sigmund Freud, the telephone switchboard of the early connectionists, and the computer-based models of memory and of present-day connectionist approaches come to mind as some examples. In the early years of this century, Hull was influenced by the emerging technology of machines that could carry out some rudimentary functions of human cognition (e.g., calculation of algebraic outcomes). Indeed, entries made in his private notebooks (Hull, 1962), his construction of a machine to aid in the calculation of correlation coefficients (Hull, 1925a, 1925b), early publications that described electromechanical devices that could "learn" (Baernstein & Hull, 1931; Hull & Baernstein, 1929; Krueger & Hull, 1931), and comments in a later paper about having constructed a machine made of "sliding disk segments of sheet metal" designed to "exhibit the conclusions logically flowing from all of the known syllogisms" (Hull, 1935a, p. 219) illustrate his deep interest in machine technology and its possible relevance for understanding psychological processes. An example of his thinking is provided in the following quotation from a 1931 paper titled "An Electro-Chemical Parallel to the Conditioned Reflex." This passage advocates the value of designing and constructing machines (robots) that simulate adaptive behavior as part of the behavioristic program to banish mentalistic explanations from behavior theory. It is believed that the construction and study of models of the type described above [i.e., physical devices capable of learning] will aid in freeing the science of complex adaptive mammalian behavior from the mysticism which ever haunts it. The belief is very widespread and persistent that certain complex forms of adaptation cannot take place by any imaginable concatenation of materials without the mediation of some nous, entelechy, soul, spirit, ego, mind, consciousness, or Einsicht. There is, on the other hand, the opposed belief that the above explanatory concepts are but the names of disembodied functions (ghosts) which, insofar as they have any objective existence, are themselves in need of explanation and may conceivably be duplicated by adroit concatenation of materials. The conflict between these points of view is probably at a permanent impasse unless the second party can bring forth tangible evidence of synthetic achievement in this direction. That a considerable degree of success in this practically virgin field will reward a vigorous effort can scarcely be doubted. As progress is made by the second group it may be anticipated that the first will gradually retreat to the more and more inaccessible parts of the psychological terrain. How successful the attempts to construct "psychic" machines will prove, time alone will tell. It will be too early to venture a final judgment until as much labor and ingenuity have been expended in this attempt as have been devoted to the development of the steam engine, the electric motor, or the printing processes. Unless some practical use for such adaptive mechanisms should develop, this will naturally be a very long time. It is not inconceivable, however, that in the demands for higher and higher degree of automaticity in machines constantly being made by modern industry, the ultra

5

CLARK L. HULL'S BEHAVIORISM

123

automaticity of the type of machine here considered, may have an important place. In that event the exploration of the potentialities lying in this radically new order of automaticity would be comparatively rapid. (Krueger & Hull, 1931, 267-268; emphasis in original)

Many modern-day theorists in psychology and cognitive science will find their own thinking reflected in Hull's proposal that important advances can be made by adopting the logical and quantitative discipline necessary to construct machine simulations of behavioral phenomena. They will also empathize with Hull's discussion of the difficulties inherent in such an undertaking. For example, designing a machine that behaves very differently depending on small changes in the pattern of stimulation to which it is subjected was seen by Hull as a major problem (e.g., Hull, 1929; Krueger & Hull, 1931). Subsequent developments in the study of "compound stimulus learning" in conditioning experiments (including stimulus configuration) and "pattern recognition" in cognitive studies are illustrative. Of course, the materials available for constructing actual "psychic machines" were extremely limited in Hull's day, and he devoted much of the rest of his career to constructing "virtual machines" (theoretical systems) that simulate (predict) behavioral phenomena of interest. Just as the design of an actual machine calls for adherence to strict logical and quantitative rules in combining a set of physical materials, Hull's theoretical systems attempted to use the same approach in combining a set of conceptual terms. The success of behavioral predictions from his systems were the means by which the adequacy of his theoretical "machines" was to be judged. B. PLAN FOR DEVELOPING A BEHAVIOR THEORY

About the time he moved from the University of Wisconsin to Yale University, in 1929, Hull began to put into practice a "master plan" for developing a behavior theory. At Yale, he joined the Institute of Human Relations, where psychologists, sociologists, economists, and cultural anthropologists pursued the ideal of a unified approach to social science, and it was in this atmosphere that Hull began his major theoretical work. He proposed to publish three key volumes that would cover "in an elementary manner the range of ordinary mammalian behavior" (Hull, 1952a, p. vii). His orientation is well expressed by the following passage from his autobiography, in which, having acknowledged that his approach was probably unfluenced by his early training in the physical sciences, he writes: I came to the definite conclusion around 1930 that psychology is a true natural science; that its primary laws are expressible quantitatively by means of a moderate number of ordinary equations; that all the complex behavior of single individuals will ultimately be derivable as secondary laws from (1) these primary laws together with (2) the conditions under which behavior occurs; and that all the behavior of groups as a whole, i.e., strictly social behavior as such, may similarly be derived as quantitative laws from the same primary equations. With these and similar views as a background, the task of psychologists obviously is that of laying bare these laws as quickly and accurately as possible, particu-

1 24

RASHOTTE AND AMSEL

larly the primary laws. This belief was deepened by the influence of my seminar students, notably Kenneth W. Spence and Neal E. Miller. It has determined the most of my scientific activities ever since, and the longer I live the more convinced I am of its general soundness. (Hull, 1952b, p. 155)

To undertake this task, Hull decided to write a series of theoretical papers that would address various issues on which he wished to attract comment. The ideas in such papers would then be refined and presented in a unified volume, along with experimental evidence, to stand as the principles from which he would predict behavior. Hull described this work as really amounting to a case of "mechanical design" (Hull, 1962, p. 833), and the first of his papers appeared in the Psychological Review in 1929 (Hull, 1929). In all, he wrote 21 articles and two brief notes on learning theory in that journal through 1950. These works include theoretical accounts of such topics as complex cognitive processes (e.g., knowledge, purpose, foresight, directing ideas), Gestalt phenomena (e.g., the appearance of novel responses in problem solving, the behavior of young children in spatial situations), species and individual differences, and behavior in social situations. We think that these Psychological Review articles best represent his orienting attitudes and the true scope of his effort to develop a behaviorism. All of these articles have been reprinted in a single volume, with commentary added by us (Amsel & Rashotte, 1984). Readers of the present chapter might consult that earlier volume for more biographical details concerning Hull, and for our comments on the ways ideas in his papers relate to previous and subsequent developments in psychology, as we saw it in the early 1980s. Briefly, the intellectual roots of Hull's planned behavioristic theory lay in a diverse set of important ideas concerning behavior and the nature of theory. A central focus of the plan was to blend the conceptual and experimental advances in Edward L. Thorndike's (1898, 1911) S –R psychology and Ivan P. Pavlov's (1927) approach to behavioral plasticity into a unified account. Both of these approaches were mechanistic, of course, as suited Hull's taste. He also included a strong focus on behavior as an adaptive process, and on the role of motivation in energizing and directing behavior, reflecting the influences of Charles Darwin and Sigmund Freud, respectively. Finally, Hull emulated the theoretical form of Isaac Newton's Principia, which had such a powerful effect on physics. In many of his writings, Hull attempted to outline his theoretical arguments as formal and quantitative statements within a hypothetico-deductive methodology. He made heavy use of the intervening variable, a model Edward C. Tolman had introduced into psychology (Tolman, 1938), and he endorsed Bridgman's operationalism as a requirement in the definition of these variables. Later, he and others in the Hullian group wrote extensively on the nature and advantages of this theoretical form over vaguely stated theories or atheoretical approaches. Arriving on the scene after John B. Watson had made behaviorism wellknown and controversial in American psychology, Hull's plan for an S –R behaviorism was very ambitious. It aspired to predict the behavior of individuals in isolation, as well as in group settings. It aspired to conceptualize the bases for

5

CLARK L. HULL'S BEHAVIORISM

125

adaptive behavior in a broad sense, including certain cognitive processes and the performance differences between species and individuals. It aspired to be logically rigorous and mathematical as a way of ensuring that its assumptions and predictions were clear and available for comparison with competing theories. The first volume of Hull's planned three-volume work was intended to be a primer of behavior theory that would simply set out the principles of the formal system to be used in approaching specific topics in behavior theory. That volume, Principles of Behavior: An Introduction to Behavior Theory, published in 1943 when Hull was 58 years old, is often cited (incorrectly, we think) as the core of his theoretical work. It made little, if any, attempt to elaborate the complex behavioral implications of the system. Indeed, the book was devoted to an analysis of classical and instrumental conditioning to the exclusion, even, of the more complex analyses of purpose, foresight, and so on characteristic of Hull's own earlier work in the 12 Psychological Review articles he had published up to that time. He had used the formal postulate style in an earlier monograph, Mathematico-Deductive Theory of Rote Learning (Hull et al., 1940), in which he attempted to explore the logical implications of conceptualizing rote learning in terms of Pavlovian processes. He considered that work a "dress rehearsal" for Principles of Behavior, and both works illustrate one way that behaviorism might utilize a highly formalized theoretical approach. Immediately after publication of Principles, Hull began working on the second volume in the series and in 4 years had completed a draft of 12 chapters. This second book was intended to demonstrate the derivation of more complex phenomena of behavior from the already-published postulates of the system in his "primer." However, difficulties in quantifying some of the theoretical variables, and frail health, delayed publication, and the book, A Behavior System: An Introduction to Behavior Theory Concerning the Individual Organism, which was finally completed in his 68th year, appeared in 1952, shortly after his death. A sample of its chapter titles conveys the range of phenomena to which the postulates were applied: "Simple Trial-and-Error Learning," "Discrimination Learning," "Learning within the Individual Behavior Link," "Behavior in Relation to Objects in Space," "Multidirectional Maze Learning," "The Problem-Solving Assembly of Behavior Segments," "Value, Valuation, and Behavior Theory." Some of these chapters were updated versions of papers Hull had published earlier, some had circulated in the psychological community for several years in the form of his seminar proceedings, and the chapter on discrimination learning borrowed heavily from classic papers by his close associate Kenneth Spence (1936, 1937, 1938, 1942). The third volume Hull had planned was never written. Of it, Hull said in his autobiography, "for many years I have felt that by far the most important portion of the system for civilization in general would be found in the third volume, which would concern the strictly social relationships among subjects" (1952b, p. 162). As noted previously, he believed that the principles underlying social behavior were not fundamentally different from those underlying individual behavior and was optimistic that the elementary phenomena of mam-

126

RASHOTTE AND AMSEL

malian social behavior could be derived from the primary "laws" stated in Principles. Obviously, this unfinished part of Hull's work was intended as an interface between psychology and the disciplines of sociology, economics, and cultural anthropology, all of which were represented in the faculty of the Institute of Human Relations at Yale. Kenneth W. Spence played an intimate role in the development of Hull's theorizing (Amsel & Rashotte, 1984), and after Hull's death Spence assumed the central position in the Hullian movement. In the 10 years or so preceding Spence's death, in 1967, the more formal parts of the theory became known as the Hull–Spence approach (e.g., Logan, 1959). Well before Hull's death the major experimental work testing the theory was done at the University of Iowa by Spence and his students. Partly as a result of the work at Iowa, Spence favored revisions in certain portions of the theory, and these were reflected in his own most unified theoretical statement made in his Silliman Lectures at Yale, published as Behavior Theory and Conditioning (Spence, 1956). C. PERSPECTIVE ON HULL'S ATTEMPT TO MAKE A FORMAL BEHAVIOR THEORY Hull's endorsement of formal theorizing stands out in the initial chapters of Principles of Behavior, where we find a discussion of the deductive nature of successful scientific theories in the natural sciences and a commentary on the application of this approach in the behavioral sciences: Following the natural-sciences model, the behavioral scientist elaborates a set of postulates, or first principles, and uses them as premises in deducing, by rigorous logic, inferences or theorems about behavioral phenomena. The postulates are abstract principles derived from available empirical evidence and influenced by shrewd guesses about the process underlying behavior. These postulates often involve hypothetical entities ("intervening variables"), invented by the theorist to organize his thinking about the relationships among experimental manipulations and measurements (independent and dependent variables) related to behavioral phenomena of interest. The theory can then be evaluated by translating the deductions from the theory into experimental operations and seeing how it fares in the laboratory. This theoretical approach advocated in psychology by Tolman and then by Hull had great appeal in the abstract. In practice, however, the difficulties proved overwhelming. First of all, the systematic gathering of empirical evidence about learning (not to say behavioral phenomena in general) began only late in the nineteenth century, and the experimental methods were often crude and overly complex in the early work. Accordingly, when Hull was writing Principles, about 60 years ago, the empirical base with which he started was very weak and he relied heavily on shrewd conjectures in the formulation of his postulates. (In a later section of this chapter, concerned with reinforcement and reward in Hullian theory, we illustrate how such deficiencies came to light as a result of experimental work and how the theory changed as a result.) Second, the absence of a

5

CLARK

L. HULL'S BEHAVIORISM

127

sound empirical data base for the known phenomena of learning made it difficult for Hull to be as meaningfully quantitative as he would have liked to be. Nevertheless, in Principles, he went through the motions of quantification in the case of some constructs (e.g., excitatory potential SER), apparently to point the direction for future endeavors. Finally, Hull's overall theoretical objective (Hull, 1952a, p. vii) to state the principles that allow deductions of "the range of ordinary mammalian behavior" (even if only "in an elementary manner") was extremely ambitious for the 1940s; indeed, the objective seems unattainable for the 1990s. Hull realized, and stated early in his second book, A Behavior System (1952a), that, in practice, rigorous deductions about even moderately complex learning phenomena could not be made from the original postulate set because it was incomplete and not sufficiently quantitative. In fact, he characterized the deductions as "informal," and commented that it was probably too early to attempt rigorous deductions about even a limited range of behaviors. The formal style and the scope of Hull's theorizing in Principles was clearly premature, at least in the sense that it was based on too little empirical work. Of those theorists who came earliest under his influence, he was emulated in style but not in scope by K. W. Spence, and in neither style nor scope by N. E. Miller and 0. H. Mowrer, although all three worked with many of the basic concepts Hull developed. The later work of Spence and his students harked back in some respects, and particularly in its emphasis on motive–incentive factors in behavior, to Hull's papers of the thirties on purpose and foresight. From the present perspective, Hull's own final assessment of the formal theoretical style seems correct: It was (and still seems to be) premature to undertake formal theorizing covering a wide range of areas or topics even in an area about which much is known, the psychology of learning. The present inclination is to make theory "in smaller pieces," to limit theoretical integration to behavioral phenomena of lesser scope, and to investigate the neural mechanisms underlying such phenomena. But, as we shall also see, important spin-offs from Hull's theoretical efforts were still occurring, to go along with the tradition of appreciation for the nature of theory and its relation to experimentation (Bergman & Spence, 1944; Hull, 1943a, 1943b, 1943c; Miller, 1959; Spence, 1948, 1952b, 1952c, 1957). Despite severe criticism (Koch, 1954) there is little doubt that Hull ranks as one of the most influential theorists in modern psychology. D. PHYSIOLOGY AND HULL–SPENCE THEORY

In view of the great interest and progress being made today in the neurobiology of learning, memory, and performance, it is of interest to note Hull's attitudes toward including physiological mechanisms in behavior theory, and Kenneth Spence's long-running disagreement with Hull on this matter. As the twentieth century draws to a close, some research programs based on evolved versions of Hull–Spence theory include a focus on neural mechanisms underlying behavior theory. We draw attention to one of these programs later in this

128

RASHOTTE AND AMSEL

chapter in consideration of Amsel's recent work in developmental neurobiology of learning and memory. Here, we present excerpts from private correspondence between Hull and Spence to characterize the tensions within the Hull– Spence system concerning this issue. Hull was in favor of using physiological terms, partly to attract students whom he assumed would find speculations about the physiological nuts and bolts of the system to be more easily grasped than the abstract mathematical formulations he sought to develop. (This is, perhaps, a carryover from his own deep interest in the structure of machines that carry out psychological functions.) Spence, on the other hand, argued that such physiological speculations detract from behavior theory, much the same type of argument that is often associated with Skinner's behaviorism (see Ringen, this volume). From the present-day perspective, neural network approaches to learning would seem to be close to the spirit of Hull's thinking with, of course, the benefit of modern computational machines and major advances in neurobiology. Spence's position is illustrated in a letter to Hull in 1943. Spence had received an autographed copy of the newly published Principles and speculates about the probable response by reviewers: I shall await with interest the reaction of the reviewers. As I have indicated several times, I feel quite sure that most of the people who read the book will completely miss the point and will go off into picayunish criticism of some of the neurophysiological "notions" that you have elaborated. This, of course, represents our old difference in viewpoint. I have always been very unhappy about the fact that you have been inclined to throw in hypotheses as to the mediational mechanisms underlying the abstract mathematical concepts. I can just see the critics bemoaning the extent to which you hypostatize little things inside the brain. In other words I fear that you will have the non-physiologically minded people jumping on your theorizing as being nothing but neurophysiological speculation and the physiologically minded people for having made what from their point of view would be naive neurophysiological speculation. To me, of course, the meat of the discussion is the framework of mathematical theory (constructs and postulate relations) and I would have been much happier if the whole discussion had been kept on a very abstract level. (Kenneth W. Spence to Clark L. Hull, September 8, 1943, Kenneth Spence Papers, Archives of the History of American Psychology, University of Akron.)

Hull responded promptly, noting that he was "plugging along" on the manuscript for his second book (ultimately, A Behavior System, which appeared almost a decade later), before commenting on Spence's earlier letter: Thanks much for your remarks about the new book. I think that I am following out your preference substantially in the writing of the second volume, as I think you will find little or nothing of the objectionable physiological nature in the Chapter II. The same, I think, will be true in all of the others. I simply take these equations and go on from there, paying no attention whatever to the physiological suggestions which I sprinkled more or less through the first volume. I am, of course, not very much concerned with what the reviewers say; what I am interested in is what the graduate students say. Those are the boys who will decide the matter. I would be very much interested to have relayed to me the reactions of various types of graduate students, both the Lewinian type and your own students. As you know, my motive in introducing the sub-molar physiological suggestions

5

CLARK L. HULL'S BEHAVIORISM

129

was the hope that it would make it somewhat easier for relatively naive graduate students to secure a sense of reality from the theory, which many people have difficulty in doing from the mere inspection of equations. (Clark L. Hull to Kenneth W. Spence, September 21, 1943, Kenneth Spence Papers, Archives of the History of American Psychology, University of Akron.)

II. S—R ANALYSIS OF COGNITIVE PROCESSES A. KNOWLEDGE, FORESIGHT, DIRECTING IDEAS, AND PURPOSE The adequacy of an S –R analysis of behavior was severely challenged by behavioral phenomena that implied that the behavior of animals is at least sometimes guided by future events rather than by the prevailing stimulus conditions. A commonplace example is that a laboratory rat performs a response that prevents the occurrence of a future event or chooses a new pathway that allows it to reach a distant goal in the shortest time when its established pathway to the goal is blocked. In common language, behaviors such as these indicate "foresight," "knowledge," "purpose." It was not obvious that S –R psychology, with its insistence on behavioral control by prevailing stimulus conditions, could conceptualize adequately such instances of behavior that appeared to be controlled by the not-here and the not-now. Hull provided an S –R account of foresight, knowledge, and purpose in a series of articles published in the Psychological Review in the early 1930s. These papers introduced several concepts that remained influential in S –R psychology, even though Hull, himself, neglected much of the content of these papers in his later work, particularly in the statement of his theory in Principles of Behavior in 1943. In these early theoretical papers Hull exhibited a notable tendency to employ certain principles in a loosely structured argument to deduce from stimulus-response language the behavioral phenomena attributed to foresight and purpose. In this respect, these papers anticipated Hull's later attempt to build a formal theory with rigorous deductive logic. These early papers also were characterized by an emphasis on the biologically adaptive significance of processes, such as foresight and purpose, and provide a good example of the Darwinian orientation that was particularly prominent in Hull's early writings. It will become clear in the following discussion that, while Hull appealed to a variety of principles in these papers, his analysis of purposive behavior proved particularly significant for future theoretical developments because it introduced the fractional anticipatory goal response, r G —sG. The proposal was that the stimuli present while an organism is striving to reach a goal will, themselves, come to evoke some components of the goal response in anticipatory form (rG) through Pavlovian conditioning; and, that such responses generate propriocep-



130



RASHOTTE AND AMSEL

tive feedback stimuli (s G) that are then available to enter into associations and direct behavior. With this concept, Hull formalized his view that Pavlovian conditioned reflexes play an important, if not a dominant, role in guiding behavior in Thorndikeian learning situations. In 1929, Hull published the first of his 21 Psychological Review articles. [All of these have been collected in their original form and pagination, and republished on the centennial of Hull's birth, with extensive commentary (Amsel & Rashotte, 1984).] It concerned the functional significance of Pavlov's conditioned reflex and was distinctively Darwinian in tone, emphasizing the significance of Pavlovian excitation and inhibition in achieving the fine behavioral adjustments necessary for survival. For example, it included consideration of signaled avoidance learning in which the experimental animal takes protective action in advance of an impending aversive event, as if it anticipates the harmful event. In important papers published in the two succeeding years, Hull (1930, 1931) emphasized that animals with "knowledge" and "foresight" enjoyed additional adaptive advantages. These papers served to illustrate a way in which S –R theory could conceptualize these cognitive phenomena in mechanistic terms, and they were perhaps the first conditioning-based accounts of "purposive behavior." Hull's analysis was made with reference to a basic case in which a sequence of behaviors (R1 –R2 –R3 –R4 –R5) occurs because the stimulus properties of the world are arranged in a way that evokes those responses in that sequence (i.e., S i –52 -53 -54 -55 . This situation is shown in the following diagram taken from Hull (1930): The World:

The Organism:

3 S3

2

R

S4

s5-

2 R 3 R 4 R

5

If this sequence of S –R events is repeated several times, certain principles come into play that illustrate how an organism gains a rudimentary knowledge of its world, and shows foresight and purpose. One principle, response-produced proprioceptive feedback, well supported by physiological studies even in 1930, is that each response produces a distinctive, internal feedback stimulus that has the potential to operate as a cue like all other stimuli. The resulting state of affairs, reproduced in the following diagram, is that when the response sequence is activated by the external stimulus sequence, associations form between the response-produced, internal feedback stimuli and the responses that follow them. These associations are symbolized as dotted arrows in the following diagram:

5

131

CLARK L. HULL'S BEHAVIORISM

The World:

The Organism:

S1

S 2

4 _—S 3•

Ri--si..-PR2_4..s.2-4R3,...S-3-1R4,s-4-4R5___

The second principle, redintegration, states that any and all stimuli present at a given time acquire the capacity to evoke the responses that follow them. This further principle leads to the deduction that once these feedback stimuli occur in conjunction with certain responses, they can evoke the responses in the sequence, even in the absence of external stimuli. In the event, for example, the external stimulus sequence were interrupted at S I , the entire response sequence would run off anyway, as shown in the following: The World: S1

The Organism: R1-*-s1-•.-R2--•-s2 The third principle holds that the response sequence can run off at a faster tempo when cued by response-produced stimuli than by external stimuli. This principle of "foreshortening" makes it possible for R 2, say, to appear in apparent anticipation of the external stimulus S 2. But the preceding diagram shows that this anticipation is more apparent than real because R2 is in fact cued by a preceding stimulus, the feedback stimulus (s i ) from R1 . Anticipatory responses or "intrusions" such as these are often taken as evidence of foresight, and following is a simple S -R account of how they develop. Hull was impressed with an economic imperative that energy expenditure be reduced to a minimum in the context of adaptive biological functioning — a fourth major principle evident in his early work. One instance of this principle of behavioral parsimony is the tendency for responses to diminish in magnitude to a level just adequate for the production of some biological adaptive consequence. Consider the operation of such a principle in the case of the response sequence described earlier. Suppose that it is adaptive to perform R3 prior to the occurrence of S3, and that a successful outcome of the response sequence depends only on R3 . The third principle provides the basis for how R 3 comes to antedate S3 : the response sequence runs off at a faster tempo than the external stimulus sequence because the responses in the sequence are links, chained together by response-produced feedback stimuli. The fourth principle holds that the intensity of R 1 and R2 may diminish greatly because the sole function of these responses is to produce feedback stim-

1 32

RASHOTTE AND AMSEL

uli that will allow the chain of responses to run off. Indeed, R1 and R2 may diminish even to a covert level, so long as their intensity at that level is sufficient to produce feedback stimuli for the next response in the sequence. Pure stimulus act was the term employed by Hull for responses whose principal function is to produce stimuli that control other responses. He viewed these acts as being of the greatest biological significance, because the stimulation they provided for the control of behavior was ultimately independent of the stimulation provided by the external environment. The fact that behaviors might seem to appear spontaneously or to be unrelated to the prevailing stimulus conditions was not, in Hull's view, necessarily an embarrassment for S –R theory. In fact, such behaviors fit rather easily into the S –R schema, when analyzed in terms of the set of principles he invoked. Pure stimulus acts, particularly those that have receded to a covert level, were viewed as rudimentary forms of directing ideas or thoughts, and were particularly obvious analogues of internal mechanisms that control behavior. Relying on this analysis, Hull argued that "knowledge" could be conceptualized as the running off of a sequence of behavior (perhaps at a covert level) without stimulus support from the world that normally evokes it, and that, when such internalization occurs, it indicates "that a functional parallel of this action segment of the physical world has become a part of the organism. Henceforth, the organism will carry about continuously a kind of replica of this world segment. In this very intimate and biologically significant sense the organism may be said to know the world" (Hull, 1930, p. 514). In this same paper, he proposed that "foresight" be viewed as the running off of a response sequence at a faster tempo than is possible when that sequence depends solely on stimuli from the external world. Hull's analysis of "purpose" in the paper "Goal Attraction and Directing Ideas as Habit Phenomena" (Hull, 1931) was based on yet another mechanism that was a brilliant invocation of the idea that Pavlovian conditioned responses can be significant factors in the control of instrumental behavior. In accepting this idea, Hull joined other analytical thinkers of the time who recognized that Thorndikeian response–reinforcer procedures can always be said to include a stimulus–reinforcer relation of the Pavlovian sort. Responses that produce the reinforcer in Thorndikeian conditioning are necessarily performed in some context of stimulation that requires that a Pavlovian stimulus – reinforcer be embedded within that Thorndikeian procedure. Hull gave much careful consideration to the manner in which Pavlovian conditioning might develop within— and influence—performance in Thorndikeian learning. For this purpose, he introduced the concept of the fractional anticipatory goal response (r G — sG), which was a Pavlovian mechanism designed to account for anticipation or purpose in S –R learning theory. Behavior is said to be purposive when responses leading to a goal are directed or guided in some way by that goal. The difficulty to be overcome in an S –R theory is that, in these cases, the goal occurs only after the response has been made.



5

133

CLARK L. HULL'S BEHAVIORISM

In his analysis, Hull first considered the features of behavior that might lead an observer to describe it as purposive. In the case of a rat learning to run through a multiple T-unit maze for food, responding is variable in the early trials but later is said to take on a "purposeful" quality because the rat (a) speeds up as it approaches the goal; (b) begins turning movements, in anticipation of reaching the final choice point, often resulting in incorrect choice of maze arms in early trials; and (c) begins to make masticatory movements before the goal is reached. While the acceleration in responding as the rat approaches the goal was easily accounted for by the goal-gradient hypothesis, which holds that responses will be more firmly connected to stimuli the closer the S –R event is to the reinforcer (i.e., the goal), the remaining two characteristics of "purposive" behavior do seem to reflect the anticipation of a future event. As the following discussion shows, it was these anticipatory characteristics of behavior that Hull explained by appealing to Pavlovian conditioning. Briefly, Hull's analysis portrayed the Thorndikeian response as a sequence of S –R connections including a final one in which the goal stimulus (S4 represents the food-pellet goal stimulus in the following diagram; designated elsewhere as S G) evokes the goal response, RG (e.g., grasping, chewing, swallowing). The top two rows of the following diagram show this aspect of Hull's analysis:

Ttit!

r 1(1

N

I

\

\.,

1

\

1

\ i<

The Organism

_ 1

/ '1

/1,i I / / 1 S / / `')

\

4..A.

I

I I

/I

I

i

1

/ ,,n

r c--4- s cs, L._.

.._1\

, '

I

/ti / / I/

1 i

.„._,

if

/1

/1

S ;

`)i)

\ /1

\(.

.{‘

/ , \

I I

1

/ ,

,

i•-rc,--4-' --i•--tG-4-')G -1 - -

..._J

1_

_____J

1

r l'k

S

__...1

The portion of the diagram labeled S D represents the stimulus condition that arises from the animal's motivational state. This is the constant portion of the stimulus complex that persists throughout the sequence of S –R events, and is present when the goal is reached. Through Pavlovian conditioning, S D comes to evoke a fractional part of the final goal response (rG –sG), and because S D is constant throughout the response sequence, it should elicit this rG –sG throughout the sequence, although most strongly in the portions nearest the goal. In this manner, Hull provided a straightforward account, in Pavlovian terms, of how the goal, or a fractional representation of it, can be available to the organism before the goal is physically present.

1 34

RASHOTTE AND AMSEL

The manner in which rG –sG comes to guide behavior is also reasonably straightforward. The s G portion of rG –sG represents the proprioceptive feedback stimulation associated with occurrence of the conditioned goal response. Because sG is present while the S –R segments of the Thorndikeian response are occurring, it becomes associated with those responses in each S –R segment through a simple contiguity principle (dashed arrows in the diagram). Once associated with the Thorndikeian responses, sG would direct their occurrence in the same way as other stimuli do. The rG –sG mechanism can, in principle, account for masticatory movements as the hungry rat approaches food at the goal. Such movements can be regarded as fractional Pavlovian conditioned portions of the consummatory response at the goal, just as salivation is a measurable fraction of the response to the food-in-mouth unconditioned stimulus (US), or to its anticipation, in a Pavlovian experiment. By simple extrapolation, this principle accounts for anticipatory turning movements in a multiple-unit maze before the rat reaches a choice-point in the maze; it accounts, particularly, for the fact that in a multiple-unit maze, the anticipatory turns are likely to be in the same direction as the final goal turn, especially in the latter units of the maze. In short, rG–sG seems to provide a plausible account of significant aspects of purposive behavior entirely within the framework of simple conditioning principles. Hull's analysis of purpose was a brilliant coup. It allowed S –R theory to invade the territory of cognitive learning theory and even to compete successfully with cognitive theory in predicting experimental results. Consider one case in which Hull's S –R analysis triumphed over Tolman's cognitive position. Edward C. Tolman (see Innis, this volume) conceptualized learning as the formation of cognitions about how one event leads to another, and viewed learned behavior as reflecting the operation of cognitions about the consequences of behavior in a given environment (Tolman, 1932). In one test of his position (Tolman, 1933), Tolman trained hungry rats to choose the white-curtained entrance in a black–white visual discrimination to gain access to a food box. An incorrect choice (black) resulted in confinement in a box without food where shock was given. After the rats were well trained in choosing the white -----> food box alternative, they were placed directly in the food box without running through the alley and received an electric shock. The point of this manipulation was to change the significance of the goal box so that, when, on a subsequent trial, the rat was placed at the beginning of the runway, its cognitions —its "insightful" behavior, in Tolman's language should now lead it to expect shock at the end of the runway. Consequently, the rat should refuse to run. In the test, to Tolman's considerable surprise, and in his picturesque language, "[the rats] immediately dashed off gaily and just as usual through the whole discriminationapparatus and bang whack into the very food compartment in which [they had] just been shocked" (Tolman, 1933, p. 250). Although Tolman suggested some possible reasons for the failure of the rats' cognitions in this situation, he conceded that the data might be a victory for S –R theory. He had reasoned that S –R theory should predict that the rats would fail to run only when the reaction

5

CLARK

L. HULL'S BEHAVIORISM

135

of running to the goal was followed by shock. (In fact, Tolman later trained rats under the latter conditions, and found that they did fail to run in the test.) In an example of how Hull's S –R analysis could make powerful predictions of a "cognitive" sort, Neal Miller (1935) used the logic of Hull's fractional anticipatory goal response to set up conditions in which rats would indeed be expected to manifest "foresight." Shocked at the goal, even without having run to it, they would show evidence of "cognitive" functioning that they anticipated the shock that would occur at the end of the runway. Miller reasoned that shock at the goal would disrupt running on a subsequent test trial only if stimuli in the goal box evoked a response that afforded a characteristic pattern of interoceptive stimulation. In that case the anticipatory goal response (r G) in the runway would have a distinctive feedback stimulus (sG) similar to that present when the animal was in the goal box. Consequently, the goal–shock experience should condition reactions evoked by shock to the distinctive interoceptive stimuli and s G should transfer the reactions to shock from the goal box to the alley and inhibit running in the alley. Miller noted that in Tolman's experiment the goal response may not have been sufficiently distinctive to provide a characteristic feedback stimulus in the goal box. This lack of distinctiveness would account for Tolman's failure to demonstrate foresight in his rats. Miller conducted an experiment in which the animals were required to make a highly distinctive goal-approach response at the end of the runway, such as climbing up and making a sharp turn to the right to enter a narrow tunnel where food was available. Under these conditions, the results were as Tolman had expected on the basis of a cognitive theory, but had been unable to demonstrate in his experiment: Rats shocked at the goal while performing a distinctive goal response showed less vigorous running to the goal for food when subsequently released at the start of the runway, even though the runway training for food and the shock training in the goal were given independently. Miller reported several other manipulations in this elegant paper that strengthened the foothold of S –R theory in the domain of the cognitive. These experiments by Tolman and Miller are early examples of what is now called the "reinforcer devaluation" experiment, which has figured in some recent theoretical efforts to understand the nature of reinforcement (e.g., Colwill & Rescorla, 1986). A recollection by B. F. Skinner provides an interesting historical note concerning Hull, the rG –sG mechanism, and one of Skinner's own efforts to deflate assertions about the special nature of "cognitive" processes. Here are Skinner's words: In 1937 someone had said that the use of tokens was a kind of symbolic behavior beyond the reach of rats, and I countered by teaching a rat . . . to pull a chain to release a marble from a rack, pick the marble up in its forepaws, carry it across the cage, and drop it down a slot. . . . When Hull visited my laboratory . . . in September, 1937, he saw [the rat] at work and noticed that the rat licked the marble as it carried it across the cage. "Anticipatory goal behavior," he called it. I had another explanation: We never washed the marbles and they became rather tacky and no doubt had some flavor if not nutritional value. (Skinner, 1977, p. 1007; emphasis in original)

1 36

RASHOTTE AND AMSEL

Skinner also pointed out that the heralded later discovery that goal-directed instrumental response sequences are sometimes disrupted by intrusion of "inappropriate" anticipatory goal responses (Breland & Breland, 1961) may have been anticipated by Hull decades earlier during this visit. Certainly, the eventual realization that such "misbehavior," as the Brelands termed it (Breland & Breland, 1961), originates in the Pavlovian contingencies inherent in Thorndikeian training situations is in line with this assertion. Later in this chapter, we see that Hull's rG –sG mechanism subsequently became a central organizing idea in Hullian accounts of reward and nonreward learning, in ways that were not anticipated in the original paper on "purpose." B. GESTALT PHENOMENA The Gestalt psychologists Wolfgang Kohler and Max Wertheimer, the American "sign-Gestalt" learning theorist Edward C. Tolman, and the field theorist Kurt Lewin were among the most vocal critics of Hull's version of behaviorism as it was being developed. Hull's responses to these criticisms provided the texture of much of his work in the 1930s and beyond. Some of his Psychological Review papers reflect attempts to deal with behavioral phenomena that were commonly interpreted as clear indicators of "Gestalt" processes, which, for many, fell outside the explanatory scope of S –R behaviorism. Papers dealing with the appearance of sudden "insight" in the solving of problems and demonstrations of "reasoning" by rats (Hull, 1935a, 1935b), and with the way "detour" problems are solved (Hull, 1938), are illustrative. More generally, Hull struggled to work out a logical way to decide which of several theoretical approaches to these and other phenomena was most fruitful (e.g., Hull, 1935a, 1935b). A more complete discussion of these issues from our perspective can be found in Amsel and Rashotte (1984). We comment here on two aspects of the Hullian approach to Gestalt phenomena. One concerns Hull's proposal to utilize "habit-family hierarchies" as the basis for some theoretical accounts. This idea provided S –R theory with a theoretical mechanism underlying flexibility in learned response outcomes. From today's perspective, these hierarchies bear some resemblance to the associative networks found in "neural network" approaches to learning and performance. (In the fifth chapter of Principles of Behavior Hull provides an interesting discussion of hierarchies of unlearned response tendencies and their importance in a mechanistic behavior theory.) The other aspect of Hull's approach we will discuss is the famous theory proposed by Hull's student and close associate, Kenneth Spence (1936, 1937), to account for the Gestalt phenomenon of "transposition." C. HABIT–FAMILY HIERARCHIES (NETWORKS OF ASSOCIATIONS) To account for complex behavior in which there is response variability, and in which a successful response suddenly appears without it having been trained in

5

CLARK

137

L. HULL'S BEHAVIORISM

that situation, Hull introduced an ingenious associative mechanism that combined two types of stimulus-response "compound habit mechanisms" (Hull, 1934a, 1934b). Figure 5.1 shows the separate "divergent" and "convergent" habit mechanisms, and their combination into the hybrid "habit-family hierarchy." For Hull, a compound habit mechanism involved multiple associative tendencies. The divergent mechanism (shown as a) is characterized by a set of independent excitatory response tendencies that can be visualized as radiating out in a

Part a

5

am opb far arab vow am tam 1,4110 . IR°

R4

Part b

Part c S.-

5A

MIN aft me. awe so vim

IRG O. ie. 09

.

Sn ,,e n So /

k is 5 R4

a

P

\\*. 5

eft ow

eft

WI NM 1/..

FIGURE 5. 1 Parts a, b, and c represent the diagrams shown as Figures 1, 2, and 8, respectively, in Hull's (1934a) "The Concept of the Habit-Family Hierarchy and Maze Learning: Part r' (Paper 6 in Amsel & Rashotte, 1984).

138

RASHOTTE AND AMSEL

fan-shaped pattern from a single stimulus. These response tendencies are in competition in the sense that they cannot be executed simultaneously, and they provide a basis for flexibility in responding in a familiar stimulus situation. The convergent mechanism (shown as b) provides for functional equivalence among several stimuli because they all evoke the same response (R G). Hull proposed that once this mechanism is established, it would be understandable that a new response established under one of these stimulus conditions (e.g., s in) could appear with little or no delay when another of the stimulus conditions occurs (e.g., sr). The habit–family hierarchy (shown as c) is a combination of the two individual mechanisms, joined by "a number of distinct behavior sequences which are supposed to differ greatly from one another both as to the qualitative nature of their activity and as to the length of sequence." (Hull, 1934a, p38) The length of the sequences in the diagram represents inversely the excitatory strength of each at point S A : The shortest response to reach the goal is the top one (initiated by R1 ); the least direct sequence is the bottom one (initiated by R4). Obviously, in this network of excitatory tendencies Hull has provided an ingenious way within the framework of S –R theory for considerable flexibility in behavior. Should one of the pathways to the goal be blocked, for example, other alternatives are available in an orderly fashion. Hull exploited this idea in many settings and noted that it serves as "the dominant physical mechanism which . . . provides the basis for a purely physical theory of knowledge" (Hull, 1934a, pp. 40-41). Tolman, in his good-natured way, commented about the attractiveness of Hull's theoretical diagrams: You are all familiar with such diagrams. They are very clever and can be invented, as I know to my cost, to explain practically any type of behavior, however far distant from an instance of conditioning such a behavior might at first sight appear. I have, therefore, the greatest respect for them. And, even though I argue against them, I find myself continually being intrigued and almost ready to change my mind and accept them and Hull after all. (Tolman, 1938, pp. 13-14)

Indeed, Hull was at his most creative in these proposals by which he extended the S –R approach to some important behavioral phenomena studied by Gestalt psychologists, such as the demonstrations by Wolfgang Kohler and by Norman R. F. Maier that novel behavioral solutions can appear suddenly when animals are striving to achieve a goal (Hull, 1935a, 1935b), and Kurt Lewin's findings concerning how young children solve the problem of reaching a goal when the direct path is blocked by a detour (Hull, 1938). More generally, Hull's habit– family hierarchies provided S –R behaviorism with a significant new theoretical tool for conceptualizing complex behavior. D. KENNETH SPENCE'S ANALYSIS OF TRANSPOSITION One of the major challenges to the stimulus–response analysis of learning arose out of learning experiments that appeared to show that animals do not

5

CLARK L. HULL'S BEHAVIORISM

139

learn specific responses to specific stimuli in simultaneous discriminations but to relations between the stimuli. In this challenge, as well as in some others for which discrimination learning was the vehicle, Gestalt psychologists and Tolmanian behaviorists found themselves on the same side. For example, in an early study reported by Wolfgang Kohler (1939), chimpanzees were reinforced for responding to a light-gray but not to a darker-gray stimulus card. In a subsequent test the originally positive (light-gray) card was paired with a new lighter-gray card, and the chimpanzees responded to the new card and not to the previously positive stimulus. Choice of the new card in this test suggested that during original training the animals had learned to respond to a relationship ("lighter than"), and not to the specific shades of gray. This phenomenon was termed transposition because it seemed analogous to what happens when music is transposed from one key to another: All of the elements (notes) are changed but the overall relationship is preserved, and the music is still recognizable as the same. In the mid-1930s, Kenneth W. Spence published a series of classic papers on discrimination learning. In one of these papers, an S —R analysis of relational learning was elaborated (Spence, 1937). Spence pointed out that transposition does not always occur, and that "relational" theories of transposition, which emphasized the role of abstract relative judgments by animals, were not satisfactory because they failed to specify when transposition would fail. Furthermore, simply saying that discrimination learning is relational does not provide a basis for the systematic experimental analysis of transposition. Spence (1936, 1937) proposed an extension of his earlier theory, a kind of Pavlovian model in which discrimination learning was seen as reflecting the divergence of excitatory associative strength to the reinforced stimulus (S +) and inhibitory associative strength to the nonreinforced stimulus (S —). The great advantage of this "absolute" over a "relational" theory was that it made rather specific predictions about the outcomes of relatively complex discrimination learning experiments. In so doing it illustrated the power of the S —R approach in specifying variables for effective experimental analysis. The details of Spence's theory are illustrated in terms of hypothetical discrimination tests among shades of gray. Spence's assumption was that gradients of excitation and inhibition generalize along a continuum of gray stimuli as is the hypothetical case after a discrimination has been formed between gray shades 8 (S +) and 11 (S —) on the basis of reinforcement of responses to S + and nonreinforcement of responses to S — . From the point of view of "relational" theory, the animals should learn the abstract rule that the "lighter than" of a pair of gray stimuli on this dimension is the correct (positive) stimulus. From the point of view of Spence's S —R analysis, however, the reinforcement of responses to S + yields increments in excitatory strength to that stimulus, while the nonreinforcement of responses to S — yields increments in inhibitory strength to that stimulus. The asymptotic levels of excitatory and inhibitory strengths to S + and S — are given by the heights of the appropriate curves directly above S + and S — , re-

140

RASHOTTE AND AMSEL

HIGH Strength of Association

..... ""

I(

LOW

/ 1

/

/

1

1

1

1

1

1

I

I

I

I

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 4

Light

4

s S+ SHADE OF GREY

-

Dark

FIGURE 5.2 Hypothetical gradients of excitation (solid line) and inhibition (dotted line) after discriminative training with stimuli of two shades of gray. Responses to gray-shade 8 (S +) were reinforced: responses to gray-shade 11 were nonreinforced. The net excitatory strength at each shade of gray is the algebraic summation of the excitatory and inhibitory association strength at that shade. The net strength at selected stimuli is shown as a vertical arrow (after Spence, 1937).

spectively. As Figure 5.2 indicates, generalization gradients of excitation and inhibition to surrounding shades of gray were assumed by the theory, so that most stimuli on this continuum of brightness have both excitatory and inhibitory strength. Spence's theory made additional assumptions: (1) that the net associative strength at any point on the continuum is the algebraic summation of the excitatory and inhibitory associative strengths (length of double-headed arrow), and (2) that when confronted with any pair of stimuli the animal's response will be to the stimulus with the absolutely greater net excitatory strength. As indicated, the net excitatory strength to the positive training stimulus S+ (shade 8) is obviously greater than to S — (shade 11), and the animal will respond to S + . If, in a transposition test, stimulus 5 is pitted against the former S + (stimulus 8), the theory predicts responding in a relational manner: the net excitatory strength is greater at S5 than at Sg. The theory also predicts, however, that transposition will fail when, for example, S3 is tested against S 8 or S5 . In these cases, responding should be to the stimulus closer or identical to the original positive training stimulus. At the time Spence's S —R account of transposition was proposed there was little, if any, experimental evidence about the nature of gradients of excitation and inhibition in discrimination learning, and there were no tests along the kinds of extended dimensions and hypothesized interactions of such gradients in discrimination learning, as they related to the phenomenon of transposition. The theory was attractive because it provided a set of mechanisms for the interpretation of transposition that were strictly deduced from a small number of assumptions; however, the major prediction that transposition would break down if one went far enough out on a dimension seemed very much against common sense. While Spence and his students performed several experiments that found evidence for the reversal of transposition, there was, of course, still considerable

5

CLARK

L. HULL'S BEHAVIORISM

141

debate about how best to interpret the phenomenon of transposition (e.g., Ehrenfreund, 1952; Mackintosh, 1974). In subsequent years, the ability to achieve sharp, symmetrical generalization gradients in discrimination experiments with pigeons revived interest in the quantitative features of Spence's theorizing about gradients of excitation and inhibition and the assumption of algebraic summation (e.g., Hearst, 1968). In the 1930s the principal significance of Spence's theory was that it provided a further instance of how a variety of behavioral phenomena could be deduced from a small number of assumptions compatible with S –R theorizing in the tradition of Thorndike and Pavlov. Spence's theoretical and experimental work on discrimination learning (e.g., Spence, 1938, 1940, 1942, 1952c, 1960a) was, and has remained, the most effective counterbalance to cognitive-oriented interpretations of discrimination learning. More generally, Spence's brilliant theoretical writings on this subject and others had an important influence on Hull's thinking, particularly on the more formal statements of Hullian theory in 1943 and later (Amsel & Rashotte, 1984). E. APPLICATION TO PSYCHOTHERAPY AND SYMBOLIC PROCESSES As we have noted, Hull's appointment to the Institute of Human Relations at Yale University, in 1929, provided the opportunity for interactions with sociologists, cultural anthropologists, and other behavioral scientists. One result of these interactions took the form of attempts to employ principles of simple learning in the analysis of complex behavior. For example, Neal E. Miller and John Dollard published books titled Social Learning and Imitation (Miller & Dollard, 1941) and Personality and Psychotherapy (Dollard & Miller, 1950), both of which drew heavily on the basic Hullian approach. In many respects these books stand as earlier versions of Hull's intended third volume, which was of course never written. The opening paragraph of Personality and Psychotherapy conveys the spirit of this kind of undertaking: This book is an attempt to aid in the creation of a psychological base for a general science of human behavior. Three great traditions, heretofore followed separately, are brought together. One of these is psychoanalysis, initiated by the genius of Freud and carried on by his many able students in the art of psychotherapy. Another stems from the work of Pavlov, Thorndike, Hull, and a host of other experimentalists. They have applied the exactness of natural-science method to the study of the principles of learning. Finally, modern social science is crucial because it describes the social conditions under which human beings learn. The ultimate goal is to combine the vitality of psychoanalysis, the rigor of the natural-science laboratory, and the facts of culture. We believe that a psychology of this kind should occupy a fundamental position in the social sciences and humanities — making it unnecessary for each of them to invent its own special assumptions about human nature and personality. (Dollard & Miller, 1950, p. 3)

An important consequence of these efforts to apply Hullian learning principles to complex phenomena was that many of the basic concepts of Hullian the-

142

RASHOTTE AND AMSEL

ory had to be "liberalized" (Miller, 1959). For example, Hull's (1943b) view that need reduction is the source of reinforcement for learned S –R associations proved overly restrictive since many instances of reinforcement (e.g., a small food pellet) were not seen to be related to obvious reduction in bodily needs. Accordingly, for example, Miller and Dollard liberalized Hull's position and proposed that the basis of reinforcement is the reduction of any strong stimulus (e.g., the response of pulling down a window blind is reinforced by the elimination of glare). Reinforcement can result from the termination of "arbitrary" stimuli that, through classical conditioning, come to play the same functional role as do primary drives such as hunger and thirst the principle of "secondary reinforcement" around which considerable research had already been initiated. Hull's other tightly formulated principles were similarly strained in learningbased analyses of such phenomena as imitation, copying, reasoning, symptom learning, Freudian repression, and techniques of psychotherapy. Furthermore, the analysis of neurotic behavior patterns and psychotherapy was strongly influenced by Miller's brilliant analyses, from animal learning experiments, of the establishment, resolution, and displacement of conflicts (e.g., Miller, 1944, 1959). It is interesting that although Miller and Dollard's analysis of neurosis and psychotherapy was influential in alerting practitioners to the importance of learning processes, it never attracted a fervent band of adherents, dedicated to revolutionizing the practice of psychotherapy. The person with a Hullian orientation who was, perhaps, most identified with this movement was Joseph Wolpe, with his early writings on reciprocal inhibition and the resultant development of systematic desensitization, a conditioning therapy (e.g., Wolpe, 1958). It is not possible to elaborate here on the many early contributions that others made to an analysis of complex behavior within the general tradition of Hullian thought. We must, at least, call attention to the many influential contributions of 0. H. Mowrer to S –R analyses of neurotic behavior, psychotherapy, language, and symbolic processes (e.g., Mowrer, 1939, 1940, 1953, 1960a, 1960b); to D. E. Berlyne's Hullian-oriented theoretical analyses of exploratory behavior, thinking, and aesthetic behavior (Berlyne, 1960, 1965, 1971); to Charles Osgood's S –R analyses of language and symbolic processes (Osgood, 1953); and to J. G. Taylor's thoroughgoing account of perception from an S –R perspective (Taylor, 1962).

III. CONCEPTUALIZATIONS OF REINFORCEMENT AND REWARD: EVOLUTION OF HULL'S 1943 THEORY The term reward appears only once in the index of subjects in Hull's Principles. (The terms punishment and fear do not appear in the index at all.) The concept of reinforcement, of course, plays the major role. It is defined as the

5

CLARK L. HULL'S BEHAVIORISM

143

diminution of a need (whether appetitive or aversive) that is necessary for the formation and strengthening of habit. Since the term reward was not used by Hull in 1943, it may seem odd that we choose to trace its meaning from its neglect by Hull to its development by others who worked in Hull's tradition. The point is that the discussion that follows addresses itself to the concept of reward and not necessarily to reinforcement, and this conceptualization of reward turns out to be very close to the meaning it had for Hull in 1943 —"incentive." A. "REINFORCEMENT" IN HULL'S 1943 THEORY

For the reader unfamiliar with the formal statements of Hull, we quote from Principles the major features of Postulate 4, to which we have already alluded: Whenever an effector activity (r —> R) and a receptor activity (S —> s) occur in close temporal contiguity (:C r), and this :Cr is closely associated with the diminution of a need (G) or with a stimulus which has been closely and consistently associated with the diminution of a need (G), there will result an increment to a tendency (AsHR ) for that afferent impulse on later occasions to evoke that reaction. The increments from successive reinforcements summate in a manner which yields a combined habit strength (SHR) which is a simple positive growth function of the number of reinforcements (N). (Hull, 1943b, p. 178)

For Hull, behavior was not related directly to habit strength but to effective excitatory potential, S ER (excitatory potential minus total inhibitory strength). This feature of the theory is expressed in Postulate 7, which specifies that all "primary drives" contribute to the total drive strength (D), which multiplies the current value of SHR to determine the strength of S ER, and, consequently, the strength of responding in a given situation: generally, S E R = f( S H R ) X f(D): Any effective habit strength ( SHR) is sensitized into reaction potentiality ( SER) by all primary drives active within an organism at a given time, the magnitude of this potentiality being a product obtained by multiplying an increasing function of SHR by an increasing function of D. (Hull, 1943b, p. 253)

Hull was led to this conceptualization by a number of experimental findings, but particularly by the data of Perin (1942), which are shown in Figure 5.3 as Hull portrayed them (p. 228) in Principles. Since neither magnitude nor delay of reinforcement was varied in Perin's experiment, the group differences at the end of acquisition were taken to reflect the strength of SHR as a function of number of reinforcements. It is clear that Hull takes the relation between number of responses in extinction and number of reinforcements in acquisition to be a simple positive growth function, as would follow if S HR increased in the manner required by Postulate 4. Further, the figure shows that when drive is varied in extinction (reduced from the acquisition level to 3 hours for one group and left unchanged for the other), the curves reach different asymptotes as would be expected if SER was the product of D and SHR (Postulate 7).

1 44



RASHOTTE AND AMSEL

60

22 HOURS HUNGER

g zo^ o u

i---

c.) z

8 40 U D G 0 cc

a_ 30 0 '–

co z 0 i 20 t.) 6 re re w

co 2

3 HOURS HUNGER

10

D

z I

10

J

20

60 30 40 50 70 NUMBER OF REINFORCEMENT REPETITIONS (N)

1

1

80

90

FIGURE 5.3 Number of unreinforced bar presses required by differently motivated groups of rats to reach a criterion of extinction, plotted as a function of the number of times the bar press was reinforced with food during acquisition. (Hull assumed that, under certain circumstances, this response measure would reflect the strength of SER.)

B. DIFFICULTIES RECOGNIZED BY HULL IN 1943

Hull obviously knew that the 1943 treatment of the Perin data was an oversimplification or, at the very most, that this conceptualization would be useful only under certain restricted (ideal) experimental conditions. For example, Perin used reactions to extinction (n) as the dependent variable, but there were already in existence important data by L. G. Humphreys (1939a, 1939b), of which Hull was aware (1943b), that restricted the use of resistance to extinction as a measure of habit strength to cases in which reinforcement is continuous: Humphreys had already shown in a series of experiments that resistance to extinction is greater following a relatively small number of reinforcements if those have occurred in the context of "partial reinforcement," the quasi-random intermixture of reinforcements and nonreinforcements. [Hull's own theoretical writings never dealt adequately with the so-called partial reinforcement extinction effect (PREE), and it fell to others, such as Amsel (e.g., 1958, 1962) and Capaldi (e.g., 1966), to provide influential theoretical treatments of partial reinforcement, and in Amsel's case, also discrimination learning, in the context of modified Hullian theory.] A second limitation on the applicability of the theory was in Hull's treatment of response latency (or, inversely, speed) as the dependent variable. The 1943

5

CLARK L. HULL'S BEHAVIORISM

145

theory specified that response latency declines as a negatively accelerated function of number of reinforcements (N), assuming a constant drive level (D). Hull recognized that this relationship can hold only when the promptness of reinforcement depends on the promptness of responding, as it does ordinarily in the case where, for example, a rat runs from a start box through a runway to obtain food at the goal. There are cases, however, in which reinforcement and the rapidity of the response are negatively correlated, such as when the rat must learn to delay leaving the start box to obtain reward, or is reinforced for some "self-imposed" ritual of slow responding (actually "time-taking"). In cases of this sort, frequency of reinforcement increases as latency (time to respond) increases. Hull recognized this as a further limitation of the theory's predictiveness. In later years, Frank A. Logan (1960), who worked with Hull on his last book, developed what he called a "micromolar" approach in which the concept of negatively correlated reinforcement figured prominently. This approach further increased the flexibility of theory in the Hullian tradition, particularly on the dependentvariable side. The oversimplicity of Hull's habit-strength function, which asserted that habit strength grew as a function of the number of reinforcements, SHR = f(N), was further questioned in our work (Amsel & Rashotte, 1969; Rashotte & Amsel, 1968); we showed that not only did rats learn idiosyncratic rituals to "take time" under one of Logan's negatively correlated reward conditions, but that these rituals emerged in extinction, even after prolonged intervening training on a simple continuous reinforcement schedule (CRF) in which the rituals were eliminated. C. THE ELLIOTT —CRESPI—ZEAMAN EXPERIMENT

Another difficulty for Hull's SHR = f(N) arose just before Principles of Behavior appeared in print. L. P. Crespi (1942), in a result partly anticipated by Elliott (1928) and soon replicated by Zeaman (1949), reported two effects that were troublesome for Hull's concept of reinforcement. First, rats trained to run to a large reward abruptly slowed down to a level below that of a small-reward control group when the reward size was reduced (Crespi's "depression effect"); and rats trained to a small reward abruptly increased their speed when reward size was increased ("elation effect"). The Elliott–Crespi–Zeaman findings posed three problems for Hull's 1943 theory. First, because habit strength is a function of reward magnitude, larger reward is expected to yield a higher level of S HR to multiply D, and therefore a high level of performance (e.g., running speed). This much was found in the first phase of the Crespi–Zeaman experiments. However, because SHR is assumed to represent a relatively permanent change in associative strength, decreases in reward size in the second phase of the experiments should have little immediate effect on response strength (running speed). Instead, Crespi and Zeaman reported pronounced decreases in speed after a single trial at the larger reward value, and the depressed running speed recovered to a new asymptotic level ap-

146

RASHOTTE AND AMSEL

propriate to the smaller reward only after four trials. The shift from small to large reward also produced immediate shifts toward higher asymptote than did a large reward from the outset. (This last finding has been harder to replicate.) The clear implication of these findings, in any case, was that magnitude-of-reinforcement (reward– size) manipulations did affect behavior, but not solely through SHR, as the 1943 theory required. Pressured by these findings and by critiques of the theory's handling of reinforcement, Hull revised his conceptualization of reinforcement in later (1951, 1952a) versions of the postulates. The revised conceptualizations took magnitude and delay of reinforcement out of the determination of habit strength and made SHR a function only of the number of reinforcements. The action of magnitude and delay of reinforcement were now expressed through two new theoretical factors, representing incentive motivation (symbolically, K and J): K and J were now incentive–motivational factors in the sense that, added to D, they multiplied habit SHR to determine excitatory potential SER. The split-off of associative–reinforcing function from motivational–incentive function was now complete. The "law" of habit formation, SHR = f(N), expressed habit strength to be a function only of the number of reinforcements (N). Reaction potential ( SER) was still determined by SHR along with drive strength (D), but now added to D, as undifferentiated incentive motivational factors, were K and J, and new computational formulas were provided for determining the values of K and J and SER (Hull, 1951). Hull's new conceptualization of reaction potential took care of most aspects of the Crespi–Zeaman experiment. For example, his new equations for K (Postulate 7) made it possible for a change in reward size (either upward or downward, even if it occurred only once) to cause the value of K to change substantially toward the value required by the new reward size. And because the changed values of K and J entered multiplicatively into the equation for SER, a changed reward value (K) or delay in reward (J) could cause behavior such as response speed in a runway to move to a new asymptotic level appropriate to that reward in very few trials. The revised theory did not handle the overshooting (elation) and undershooting (depression) effects Crespi identified in shifts from small to large reward and large to small reward, respectively. Crespi held that these effects were emotional and should disappear with continued training. The work on successive negative contrast (SNC) shows clearly that this is the case in the shift from large to small reward size (see Amsel, 1992, for a review). Hull accepted this analysis but did not feature it in his theorizing because he was uncertain about the generality of the phenomena in question. Hull's own discussion of reward-shift in A Behavior System (1952a) centered on the latent-learning experiment. The classic latentlearning experiment is one in which animals are run through a maze for several trials without reward, and are then shifted to a series of large rewards (e.g., Blodgett, 1929). In some of these experiments (Tolman & Honzik, 1930) a downward shift in reward is also included. In these experiments, as in those of Crespi and

5

CLARK

L. HULL'S BEHAVIORISM

147

Zeaman, upward shifts are followed by abrupt improvement in performance (reduction in errors in the maze) and downward shifts result in increased errors. However, as Hull observed, "overshooting" is found in neither case. The entire history of reward-shift experiments and the theoretical treatment they received by Hull and by many other theorists has recently been reviewed in great detail (Flaherty, 1996). D. SPENCE'S CONTRIBUTIONS TO REWARD-INCENTIVE THEORY An important contributor to Hull's formal theorizing from the beginning, Kenneth W. Spence continued the development of what came to be called Hull–Spence theory in important ways. A significant part of the empirical and theoretical work of Spence and his students at Iowa was directed at motive–incentive factors in behavior, and this emphasis was reflected in Spence's Silliman Lectures at Yale University, published as the book Behavior Theory and Conditioning (1956). In Chapter 5 of this book, Spence's treatment of "Reinforcement in Instrumental Learning" is primarily a discussion of how the theory must be adapted to handle new data on magnitude and delay of reward. It was characteristic of Spence to let vigorous experimental analysis guide the development of his theorizing, and the problem of magnitude of reward in instrumental learning is a good example of that style. In 1956 Spence reported several experiments, conducted in his laboratory, which appear to replicate all aspects of the Crespi–Zeaman experiments except "overshooting" of the asymptote for large reward following a shift from small to large reward (Spence, 1956). The problem for Spence was that the number of trials before the shift from small to large magnitude of reward in the Crespi–Zeaman experiments was so limited that running speed had not reached asymptote in the preshift phase. Consequently, the apparent "elation effect" following the shift seemed attributable to the uninteresting fact that speed was simply approaching its true asymptote. A period of intensive experimental work on this problem left Spence with a firmer empirical base for the theoretical treatment of reward magnitude. There remained three facts to be accounted for in a theory of the action of rewards: (a) rate and asymptote of responding are higher for large reward than for small reward; (b) performance level shifts quickly following increases and decreases in reward size; and (c) the Crespi–Zeaman "depression effect" occurs —response speed falls below the low-reward asymptote following a shift from high to low reward. At this point, there is an obvious return to the spirit of Hull's writings in the 1930s where rG –sG played such a central theoretical role. Indeed Spence had prepared the reader for his view of reward–incentive effects in Chapter 2 of Behavior Theory and Conditioning (see Figure 5.4): Spence's position, like Hull's earlier one, was that in every case of instrumental learning there is an inherent classical conditioning component. Further, according to the Pavlovian principle of stimulus substitution, different reinforcers,

148

RASHOTTE AND AMSEL

A. SA

B. SA SA

II I rg'Sr '1– -"I'

II 1 I I

SG

/

\

rgSg— --4. 40 Rwc RG

FIGURE 5.4 Diagrammatic representation of the conditioning processes involved in instrumental learning. A. Early in training. This diagram shows how classical conditioning of the consummatory response (R G ), represented by arrows marked II, is an integral part of instrumental learning. The arrows marked I represent the instrumental conditioning of the locomoting response. B. Later in training. The stimulus cues of the alley (S A , S A') are assumed to evoke an anticipatory, fractional portion of the consummatory response, denoted r g. The interoceptive cue (s g ) produced by r g, in turn, is assumed to acquire a habit loading for the instrumental locomoting response.

such as food or water, should produce qualitatively different R Gs, and therefore different conditioned rgs. As in Hull's classic conceptualization, r g is assumed to produce distinct feedback stimulation, symbolized sg, and the mechanism is designated rg —sg. What is new in this portion of Spence's theorizing is that he gives rg motivational properties, something Hull had not done earlier, while retaining in sg the major associative mechanism of expectation that Hull had conceptualized 25 years earlier. More formally, Spence (1956) proposes that incentive motivation, K, represents, quantitatively, the strength of rg –sg. In this way, rg –sg is tied directly to one of the theoretical variables that determine the value of SER (henceforth E, as Spence omits the subscripts S and R). Further, Spence's account restricts the effects of reward in instrumental appetitive learning to K and to the associative properties of s g. The growth of H (sHR) is determined simply by contiguity, the number of S –R conjunctions. We can recognize this position as the ultimate form of an evolution of thinking in which positive reinforce-

5

CLARK L. HULL'S BEHAVIORISM

149

ment (or reward) plays an ever-decreasing role as an associative variable, as it did under Hull's strong reinforcement principle of 1943 . In recognition of the essential similarity of D and K as general motivational factors, and as a result of other experiments, Spence held D and K to be additive [E = H X (D + K)], not multiplicative, in determining excitatory potential. In the revised postulates (1952a) Hull's formula had been SER = SHR X D X K. It remained for Spence to account for the depression effect, that response strength falls below the low-reward asymptote following a shift from large to small reward. He elected, as Crespi had before, to attribute this "undershooting" of the asymptote to an emotional reaction, "frustration," and he regarded frustration as inhibitory. There was no specific mechanism for the relation of frustration to inhibition until some years later when Spence (1960b) endorsed the theoretical view of his student Amsel (1958), who proposed such a mechanism in what is now known as Frustration Theory. Spence's treatment of the role of reward in instrumental learning stimulated a number of experiments, because it identified several experimental operations that should influence performance through K. In particular, the theoretical strategy of specifying rg –sg3 as the basis of K encouraged tests of the theory in which explicit classical conditioning manipulations were made on the baseline of instrumental learning experiments. For example, Spence (1956) reported experiments that attempted to vary strength of conditioning of rg to stimuli of the SA variety by manipulating the time the rat was allowed to eat in the goal box. Although experiments such as these suggest that the concept of rg –sg as the basis for K-value stimulated research, it was often impossible to employ the logic of the approach as the basis for unequivocal predictions about instrumental responding. The problem lay in the lack of precise understanding of the basis for conditioning of rg –sg : the runway was conceptualized as a sequence of exteroceptive stimuli preceding the goal-box stimulus and food; the theory assumed that the goal-box stimuli were conditioned to rg because of their close temporal proximity to the food US, and that stimuli earlier in the runway evoked rg through generalization. Accordingly, the strength of r g –sg at any point in the runway should be determined by the similarity between runway cues at that point and the goal-box stimuli. In Spence's view, these assumptions were tentative and needed refinement. For one thing, as Spence pointed out, proprioceptive as well as exteroceptive cues would surely also affect the quantitative details of generalization of rG –sG across the runway. Another difficulty, which became more and more apparent in the sixties and later, arose from the growing evidence from studies of successive-compound conditioning that processes far more complex than generalization influence the strength of conditioning to stimuli in a sequence preceding a US (e.g., Baker, 1968; Egger & Miller, 1962; Holland, 1992; Razran, 1971; Rudenko, 1974; Wickens, Born, & Wickens, 1963). Additional 3 Subscripts are sometimes lowercase (g or f) and sometimes uppercase (G or F). This follows the author's usage in each instance.

1 50

RASHOTTE AND AMSEL

difficulties with the rg –sg analysis had also been presented by Logan, in his book Incentive (1960). E. AMSEL'S FRUSTRATION THEORY: A CONDITIONING MODEL OF EFFECTS OF REWARD AND NONREWARD As we have noted, both Hull and Spence accepted the view that "frustration" (or some other emotional factor) accounted for the depression of running speed below a small-reward asymptote when magnitude of reward was shifted from large to small, but neither theorist provided a detailed account of how frustration might enter into the general structure of the theory. Other researchers had also reported signs of "emotional" upset in animals at the beginning of extinction, and some had suggested that emotion should be included in making predictions about the strength of responding in extinction (e.g., Skinner, 1950). As early as 1951, at a meeting of the Southern Society for Philosophy and Psychology, Amsel proposed a conceptualization of the role of anticipatory frustration, in the context of the rG –sG mechanism, as a third factor to be added to Hull's two-factor theory of inhibition. In this paper Amsel was extending Hull's two-factor theory of inhibition and proposing a third factor, anticipatory frustration (rF –sF), as important in understanding a variety of nonreinforcement-related inhibitory effects. He addressed himself specifically to the Crespi–Zeaman effect, which we have already discussed, but included a discussion of other "paradoxical" effects that occur when rewards and nonrewards are intermixed in learning. In the same year, J. S. Brown and I. E. Farber (1951) advanced a theory of frustration, also in a Hullian framework. Their elegant, but complex, theory related frustration to conflicting response tendencies, as well as to the operation of nonreinforcement and thwarting. The theory was appended to a more general conceptualization of emotions as intervening variables, and it is, perhaps, for this reason that it did not have the impact on experimental work it might otherwise have had. Amsel subsequently elaborated his views in a series of empirical and theoretical papers (Amsel, 1958, 1962, 1967) that included Amsel and Roussel's (1952) demonstration of the "Frustration Effect" (FE), an account of the primary or energizing (D-like) effects of frustration. The whole approach came to be known as Frustration Theory. It accounted not only for the primary (activating) effects of frustration resulting when an expected reward does not materialize, but also for the partial reinforcement extinction effect (PREE) and, ultimately, for a family of other paradoxical effects of reward schedules in both discriminative and nondiscriminative instrumental learning (see Amsel, 1986, 1992). In informal terms, Amsel's theory assumed that when nonreward, reduced reward, or delayed reward occurs in place of an expected reward, the animal experiences a temporary aversive motivational state, primary frustration. With the conditions that produce primary frustration specified, the theory outlined three properties of frustration. The first, primary frustration (RF), is a hypothetical un-

5

CLARK

L. HULL'S BEHAVIORISM

151

conditioned reaction to the frustrating event. The theory specifies that RF will exert a transient motivational (energizing) effect on responses with which it coincides, increasing, particularly, the vigor with which these responses are performed. The behavioral manifestation of such a consequence of R F (the FE) is an effect that has been documented by many (perhaps hundreds of) empirical studies, most of them employing some variant of the double-runway apparatus first used by Amsel and Roussel (see Amsel, 1992, for a more extended account). The second property of frustration is the frustration drive stimulus (S F), a feedback stimulus from primary frustration that acts to cue, guide, and direct behavior (i.e., it acts associatively; Amsel & Prouty, 1959; Amsel & Ward, 1954). The third property of frustration, and the one that was actually proposed first, and that bears perhaps the heaviest theoretical burden, is conditioned (anticipatory) frustration (rF –sF). This factor refers to the manner in which frustration influences responses that precede the frustrating event, and here the theory relies on the logic derived from its Pavlovian antecedents: With repeated occurrences, stimuli (conditioned stimuli, or CSs) accompanying primary frustration (RF) come to evoke a classically conditioned form of R F, designated rF –sF. As with rG –sG, of which it is an aversive counterpart, rF –sF is evoked first by stimuli in the region of the goal event, and later moves forward in time into the instrumental response. This conditioned response (r F –sF) is assumed to increase in strength as a function of trials, reaching an asymptotic strength appropriate to the strength of RF. The role of rF –sF in instrumental responding was specified most clearly in later versions of the theory that emphasize that its function varies with its strength (Amsel, 1967). While rF contributes to incentive motivation, K, at all values of its strength, it provides an aversive guiding stimulus (sF) only at values above some threshold. At weak intensities, therefore, the role of r F is only an invigorating one. As rF becomes stronger, however, s F becomes an aversive stimulus and evokes unconditioned and conditioned responses that are antagonistic to the referent instrumental response. Obviously, such competing responses evoked by rF –sF provide a plausible account — Amsel provided this account in his 1951 paper— of depression of running speed below the small-reward asymptote following a shift from large to small reward in the Crespi–Zeaman experiment. Spence might have used such a mechanism in 1956 as the basis for his inhibition (I), which, it will be recalled, represented the effects of competing responses and subtracted from E to weaken performance. He did later adopt r F –sF as the basis for I (Spence, 1960b). Perhaps the most referenced portion of Amsel's theory concerns the influence of rF –sF when the instrumental response continues to be performed in its presence. There are two cases, depicted in Figure 5.5. The first is the partial reinforcement experiment in which reward for an instrumental response is restricted to a random percentage (usually half) of trials. The four-stage hypothesis (Amsel, 1958) outlines a sequence of events leading to the development of persistence. The second case concerns the presence of differential external stimuli (S +

152

RASHOTTE AND AMSEL

PARTIAL REWARD REWARD TRIALS

A

0 Z

SA

Z

Cr 4 LLJ J

SA

SR

\>1n RA ,

, ,

tfi sit4RApp *

RR

/ RAPP

JNon

\ RNon R

App

RAPP

. . 'Apo

. 4. ''A , rR- 'setA pp . .F

SA

SNon R \-.),./..: \ ....., 4, elf RR

'Fi- si/App 4 R F

SNon R 4. rfr s

r

W V' < (/)

rF; $F

rF:$F-4RAV

3

4A

Conflict

1 ''' RApp 'Ft- $1,V

RNon R

A

0 (/)

NONREWARD TRIALS

SNon R

\

SR

2 1 \:?C\

DISCRIMINATION LEARNING

NONREWARD TRIALS REWARD TRIALS



♦ RAV

4 Conflict 'RI' $ R-- 4 RAI*

f

r - sf SA

42 - $R--"RApp

FIGURE 5.5 Diagrammed sequence of hypotheses relating frustrative nonreward to stages of partial reward and discrimination learning. (adapted from Amsel, 1958).

and S —), and the role of rF —sF as a mediating mechanism in the formation of simultaneous or successive-trial discrimination learning. The assumption that rF —sF becomes conditioned to the instrumental response has provided theoretical leverage in topic areas beyond the scope of Hull's original theory, and indeed beyond the problem areas to which frustration theory was initially applied (see Amsel, 1992). The conditioning model theory of Amsel has integrated the explanations of a number of phenomena besides the invigorating effect of primary frustration (the FE), the Crespi—Zeaman effect, and the partial reinforcement extinction effect (PREE). The theory was extended in 1962 to account for a variety of phenomena (see Appendix, Amsel, 1992, pp. 233ff): emotional—motivational factors in discrimination learning; the retention and durability of persistence; the overlearning —extinction effect; the overlearning —reversal effect in discrimination learning; the phenomenon of subzero extinction (a finding analogous to the depression effect); and the paradoxical Haggard—Goodrich partial reinforcement acquisition effect (PRAE), in which partially rewarded animals respond more vigorously in early stages of the instrumental response-chain in acquisition than do continuously rewarded animals. Also accounted for are the action of certain drugs such as alcohol and sodium amobarbital to attenuate the PRAE and the PREE; the phenomenon of behavioral contrast; the appearance of certain "adjunctive" behaviors including schedule-induced polydipsia in Skinnerian—operant experiments; the role of the limbic system, particularly the role

5

CLARK L. HULL'S BEHAVIORISM

153

of the amygdala and the hippocampus, in short-term memory –based learning in certain "paradoxical" phenomena such as the PREE; transfer of persistence across situations and motivational–reinforcement conditions, including the effects of early experience on later behavior; and the phenomenon of "regression": return to earlier successful modes of behavior. Amsel (1972, 1992) extended the theory to provide a more general account of persistence, viewing partial reinforcement as only one instance of a more general case. This extended theory relies heavily on the proposal that stimuli that initially disrupt instrumental responding (e.g., stimuli to which subjects "habituate") become counterconditioned to the evocation of the instrumental responding they initially disrupt, with the consequence that instrumental responding will, in the future, be relatively unaffected by these and other frustrative or distracting stimuli. This general theory of persistence stems from the work of Amsel and his students and others (e.g., Terris, German, & Enzie, 1969; Wagner, 1966) on transfer of persistence. In the last 20 years, this and other work has led, quite naturally, to an increasing interest by Amsel and his students in the study not only of the PREE, but also persistence and a number of other behavioral effects based on a variety of reward schedules, all of this from a strong ontogenetic –developmental perspective, but also with a behavioral neuroscience emphasis, particularly on hippocampal function (see Amsel, 1986, 1992, 1994). The material reviewed in the preceding subsections provides one illustration of how behavior theory of the Hullian variety initiated and responded to empirical findings. In this case, where some important changes in the conceptualization of reinforcement, reward, and nonreward in Hullian theory are reviewed from 1943 onward, we can see that Hullian behaviorism has been strongly disposed to refine its initial formulations and to develop new ones within the S –R context as the empirical evidence requires.

CONCLUDING COMMENTS Hull's S –R behaviorism was a major addition to the intellectual achievements of psychology during the twentieth century. Our present review of selected aspects of his efforts characterizes some important features of his approach. However, a firsthand reading of his theoretical papers in the Psychological Review (Amsel & Rashotte, 1984) and of his classic books such as Principles of Behavior (Hull, 1943b) and A Behavior System (Hull, 1952a) remains the best way to grasp his thinking and the possibilities it raises for behavior theory even today. What is the enduring value of Clark L. Hull's work? We addressed this issue several years ago in a chapter, titled "Postscript: The Decline and Lasting Influence of Clark Hull," which followed our extended commentary on his 21 Psychological Review papers (Amsel & Rashotte, 1984). Our comments at that time remain apt. Therefore, we conclude with an excerpt from our earlier postscript:

1 54

RASHOTTE AND AMSEL

What is most important in Hull's surviving influence is perhaps the scientific style and tradition: the willingness to provide explicit and falsifiable predictions from a set of hypotheses derived from the existing literature; the enormous body of experimental results, certainly not always confirmatory, these hypotheses have generated; the economy of a family of interrelated constructs, empirically derived; the recognition in the context of general lawfulness that if explanations are as complex (have as many degrees of freedom) as the phenomena they seek to explain, they offer no explanatory advantage; the emphasis not only on the purely associative but also the motivational processes (the latter almost completely neglected since the return of cognitive structuralism in the 1960s); the hope for and attempt at quantification in learning theory, still to this day a science of ordinal differences and interactions. If not the content—and much of this certainly survives — then the approach, the strategy, and the example of dedication are identifiable aspects of Hull's legacy. (Amsel & Rashotte, 1984, p. 507)

ACKNOWLEDGMENTS We express our deep thanks to Anna Tapsak, at the University of Texas, Austin, who provided us with excellent and timely technical assistance in preparing this chapter. We are indebted to Ruth Hull Low and Janet Taylor Spence for permission to quote from private correspondence written by Clark Hull and Kenneth Spence, respectively. We thank Kindler Verlag for permission to incorporate adapted portions of an earlier chapter we wrote that appeared in a German language version: Amsel, A, & Rashotte, M. E. (1977). A perspective on S –R learning theory in America with particular reference to Clark L. Hull, his precursors, and his followers. In Zeier, H. (Ed.). The psychology of the 20th century: Vol. 4. Pavlov and his followers: From Classical conditioning to behavioral therapy. Zurich: Kindler Verlag (in German).

REFERENCES Amsel, A. (1958). The role of frustrative nonreward in noncontinuous reward situations. Psychological Bulletin, 55, 102 –119. Amsel, A. (1962). Frustrative nonreward in partial reinforcement and discrimination learning: Some recent history and a theoretical extension. Psychological Review, 69, 306-328. Amsel, A. (1967). Partial reinforcement effects on vigor and persistence. In K. W. Spence & J. T. Spence (Eds.), The psychology of learning and motivation (Vol. 1, pp. 1-65). New York: Academic Press. Amsel, A. (1972). Behavioral habituation, counterconditioning, and a general theory of persistence. In A. H. Black & W. F. Prokasy (Eds.), Classical conditioning II: Current research and theory (pp. 409-426) New York: Appleton-Century-Crofts. Amsel, A. (1986). Developmental psychobiology and behaviour theory: Reciprocating influences (Daniel E. Berlyne memorial lecture). Canadian Journal of Psychology, 40, 311-342. Amsel, A. (1992). Frustration theory: An analysis of dispositional learning and memory. Cambridge, England: Cambridge University Press. Amsel, A. (1994). Précis of Frustration theory: An analysis of dispositional learning and memory, Cambridge University Press, 1992. Psychonomic Bulletin and Review, 1, 280-296. Amsel, A., & Prouty, D. (1959). Frustrative factors in selective learning with reward and nonreward as discriminanda. Journal of Experimental Psychology, 57, 224-230. Amsel, A., & Rashotte, M. E. (1969). Transfer of experimenter-imposed slow-response patterns to

5

CLARK L. HULL'S BEHAVIORISM

155

the extinction of a continuously rewarded response. Journal of Comparative and Physiological Psychology, 69, 185 -189. Amsel, A., & Rashotte, M. E. (1984). Mechanisms of adaptive behavior: Clark L. Hull's theoretical papers, with commentary. New York: Columbia University Press. Amsel, A., & Roussel, J. (1952). Motivational properties of frustration: I. Effect on a running response of the addition of frustration to the motivational complex. Journal of Experimental Psychology, 43, 363-368. Amsel, A., & Ward, J. S. (1954). Motivational properties of frustration: II. Frustration drive stimulus and frustration reduction in selective learning. Journal of Experimental Psychology, 48, 37-47. Baernstein, H. D., & Hull, C. L. (1931). A mechanical model of the conditioned reflex. Journal of General Psychology, 5, 99 -106. Baker, T. W. (1968) Properties of compound conditioned stimuli and their components. Psychological Bulletin, 70, 611 -625. Bergman, G., & Spence, K. W. (1944). Operationism and theory in psychology. Psychological Review, 51, 47-68. Berlyne, D. E. (1960). Conflict arousal and curiosity. New York: McGraw-Hill. Berlyne, D. E. (1965). Structure and direction in thinking. New York: John Wiley & Sons. Berlyne, D. E. (1971). Aesthetics and psychobiology. New York: Appleton-Century-Crofts. Blodgett, H. C. (1929). The effect of the introduction of reward upon the maze performance of rats. University of California Publications in Psychology, 4, 113-134. Breland, K., & Breland, M. (1961). The misbehavior of organisms. American Psychologist, 16, 681- 684. Brown, J. S., & Farber, I. E. (1951). Emotions conceptualized as intervening variables-with suggestions toward a theory of frustration. Psychological Bulletin, 48, 465-495. Capaldi, E. J. (1966). Partial reinforcement: A hypothesis of sequential effects. Psychological Review, 73, 459-477. Colwill, R. M. & Rescorla, R. A. (1986). Associative structures in instrumental learning. In G. H. Bower (Ed.), The psychology of learning and motivation (Vol. 20, pp. 55-104). New York: Academic Press. Crespi, L. P. (1942). Quantitative variation of incentive and performance in the white rat. American Journal of Psychology, 55, 467 - 517. Dollard, J., & Miller, N. E. (1950). Personality and psychotherapy. New York: McGraw-Hill. Egger, M. D., & Miller, N. E. (1962). Secondary reinforcement in rats as a function of information value and reliability of the stimulus. Journal of Experimental Psychology, 64, 97-104. Ehrenfreund, D. (1952). A study of the transposition gradient. Journal of Experimental Psychology, 43, 81-87. Elliott, M. H. (1928). The effect of change of reward on the maze performance of rats. University of California Publications in Psychology, 4, 19-30. Flaherty, C. E (1996). Incentive relativity. Cambridge, England: Cambridge University Press. Hearst, E. (1968). Discrimination learning as the summation of excitation and inhibition. Science, 162, 1303 -1306. Holland, P. C. (1992). Occasion setting in Pavlovian conditioning. In D. Medin (Ed.), The psychology of learning and motivation (Vol. 28, pp. 69-125). San Diego, CA: Academic Press. Hull, C. L. (1925a). An automatic correlation calculating machine. Journal of the American Statistical Association, 20, 522 - 531. Hull, C. L. (1925b). An automatic machine for making multiple aptitude forecasts. Journal of Educational Psychology, 26, 593 -598. Hull, C. L. (1929). A functional interpretation of the conditioned reflex. Psychological Review, 36, 498 - 511. Hull, C. L. (1930). Knowledge and purpose as habit mechanisms. Psychological Review, 37, 511-525. Hull, C. L. (1931). Goal attraction and directing ideas conceived as habit phenomena. Psychological Review, 38, 487-506.

156

RASHOTTE AND AMSEL

Hull, C. L. (1934a). The concept of the habit-family hierarchy and maze learning: Part 1. Psychological Review, 41, 33-54. Hull, C. L. (1934b). The concept of the habit-family hierarchy and maze learning: Part 2. Psychological Review, 41, 134 -152. Hull, C. L. (1935a). The conflicting psychologies of learning-a way out. Psychological Review, 42, 491-516. Hull, C. L. (1935b). The mechanism of the assembly of behavior segments in novel combinations suitable for problem solution. Psychological Review, 42, 219-245. Hull, C. L. (1938). The goal gradient hypothesis applied to some "field force" problems in young children. Psychological Review, 45, 271- 299. Hull, C. L. (1943a). A postscript concerning intervening variables. Psychological Review, 50, 540. Hull, C. L. (1943b). Principles of behavior: An introduction to behavior theory. New York: Appleton-Century-Crofts. Hull, C. L. (1943c). The problem in intervening variables in molar behavior theory. Psychological Review, 50, 273-291. Hull, C. L. (1951). Essentials of behavior. New Haven, CT: Yale University Press. Hull, C. L. (1952a). A behavior system: An introduction to behavior theory concerning the individual organism. New Haven, CT: Yale University Press. Hull, C. L. (1952b). Autobiography. In E. G. Boring, H. S. Langfeld, H. Werner, & R. M. Yerkes (Eds.), A history of psychology in autobiography (Vol. 4, pp. 143-162). Worcester, MA: Clark University Press. Hull, C. L. (1962). Psychology of the scientist: IV. Passages from the 'Idea Books' of Clark L. Hull. Perceptual and Motor Skills, 15, 807-882 (monograph supplement). Hull, C. L., & Baernstein, H. D. (1929). A mechanical parallel to the conditioned reflex. Science, 70, 14-25. Hull, C. L., Hovland, C. I., Ross, R. T., Hall, M., Perkins, D. T., & Fitch, F B. (1940). Mathematicodeductive theory of rote learning. New Haven, CT: Yale University Press. Humphreys, L. G. (1939a). Acquisition and extinction of verbal expectation in a situation analogous to conditioning. Journal of Experimental Psychology, 25, 294-301. Humphreys, L. G. (1939b). The effect of random alternation of reinforcement on the acquisition and extinction of conditioned eyelid reactions. Journal of Experimental Psychology, 25, 141-158. Koch, S. (1954). C. L. Hull. In: W. K. Estes et al. (Eds.), Modern learning theory (pp. 1-176). New York: Appleton-Century-Crofts. Kohler, W. (1939). Simple structural functions in the chimpanzee and in the chicken. In E. D. Ellis (Ed.), A source book of Gestalt psychology (pp. 217-227). New York: Harcourt, Brace. (Translation; originally published in German in 1918: Aus der Anthropoidenstation auf Teneriffa. IV. Nachweis einfacher Strukturenfunktionen beim Schimpansen and beim Haushuhn: Ober eine neue Methode zur Untersuchung des bunten Farbensystems. Abh. Preuss. Akad. Wiss., Berlin, pp. 1-101.) Krueger, R. G., & Hull, C. L. (1931). An electro-chemical parallel to the conditioned reflex. Journal of General Psychology, 5, 262-269. Logan, F. A. (1959). The Hull-Spence approach. In S. Koch (Ed.), Psychology: A study of a science (Vol. 2, pp. 293-358). New York: McGraw-Hill. Logan, F. A. (1960). Incentive: How the conditions of reinforcement affect the performance of rats. New Haven, CT: Yale University Press. Mackintosh, N. J. (1974). The psychology of animal learning. New York: Academic Press. Miller, N. E. (1935). A reply to "Sign-gestalt or conditioned reflex?" Psychological Review, 42, 280-292. Miller, N. E. (1944). Experimental studies of conflict. In J. M. Hunt (Ed.), Personality and the behavior disorders (pp. 431-465) New York: Ronald Press. Miller, N. E. (1959). Liberalization of basic S-R concepts: Extensions to conflict behavior, motivation, and social learning. In S. Koch (Ed.), Psychology: A study of a science (Vol. 2, pp. 196-292). New York: McGraw-Hill.

5

CLARK L. HULL'S BEHAVIORISM

157

Miller, N. E., & Dollard, J. (1941). Social learning and imitation. New Haven, CT: Yale University Press. Mowrer, 0. H. (1939). A stimulus-response analysis of anxiety and its role as a reinforcing agent. Psychological Review, 46, 553-566. Mowrer, 0. H. (1940). Experimental analogue to "regression" with incidental observation on "reaction formation." Journal of Abnormal and Social Psychology, 35, 56-87. Mowrer, 0. H. (1953). Motivation and neurosis. Current Theory and Research in Motivation (pp. 162-185). Lincoln: University of Nebraska Press. Mowrer, 0. H. (1960a). Learning theory and behavior. New York: John Wiley & Sons. Mowrer, 0. H. (1960b). Learning theory and the symbolic processes. New York: John Wiley & Sons. Osgood, C. E. (1953). Method and theory in experimental psychology. New York: Oxford University Press. Pavlov, I. P. (1927). Conditioned reflexes: An investigation of the physiological activity of the cerebral cortex. (G. V. Anrep, Trans.). Oxford, England: Oxford University Press. Perin, C. T. (1942). Behavior potentiality as a joint function of the amount of training and the degree of hunger at the time of extinction. Journal of Experimental Psychology, 30, 93-113. Rashotte, M. E., & Amsel, A. (1968). Transfer of slow-response rituals to the extinction of a continuously rewarded response. Journal of Comparative and Physiological Psychology, 66, 432-443. Razran, G. (1971). Mind in evolution: An east-west synthesis of learned behavior and cognition. Boston: Houghton-Mifflin. Rudenko, L. P. (1974). On the functional structure of conditioned reflexes to serial stimuli in dogs. Acta Neurobiologica Experimentalis, 34, 69-79. Skinner, B. F. (1950). Are theories of learning necessary? Psychological Review, 57, 193-216. Skinner, B. F. (1977). Herrnstein and the evolution of behaviorism. American Psychologist, 32, 1006-1012. Spence, K. W. (1936). The nature of discrimination learning in animals. Psychological Review, 43, 427 -449. Spence, K. W. (1937). The differential response in animals to stimuli varying within a single dimension. Psychologial Review, 44, 430-444. Spence, K. W. (1938). Gradual versus sudden solution of discrimination problems by chimpanzees. Journal of Comparative Psychology, 25, 213 -224. Spence, K. W. (1940). Continuous versus non-continuous interpretations of discrimination learning. Psychological Review, 47, 271 -288. Spence, K. W. (1942). The basis of solution by chimpanzees of the intermediate size problem. Journal of Experimental Psychology, 31, 257 - 271. Spence, K. W. (1948). The postulates and methods of behaviorism. Psychological Review, 55, 67 -78. Spence, K. W. (1952a). Clark Leonard Hull, 1884-1952. American Journal of Psychology, 65, 639 - 646. Spence, K. W. (1952b). Mathematical formulations of learning phenomena. Psychological Review, 59, 152 -160. Spence, K. W. (1952c). The nature of response in discrimination learning. Psychological Review, 59, 89-93. Spence, K. W. (1956). Behavior theory and conditioning. New Haven, CT: Yale University Press. Spence, K. W. (1957). The empirical basis and theoretical structure of psychology, Philosophy of Science, 24, 97 -108. Spence, K. W. (1960a). Conceptual models of spatial and non-spatial selective learning. In: K. W. Spence, Behavior theory and learning: Selected papers (pp. 366-392). Englewood Cliffs, NJ: Prentice-Hall. Spence, K. W. (1960b). The roles of reinforcement and non-reinforcement in simple learning. In K. W. Spence, Behavior theory and learning: Selected papers (pp. 91-112). Englewood Cliffs, NJ: Prentice-Hall.

158

RASHOTTE AND AMSEL

Taylor, J. G. (1962). The behavioral basis of perception. New Haven, CT: Yale University Press. Terris, W., German, D., & Enzie, R. (1969). Transitional resistance to the effects of aversive stimulation. Journal of Comparative and Physiological Psychology, 67, 264-268. Thorndike, E. L. (1898). Animal intelligence: An experimental study of the associative processes in animals. Psychological Monographs, 2, no. 4. Thorndike, E. L. (1911). Animal intelligence: Experimental studies. New York: Macmillan. Tolman, E. C. (1932). Purposive behavior in animals and men. Berkeley and Los Angeles: University of California Press. Tolman, E. C. (1933). Sign-gestalt or conditioned reflex? Psychological Review, 40, 246-255. Tolman, E. C. (1938). The determiners of behavior at a choice point. Psychological Review, 45, 1 – 41. Tolman, E. C., & Honzik, C. H. (1930). Introduction and removal of reward and maze performance in rats. University of California Publications in Psychology, 4, 257-275. Wagner, A. R. (1966). Frustration and punishment. In R. N. Haber (Ed.), Current research in motivation (pp. 229-239). New York: Holt, Rinehart & Winston. Wickens, D. D., Born, D. G., & Wickens, C. D. (1963). Response strength to a compound conditioned stimulus as a function of the element interstimulus interval. Journal of Comparative and Physiological Psychology, 56, 727 – 731. Wolpe, J. (1958). Psychotherapy by reciprocal inhibition. Stanford, CA: Stanford University Press. Zeaman, D. (1949). Response latency as a function of the amount of reinforcement. Journal of Experimental Psychology, 39, 446-483.