Understanding strategy selection

Int. J. Human-Computer Studies (2001) 54, 137}154 doi:10.1006/ijhc.2000.0434 Available online at http://www.idealibrary.com on

Understanding strategy selection MAXWELL J. ROBERTS AND ELIZABETH J. NEWTON Department of Psychology, University of Essex, Wivenhoe Park, Colchester, Essex, CO4 3SQ, UK. email: [email protected]. (Received 11 August 2000 and accepted in revised form 22 August 2000) This paper explores several issues associated with explanations of why di!erent people use di!erent strategies for learning and inference tasks. It is suggested that although the concept of cognitive style is a useful starting point, it is unable to account for many "ndings in the literature, and that any model of strategy usage that con"nes itself to mechanisms governing strategy selection is incomplete. In addition, it is necessary to take account of strategy availability: Which strategies do people possess, and how do people discover new strategies? Several "ndings in the literature indicate that strategy discovery is related to general abilities. Speci"cally, those who are best able to execute a current strategy are those who are the most likely to identify new, more e!ective methods. It is suggested that many "ndings that support the notion of cognitive style can be reinterpreted in this light.  2001 Academic Press KEYWORDS: reasoning; problem solving; individual di!erences; abilities; strategies; cognitive styles

1. Understanding strategy selection Whenever one observes people making inferences and solving problems, the contrast between what occurs in the real world and what is claimed to take place in the psychology laboratory is striking. In the real world, the sheer variety of strategies that are often applied, even to a simple task, is a testament to human thought and imagination. However, in the laboratory, this creativity is usually sti#ed or ignored. Diversity of mental process is upsetting to most cognitive psychologists, who typically work by proposing theories in the form of hypothesized sets of mental processes, which are intended to apply to every single person without exception. This makes the task of a researcher a simple one; either to show that a theory is an adequate account for most people or, if there is competition, to show that the theory is better than the alternatives. Often, this is expressed as a dichotomy: either theory X accounts for all people or theory Y, but not both. One example of a "eld where this has taken place is deduction, where there has been considerable squabbling between rival camps. Some researchers assert that deduction rules underlie all such reasoning. These are analogous to simple propositional, or logical rules (e.g. Rips, 1994). Others assert that mental models underlie all such reasoning. These are analogous to imagined diagrams of states of a!airs (e.g. Johnson-Laird & Byrne, 1991). However, the problem with taking either extreme stance is how to encompass those individuals whose data are out of step with others, and who may even match the 1071-5819/01/010137#18 $35.00/0

 2001 Academic Press

138

M. J. ROBERTS AND E. J. NEWTON

predicted patterns of data for a rival theory. Usually, such people are dismissed as experimental error. Hence, it is asserted that even these individuals think in the speci"ed way, and without the error, their data would be in line with the majority. However, Roberts (1993, 2000) has argued that positive evidence is required before mis"ts can be dismissed in this way. Without this, it simply does not make sense to claim that all people are identical. In addition, Roberts has argued that for any cognitive domain, the presence of reliably identi"able individual di!erences in strategy usage renders any proposed theoretical dichotomy irrevocably false. If a single, exclusive, universal theory has been posited, but people whose data do not "t the theory for reasons other than experimental error are ignored, then the theory is wrong. At best it only accurately accounts for the behaviour of the majority, and is wrong for the minority. At worst a monstrous hybrid model has been created which accounts for the behaviour of no one individual. Overall, Roberts (1993, 2000) concluded that for any task involving reasoning, problem solving or judgement, where individual di!erences in strategy usage are almost inevitable, research which assumes that all people use identical methods must inevitably fail. Instead, e!ort should be directed towards understanding the individual di!erences themselves. Many others have also complained about the lack of interest in these issues (e.g. Newell, 1973; Smith, 1989). Unfortunately, a change in emphasis towards the need to understand how people di!er, as well as how they are the same, requires a paradigm shift in mainstream cognition research, and this has slowed the process of accepting and understanding their importance. As just one example, an important UK textbook on cognitive psychology (Eysenck & Keane, 2000) does not have an index entry for individual di+erences despite entering its fourth version. However, although this research continues to be a minority pursuit, many "ndings have accumulated and clear patterns are beginning to emerge. The purpose of this paper is to provide a selective overview of some of the most important "ndings in this area, and attempt to show how they can be related so that a framework for understanding individual di!erences in strategy usage can emerge. Space considerations mean that studies into the cognitive development of children*which can be conceptualized as an attempt to understand di!erences in strategy usage between di!erent age groups*must be omitted. Before beginning, it is necessary to de"ne strategy. Siegler and Jenkins (1989) capture the essentials. A strategy is &&any procedure that is nonobligatory and goal directed'' (p. 11), and thus can be seen as a set of cognitive processes which in theory could be modi"ed*for example, through discovery or instruction*or completely dispensed with. Hence, while the constituent processes of certain cognitive activities may be relatively immutable*for example low-level visual perception*the processes used for other activities may be modi"ed at short notice, and a set of self-contained modi"able cognitive processes would constitute a strategy.

1.1. EXAMPLES OF INDIVIDUAL DIFFERENCES IN STRATEGY USAGE

The examples in this section all demonstrate substantial individual di!erences in strategy usage, but leave open the question of why people di!er. The most dramatic demonstrations are probably those of Pask and colleagues (e.g. Pask & Scott, 1972; Pask, 1976, 1988; Scott, 1993). This work is related to studies on concept attainment by Bruner (1974), but the intention of Pask and colleagues was to investigate learning and

UNDERSTANDING STRATEGY SELECTION

139

understanding in a richer environment. One of the best-documented tasks requires people to learn an arti"cial taxonomy of imaginary animals. Two major categories of strategy were identi"ed: holist and serialist. Although the descriptions that follow represent extreme manifestations of each strategy, and elements of both are usually required for success, the learning of most people was found to be dominated by one or the other.- Holist strategy users preferred to seek information in order to obtain a global, overall picture of how the various elements of a domain related to each other. In order to do so, they would often attempt to advance their knowledge on several fronts simultaneously. Hence, even when learning speci"c rules for distinguishing animals, they would often attempt to test several at once. Occasionally, when taken to an extreme, this strategy would lead to globetrotting. Here, there was an appreciation of the need to understand the overall picture but, for example, there was a tendency to draw inappropriate analogies due to the failure to appreciate individual rules. Serialist strategy users by contrast preferred a step-by-step approach to seeking information, building an overall picture by mastering each individual rule before moving on to the next. This approach varied in systematicity from person to person, but could lead to improvidence, where the relationship between rules was not appreciated and the overall picture was not established. For example, analogies were not drawn in order to transfer knowledge and hence speed learning. Having found that the strategies were relatively stable and could be reliably identi"ed, a further study was conducted by Pask and Scott (1972) in which people were classi"ed by their preferred learning strategy. They were then presented with a programmed instruction scheme for learning the taxonomy that was either congruent or incongruent with this. Massive di!erences in learning were found irrespective of actual strategy classi"cation: those with a matched scheme learnt e!ectively and those with a mismatched scheme learnt poorly. Taking the studies as a whole, it was found that although a few people were versatile learners, i.e. were able to switch between strategies relatively e!ortlessly, most people had a clear dominant strategy that was consistent across di!erent tasks, and were remarkably reluctant to change strategy even when this was paired with an incongruent learning scheme. These "ndings have several important implications for psychologists. They are probably the most prototypical demonstration of the existence and importance of cognitive styles as determinants of strategy usage (see the next section). For cognitive psychologists, they demonstrate that individual di!erences in ways of thinking are not minor methodological irritations, but can have important consequences for performance. For applied psychologists, there is clearly a need at the very least to be aware of potential mismatches between learning strategy and instructional format, which need not necessarily be in the classroom. If people use di!erent strategies to learn and make inferences, then there are two possible courses of action. Either people should be matched with materials appropriate to their personal strategies, or people using inappropriate strategies should be taught to change. However, even 25 years on, large-scale attempts to match instructional material to preferred learning strategy are very rare, despite the clear bene"ts of a match and the dire consequences of a mismatch. -In Pask's terminology, people who show a strong tendency to use holist strategies are known as comprehension learners while people who show a strong tendency to use serialist strategies are known as operation learners. For simplicity, these people will be referred to as holists and serialists throughout this essay.

140

M. J. ROBERTS AND E. J. NEWTON

Individual di!erences in strategy usage also have important implications for psychologists' understanding of general abilities. This is because attempts to map and understand the structure of human ability are typically based upon the factor analysis of psychometric tests, and it is assumed that the overall patterns of factors and test score loadings apply to every individual (see Kline, 1991 for a review). Unfortunately, this assumption only holds if all individuals have used identical strategies to solve the test items, and this has often been found not to be the case. For example, psychometric tests of spatial ability frequently involve the presentation of two-dimensional patterns representing #attened shapes; usually cubes. The intention of the test constructor is that all people will attempt mentally to fold a pattern in order to form a three-dimensional mental image of a solid. Several studies have attempted to verify this assumption by identifying the processes that people use. Researchers typically utilize various combinations of concurrent and retrospective verbal reports, gaze-direction tracking, people's drawings and their speed and accuracy at di!erent types of item. The usual "nding is that these tests are not strategically pure (e.g. Barratt, 1953; French, 1965; Just & Carpenter, 1985). Instead of imagery, many people are found to use various analytic strategies. For example, it is possible to infer from a #attened, unfolded cube, which sides will be adjacent, and their orientation, without having mentally to fold it. Furthermore, such people often outperform those who use imagery strategies and, when a range of tests are presented and factor analysed, the two types of person typically have di!erent patterns of factor loadings. This "nding is not con"ned to spatial ability tests. For example, Hunt (1974) identi"ed and simulated two alternative strategies for solving Raven's progressive matrices*an important test of general intelligence. The perceptual strategy solved items by, for example, extending lines and making superimpositions. The analytic strategy solved items by inferring the underlying rules and applying them. In terms of performance, the analytic strategy was vastly superior. In the light of these "ndings, the meaning of psychometric test scores becomes ambiguous. For example, a person with a moderate spatial ability test score may be performing well with an ine!ective strategy, or performing poorly with an e!ective strategy. In general, performance will depend not just upon speed and precision of cognitive processes, but also upon appropriateness of strategy selection (see also Baron, 1978). Because of this, correlations between psychometric test scores and other tasks are not easy to interpret. For example, the best performers at a spatial ability test may not be using imagery. Hence, even where spatial ability is found to be highly correlated with performance at another task, it cannot be inferred from this that people solve this task by the use of imagery [see Roberts, Wood & Gilmore (1994), for further discussion]. However, as we shall see later, an analysis of why people use di!erent strategies will show that this problem is not as serious as it could be. Overall, these "ndings raise the question of whether current conceptualizations of human ability are accurate for all or even some people; an issue that test constructors have been very reluctant to embrace. Instead, researchers have preferred to produce tests of high strategic purity and subject these to factor analysis. Unfortunately, these tend to consist of relatively simple tasks, and Lohman and Kyllonen (1983) summarize the consequences of oversimpli"cation. In general, as a task is simpli"ed: (1) the diversity of strategies used for solving it decreases; (2) imagery strategies become more prevalent and analytic strategies become less prevalent; (3) the factorial purity of the task increases,

UNDERSTANDING STRATEGY SELECTION

141

i.e. fewer distinct ability components can be identi"ed as contributing to task performance; and (4) the task is less able to predict outcomes in various applied settings. Despite all e!orts, even current tests are not necessarily strategically pure, and Lohman and Kyllonen suggest that &&tests often measure di!erent abilities for di!erent students depending on how they have been solved'' (p. 121). These overall "ndings point towards possible reasons why researchers "nd weaker than expected links between ability and measures that are expected to be associated with ability. Hence, in the quest to devise pure models of ability, and in the belief that strategies form unwanted pollution that must be eliminated at all costs, researchers may have produced inappropriate models of the structure of ability, and inappropriate means to measure abilities. Individual di!erences in strategy usage have been found in numerous other studies, even using simple deduction tasks often assumed to be strategically pure. For example, Sternberg and Weil (1980) investigated linear syllogisms (e.g. if A is taller than B, B is taller than C and C is taller than D then who is tallest?). They identi"ed verbal, spatial, mixed and task-speci"c short-cut strategy users, as well as "nding that some people were unable to follow instructions to use speci"c strategies. Ford (1994) has identi"ed both verbal and spatial strategy users for categorical syllogisms (e.g. if all A are B and some B are C then what follows?) and Gilhooly, Logie, Wetherick and Wynn (1993) have identi"ed users of numerous di!erent task-speci"c short-cut strategies also for the same task. However, it would be premature to conclude from the studies described that all tasks in the domains of learning, reasoning and problem solving will be prone to individual di!erences in strategy usage. Instead, researchers in these "elds should be alert to this possibility. Only an analysis based upon each individual's patterns of performance could show that a given task is strategically pure; summaries of performance across people can conceal minority strategies with ease [for example, see Siegler (1996) for demonstrations in the domain of children's arithmetic].

1.2. COGNITIVE STYLE AS A PREDICTOR AND AN EXPLANATION OF STRATEGY USAGE

The studies described above demonstrate that individual di!erences in strategy usage can easily be observed given suitable conditions, and care and precision on the part of the researcher. Some immediate implications have also been discussed, and it is worth taking a step back at this point in order to consider wider theoretical questions. Speci"cally: How are strategies acquired? and How do people choose between them? Answers to these may point towards appropriate actions to take, in both theoretical and applied settings, when faced with individual di!erences. Di!erences in strategy usage are rarely arbitrary in terms of task performance, indeed inappropriate choices can be disastrous (Pask & Scott, 1972). Where people sabotage their chances of success in this way, "nding answers to these questions becomes all the more important. In a sense, the easiest way in which to answer these questions is not to attempt to answer them at all. Hence, it can be asserted that di!erent strategies are manifestations of di!erent cognitive styles [for a recent review, see Sternberg (1997) and also Schmeck (1983) and Riding, Glass & Douglas (1993)]. In other words, di!erent people prefer to seek information and/or to represent and process it in di!erent ways, and hence prefer to use di!erent strategies. Thus, presenting tasks in ways which are incongruent with people's preferences will result in degraded performance. The cognitive style explanation,

142

M. J. ROBERTS AND E. J. NEWTON

if valid, is one that is important not just for learning and instruction, but also outside of the classroom too. For example, when teaching computing methods, the "rst author has encountered many people who have objected strongly to graphical user interfaces, preferring the command line, sometimes apparently thriving on this. Up till now, people could choose between di!erent styles of interface, but it is not clear whether this will remain possible even in the foreseeable future. There is therefore the risk that by ignoring individual di!erences, the quest for making computers easier to use has made them considerably harder to use for some [see also Sein & Bostrom (1989) for empirical research into this topic]. Framing strategy usage in terms of stylistic preference, and coupling this with the lack of #exibility that has been shown in several studies, inevitably leads to the question of whether some styles are inherently better than others. This is a question that advocates of cognitive style usually decline to answer, instead asserting that although there might be individual di!erences in e$ciency when particular tasks are investigated, when the whole range of tasks that people face is considered, di!erent tendencies are likely to be equally e!ective. While commendably egalitarian, this assertion should be treated with caution as it is di$cult to evaluate empirically. In addition, the concept of style is very di$cult to separate from the concept of ability (e.g. Entwistle, 1979; Schmeck, 1983). However, the possibility that everyone has a potential niche in which they can #ourish is attractive, and so the concept of cognitive style deserves be taken seriously. One practical problem with a cognitive style account of strategy usage is the sheer number of dichotomies posited in the literature. This has grown steadily over the years, and there has been little attempt by researchers to integrate the various constructs. The problem is therefore one of determining: (1) which styles are genuine, rather than being manifestations of, for example, di!erent levels of verbal ability, spatial ability, or intelligence; (2) which styles are independent, as opposed to being similar to other styles identi"ed and named by rival research groups; and (3) which styles are important for the domain of interest. Thus, a researcher may have to decide whether people are ,eld dependent or ,eld independent [see Linn & Kyllonen (1981), for a sceptical review of this style], assimilators or explorers (Goldsmith, 1986), adaptors or innovators (Kirton, 1976), visualizers or verbalizers (e.g. Riding et al., 1993), holists or serialists (cf. Pask & Scott, 1972), convergent or divergent thinkers (e.g. Hudson, 1966) and should also consult the literature on the relationship between various personality variables, such extraversion and neuroticism, and problem solving (e.g. Weinman, 1987). This list is by no means comprehensive, and even if just a few style dimensions are valid, this will cause impracticalities. For example, just four valid dichotomies will result in 16 di!erent combinations of styles, leading to a requirement for considerable numbers of subjects, and resulting in inevitable higher-order interactions and empty cells, hence making research di$cult to conduct and interpret in any setting. Another problem with a cognitive style account of strategy usage concerns the studies that have shown that instructing people to use non-preferred strategies can be di$cult. While this could be taken to re#ect stylistic avoidances, a closer look at people's behaviour hints at a more complex picture than a cognitive style account can capture. One example is discussed by Siegler (1996, p. 153) in which children were successfully taught a rehearsal strategy which improved their memory. However, very few children continued to use this strategy when given the option not to. Crucially, many

UNDERSTANDING STRATEGY SELECTION

143

non-adopters attributed their increase in success, not to their utilization of more e!ective procedures, but to the greater e!ort required in executing them. Another example is from Roberts (1991), where people were instructed to use a highly e$cient strategy for solving a series of compass point directions problems (see later for a full description of this task). Several people commented before commencement that they did not like to think in the way required by the imposed strategy and expressed misgivings about using it. However, Newton and Roberts (2000) found that after some practice with the e$cient strategy, subjects' subsequent selection of it in preference to alternatives was almost unanimous. In all of these examples, people appeared to be choosing between their own natural strategy*with which they were familiar and reasonably competent*and a new strategy*with which they were unfamiliar and unpractised (cf. Sternberg & Weil, 1980). Even though the imposed strategy had a competitive edge in these circumstances, a reluctance to use it could re#ect a lack of con"dence in its value, rather than the need to avoid an alien style of thinking. Paradoxically, the simplicity of the cognitive style account of strategy usage leads to its "nal downfall. It is untrue to assert that this o!ers an explanation of behaviour, instead it simply uses jargon to redescribe behaviour. This is emphasized when considering the connotations of the word style, which implies both choice and -exibility. Choice is important because it is scarcely reasonable to talk about preference unless there are two or more options for each individual to choose between. Flexibility is implied because if people's choices re#ect mere preferences, then when a preference becomes particularly disadvantageous, it would be reasonable to expect a non-preferred, but locally favourable option, to be chosen on a temporary basis. The fact that #exibility is often not observed implies two possibilities. On the one hand, people may have choice but lack #exibility. If this is the case then cognitive property would be a more appropriate phrase than cognitive style, and researchers must ask why some people can be locked in ways of thinking that can be disadvantageous to them. On the other hand, people may have #exibility but lack choice, in which case researchers must ask why not all strategies are available to all people. These questions suggest that the concept of cognitive style should be regarded as a springboard from which to begin the understanding of individual di!erences in strategy usage, rather than the "nal answer itself. In the next section, work will be discussed which is relevant to this issue.

1.3. STUDIES THAT SUGGEST REASONS FOR INDIVIDUAL DIFFERENCES IN STRATEGY USAGE

One of the most well-known pieces of work that comes under this heading is that of MacLeod, Hunt and Mathews (1978), in which solution strategies for a sentence}picture veri,cation task were investigated. For this, people are given a series of trials in which simple sentences describe two objects (e.g. the cross is above the star, the star is not below the cross). Each sentence is then followed by a picture of a cross and a star. The task is to determine whether each picture is a true depiction of its sentence. Previous researchers attempted to show that the solution processes used were identical for all people and were based upon verbal propositions. MacLeod, et al., investigated individuals' patterns of mean decision times for various types of trial, and instead identi"ed two strategies for solving them. For the verbal strategy, the sentence is encoded verbally,

144

M. J. ROBERTS AND E. J. NEWTON

and the picture is converted into a proposition (e.g. &&cross above star'') and the two verbal representations are compared. For the spatial strategy, the sentence is converted into a spatial representation*usually a mental image*and directly compared with the picture on the display. The main "nding was that people who adopted the spatial strategy showed a strong tendency to have high spatial ability, while people who adopted the verbal strategy showed a strong tendency to have low spatial ability. Verbal ability was equally high for both strategy groups, who were university students. In a follow-up study, Mathews, Hunt and MacLeod (1980) found that it was possible to instruct people to adopt alternative strategies. However, their actual data imply that there was considerable reluctance to do so by many. Overall, "nding that people choose strategies appropriate to their abilities is very important for adding substance to the visualizer}verbalizer cognitive style, in which people are said to prefer either spatial or verbal representations of information. Speci"cally, although the verbal and spatial strategies above are similar in terms of numbers of processing steps, high spatial ability gives the spatial strategy a slight edge while low ability makes this less attractive. Hence, either people may have selected on the basis of a cost}bene"t analysis, comparing the two strategies for e!ectiveness, and thus making best use of resources. Alternatively, because of their ability-based preferences, people with high spatial ability may have assembled a repertoire of spatial strategies over the years that enabled them to make full use of their superior ability, while people with low ability may have assembled a repertoire of non-spatial strategies that enabled them to avoid their de"ciency.- Hence, the decision was made on the basis of strategy availability as determined by ability and past experience, thus making use of best resources. Although it is not possible to distinguish between these explanations from the results of MacLeod, et al. (1978), both accounts provide a sound grounding for cognitive style. Hence, rather than re#ecting arbitrary prejudices, styles are based upon the desire of people to maximize their performance given the constraints of their ability.? On balance, the studies discussed earlier, showing that people can be reluctant to use a non-preferred strategy even when this would be advantageous, suggest that the cost}bene"t analysis account is the less likely of the two. Despite the elegance of these "ndings, and their many implications for both theoretical and applied psychology, they have never really been followed up with any enthusiasm. Instead, later work on sentence}picture veri"cation has criticized the realiability and validity of the strategy classi"cation systems used previously (e.g. Marqeur & Pereira, 1990; Roberts et al., 1994). A di!erent approach to understanding strategy usage has been taken by Lohman, Kyllonen and colleagues (e.g. Kyllonen, Lohman & Snow, 1983; Lohman & Kyllonen, -A frequent error when interpreting "ndings in this "eld is to assume that, in terms of ability as opposed to style, people are either high in verbal or high in spatial ability. In fact, all patterns of abilities can be observed, but given that verbal and spatial ability tests tend to be correlated, the two most common patterns of extremes are either high in both verbal and spatial ability, or low in both verbal and spatial ability. ?It could be argued that &&explaining'' strategy usage on the basis of ability simply moves the unexplained component elsewhere*why do people di+er in ability?*exactly the same problem as was identi"ed for cognitive style accounts. However, it can be argued that the understanding of the origins of individual di!erences in cognitive styles is a less tractable problem than understanding individual di!erences in general abilities and their development, whether these are ultimately found to be due to genetic factors, the environment or both.

UNDERSTANDING STRATEGY SELECTION

145

1983; Kyllonen, Lohman & Woltz, 1984). Having identi"ed numerous instances of strategic impurities in the solution of spatial ability tests, they attempted to understand the exact relationship between spatial ability, strategy usage and task performance. Several tasks were investigated, which typically involved various operations such as folding, rotation and matching. Overall, it was found that people tended to vary in strategy from trial to trial, so that it was rarely possible to categorize a person as preferring a single strategy. However, it was also found that the best performers were more #exible in their strategy usage, and were not necessarily performing best because they were better able to form rich, detailed spatial representations. Instead, as problem di$culty increased, the better performers were more likely to use simpli"cation strategies which reduced the demands of representing the task components. Similar "ndings are also described by Cooper and Mumaw (1985) in which high spatial ability people were less likely than low spatials to use an imagery strategy for a complex visual matching task, and more likely to use an analytic scanning strategy which was much less demanding in terms of spatial ability. As the "ndings stand, they sit awkwardly with those of MacLeod, et al. (1978) particularly if a cognitive style interpretation is to be applied. For MacLeod, et al., high spatial people solved the problems spatially, while low spatials avoided this. For the Lohman and Kyllonen studies, high spatials tended to use strategies that reduced the visual component of the harder problems, simplifying them and making them less demanding, while low spatials were considerably less likely to do so. A visualizer}verbalizer dimension should predict the reverse. Although an additional style could be posited to account for this, the ultimate outcome of adding styles to de#ect anomalous results is problems of unfalsi"ability and intractability. Hence, there would be no obvious basis for specifying in advance which style(s) would be expected to predict strategy usage for any given task. Given the di$culty in reconciling the above results within a single style dimension, we therefore need to consider non-stylistic reasons for the relationship between ability and strategy usage observed in the Lohman and Kyllonen studies. Perhaps the best performers had a superior ability to represent information spatially, and this enabled them to discover simpli"cation strategies that were particularly e!ective for performing the tasks, and these replaced the imagery strategies at which these people were also superior. Alternatively, perhaps the best performers were those who, due to more past experience in spatial domains, had acquired many strategies, some of which were particularly useful for performing the investigated tasks. In other words, did high spatial ability result in a larger strategy repertoire*an ability-based account*or did a larger strategy repertoire result in high spatial ability*a knowledge-based account? The knowledgebased explanation would be compatible with the view by some psychologists that all observed di!erences in performance are entirely due to di!erences in motivation, practice and domain-speci"c knowledge. Di!erences in general ability, i.e. the global e!ectiveness with which the cognitive architecture operates, are asserted to be irrelevant to task performance (e.g. Simon, 1990; Ericsson & Charness, 1994). However, knowledge-based explanations of performance and strategy usage are not without problems, and the importance of general abilities cannot be dismissed on the basis of the "ndings so far (e.g. Roberts & Stevenson, 1996; Roberts, Gilmore & Wood, 1997; Newton & Roberts, 2000).

146

M. J. ROBERTS AND E. J. NEWTON

One important pointer towards the best explanation for the "ndings of the Lohman and Kyllonen studies comes from work by Siegler and colleagues. This has been concerned with the events which trigger strategy discovery and strategy change in many domains, such as children performing arithmetic (e.g. Siegler, 1992, 1995, 1996; Siegler & Jenkins, 1989). One important aspect of the methodology used is the intensive observation of subjects individually over an extended period of time, using trial-by-trial analysis (this is termed the microgenetic method, see Siegler & Crowley, 1991). This allows for the study of the underlying processes which accompany the discovery of a new strategy, rather than simply observing the start and end points of the process. Using this methodology in order to observe 5-year olds performing addition, Siegler and colleagues have found that even for these relatively simple tasks, a wide range of strategies can be employed. It was also found that at the level of the individual, development does not resemble a staged progression in strategy complexity. Instead, newly discovered and existing strategies intermingle, and the more e!ective strategies usually increase in frequency only gradually. Hence, gross changes between stages of cognitive development should be understood in terms of the accumulation and modi"cation of individual strategies with experience. In addition, a further suggestion is that strategy discovery processes are not con"ned to any particular age group, and apply equally to adults and children. In other words, the discovery and selection of any strategy, irrespective of age, domain or complexity, may be understood in terms of the same basic mechanisms. Siegler's work was in part motivated by Van Lehn's (1988) assertion that &&learning occurs only when an impasse occurs. If there is no impasse, there is no learning'' (pp. 31}32). In this context, an impasse is de"ned as a point at which current existing strategies are no longer able to play a role in solving the problem. Hence, if impasses drive strategy discovery, it would be expected that, for example, a series of incorrect answers would be followed by a change in strategy, or that a new strategy would be observed for the "rst time immediately after a particularly di$cult problem. Conversely, an adequately functioning strategy would not be expected to be replaced by a new one at any point. Of particular interest to Siegler and colleagues was the events leading to the discovery of the highly e!ective min strategy for addition, in which a sum is calculated by counting up from the larger addend. For example, 2 and 7 are added together by starting from 7 and counting two upwards: &&7, 8, 9''. It was found that the discovery of this was accompanied by an increase in the time taken to solve problems. However, although this would be compatible with an impasse triggering an attempt to change strategy, increased errors were not observed on problems prior to strategy discovery, and there was no other evidence that these problems were more di$cult than normal. Hence, it was shown that new strategies can be generated despite existing ones performing adequately. It is also possible to devise particularly di$cult impasse problems for this task, which can only be solved e!ectively by using the min strategy. For example, if this were not used, 24#2 would trigger an impasse. When children were presented with these, the min strategy was only used if it had already been discovered during the solution of the easier problems. Where this was the case, its frequency increased dramatically for all problems after the impasse problems. Overall, impasses were not found to be related to strategy discovery: although they may motivate a change in strategy, this can only take place if the replacement has already been discovered. Hence, while it could be argued that impasses

UNDERSTANDING STRATEGY SELECTION

147

are necessary to drive learning for the case of arti"cial intelligence, this is almost certainly not true for human cognition. If we accept that the longer solution times just prior to the discovery of a new strategy re#ect an increase in cognitive activity, but that this is not impasse related, then an alternative activity must be posited. One possibility is that some form of evaluation process is taking place, for example, by comparing a newly discovered strategy to an existing one, both for e!ectiveness at solving the problem and for ease of use (see also Roberts et al., 1997). At "rst sight, there should be no need to evaluate the validity of a newly discovered strategy, as indicated by Siegler and Jenkins' (1989) "nding that, provided that children understood a task, illegal strategies were not discovered. From this, they suggested that the discovery of a new strategy is constrained by a goal sketch. This is a knowledge structure which incorporates all of the subgoals necessary for a strategy to be legitimate, and therefore is able to keep candidate strategies on track (but see also Newton & Roberts, 2000). Evidence for this also comes from Siegler and Crowley (1994), who found that children were able to evaluate strategies which were conceptually too advanced for them to use themselves, implying that they were able to determine the e!ectiveness with which a strategy may obtain its goals in isolation from their ability to discover or execute the strategy. Overall, Siegler and colleagues have shown that strategies develop during problem solving. They are not static pieces of knowledge, and instead are dynamic and ever changing. Although impasses may motivate change, they are not a direct part of the strategy discovery process. Instead, the discovery of new strategies can result from an individual's observations about a task; noticing interesting relationships which can be capitalized on in the future in order to economize on cognitive e!ort. This is likely to be motivated by a desire for cognitive e$ciency and elegance. More speci"cally, one possibility is that the procedure by which new strategies are discovered is based upon the identi"cation and deletion of redundant steps as a result of experience. However, although ruling out impasses, the exact mechanisms that trigger strategy discovery have not been established (see Crowley, Shrager & Siegler, 1997 for suggestions), nor have individual di!erences in strategy discovery been fully explained. In retrospect, impasses are highly unlikely to drive strategy discovery. If all learning were impasse driven, this would imply that individual di!erences in performance at all tasks should disappear with time. The learning of good performers, who would experience few impasses, would proceed slowly, while the learning of poor performers, who would experience many impasses, would be relatively rapid. This is the opposite of what is normally observed, and as Roberts et al. (1997) suggest, those who are most likely to experience impasses are those who are least likely to be equipped to discover new strategies in order to overcome them. Evidence for this comes from several studies in which it has been found that the people who are the most likely to discover new, more e$cient strategies for solving a task are those who execute the old, less e$cient strategies the most e!ectively. According to an impasse-based theory, the best performers should be least likely to discover new strategies. For example, Wood (1969, 1978) found that the people who discovered short-cut strategies for solving linear syllogisms were those who were initially best at solving them by using imagery, while Galotti, Baron and Sabini (1986) found that good reasoners at categorical syllogisms were more likely to discover short-cuts than bad reasoners.

148

M. J. ROBERTS AND E. J. NEWTON

One piece of work where these ideas were developed is Roberts et al. (1997). They investigated a compass point directions task in which people were presented with several trials, each consisting of a set of compass point directions presented simultaneously (e.g. one step east, one step north, one step north, one step west, one step south). The task was to deduce the "nal compass point of a person relative to the starting point after taking them. There are two strategies for solving these problems. For the spatial strategy, the full path must be traced, either with a "nger or mentally; subjects are not permitted to use pencil and paper for the standard task. This strategy is slow, inaccurate and frequently stressful to execute. The cancellation strategy is relatively non-spatial in its execution; opposites are simply cancelled, with those that remain constituting the correct answer. This strategy is considerably faster, more accurate and less stressful to execute for the vast majority of people, irrespective of their abilities. Roberts, et al., intended to test to destruction the concept of the visualizer}verbalizer cognitive style, as implied by MacLeod et al. (1978). If people with high spatial ability continued to use the ine$cient spatial strategy, missing the bene"ts of cancellation, while low spatials used cancellation, thus avoiding using ability that they did not possess, then this would be a very powerful demonstration of the existence and importance of this cognitive style. Unfortunately, the reverse was found: people with high spatial ability showed a strong tendency to use cancellation, while people with low spatial ability showed a strong tendency to use the spatial strategy. In addition, a further task was presented in which compass point directions for two people were given, and the task was to decide where one would end up relative to the other. The intention was to devise a directions tasks in which cancellation was unlikely to be adopted. This was successful, and here it was found that the high spatials outperformed the low spatials. Thus, evidence was obtained to suggest that, for compass point directions tasks, high spatials are better than low spatials at executing spatial strategies. At "rst sight, the overall "ndings appear to be paradoxical, but this is because strategy selection accounts typically neglect issues surrounding strategy discovery and strategy possession: people cannot choose between strategies if they are not aware of alternatives. It appears that the spatial strategy is the natural strategy for this task, in the sense that all people are aware of its validity. Cancellation, where used, is discovered during experience with the task. In order to do so, and add this to the strategy repertoire, it is necessary to be able to execute the spatial strategy accurately. Hence, high spatial ability people are better able to discover that opposite steps cancel no matter how many steps intervene, and the entire process of constructing a spatial representation is redundant. People with low ability are e!ectively penalized twice: they are less able to change to cancellation precisely because they are less able to perform the spatial strategy. It is also likely that for those who discover cancellation but are unsure of its validity, it is necessary to be able to execute the spatial strategy accurately in order to evaluate cancellation: evaluation can only be achieved for these people by comparing the answers of the two strategies. Because the spatial strategy is known to be valid, a mismatch in answers*likely if the spatial strategy is executed poorly*will lead to the rejection of cancellation. A further study ruled out the knowledge-based account of strategy usage discussed earlier: allowing people to use pencil and paper vastly reduced the incidence of cancellation, even for high spatials, resulting in considerably reduced performance for them. It was therefore argued that if people had brought the cancellation strategy to the task, instead of

UNDERSTANDING STRATEGY SELECTION

149

discovering it during the task, then pencil and paper should not have prevented this strategy from being used (see also Newton & Roberts, 2000).

1.4. RECURRING FINDINGS AND TENTATIVE CONCLUSIONS

The "rst conclusion to be drawn from these studies is that accounts of strategy selection, whether ability or stylistically based, are incomplete without accounts of strategy possession: strategies can only be chosen from those that are available. For example, the results of Roberts et al. (1997) are not explicable by a visualizer}verbalizer style dimension, but are explicable in terms of the relationship between ability and strategy discovery. In order to predict and understand strategy selection, it is therefore necessary to know not just a person's current strategy repertoire, but also which strategies are likely to be added as a result of experience with a task. In the above studies, people who represented information more e!ectively were better able to discover new, more e!ective strategies. People who were less able to represent information were, somewhat ironically, less able to discover the strategies that would have enabled them to improve their poor performance. Clearly, this situation is far more complicated than could be captured by a set of cognitive style dichotomies. A person who prefers to avoid spatial thinking, perhaps due to low spatial ability, will not be able to express this preference unless aware of non-spatial strategies from which to choose. Even for high spatials, a cognitive style account is not enough: however spatial in their thinking, these people are likely to simplify or quickly dispense with spatial strategies when opportunities to use more e!ective methods present themselves. Although it might be possible in some circumstances to observe people performing in line with hypothesized cognitive styles, these situations are likely to be special cases. One example of this would be where most people know a strategy appropriate to their abilities, and using the relevant strategy will result in the best possible performance for each person. Even here, the relative lack of explanatory power of cognitive style accounts limits their interest. Elsewhere, their simplicity causes them considerable di$culty, particularly where there have been genuine attempts to falsify their predictions. Although questioning the validity of one widely accepted style*the visualizer}verbalizer distinction*does not have direct implications for the others, it is reasonable to question whether the others would withstand the same degree of scrutiny. However, if the importance of cognitive style is to be downplayed, it is necessary to o!er an alternative account of situations where inappropriate strategy usage can be linked to stylistic dimensions. For example, why did the serialists and holists observed by Pask and Scott (1972) perform so poorly when matched with incongruent instruction programmes? If cognitive styles represent mere preferences, then why do people prefer to sabotage their own performance? These types of results are puzzling: on the one hand, they are the best evidence of strong stylistic preferences, but on the other, they suggest that style and preference are descriptives that are too weak. This leads us to the next conclusion, which concerns how people discover new strategies. These are not static components stored in mental "ling cabinets to be chosen when a suitable situation arises. Although knowing the best strategy for a given situation is obviously advantageous, where this is not the case, it is clear that some people are better able to discover more e!ective strategies. As Siegler and colleagues have shown,

150

M. J. ROBERTS AND E. J. NEWTON

impasses are unlikely to be an integral part of strategy discovery. Other research supports this, and points towards the opposite conclusion by showing that those people who are best equipped to use a strategy are often those who are most likely to dispense with it and turn to other, more e!ective methods. Hence, strategy discovery depends upon what a person can learn from a task while performing it, and the more e!ectively a task is performed, the more that can be learnt. Hence, the most successful people learn the most. That people learn from success is highly relevant when returning to the results observed by Pask and Scott (1972): neither the holists nor the serialists were able to adopt a more appropriate strategy when given a task incongruent with their dominant learning strategy. While this might re#ect persistent cognitive styles, there is an alternative, speculative, explanation that is worth considering.- New learning strategies are probably di$cult to discover and, for whatever reason, the vast majority of people are unlikely to be in possession of more than one fully practised learning strategy, whether this has developed in tandem with abilities, or has been institutionalized (see Pask, 1988). When there is a mismatch between information presentation and possessed learning strategy, most people fail because they do not have the (probably unusually high) level of ability necessary in order to execute their possessed strategy e!ectively. As we have seen, success is more conducive to discovering new strategies than failure. Hence, in the Pask and Scott studies, mismatched people performed poorly and remained with an inappropriate strategy, not because they were locked in their style out of preference, but perhaps because poor performance prevented their discovering more e+ective learning strategies. In general, the various results discussed above strongly suggest that an adherence to inappropriate strategies in the face of adversity may re#ect an inability to discover alternatives rather than a desire to think in a certain way. Finally, it should be noted that most of the studies discussed above investigated spatial tasks, but it is reasonable to speculate that for other tasks, high verbal ability or high mathematical ability, as appropriate, may be important for discovering new strategies. In addition, many studies found that high spatial ability people tend to use non-spatial strategies. However, one should not conclude from this that people with high spatial ability never use spatial representations, only that under certain circumstances they may be better equipped to dispense with them if this is to their advantage. At other times, it may be more advantageous for them to continue to use spatial representations. One immediate consequence of this is that, while people should be matched with instruction and training material that is the most compatible with their abilities, there is no need to teach people to use their abilities in rigid ways. People high in spatial ability should not be taught to solve problems in spatial ways. Indeed, attempts to train high ability people have often been found to be counter-productive in the past (see Snow, 1989). Likewise, people low in spatial ability should not be taught to solve problems verbally. Instead, the overall results present the instructor with a dilemma. Instruction in every e!ective strategy would not be practical, and instruction in how to discover more e!ective methods will not succeed unless it is possible to teach people how to represent -Pask occasionally distanced himself from the concept of cognitive style when discussing his "ndings, insisting that the learning methods are strategies for learning rather than "xed styles of learning, and pointing out that with appropriate instruction, people could be taught to use either strategy (e.g. Pask, 1976, p. 133).

UNDERSTANDING STRATEGY SELECTION

151

information more e!ectively. The challenge is to teach people low in ability to overcome any barriers that this may place on the strategy discovery process.

1.5. TOWARDS A FRAMEWORK FOR UNDERSTANDING INDIVIDUAL DIFFERENCES IN STRATEGY USAGE

The conclusions described above are a useful starting point for understanding strategy usage, but one sticking point in comparing and planning studies is that there are di!erent aspects that can be investigated, and papers rarely make explicit which are being addressed. To end the discussion, the three main questions that must be answered in order to understand strategy usage are made explicit below. Clearly, more work is required in order to answer all of them, but equally clearly, the questions are tractable and research to date points towards some very interesting answers. (1) =here do strategies come from? Although apparently a question for developmental psychologists, adults discover strategies too. Only by knowing what strategies people possess or are able to discover, as constrained by their abilities, can we then go on to understand how people decide which strategies to use. Work by Wood (1978), Lohman and Kyllonen (1983), Siegler (1996), and Roberts et al. (1997) all suggest answers to this question, but this aspect of strategy usage is probably the least understood of all. (2) How do we know which strategies are valid? This question is probably the least important, although still interesting. It is in part derived from observations made by Roberts et al. (1997), where people frequently reported that they had thought about cancellation, but had rejected it as a strategy that would not yield a su$ciently accurate answer under any circumstances when compared with the spatial strategy. Siegler (e.g. 1996) suggests that strategy discovery is constrained by the goal sketch with the result that pathological strategies are unlikely to develop from valid strategies, but even if this is true, as Roberts, et al., found, people may not realize that this is the case (see also Newton & Roberts, 2000). (3) How do we choose between strategies? This question has received the most attention and is linked to the literature on metacognition (e.g. Brown, 1987; Schoenfeld, 1987). However, answers to this question alone cannot provide a complete account of strategy usage. A cognitive style account addresses this question only, likewise more elaborate metacognition accounts which suggest that people compare strategies for e!ectiveness, and are constantly monitoring their performance in order to do so. The "ndings of MacLeod et al. (1978) would "t either of these. Siegler (e.g. 1996), on the other hand, suggests that strategies can be selected more or less automatically without the need to invoke a metacognitive homunculus. Overall, it has been shown that individual di!erences in strategy usage are readily observable, often have implications for performance, and are potentially systematic and explicable, although not always in intuitively obvious ways. The assumption that individual di!erences are unimportant or constitute random error is unsafe, and no theory of psychology that claims to account for all performance can be complete without taking them into account. Although this paper has focused on the relationship between general abilities and strategy discovery and selection, the importance of prior knowledge of strategies is certainly not ruled out, and task facets are also known to be important

152

M. J. ROBERTS AND E. J. NEWTON

factors. A general theory of strategy usage will therefore almost certainly have to aim for an understanding of the interaction between all three of these.

References BARON, J. (1978). Intelligence and general strategies. In G. UNDERWOOD, Ed. Strategies in Information Processing, pp. 403}450. London: Academic Press. BARRATT, E. S. (1953). An analysis of verbal reports of solving spatial problems as an aid in de"ning spatial factors. Journal of Psychology, 36, 17}25. BROWN, A. (1987). Metacognition, executive control, self-regulation and other more mysterious mechanisms. In F. E. WEINERT & R. H. KLUWE, Eds. Metacognition, Motivation and ;nderstanding, pp. 65}116. Hillsdale, NJ: Lawrence Erlbaum. BRUNER, J. S. (1974). Beyond the Information Given. London: George Allen and Unwin. COOPER, L. A. & MUMAW, R. J. (1985). Spatial aptitude. In R. F. DILLON, Ed. Individual Di+erences in Cognition, Vol. 2, pp. 67}94. Orlando: Academic Press. CROWLEY, K., SHRAGER, J. & SIEGLER, R. S. (1997). Strategy discovery as a competitive negotiation between metacognitive and associative mechanisms. Developmental Review, 17, 462}489. ENTWISTLE, N. J. (1979). Stages, levels, styles or strategies: dilemmas in the description of thinking. Educational Review, 31, 123}132. ERICSSON, K. A. & CHARNESS, N. (1994). Expert performance: its structure and acquisition American Psychologist, 49, 725}747. EYSENCK, M. W. & KEANE, M. T. G. (2000). Cognitive Psychology: A Student1s Handbook, 4th edn. Hove: Psychology Press. FORD, M. (1994). Two models of mental representation and problem solving in syllogistic reasoning. Cognition, 54, 1}71. FRENCH, J. W. (1965). The relationship of problem-solving styles to the factor composition of tests. Educational and Psychological Measurement, 25, 9}28. GALOTTI, K. M., BARON, J. & SABINI, J. P. (1986). Individual di!erences in syllogistic reasoning: deduction rules or mental models? Journal of Experimental Psychology: General, 115, 16}25. GILHOOLY, K. J., LOGIE, R. H., WETHERICK, N. E. & WYNN, V. (1993). Working memory and strategies in syllogistic-reasoning tasks. Memory and Cognition, 21, 115}124. GOLDSMITH, R. E. (1986). Personality and adaptive-innovative problem solving. Journal of Social Behavior and Personality, 1, 95}106. HUDSON, L. (1966). Contrary Imaginations. London: Methuen. HUNT, E. (1974). Quote the Raven? Nevermore. In L. W. GREGG, Ed. Knowledge and Cognition, pp. 129}157. New York: John Wiley. JOHNSON-LAIRD, P. N. & BYRNE, R. M. J. (1991). Deduction. Hove: Psychology Press. JUST, M. A. & CARPENTER, P. A. (1985). Cognitive co-ordinate systems: accounts of mental rotation and individual di!erences in spatial ability. Psychological Review, 92, 137}172. KIRTON, M. (1976). Adaptors and innovators: a description and measure. Journal of Applied Psychology, 65, 622}629. KLINE, P. (1991). Intelligence: ¹he Psychometric
UNDERSTANDING STRATEGY SELECTION

153

MARQUER, J. M. & PEREIRA, M. (1990). Reaction times in the study of strategies in sentence} picture veri"cation: a reconsideration. Quarterly Journal of Experimental Psychology, 42A, 147}168. MATHEWS, N. N., HUNT, E. B. & MACLEOD, C. M. (1980). Strategy choice and strategy training in sentence-picture veri"cation. Journal of
154

M. J. ROBERTS AND E. J. NEWTON

SMITH, P. T. (1989). Critical notice: deadly tenacity and almost diabolical cunning. Quarterly Journal of Experimental Psychology, 41A, 877}884. SNOW, R. E. (1989). Aptitude-treatment interaction as a framework for research on individual di!erences in learning. In P. L. ACKERMAN, R. J. GLASER & R. J. STERNBERG, Eds. ¸earning and Individual Di+erences: Advances in ¹heory and Research, pp. 13}59. New York: W. H. Freeman. STERNBERG, R. J. (1997). Are cognitive styles still in style. American Psychologist, 52, 700}712. STERNBERG, R. J. & WEIL, E. M. (1980). An aptitude;strategy interaction in linear syllogistic reasoning. Journal of Educational Psychology, 72, 226}239. VAN LEHN, K. (1988). Towards a theory of impasse driven learning. In H. MANDL & A. LESGOLD, Eds. ¸earning Issues for Intelligent ¹utoring Systems, pp. 19}41. New York: Springer-Verlag. WEINMAN, J. (1987). Non-cognitive determinants of perceptual problem-solving strategies. Personality and Individual Di+erences, 8, 53}58. WOOD, D. J. (1969). ¹he nature and development of problem solving strategies. Unpublished Doctoral Dissertation, University of Nottingham. WOOD, D. J. (1978). Problem solving*the nature and development of strategies. In G. UNDERWOOD, Ed. Strategies in Information Processing, pp. 329}356. London: Academic Press. Paper accepted for publication by Editor, Prof. B. Gaines.