- Email: [email protected]
Solving the Tower of Hanoi: does mode of presentation matter?
Computers in Human Behavior 19 (2003) 579–592 www.elsevier.com/locate/comphumbeh
Solving the Tower of Hanoi: does mode of presentation matter? Jan M. Noyesa,*, Kate J. Garlandb a
Department of Experimental Psychology, University of Bristol, Bristol BS8 1TN, UK b Department of Psychology, University of Plymouth, Plymouth PL4 8AA, UK
Abstract Three studies are reported which consider different presentation modes for the Tower of Hanoi puzzle: these are computer and physical models, and a mental representation. Individuals were found to be most efficient when using a mental representation, but the costs were longer times and a greater probability of failure. This efficiency was thought to be in part due to providing a verbal protocol whilst problem-solving. Using a computer was more likely to guarantee success, but only after generating more moves, albeit in a shorter time. Features of the computer display and the alleviation of the load on working memory were thought to explain why this presentation mode was more successful. It is suggested that computers make us inefficient at the problem-solving process. # 2003 Elsevier Science Ltd. All rights reserved. Keywords: Tower of Hanoi; Problem solving; Computer-based tasks; Mental representation; Verbal protocols
The use of computers has proliferated over the past couple of decades since the launch of the first personal computer in February 1978. Despite the extensive use of computers, we still have a limited understanding of the optimum means of presenting material on computer screens especially when there is a learning element involved (Kirchner & Paas, 2001). However, this is not a new viewpoint. As long ago as 1989, Jill Larkin was writing about the extra skills needed to use a computer-based task and the fact that user efficiency will depend on the design of the interface (Larkin, 1989). It is intended that the studies reported here will help increase our understanding of problem solving in computer-based tasks. Computers promote the acquisition of declarative knowledge, i.e. the acquisition of facts (see Cagan & Kotovsky, 1997; Sarkar, Chakrabarti, & Ghose, 1998). However, problem-solving tasks could be viewed as a means of providing procedural * Corresponding author. E-mail address: [email protected] (J.M. Noyes). 0747-5632/03/$ - see front matter # 2003 Elsevier Science Ltd. All rights reserved. doi:10.1016/S0747-5632(03)00002-5
580
J.M. Noyes, K.J. Garland / Computers in Human Behavior 19 (2003) 579–592
knowledge, that is, knowing ‘how’ rather than knowing ‘that’. New technologies have not focused to any great extent on the acquisition of procedural knowledge (Chambers, 1999); hence, there is an important gap that needs to be addressed. In order to solve a problem, it is essential that an individual understands the problem. If this is the case, problem solving begins with the construction of a mental representation of the problem (Robertson, 2001). This representation is not necessarily optimal (Simon, 1978), but its construction can be aided by the way the problem is presented to the solver. It is thought that representations are influenced by the extent of the person’s understanding of the problem and their previous experiences and knowledge (Kotovsky, Hayes, & Simon, 1985). The mental representation will also depend on the loads on the problem-solver’s memory. When problem solving, the capacity of short-term, or ‘working memory’ is particularly relevant because of its influence on the forming of mental representations (Baddeley & Hitch, 1974). Information that is not held in working memory will need to be retained by the long-term memory system. Storing more knowledge in long-term memory reduces the load on working memory. This results in a greater capacity being made available for active processing. When solving problems, if the various rules have been learned and their application practiced, this information can be held in long-term memory. Thus, once the individual is familiar with the problem, s/he will be in a better position to plan how to solve the problem. There are two important issues here. First, is the role of experience in terms of aiding the forming of mental representations, reducing the memory loads and facilitating planning activities. Second, the implications for having a display of the problem that acts like an ‘external memory’ and provides the user with information about the problem at all times. It is reasonable to conclude, therefore, that an important characteristic of using a computer is that it reduces the load on working memory. However, Van Oostendorp and Walbeehm (1995) went further and separated working memory into two types. They suggested that computer displays ‘‘offer the user an external (working) memory in addition to the internal (working) memory he or she already possesses’’ (p. 19). The use of the word ‘memory’ is of interest here, since it might be argued that the display screen is merely providing an external representation of the problem rather than a memory. However, whatever the terminology, there are many advantages to having this situation when problem-solving: 1. It reduces the load on internal working memory [in particular, the visuospatial scratchpad (Baddeley & Hitch, 1974)]. 2. Storing less information in the internal working memory means that there is less chance of forgetting information. This reduces the chance of the problemsolver making errors. 3. Problem-solvers may consider the task to be less cognitively complex, because of the reduced load on working memory. Hence, they feel more confident about solving the problem. 4. It allows the user to become more focused on solving the problem as opposed to remembering the rules (described by Van Oostendorp and Walbeehm as the ability to use more ‘concentration units’).
J.M. Noyes, K.J. Garland / Computers in Human Behavior 19 (2003) 579–592
581
For many individuals, talking through solving the problem will have a positive effect upon performance. This may be because hearing what you have just said reduces the load on working memory; thus, providing more capacity for solving the problem (Robertson, 2001). It is also a form of rehearsal that aids transfer of information to the long-term store. In their simplest form, verbal protocols require individuals to report their thoughts as they carry out the task. This is particularly appropriate for tasks that involve sequential processing, because this mirrors the consecutive nature of the thought processes. It is then relatively easy to talk through solving the problem. Verbal protocols, and in particular, the so-called ‘thinkingaloud’ techniques, have been shown to aid problem solving and this benefit has been well-documented (Ahlum-heath & DiVesta, 1986; Berry, 1983; Ericsson & Simon, 1993). More recently, De Mul and Van Oostendorp (1996) and Van Oostendorp and De Mul (1999) studied the think-aloud technique in the assessment of an interface for display-based, problem solving, and found that it offered positive benefits in terms of the amount of material learned. However, Davies (2000) argued that for verbalization to be beneficial, it has to be accompanied by evaluation. Evaluation results in an explicit representation of the strategies used to solve the problem, and this is advantageous to the problem-solver. One of the issues is whether evaluation without verbalization has the same benefits. Davies argued that verbalizing may result in more processing time being directed towards the problem-solving task. Consequently, if the effort is towards thinking aloud, then the process of evaluation may be independent of verbalization processes. He also put forward the suggestion that non-verbal evaluation of moves (as indicated by a key press on a computer) is the same as a verbal evaluation. Hence, carrying out a problem-solving task on a computer may have the same benefits as thinking aloud whilst carrying out the same task in one’s head or on paper, etc. This fits with the hypothesis put forward by Berardicoletta, Dominowski, Buyer, and Rellinger (1995). They suggested that successful problem-solving was not due to verbalization per se, but due to the meta-cognitive processing resulting in increased effort to produce an explanation for trying to reach a solution. In terms of collecting verbal protocols, there are a number of points that emerge from the literature relevant to the above discussion. Nisbett and Wilson (1977) argued that verbal protocols provide spurious evidence, as there is no introspective access to higher cognitive processing. (They even went as far at to suggest that they are no more valid than the speculations of external observers.) It is therefore debatable whether protocols accurately reflect underlying processes. A further problem, as Ericsson and Simon (1980) pointed out, is that the production of verbal protocols may significantly distort the underlying cognitive processing. However, verbal protocols can be validated if there is a close correspondence between what the problem-solver is reporting and their behaviour, working memory (as opposed to long-term memory) is being used, and prompting by an observer is kept to a minimum. The Tower of Hanoi is a well-known problem-solving task that has been used many times in an experimental setting (see Anderson & Douglass, 2001). This particular
582
J.M. Noyes, K.J. Garland / Computers in Human Behavior 19 (2003) 579–592
puzzle consists of a search process within a problem space about which the problemsolver has very little specific domain knowledge (Newell & Simon, 1972). Solvers need to acquire additional knowledge to decompose a goal into sub-goals. They need to learn how to evaluate the outcomes of their actions in order to sort the actions that they carry out in terms of their contribution to solving a sub-goal (and ultimately, the overriding goal of solving the Tower of Hanoi puzzle). This task was selected because it represents a well-defined problem with clear initial and goal states, operators and restrictions (Robertson, 2001). Furthermore, it is a relatively straightforward task with a set of very simple instructions that can be easily represented. The Tower of Hanoi puzzle comprises a number of vertical pegs, and doughnutshaped disks of graduated sizes that fit onto these posts. At the start of the problemsolving exercise, all the disks are arranged in pyramid form on one of the end pegs with the largest disk on the bottom. The ‘problem’ is to move all of the disks from this end peg to the other end peg subject to a number of constraints. These are: (1) only one disk can be moved at a time; (2) a disk cannot be moved to be placed on a disk that is smaller than itself, and (3) no disk can be put aside. Any number of disks can be used; the minimum number of moves is 2N 1, where N equals the number of disks. However, five disks and three pegs provide a problem of sufficient difficulty that can be solved within a relatively short period, as only 31 moves need to be carried out. This particular transformation puzzle has been widely used for a number of decades (see Simon, 1975, for studies using the Tower of Hanoi in the 1960s and 1970s). In summary, it was hypothesized that for a simple, problem-solving task such as the Tower of Hanoi having access to a model of the problem will benefit performance in terms of more successful problem-solving (i.e. completion of the puzzle), and more efficient problem-solving (i.e. fewer moves and faster times).
1. Method 1.1. Design and participants
In Experiment 1, a within-subjects design was employed for the form of presentation of the problem-solving task. These were a computer model, a physical model and a mental representation (these will be referred to as Computer, Physical and Mental, respectively). The order in which the three forms of presentation were attempted was counterbalanced. A total of 39 Level 1 undergraduate students from the University of Bristol, UK carried out the experiment as part of a course requirement. The 10 men and 29 women had an age range of 18.00–23.58 years (M=19.47, S.D.=0.99). In Experiment 2, a within-subjects design was employed and only two forms of the problem-solving task were used: namely, the Computer and Mental conditions. (The Physical model was not used because of the larger number of unsuccessful
J.M. Noyes, K.J. Garland / Computers in Human Behavior 19 (2003) 579–592
583
completions in this condition.) The order in which participants undertook the two conditions was counterbalanced with equal numbers attempting each condition first. A total of 76 Level 1 undergraduate students from the University of Bristol carried out the experiment as part of a course requirement. The 20 men and 56 women had an age range of 18.42–25.33 years (M=19.76, S.D.=0.99). In Experiment 3, a within-subjects design was employed with the Computer and Mental conditions. However, analysis of the first attempt at the problem was completed as a between-subjects design. A total of 56 Level 1 undergraduate students from the University of Bristol carried out the experiment as part of a course requirement. The 10 men and 46 women had an age range of 18.00–36.92 years (M=19.84, S.D.=2.67). In all three experiments, no participants had attempted the Tower of Hanoi puzzle previously. This is important since Davis and Klebe (2001) found that young people in their 20s and 30s improved over four successive trials at solving the Tower of Hanoi (unlike individuals in their 80s who were significantly impaired when carrying out repeated attempts). Also, none of the participants were known or observed to have any visual or other impairment that may have affected their ability to complete the task. 1.2. Materials The Computer group received their problem-solving task via a 15 inch, SVGA monitor (65 Hz refresh rate) powered by a Memax ‘x86 Family model’ personal computer with 32 Mb RAM, and using the Microsoft (MS) Windows NT version 4.0 operating system. In the Computer condition, the ‘Tower of Hanoi’ (Version 1) program authored by Frantisek Folber was used. The Physical group was provided with a paper model of the Tower of Hanoi, while the Mental group had no physical aids or access to them. 1.3. Procedure Participants were seated, either in front of a computer or at a desk. Initially, they were read a description of the problem, noting especially the move constraints, and information about the verbal protocol. Each participant was given up to 15 min to solve the problem. They were told they could stop at any time. Although the procedure was essentially the same for all three experiments, there were slight differences between the conditions; for example, in the Computer condition, timing data was collected by the system. This commenced either when any key was pressed to activate the presentation on the computer screen, or when the experimenter signaled that the experiment was about to start. As soon as the participant had correctly solved the Tower of Hanoi problem, the computer indicated the time and the number of moves. In the Mental condition (and the Physical condition used in Experiment 1), these data were collected by the experimenter. In addition, the experimenter collected verbal protocols. At the end of the third experiment, participants were
584
J.M. Noyes, K.J. Garland / Computers in Human Behavior 19 (2003) 579–592
given a questionnaire on strategies used to solve the problem. No prompting was given by the experimenter (in keeping with the suggestion of Ericsson & Simon, 1980). Participants were subsequently debriefed. 2. Results 2.1. Experiment 1 Successful completions. The overall mean for successful completions of the Tower of Hanoi puzzle was 85% with unsuccessful attempts being highest in the Physical condition (23%) followed by the Mental (13%) and Computer conditions (8%). Number of moves. The mean number of moves was 42.45 (S.D.=13.41). The Computer and Physical groups took a similar number of moves (M=43.88, S.D.=12.23) and (M=45.57, S.D.=18.17) respectively, when compared with the Mental group (M=38.15, S.D.=9.83). A one-way ANOVA confirmed that this difference was significant, F(2, 46)=4.35, P=0.019. A Bonferroni pairwise comparison indicated a significant difference between the Mental and Physical conditions (P=0.032). Times taken. The mean time taken was 346.75 s (S.D.=218.68). No significant differences were found (P=0.083) between groups. (Physical group, M=291.40 s, S.D.=211.95; Computer group, M=338.69 s, S.D.=197.67; Mental group, M=404.12 s, S.D.=246.42.) Time per move. The mean time taken per move was 5.94 (S.D.=2.84) for the Physical group, 6.91 (S.D.=3.00) for the Computer group and 9.88 (S.D.=4.95) for the Mental group. A one-way ANOVA confirmed that this difference was significant, F(2, 44)=11.42, P=0.001.
2.2. Experiment 2 Successful completions. The number of successful completions of the puzzle was 83% of all attempts. A comparison across the two conditions indicated that all participants in the Computer condition completed the task, compared to only twothirds of the individuals in the Mental group. Number of moves. The mean number of moves in the two conditions study was 52.47 (S.D.=21.68). The difference between the Computer group (M=54.43, S.D.=22.21) and the Mental group (M=49.36, S.D.=21.14) was not significant (P=0.084). Times taken. The mean time taken across the two conditions was 361.84 s (S.D.=199.62). In contrast with the number of moves being taken, the Computer group was faster (M=289.83, S.D.=161.00), when compared with the Mental group (M=476.39, S.D.=238.23) with the difference being significant, t(42)= 5.53, P=0.001. Time per move. The mean time taken per move was 5.37 (S.D.=2.34) for the Computer group and 10.04 (S.D.=5.13) for the Mental group with the difference being significant, t(42)= 6.85, P=0.001.
J.M. Noyes, K.J. Garland / Computers in Human Behavior 19 (2003) 579–592
585
2.3. Experiment 3 Successful completions. The number of successful completions of the puzzle was 68% of all attempts. A comparison across the two conditions indicated that 70% of participants in the Computer group were completing the task, compared with only 50% of individuals in the Mental group. Number of moves. The mean number of moves in the two conditions was 55.75 (S.D.=17.95). The difference between the Computer group (M=58.14, S.D.=22.28) and the Mental group (M=49.55, S.D.=13.63) was not significant (P=0.181). Times taken. The mean time taken across the two conditions was 374.92 s (S.D.=188.45). In contrast with the number of moves being taken, the Computer group were faster (M=305.51, S.D.=161.82) when compared to the Mental group (M=554.77, S.D.=215.07), with the difference being significant, t(20)= 6.00, P=0.001. Time per move. The mean time taken per move was 5.29 (S.D.=2.11) for the Computer group and 9.50 (S.D.=5.37) for the Mental group, with the difference being significant, t(20)= 3.55, P=0.002.
2.3.1. First attempt An independent samples t-test was carried out on the first attempt data. Number of moves for the Computer condition was higher (M=61.56, S.D.=24.89) than the Mental condition (M=49.19, S.D.=14.12). This was significant, t(41)=1.816, P=0.044. Times taken for the Computer condition (M=344.57, S.D.=189.54) were faster than the Mental condition (M=511.00, S.D.=253.29). This was significant, t(42)= 3.07, P=0.004.
2.3.2. First versus second attempt In order to determine transfer effects, the number of moves for the second attempts at solving the problem were deducted from the number of moves taken in the first attempt. The overall mean difference was 1.11 (S.D.=21.75). When the computer condition was attempted first, M=11.23 (S.D.=16.20); however, when the mental condition was attempted first, M=8.29 (S.D.=6.02). A one-way ANOVA confirmed that this difference was significant, F(1, 25)=6.59, P=0.017. Therefore, there was an overall advantage of transferring from the first to the second condition, but this was only when the Computer condition was presented first. Attempting the Mental representation condition first actually had a detrimental effect on the subsequent solving of the computer version of the puzzle.
2.3.3. Qualitative data At the end of Experiment 3, participants were asked seven questions on how they were solving the puzzle; a summary of the key findings for three of these questions are listed here.
586
J.M. Noyes, K.J. Garland / Computers in Human Behavior 19 (2003) 579–592
1. What did you do differently in the two conditions? Did not do anything different/tried to do the same (14 responses) Less trial and error in second condition (use of sub-goals/tactics) (13 responses) More thought/forward planning was needed in the Mental condition (5 responses) One participant said more trial and error was used in the Computer condition, as no forward planning was needed as disks could be seen. 2.
What aspects of your problem-solving behaviour in the first condition were of benefit in the second condition? Remembering move sequences (11 responses) General familiarization (11 responses) Little/nothing/none (10 responses) Remembering strategies (5 responses) Use of sub-goals (3 responses)
3.
What aspects of your problem-solving behaviour in the first condition were NOT of benefit in the second condition? Nothing/none/do not know (13 responses) Trial and error approach (especially for Mental representation) (6 responses) Having an image (3 responses) Ability to visualize (2 responses) Lack of image (1 response)
3. Discussion In the first experiment, participants made fewer moves using the mental representation than the physical and computer models, and more people gave up when trying to solve the puzzle using the physical model. The finding that participants were able to solve the Tower of Hanoi in fewer moves with a mental representation is interesting. It suggests that problem-solving ‘in the head’ is more efficient than using a computer. The number of moves taken in this first experiment resulted in a range across the three conditions with the physical condition taking the most moves followed by the computer and mental representations. The computer presentation of the Tower of Hanoi puzzle provides a means of representing the problem pictorially. Thus, it provides an ‘intermediate representation’ between the physical and mental models. This would suggest that the physical model should provide the most benefits in terms of helping us solve the problem. Being able to feel the disks, and physically move them around should have reduced the number of moves. The advantages of making an abstract representation into a more concrete one has been shown to be of
J.M. Noyes, K.J. Garland / Computers in Human Behavior 19 (2003) 579–592
587
benefit in problem-solving (Beveridge & Parkins, 1987). For example, Reed and Ettinger (1987) found that tables were useful for problem solvers because it provided them with concrete information. However, the explanation in this research probably lies with the design of the computer-generated Tower of Hanoi puzzle. Manipulation of the computer version was very easy and involved a ‘drag and drop’ mouse operation to move the disks on the screen. Thus, individuals could very quickly elicit the desired moves; perhaps, this ease of operation resulted in them not focusing on reaching the end-point by the most efficient means, and as a result, a ‘trial and error’ approach was being adopted. As Larkin (1989) pointed out, the computer program for the Tower of Hanoi is flexible in that it allows the problem to be solved in a variety of ways. Hence, problem-solvers do not have to be too careful about making sure the next move is the right one. However, the advantage demonstrated in Experiment 3 of transferring from the computer to the mental representation of the problem (rather than vice versa) might be explained by the benefits accrued from using the more concrete version of the puzzle. The transfer effects are not easy to explain. In isomorphic problems, experience with solving a problem facilitates further attempts at solving the same problem. This has been well documented (see Luger & Bauer, 1978; Reed, Ernst, & Banerji, 1974). Hence, it would have been expected that individuals on their second attempts would be able to solve the problem in a fewer number of moves. Although this occurred when moving from the computer to the mental representation, it did not happen when participants transferred from the mental to the computer version of the Tower of Hanoi. This latter finding is counter-intuitive, as the extra cognitive effort that went into solving the mental representation first should have benefited any second attempt. However, Reed et al. stated that facilitation only occurs if the second problem is easier than the first; perhaps the computer model did not present participants with a problem of equivalent difficulty. The display may have provided interference with the result that participants did not fully apply themselves to the problem-solving task in this medium. Several participants reported how they had visualized the Tower of Hanoi puzzle in the mental representation condition. This may have then made it difficult for them to solve the problem when confronted with a computer-generated model. In all three experiments, participants were faster when using the computer version of the puzzle in terms of moves per second. Again, the design of the computer display may explain these results: the use of the mouse allowed participants to move the disks with speed and relative ease. It has been suggested that working memory may be involved in successful performance of the Tower of Hanoi puzzle (Welsh, Cicerello, Cuneo, & Brennan, 1995). When attempting to solve the puzzle, individuals reach the limits of their working memory; they are only able to ‘see’ the state of the disks on the pegs a few moves ahead. Further, they may not remember the moves previously made, so are unable to apply strategies that may have proved previously to be successful in moving the disks. One of the reasons why Welsh et al. suggested working memory was involved, arose as a result of their observations of individuals solving the Tower of Hanoi. Interestingly, they found in their work with a four disk (15-move) puzzle that there
588
J.M. Noyes, K.J. Garland / Computers in Human Behavior 19 (2003) 579–592
were certain points, that is, at the first, fifth and nineth moves, where errors were likely to be made. They also measured pause times and found that even optimal problem-solvers increased their pause time before these junctures. When asked what they were doing, participants replied that they were planning moves during these pause times (although whether or not they reported this planning activity did not correlate with successful performance). Although pause times were not measured in the experiments reported here, they may have been a useful adjunct in explaining the differences in the results. Furthermore, there was some evidence from the verbal reports that some individuals were pausing around moves 14–17. In terms of working memory loads, using the computer version of the Tower of Hanoi should have placed participants at an advantage since the on-screen visualization of the puzzle would have reduced these loads (Van Oostendorp & Walbeehm, 1995). Information that needed to be remembered and that otherwise would have had to be held in working memory was displayed on the screen for as long as the problem-solver desired. For example: The goal of moving all the disks to the other end peg is evident. Information relating to the state of the problem is always present. The display ‘performs’ any legal action required, and in turn, it simply blocks moves that are not allowed, for example, trying to move a disk that is not on the top of the peg, or trying to move a disk to a location where the resident disk is smaller, or trying to put a disk to one side. If moves are made in error, they are immediately visible on the screen, and thus, the problem-solver is able to continue from the current state. In the case of the Tower of Hanoi puzzle, making the moves relies on the internal working memory, but the person also needs to apply the restriction rules to making the moves. This information, although not shown on the screen in the same way as the puzzle, is ‘present’ in the computer; hence, the user has access to an ‘external (working) memory’. Van Oostendorp and Walbeehm described this as ‘intermediate information’ that is being kept in an external working memory. Further, there is little cost for interruption of carrying out the task as information is not lost, for example, if concentration momentarily lapses. This may help explain the greater number of moves when using the computer version. Participants had so much information present on the display screen that there was no need to be totally focused on solving the problem. In essence, it could be argued that display-based problem solving reduces the complexity of the mental processes involved by reducing the loads on working memory. Certainly, participants using the mental representation reported the following: ‘‘very difficult to visualize in head and keep strategy’’, ‘‘easy to forget where blocks were’’, and ‘‘saying name out loud interferes with problem-solving’’. Although a relatively simple puzzle, Simon (1975) wrote about the ‘sophisticated perceptual strategy’ that was needed to solve the Tower of Hanoi. Each move is directly cued by the visible state of the physical problem; hence, at any point, the users need to have access to a representation of the problem state. This, of course, is
J.M. Noyes, K.J. Garland / Computers in Human Behavior 19 (2003) 579–592
589
much easier if you are looking at the representation of the puzzle on the computer. Larkin (1989) suggested that it is possible merely to look at the display and to do what is implied in terms of carrying out the next step in the puzzle without ‘effortful mental calculation’ or the need for storage of information. This may be borne out by the fact that more people solved the problem when using the computer presentation, that is, they kept going (and indeed, were able to keep going) until they reached the goal state. These assertions about the computer-generated version of the problem may help explain the differences we found between the modes of presentation. Given the many benefits of the computer presentation of the puzzle, it is curious that people were solving the problem ‘in their head’ with a fewer number of moves. Robertson (2001) stated that the first activity carried out by a person when confronted with a problem is the generation of an internal representation of the particular problem. When presented with a computer model for the Tower of Hanoi, there is no need to make any effort to form your own mental representation, because there is an external representation on the display screen. Consequently, the problemsolver faced with the computer version of the problem is immediately at a disadvantage because they are not having the benefit of having to apply themselves to beginning to solve the problem. Further, the computer’s representation of the problem may not match their internal representation of the problem. In effect, the computer model may be providing so-called ‘cognitive clutter’ that is interfering with the optimum route for problem solving. In contrast, solving the problem using only a mental representation allows you to build a strong representation of the problem, and this results in more efficient problem solving. Moreover, as already discussed, the transfer effects of moving between the two conditions support this. A further explanation may lie in the use of verbal protocols. Individuals solving the puzzle using the mental representation version of the Tower of Hanoi were required to talk through the moves they were making, that is, to think aloud. This form of protocol analysis is particularly appropriate to transformation problems such as the Tower of Hanoi. These problems create situations, which the person finds easy to verbalize (Byrne, 1977). Further, it has also been previously used when solving the Tower of Hanoi. For example, Simon and Hayes (1976) concluded that ‘thinking aloud’ whilst the participants were actually working out the Tower of Hanoi not only provided a rich source of information in understanding how people solve problems, but enhanced the problem-solving skills of the individual. It is a reasonable assumption, therefore, that providing the verbal protocol by thinking aloud whilst solving this particular puzzle was of benefit to participants solving the problem ‘in their head’. However, examination of the qualitative data does not indicate this as participants reported generating the verbal protocols as ‘getting in the way’. Although some participants felt this, it may still have enhanced their solving of the puzzle, and might help explain why people were so efficient at solving the Tower of Hanoi when they had only a mental representation. The studies reported here do not provide any information concerning the strategies used by the problem-solvers, although some inferences can be made from both the verbal protocols and the qualitative data collected in the third experiment. In contrast, Simon (1975) carried out a detailed analysis of strategies employed to solve
590
J.M. Noyes, K.J. Garland / Computers in Human Behavior 19 (2003) 579–592
the Tower of Hanoi. Similarly, Anzai and Simon (1979) carried out an in-depth study of one person solving a five disk, Tower of Hanoi puzzle, four times during a one and a half-hour period. Interestingly, it was found that each time this participant attempted the puzzle, she adopted a different strategy that was more efficient than the previous one. Cagan and Kotovsky (1997) in their studies on problem-solving behaviour observed that individuals begin by searching more or less randomly. Over time they become more deterministic as they learn more about the problem. They described this as a two-phase aspect of problem-solving behaviour. Although we do not have the empirical evidence, perhaps problem-solvers using the computer and physical versions spent longer in the first phase, and subsequently, generated more moves. Certainly, some of the raw data indicates that a few individuals were making a large number of moves (in excess of 100) to solve a problem that only required 31 moves. Given the efficiency that our participants solved the puzzle with only a mental representation, this suggests that they are more likely to have adopted a strategy than when using the computer version. The qualitative data does provide some evidence for this. For example, ‘trial and error’ was cited as a strategy being used, but was being discarded on the second attempt as it was not found to be particularly beneficial. This was more likely to be the case in the mental representation condition. Remembering the moves was also an aspect of the first attempt that people applied in the second, and perhaps the greater cognitive effort needed to solve the mental representation meant that people were more able to remember the moves. However, it is also evident from this data that many participants had no particular strategy: for example, ‘‘knew vaguely what was doing, but no definite technique or strategy’’, ‘‘followed no particular strategy’’, and ‘‘no strategy apparent!’’. One of the difficulties associated with the use of any computer-generated model is the nature of the interface. This is particularly the case when considering problemsolving as the ergonomics of the display and the user’s interaction with it can influence the ease with which the problem can be solved. Larkin (1989) pointed out the importance of the interface design in influencing computational efficiency, and this has been demonstrated for the Tower of Hanoi and other isomorphs (Hayes & Simon, 1976; Kotovsky et al., 1985). The importance of the design of the interface must not be overlooked, because as Zhang (1991) pointed out the external representations of the problem provide memory aids. Hence, the design of these can change the nature of the task. The precise design of the computer-generated Tower of Hanoi will, therefore, influence the solving of the puzzle. This needs to be taken into account when generalizing from Tower of Hanoi studies. A further consideration is the extent to which wider inferences about problem solving can be made from a simple, transformation problem such as the Tower of Hanoi. The use of various types of puzzles in problem-solving research could be argued to be very remote from the day-to-day problems that humans encounter, and could, therefore, be seen to be lacking in ecological validity. However, problems in the ‘real world’ frequently involve considerable time to solve, if indeed, they are solved at all. The puzzles used in experimental research have a greater likelihood of being solved, because their well-defined structure clarifies both the initial and goal
J.M. Noyes, K.J. Garland / Computers in Human Behavior 19 (2003) 579–592
591
states, and identifies constraints (both legal operators and restrictions). In addition, they have discrete stages that can be easily observed, and ‘moves’ tend to be completed at a pace suitable for the participants to verbalize their behaviour. Hence, the simplicity of the Tower of Hanoi problem provides a good starting point, as we need first to find out about the differences between computer-generated and other forms of the problem before progressing to more complex problems. Moreover, there are a number of well-known, transformation problems (e.g. Hobbits & Orcs, book burners and book lovers, missionaries and cannibals) and it would be interesting to see if the findings reported here would be replicated with other puzzles (see Reed et al., 1974). In summary, having a physical representation of the Tower of Hanoi problem was not found to be helpful in solving the puzzle. In contrast, using a display-based computer generated model was likely to lead to success in the shortest times (but with the greatest number of moves). Various features of the display-based presentation were thought to explain these findings. However, the most efficient way of solving the problem, i.e. with the fewest number of moves, was with a mental representation of the puzzle. However, the costs are that it will take you longer and you are more likely to lose where you are in the problem, and have to give up. Mental representation of the puzzle demanded that individuals ‘talked’ through solving the problem, and this may help explain the efficiency of this approach. References Ahlum-heath, M. E., & DiVesta, F. J. (1986). The effects of conscious controlled verbalization of a cognitive strategy on transfer in problem solving. Memory and Cognition, 14, 281–285. Anderson, J. R., & Douglass, S. (2001). Tower of Hanoi: Evidence for the cost of goal retrieval. Journal of Experimental Psychology—Learning, Memory and Cognition, 27(6), 1331–1346. Anzai, Y., & Simon, H. A. (1979). The theory of learning by doing. Psychological Review, 86(2), 124–140. Baddeley, A. D., & Hitch, G. (1974). Working memory. In G. A. Bower (Ed.), Recent advances in learning and motivation (Vol. 8). New York: Academic Press. Berardicoletta, B., Dominowski, R. L., Buyer, L. S., & Rellinger, E. R. (1995). Metacognition and problem-solving—a process-oriented approach. Journal of Experimental Psychology—Learning, Memory and Cognition, 21(1), 205–223. Berry, D. C. (1983). Metacognitive experience and transfer of logical reasoning. Quarterly Journal of Experimental Psychology, 35A, 39–49. Beveridge, M., & Parkins, E. (1987). Visual representation in analogical problem solving. Memory and Cognition, 15(3), 230–237. Byrne, R. (1977). Planning meals: problem-solving on a real data-base. Cognition, 5, 285–332. Cagan, J., & Kotovsky, K. (1997). Simulated annealing and the generation of the objective function: a model of learning during problem solving. Computational Intelligence, 13(4), 534–581. Chambers, P. (1999). Information handling skills, cognition and new technologies. British Journal of Educational Technology, 30(2), 151–162. Davies, S. P. (2000). Move evaluation as a predictor and moderator of success in solutions to well-structured problems. Quarterly Journal of Experimental Psychology Section A—Human Experimental Psychology, 53(4), 1186–1201. Davis, H. P., & Klebe, K. J. (2001). A longitudinal study of the performance of the elderly and young on the Tower of Hanoi puzzle and Rey recall. Brain and Cognition, 46(1–2), 95–99. De Mul, S., & Van Oostendorp, H. (1996). Learning user interfaces by exploration. Acta Psychologica, 91(3), 325–344.
592
J.M. Noyes, K.J. Garland / Computers in Human Behavior 19 (2003) 579–592
Ericsson, K. A., & Simon, H. A. (1980). Verbal reports as data. Psychological Review, 87(3), 215–251. Ericsson, K. A., & Simon, H. A. (1993). Protocol analysis: verbal reports as data (rev. ed). Cambridge, MA: MIT Press. Hayes, J. R., & Simon, H. A. (1976). The understanding process: problem isomorphs. Cognitive Psychology, 8, 165–190. Kirchner, P. A., & Paas, F. (2001). Web-enhanced higher education: A tower of Babel. Computers in Human Behavior, 17, 347–353. Kotovsky, K., Hayes, J. R., & Simon, H. A. (1985). Why are some problems hard? Evidence from the Tower of Hanoi. Cognitive Psychology, 17, 248–294. Larkin, J. H. (1989). Display-based problem solving. In D. Klahr, & K. Kotovsky (Eds.), Complex information processing: the impact of Herbert A. Simon. Hillsdale, NJ: LEA. Luger, G. F., & Bauer, M. A. (1978). Transfer effects in isomorphic problem situations. Acta Psychologica, 42, 121–131. Newell, A., & Simon, H. A. (1972). Human problem solving. Englewood Cliffs, NJ: Prentice-Hall. Nisbett, R. E., & Wilson, T. D. (1977). Telling more than we can know: verbal reports on mental processes. Psychological Review, 84(3), 231–259. Reed, S. K., & Ettinger, M. (1987). Usefulness of tables for solving word problems. Cognition and Instruction, 4(1), 43–58. Reed, S. K., Ernst, G. W., & Banerji, R. (1974). The role of analogy in transfer between similar problem states. Cognitive Psychology, 6, 436–450. Robertson, S. I. (2001). Problem solving. Hove, UK: Psychology Press. Sarkar, S., Chakrabarti, P. P., & Ghose, S. (1998). Learning while solving problems in best first search. IEEE Transactions on Systems, Man and Cybernetics, Part A—Systems and Humans, 28(4), 535–542. Simon, H. A. (1975). The functional equivalence of problem solving skills. Cognitive Psychology, 7, 268– 288. Simon, H. A. (1978). Information-processing theory of human problem solving. In W. K. Estes (Ed.), Handbook of learning and cognitive processes (pp. 271–295). Hillsdale, NJ: LEA. Simon, H. A., & Hayes, J. R. (1976). The understanding process: problem isomorphs. Cognitive Psychology, 8, 165–190. Van Oostendorp, H., & De Mul, S. (1999). Learning by exploration: thinking aloud while exploring an information system. Instructional Science, 27(3–4), 269–284. Van Oostendorp, H., & Walbeehm, B. J. (1995). Towards modelling exploratory learning in the context of direct manipulation interfaces. Interacting with Computers, 7(1), 3–24. Welsh, M., Cicerello, A., Cuneo, K., & Brennan, M. (1995). Error and temporal patterns in Tower of Hanoi performance—cognitive mechanisms and individual differences. Journal of General Psychology, 122(1), 69–81. Zhang, J. (1991). The interaction of internal and external representations in a problem solving task. In Proceedings of 13th Annual Conference of Cognitive Science Society (pp. 954–958). Chicago, IL: LEA.
Solving the Tower of Hanoi: does mode of presentation matter? Jan M. Noyesa,*, Kate J. Garlandb a
Department of Experimental Psychology, University of Bristol, Bristol BS8 1TN, UK b Department of Psychology, University of Plymouth, Plymouth PL4 8AA, UK
Abstract Three studies are reported which consider different presentation modes for the Tower of Hanoi puzzle: these are computer and physical models, and a mental representation. Individuals were found to be most efficient when using a mental representation, but the costs were longer times and a greater probability of failure. This efficiency was thought to be in part due to providing a verbal protocol whilst problem-solving. Using a computer was more likely to guarantee success, but only after generating more moves, albeit in a shorter time. Features of the computer display and the alleviation of the load on working memory were thought to explain why this presentation mode was more successful. It is suggested that computers make us inefficient at the problem-solving process. # 2003 Elsevier Science Ltd. All rights reserved. Keywords: Tower of Hanoi; Problem solving; Computer-based tasks; Mental representation; Verbal protocols
The use of computers has proliferated over the past couple of decades since the launch of the first personal computer in February 1978. Despite the extensive use of computers, we still have a limited understanding of the optimum means of presenting material on computer screens especially when there is a learning element involved (Kirchner & Paas, 2001). However, this is not a new viewpoint. As long ago as 1989, Jill Larkin was writing about the extra skills needed to use a computer-based task and the fact that user efficiency will depend on the design of the interface (Larkin, 1989). It is intended that the studies reported here will help increase our understanding of problem solving in computer-based tasks. Computers promote the acquisition of declarative knowledge, i.e. the acquisition of facts (see Cagan & Kotovsky, 1997; Sarkar, Chakrabarti, & Ghose, 1998). However, problem-solving tasks could be viewed as a means of providing procedural * Corresponding author. E-mail address: [email protected] (J.M. Noyes). 0747-5632/03/$ - see front matter # 2003 Elsevier Science Ltd. All rights reserved. doi:10.1016/S0747-5632(03)00002-5
580
J.M. Noyes, K.J. Garland / Computers in Human Behavior 19 (2003) 579–592
knowledge, that is, knowing ‘how’ rather than knowing ‘that’. New technologies have not focused to any great extent on the acquisition of procedural knowledge (Chambers, 1999); hence, there is an important gap that needs to be addressed. In order to solve a problem, it is essential that an individual understands the problem. If this is the case, problem solving begins with the construction of a mental representation of the problem (Robertson, 2001). This representation is not necessarily optimal (Simon, 1978), but its construction can be aided by the way the problem is presented to the solver. It is thought that representations are influenced by the extent of the person’s understanding of the problem and their previous experiences and knowledge (Kotovsky, Hayes, & Simon, 1985). The mental representation will also depend on the loads on the problem-solver’s memory. When problem solving, the capacity of short-term, or ‘working memory’ is particularly relevant because of its influence on the forming of mental representations (Baddeley & Hitch, 1974). Information that is not held in working memory will need to be retained by the long-term memory system. Storing more knowledge in long-term memory reduces the load on working memory. This results in a greater capacity being made available for active processing. When solving problems, if the various rules have been learned and their application practiced, this information can be held in long-term memory. Thus, once the individual is familiar with the problem, s/he will be in a better position to plan how to solve the problem. There are two important issues here. First, is the role of experience in terms of aiding the forming of mental representations, reducing the memory loads and facilitating planning activities. Second, the implications for having a display of the problem that acts like an ‘external memory’ and provides the user with information about the problem at all times. It is reasonable to conclude, therefore, that an important characteristic of using a computer is that it reduces the load on working memory. However, Van Oostendorp and Walbeehm (1995) went further and separated working memory into two types. They suggested that computer displays ‘‘offer the user an external (working) memory in addition to the internal (working) memory he or she already possesses’’ (p. 19). The use of the word ‘memory’ is of interest here, since it might be argued that the display screen is merely providing an external representation of the problem rather than a memory. However, whatever the terminology, there are many advantages to having this situation when problem-solving: 1. It reduces the load on internal working memory [in particular, the visuospatial scratchpad (Baddeley & Hitch, 1974)]. 2. Storing less information in the internal working memory means that there is less chance of forgetting information. This reduces the chance of the problemsolver making errors. 3. Problem-solvers may consider the task to be less cognitively complex, because of the reduced load on working memory. Hence, they feel more confident about solving the problem. 4. It allows the user to become more focused on solving the problem as opposed to remembering the rules (described by Van Oostendorp and Walbeehm as the ability to use more ‘concentration units’).
J.M. Noyes, K.J. Garland / Computers in Human Behavior 19 (2003) 579–592
581
For many individuals, talking through solving the problem will have a positive effect upon performance. This may be because hearing what you have just said reduces the load on working memory; thus, providing more capacity for solving the problem (Robertson, 2001). It is also a form of rehearsal that aids transfer of information to the long-term store. In their simplest form, verbal protocols require individuals to report their thoughts as they carry out the task. This is particularly appropriate for tasks that involve sequential processing, because this mirrors the consecutive nature of the thought processes. It is then relatively easy to talk through solving the problem. Verbal protocols, and in particular, the so-called ‘thinkingaloud’ techniques, have been shown to aid problem solving and this benefit has been well-documented (Ahlum-heath & DiVesta, 1986; Berry, 1983; Ericsson & Simon, 1993). More recently, De Mul and Van Oostendorp (1996) and Van Oostendorp and De Mul (1999) studied the think-aloud technique in the assessment of an interface for display-based, problem solving, and found that it offered positive benefits in terms of the amount of material learned. However, Davies (2000) argued that for verbalization to be beneficial, it has to be accompanied by evaluation. Evaluation results in an explicit representation of the strategies used to solve the problem, and this is advantageous to the problem-solver. One of the issues is whether evaluation without verbalization has the same benefits. Davies argued that verbalizing may result in more processing time being directed towards the problem-solving task. Consequently, if the effort is towards thinking aloud, then the process of evaluation may be independent of verbalization processes. He also put forward the suggestion that non-verbal evaluation of moves (as indicated by a key press on a computer) is the same as a verbal evaluation. Hence, carrying out a problem-solving task on a computer may have the same benefits as thinking aloud whilst carrying out the same task in one’s head or on paper, etc. This fits with the hypothesis put forward by Berardicoletta, Dominowski, Buyer, and Rellinger (1995). They suggested that successful problem-solving was not due to verbalization per se, but due to the meta-cognitive processing resulting in increased effort to produce an explanation for trying to reach a solution. In terms of collecting verbal protocols, there are a number of points that emerge from the literature relevant to the above discussion. Nisbett and Wilson (1977) argued that verbal protocols provide spurious evidence, as there is no introspective access to higher cognitive processing. (They even went as far at to suggest that they are no more valid than the speculations of external observers.) It is therefore debatable whether protocols accurately reflect underlying processes. A further problem, as Ericsson and Simon (1980) pointed out, is that the production of verbal protocols may significantly distort the underlying cognitive processing. However, verbal protocols can be validated if there is a close correspondence between what the problem-solver is reporting and their behaviour, working memory (as opposed to long-term memory) is being used, and prompting by an observer is kept to a minimum. The Tower of Hanoi is a well-known problem-solving task that has been used many times in an experimental setting (see Anderson & Douglass, 2001). This particular
582
J.M. Noyes, K.J. Garland / Computers in Human Behavior 19 (2003) 579–592
puzzle consists of a search process within a problem space about which the problemsolver has very little specific domain knowledge (Newell & Simon, 1972). Solvers need to acquire additional knowledge to decompose a goal into sub-goals. They need to learn how to evaluate the outcomes of their actions in order to sort the actions that they carry out in terms of their contribution to solving a sub-goal (and ultimately, the overriding goal of solving the Tower of Hanoi puzzle). This task was selected because it represents a well-defined problem with clear initial and goal states, operators and restrictions (Robertson, 2001). Furthermore, it is a relatively straightforward task with a set of very simple instructions that can be easily represented. The Tower of Hanoi puzzle comprises a number of vertical pegs, and doughnutshaped disks of graduated sizes that fit onto these posts. At the start of the problemsolving exercise, all the disks are arranged in pyramid form on one of the end pegs with the largest disk on the bottom. The ‘problem’ is to move all of the disks from this end peg to the other end peg subject to a number of constraints. These are: (1) only one disk can be moved at a time; (2) a disk cannot be moved to be placed on a disk that is smaller than itself, and (3) no disk can be put aside. Any number of disks can be used; the minimum number of moves is 2N 1, where N equals the number of disks. However, five disks and three pegs provide a problem of sufficient difficulty that can be solved within a relatively short period, as only 31 moves need to be carried out. This particular transformation puzzle has been widely used for a number of decades (see Simon, 1975, for studies using the Tower of Hanoi in the 1960s and 1970s). In summary, it was hypothesized that for a simple, problem-solving task such as the Tower of Hanoi having access to a model of the problem will benefit performance in terms of more successful problem-solving (i.e. completion of the puzzle), and more efficient problem-solving (i.e. fewer moves and faster times).
1. Method 1.1. Design and participants
In Experiment 1, a within-subjects design was employed for the form of presentation of the problem-solving task. These were a computer model, a physical model and a mental representation (these will be referred to as Computer, Physical and Mental, respectively). The order in which the three forms of presentation were attempted was counterbalanced. A total of 39 Level 1 undergraduate students from the University of Bristol, UK carried out the experiment as part of a course requirement. The 10 men and 29 women had an age range of 18.00–23.58 years (M=19.47, S.D.=0.99). In Experiment 2, a within-subjects design was employed and only two forms of the problem-solving task were used: namely, the Computer and Mental conditions. (The Physical model was not used because of the larger number of unsuccessful
J.M. Noyes, K.J. Garland / Computers in Human Behavior 19 (2003) 579–592
583
completions in this condition.) The order in which participants undertook the two conditions was counterbalanced with equal numbers attempting each condition first. A total of 76 Level 1 undergraduate students from the University of Bristol carried out the experiment as part of a course requirement. The 20 men and 56 women had an age range of 18.42–25.33 years (M=19.76, S.D.=0.99). In Experiment 3, a within-subjects design was employed with the Computer and Mental conditions. However, analysis of the first attempt at the problem was completed as a between-subjects design. A total of 56 Level 1 undergraduate students from the University of Bristol carried out the experiment as part of a course requirement. The 10 men and 46 women had an age range of 18.00–36.92 years (M=19.84, S.D.=2.67). In all three experiments, no participants had attempted the Tower of Hanoi puzzle previously. This is important since Davis and Klebe (2001) found that young people in their 20s and 30s improved over four successive trials at solving the Tower of Hanoi (unlike individuals in their 80s who were significantly impaired when carrying out repeated attempts). Also, none of the participants were known or observed to have any visual or other impairment that may have affected their ability to complete the task. 1.2. Materials The Computer group received their problem-solving task via a 15 inch, SVGA monitor (65 Hz refresh rate) powered by a Memax ‘x86 Family model’ personal computer with 32 Mb RAM, and using the Microsoft (MS) Windows NT version 4.0 operating system. In the Computer condition, the ‘Tower of Hanoi’ (Version 1) program authored by Frantisek Folber was used. The Physical group was provided with a paper model of the Tower of Hanoi, while the Mental group had no physical aids or access to them. 1.3. Procedure Participants were seated, either in front of a computer or at a desk. Initially, they were read a description of the problem, noting especially the move constraints, and information about the verbal protocol. Each participant was given up to 15 min to solve the problem. They were told they could stop at any time. Although the procedure was essentially the same for all three experiments, there were slight differences between the conditions; for example, in the Computer condition, timing data was collected by the system. This commenced either when any key was pressed to activate the presentation on the computer screen, or when the experimenter signaled that the experiment was about to start. As soon as the participant had correctly solved the Tower of Hanoi problem, the computer indicated the time and the number of moves. In the Mental condition (and the Physical condition used in Experiment 1), these data were collected by the experimenter. In addition, the experimenter collected verbal protocols. At the end of the third experiment, participants were
584
J.M. Noyes, K.J. Garland / Computers in Human Behavior 19 (2003) 579–592
given a questionnaire on strategies used to solve the problem. No prompting was given by the experimenter (in keeping with the suggestion of Ericsson & Simon, 1980). Participants were subsequently debriefed. 2. Results 2.1. Experiment 1 Successful completions. The overall mean for successful completions of the Tower of Hanoi puzzle was 85% with unsuccessful attempts being highest in the Physical condition (23%) followed by the Mental (13%) and Computer conditions (8%). Number of moves. The mean number of moves was 42.45 (S.D.=13.41). The Computer and Physical groups took a similar number of moves (M=43.88, S.D.=12.23) and (M=45.57, S.D.=18.17) respectively, when compared with the Mental group (M=38.15, S.D.=9.83). A one-way ANOVA confirmed that this difference was significant, F(2, 46)=4.35, P=0.019. A Bonferroni pairwise comparison indicated a significant difference between the Mental and Physical conditions (P=0.032). Times taken. The mean time taken was 346.75 s (S.D.=218.68). No significant differences were found (P=0.083) between groups. (Physical group, M=291.40 s, S.D.=211.95; Computer group, M=338.69 s, S.D.=197.67; Mental group, M=404.12 s, S.D.=246.42.) Time per move. The mean time taken per move was 5.94 (S.D.=2.84) for the Physical group, 6.91 (S.D.=3.00) for the Computer group and 9.88 (S.D.=4.95) for the Mental group. A one-way ANOVA confirmed that this difference was significant, F(2, 44)=11.42, P=0.001.
2.2. Experiment 2 Successful completions. The number of successful completions of the puzzle was 83% of all attempts. A comparison across the two conditions indicated that all participants in the Computer condition completed the task, compared to only twothirds of the individuals in the Mental group. Number of moves. The mean number of moves in the two conditions study was 52.47 (S.D.=21.68). The difference between the Computer group (M=54.43, S.D.=22.21) and the Mental group (M=49.36, S.D.=21.14) was not significant (P=0.084). Times taken. The mean time taken across the two conditions was 361.84 s (S.D.=199.62). In contrast with the number of moves being taken, the Computer group was faster (M=289.83, S.D.=161.00), when compared with the Mental group (M=476.39, S.D.=238.23) with the difference being significant, t(42)= 5.53, P=0.001. Time per move. The mean time taken per move was 5.37 (S.D.=2.34) for the Computer group and 10.04 (S.D.=5.13) for the Mental group with the difference being significant, t(42)= 6.85, P=0.001.
J.M. Noyes, K.J. Garland / Computers in Human Behavior 19 (2003) 579–592
585
2.3. Experiment 3 Successful completions. The number of successful completions of the puzzle was 68% of all attempts. A comparison across the two conditions indicated that 70% of participants in the Computer group were completing the task, compared with only 50% of individuals in the Mental group. Number of moves. The mean number of moves in the two conditions was 55.75 (S.D.=17.95). The difference between the Computer group (M=58.14, S.D.=22.28) and the Mental group (M=49.55, S.D.=13.63) was not significant (P=0.181). Times taken. The mean time taken across the two conditions was 374.92 s (S.D.=188.45). In contrast with the number of moves being taken, the Computer group were faster (M=305.51, S.D.=161.82) when compared to the Mental group (M=554.77, S.D.=215.07), with the difference being significant, t(20)= 6.00, P=0.001. Time per move. The mean time taken per move was 5.29 (S.D.=2.11) for the Computer group and 9.50 (S.D.=5.37) for the Mental group, with the difference being significant, t(20)= 3.55, P=0.002.
2.3.1. First attempt An independent samples t-test was carried out on the first attempt data. Number of moves for the Computer condition was higher (M=61.56, S.D.=24.89) than the Mental condition (M=49.19, S.D.=14.12). This was significant, t(41)=1.816, P=0.044. Times taken for the Computer condition (M=344.57, S.D.=189.54) were faster than the Mental condition (M=511.00, S.D.=253.29). This was significant, t(42)= 3.07, P=0.004.
2.3.2. First versus second attempt In order to determine transfer effects, the number of moves for the second attempts at solving the problem were deducted from the number of moves taken in the first attempt. The overall mean difference was 1.11 (S.D.=21.75). When the computer condition was attempted first, M=11.23 (S.D.=16.20); however, when the mental condition was attempted first, M=8.29 (S.D.=6.02). A one-way ANOVA confirmed that this difference was significant, F(1, 25)=6.59, P=0.017. Therefore, there was an overall advantage of transferring from the first to the second condition, but this was only when the Computer condition was presented first. Attempting the Mental representation condition first actually had a detrimental effect on the subsequent solving of the computer version of the puzzle.
2.3.3. Qualitative data At the end of Experiment 3, participants were asked seven questions on how they were solving the puzzle; a summary of the key findings for three of these questions are listed here.
586
J.M. Noyes, K.J. Garland / Computers in Human Behavior 19 (2003) 579–592
1. What did you do differently in the two conditions? Did not do anything different/tried to do the same (14 responses) Less trial and error in second condition (use of sub-goals/tactics) (13 responses) More thought/forward planning was needed in the Mental condition (5 responses) One participant said more trial and error was used in the Computer condition, as no forward planning was needed as disks could be seen. 2.
What aspects of your problem-solving behaviour in the first condition were of benefit in the second condition? Remembering move sequences (11 responses) General familiarization (11 responses) Little/nothing/none (10 responses) Remembering strategies (5 responses) Use of sub-goals (3 responses)
3.
What aspects of your problem-solving behaviour in the first condition were NOT of benefit in the second condition? Nothing/none/do not know (13 responses) Trial and error approach (especially for Mental representation) (6 responses) Having an image (3 responses) Ability to visualize (2 responses) Lack of image (1 response)
3. Discussion In the first experiment, participants made fewer moves using the mental representation than the physical and computer models, and more people gave up when trying to solve the puzzle using the physical model. The finding that participants were able to solve the Tower of Hanoi in fewer moves with a mental representation is interesting. It suggests that problem-solving ‘in the head’ is more efficient than using a computer. The number of moves taken in this first experiment resulted in a range across the three conditions with the physical condition taking the most moves followed by the computer and mental representations. The computer presentation of the Tower of Hanoi puzzle provides a means of representing the problem pictorially. Thus, it provides an ‘intermediate representation’ between the physical and mental models. This would suggest that the physical model should provide the most benefits in terms of helping us solve the problem. Being able to feel the disks, and physically move them around should have reduced the number of moves. The advantages of making an abstract representation into a more concrete one has been shown to be of
J.M. Noyes, K.J. Garland / Computers in Human Behavior 19 (2003) 579–592
587
benefit in problem-solving (Beveridge & Parkins, 1987). For example, Reed and Ettinger (1987) found that tables were useful for problem solvers because it provided them with concrete information. However, the explanation in this research probably lies with the design of the computer-generated Tower of Hanoi puzzle. Manipulation of the computer version was very easy and involved a ‘drag and drop’ mouse operation to move the disks on the screen. Thus, individuals could very quickly elicit the desired moves; perhaps, this ease of operation resulted in them not focusing on reaching the end-point by the most efficient means, and as a result, a ‘trial and error’ approach was being adopted. As Larkin (1989) pointed out, the computer program for the Tower of Hanoi is flexible in that it allows the problem to be solved in a variety of ways. Hence, problem-solvers do not have to be too careful about making sure the next move is the right one. However, the advantage demonstrated in Experiment 3 of transferring from the computer to the mental representation of the problem (rather than vice versa) might be explained by the benefits accrued from using the more concrete version of the puzzle. The transfer effects are not easy to explain. In isomorphic problems, experience with solving a problem facilitates further attempts at solving the same problem. This has been well documented (see Luger & Bauer, 1978; Reed, Ernst, & Banerji, 1974). Hence, it would have been expected that individuals on their second attempts would be able to solve the problem in a fewer number of moves. Although this occurred when moving from the computer to the mental representation, it did not happen when participants transferred from the mental to the computer version of the Tower of Hanoi. This latter finding is counter-intuitive, as the extra cognitive effort that went into solving the mental representation first should have benefited any second attempt. However, Reed et al. stated that facilitation only occurs if the second problem is easier than the first; perhaps the computer model did not present participants with a problem of equivalent difficulty. The display may have provided interference with the result that participants did not fully apply themselves to the problem-solving task in this medium. Several participants reported how they had visualized the Tower of Hanoi puzzle in the mental representation condition. This may have then made it difficult for them to solve the problem when confronted with a computer-generated model. In all three experiments, participants were faster when using the computer version of the puzzle in terms of moves per second. Again, the design of the computer display may explain these results: the use of the mouse allowed participants to move the disks with speed and relative ease. It has been suggested that working memory may be involved in successful performance of the Tower of Hanoi puzzle (Welsh, Cicerello, Cuneo, & Brennan, 1995). When attempting to solve the puzzle, individuals reach the limits of their working memory; they are only able to ‘see’ the state of the disks on the pegs a few moves ahead. Further, they may not remember the moves previously made, so are unable to apply strategies that may have proved previously to be successful in moving the disks. One of the reasons why Welsh et al. suggested working memory was involved, arose as a result of their observations of individuals solving the Tower of Hanoi. Interestingly, they found in their work with a four disk (15-move) puzzle that there
588
J.M. Noyes, K.J. Garland / Computers in Human Behavior 19 (2003) 579–592
were certain points, that is, at the first, fifth and nineth moves, where errors were likely to be made. They also measured pause times and found that even optimal problem-solvers increased their pause time before these junctures. When asked what they were doing, participants replied that they were planning moves during these pause times (although whether or not they reported this planning activity did not correlate with successful performance). Although pause times were not measured in the experiments reported here, they may have been a useful adjunct in explaining the differences in the results. Furthermore, there was some evidence from the verbal reports that some individuals were pausing around moves 14–17. In terms of working memory loads, using the computer version of the Tower of Hanoi should have placed participants at an advantage since the on-screen visualization of the puzzle would have reduced these loads (Van Oostendorp & Walbeehm, 1995). Information that needed to be remembered and that otherwise would have had to be held in working memory was displayed on the screen for as long as the problem-solver desired. For example: The goal of moving all the disks to the other end peg is evident. Information relating to the state of the problem is always present. The display ‘performs’ any legal action required, and in turn, it simply blocks moves that are not allowed, for example, trying to move a disk that is not on the top of the peg, or trying to move a disk to a location where the resident disk is smaller, or trying to put a disk to one side. If moves are made in error, they are immediately visible on the screen, and thus, the problem-solver is able to continue from the current state. In the case of the Tower of Hanoi puzzle, making the moves relies on the internal working memory, but the person also needs to apply the restriction rules to making the moves. This information, although not shown on the screen in the same way as the puzzle, is ‘present’ in the computer; hence, the user has access to an ‘external (working) memory’. Van Oostendorp and Walbeehm described this as ‘intermediate information’ that is being kept in an external working memory. Further, there is little cost for interruption of carrying out the task as information is not lost, for example, if concentration momentarily lapses. This may help explain the greater number of moves when using the computer version. Participants had so much information present on the display screen that there was no need to be totally focused on solving the problem. In essence, it could be argued that display-based problem solving reduces the complexity of the mental processes involved by reducing the loads on working memory. Certainly, participants using the mental representation reported the following: ‘‘very difficult to visualize in head and keep strategy’’, ‘‘easy to forget where blocks were’’, and ‘‘saying name out loud interferes with problem-solving’’. Although a relatively simple puzzle, Simon (1975) wrote about the ‘sophisticated perceptual strategy’ that was needed to solve the Tower of Hanoi. Each move is directly cued by the visible state of the physical problem; hence, at any point, the users need to have access to a representation of the problem state. This, of course, is
J.M. Noyes, K.J. Garland / Computers in Human Behavior 19 (2003) 579–592
589
much easier if you are looking at the representation of the puzzle on the computer. Larkin (1989) suggested that it is possible merely to look at the display and to do what is implied in terms of carrying out the next step in the puzzle without ‘effortful mental calculation’ or the need for storage of information. This may be borne out by the fact that more people solved the problem when using the computer presentation, that is, they kept going (and indeed, were able to keep going) until they reached the goal state. These assertions about the computer-generated version of the problem may help explain the differences we found between the modes of presentation. Given the many benefits of the computer presentation of the puzzle, it is curious that people were solving the problem ‘in their head’ with a fewer number of moves. Robertson (2001) stated that the first activity carried out by a person when confronted with a problem is the generation of an internal representation of the particular problem. When presented with a computer model for the Tower of Hanoi, there is no need to make any effort to form your own mental representation, because there is an external representation on the display screen. Consequently, the problemsolver faced with the computer version of the problem is immediately at a disadvantage because they are not having the benefit of having to apply themselves to beginning to solve the problem. Further, the computer’s representation of the problem may not match their internal representation of the problem. In effect, the computer model may be providing so-called ‘cognitive clutter’ that is interfering with the optimum route for problem solving. In contrast, solving the problem using only a mental representation allows you to build a strong representation of the problem, and this results in more efficient problem solving. Moreover, as already discussed, the transfer effects of moving between the two conditions support this. A further explanation may lie in the use of verbal protocols. Individuals solving the puzzle using the mental representation version of the Tower of Hanoi were required to talk through the moves they were making, that is, to think aloud. This form of protocol analysis is particularly appropriate to transformation problems such as the Tower of Hanoi. These problems create situations, which the person finds easy to verbalize (Byrne, 1977). Further, it has also been previously used when solving the Tower of Hanoi. For example, Simon and Hayes (1976) concluded that ‘thinking aloud’ whilst the participants were actually working out the Tower of Hanoi not only provided a rich source of information in understanding how people solve problems, but enhanced the problem-solving skills of the individual. It is a reasonable assumption, therefore, that providing the verbal protocol by thinking aloud whilst solving this particular puzzle was of benefit to participants solving the problem ‘in their head’. However, examination of the qualitative data does not indicate this as participants reported generating the verbal protocols as ‘getting in the way’. Although some participants felt this, it may still have enhanced their solving of the puzzle, and might help explain why people were so efficient at solving the Tower of Hanoi when they had only a mental representation. The studies reported here do not provide any information concerning the strategies used by the problem-solvers, although some inferences can be made from both the verbal protocols and the qualitative data collected in the third experiment. In contrast, Simon (1975) carried out a detailed analysis of strategies employed to solve
590
J.M. Noyes, K.J. Garland / Computers in Human Behavior 19 (2003) 579–592
the Tower of Hanoi. Similarly, Anzai and Simon (1979) carried out an in-depth study of one person solving a five disk, Tower of Hanoi puzzle, four times during a one and a half-hour period. Interestingly, it was found that each time this participant attempted the puzzle, she adopted a different strategy that was more efficient than the previous one. Cagan and Kotovsky (1997) in their studies on problem-solving behaviour observed that individuals begin by searching more or less randomly. Over time they become more deterministic as they learn more about the problem. They described this as a two-phase aspect of problem-solving behaviour. Although we do not have the empirical evidence, perhaps problem-solvers using the computer and physical versions spent longer in the first phase, and subsequently, generated more moves. Certainly, some of the raw data indicates that a few individuals were making a large number of moves (in excess of 100) to solve a problem that only required 31 moves. Given the efficiency that our participants solved the puzzle with only a mental representation, this suggests that they are more likely to have adopted a strategy than when using the computer version. The qualitative data does provide some evidence for this. For example, ‘trial and error’ was cited as a strategy being used, but was being discarded on the second attempt as it was not found to be particularly beneficial. This was more likely to be the case in the mental representation condition. Remembering the moves was also an aspect of the first attempt that people applied in the second, and perhaps the greater cognitive effort needed to solve the mental representation meant that people were more able to remember the moves. However, it is also evident from this data that many participants had no particular strategy: for example, ‘‘knew vaguely what was doing, but no definite technique or strategy’’, ‘‘followed no particular strategy’’, and ‘‘no strategy apparent!’’. One of the difficulties associated with the use of any computer-generated model is the nature of the interface. This is particularly the case when considering problemsolving as the ergonomics of the display and the user’s interaction with it can influence the ease with which the problem can be solved. Larkin (1989) pointed out the importance of the interface design in influencing computational efficiency, and this has been demonstrated for the Tower of Hanoi and other isomorphs (Hayes & Simon, 1976; Kotovsky et al., 1985). The importance of the design of the interface must not be overlooked, because as Zhang (1991) pointed out the external representations of the problem provide memory aids. Hence, the design of these can change the nature of the task. The precise design of the computer-generated Tower of Hanoi will, therefore, influence the solving of the puzzle. This needs to be taken into account when generalizing from Tower of Hanoi studies. A further consideration is the extent to which wider inferences about problem solving can be made from a simple, transformation problem such as the Tower of Hanoi. The use of various types of puzzles in problem-solving research could be argued to be very remote from the day-to-day problems that humans encounter, and could, therefore, be seen to be lacking in ecological validity. However, problems in the ‘real world’ frequently involve considerable time to solve, if indeed, they are solved at all. The puzzles used in experimental research have a greater likelihood of being solved, because their well-defined structure clarifies both the initial and goal
J.M. Noyes, K.J. Garland / Computers in Human Behavior 19 (2003) 579–592
591
states, and identifies constraints (both legal operators and restrictions). In addition, they have discrete stages that can be easily observed, and ‘moves’ tend to be completed at a pace suitable for the participants to verbalize their behaviour. Hence, the simplicity of the Tower of Hanoi problem provides a good starting point, as we need first to find out about the differences between computer-generated and other forms of the problem before progressing to more complex problems. Moreover, there are a number of well-known, transformation problems (e.g. Hobbits & Orcs, book burners and book lovers, missionaries and cannibals) and it would be interesting to see if the findings reported here would be replicated with other puzzles (see Reed et al., 1974). In summary, having a physical representation of the Tower of Hanoi problem was not found to be helpful in solving the puzzle. In contrast, using a display-based computer generated model was likely to lead to success in the shortest times (but with the greatest number of moves). Various features of the display-based presentation were thought to explain these findings. However, the most efficient way of solving the problem, i.e. with the fewest number of moves, was with a mental representation of the puzzle. However, the costs are that it will take you longer and you are more likely to lose where you are in the problem, and have to give up. Mental representation of the puzzle demanded that individuals ‘talked’ through solving the problem, and this may help explain the efficiency of this approach. References Ahlum-heath, M. E., & DiVesta, F. J. (1986). The effects of conscious controlled verbalization of a cognitive strategy on transfer in problem solving. Memory and Cognition, 14, 281–285. Anderson, J. R., & Douglass, S. (2001). Tower of Hanoi: Evidence for the cost of goal retrieval. Journal of Experimental Psychology—Learning, Memory and Cognition, 27(6), 1331–1346. Anzai, Y., & Simon, H. A. (1979). The theory of learning by doing. Psychological Review, 86(2), 124–140. Baddeley, A. D., & Hitch, G. (1974). Working memory. In G. A. Bower (Ed.), Recent advances in learning and motivation (Vol. 8). New York: Academic Press. Berardicoletta, B., Dominowski, R. L., Buyer, L. S., & Rellinger, E. R. (1995). Metacognition and problem-solving—a process-oriented approach. Journal of Experimental Psychology—Learning, Memory and Cognition, 21(1), 205–223. Berry, D. C. (1983). Metacognitive experience and transfer of logical reasoning. Quarterly Journal of Experimental Psychology, 35A, 39–49. Beveridge, M., & Parkins, E. (1987). Visual representation in analogical problem solving. Memory and Cognition, 15(3), 230–237. Byrne, R. (1977). Planning meals: problem-solving on a real data-base. Cognition, 5, 285–332. Cagan, J., & Kotovsky, K. (1997). Simulated annealing and the generation of the objective function: a model of learning during problem solving. Computational Intelligence, 13(4), 534–581. Chambers, P. (1999). Information handling skills, cognition and new technologies. British Journal of Educational Technology, 30(2), 151–162. Davies, S. P. (2000). Move evaluation as a predictor and moderator of success in solutions to well-structured problems. Quarterly Journal of Experimental Psychology Section A—Human Experimental Psychology, 53(4), 1186–1201. Davis, H. P., & Klebe, K. J. (2001). A longitudinal study of the performance of the elderly and young on the Tower of Hanoi puzzle and Rey recall. Brain and Cognition, 46(1–2), 95–99. De Mul, S., & Van Oostendorp, H. (1996). Learning user interfaces by exploration. Acta Psychologica, 91(3), 325–344.
592
J.M. Noyes, K.J. Garland / Computers in Human Behavior 19 (2003) 579–592
Ericsson, K. A., & Simon, H. A. (1980). Verbal reports as data. Psychological Review, 87(3), 215–251. Ericsson, K. A., & Simon, H. A. (1993). Protocol analysis: verbal reports as data (rev. ed). Cambridge, MA: MIT Press. Hayes, J. R., & Simon, H. A. (1976). The understanding process: problem isomorphs. Cognitive Psychology, 8, 165–190. Kirchner, P. A., & Paas, F. (2001). Web-enhanced higher education: A tower of Babel. Computers in Human Behavior, 17, 347–353. Kotovsky, K., Hayes, J. R., & Simon, H. A. (1985). Why are some problems hard? Evidence from the Tower of Hanoi. Cognitive Psychology, 17, 248–294. Larkin, J. H. (1989). Display-based problem solving. In D. Klahr, & K. Kotovsky (Eds.), Complex information processing: the impact of Herbert A. Simon. Hillsdale, NJ: LEA. Luger, G. F., & Bauer, M. A. (1978). Transfer effects in isomorphic problem situations. Acta Psychologica, 42, 121–131. Newell, A., & Simon, H. A. (1972). Human problem solving. Englewood Cliffs, NJ: Prentice-Hall. Nisbett, R. E., & Wilson, T. D. (1977). Telling more than we can know: verbal reports on mental processes. Psychological Review, 84(3), 231–259. Reed, S. K., & Ettinger, M. (1987). Usefulness of tables for solving word problems. Cognition and Instruction, 4(1), 43–58. Reed, S. K., Ernst, G. W., & Banerji, R. (1974). The role of analogy in transfer between similar problem states. Cognitive Psychology, 6, 436–450. Robertson, S. I. (2001). Problem solving. Hove, UK: Psychology Press. Sarkar, S., Chakrabarti, P. P., & Ghose, S. (1998). Learning while solving problems in best first search. IEEE Transactions on Systems, Man and Cybernetics, Part A—Systems and Humans, 28(4), 535–542. Simon, H. A. (1975). The functional equivalence of problem solving skills. Cognitive Psychology, 7, 268– 288. Simon, H. A. (1978). Information-processing theory of human problem solving. In W. K. Estes (Ed.), Handbook of learning and cognitive processes (pp. 271–295). Hillsdale, NJ: LEA. Simon, H. A., & Hayes, J. R. (1976). The understanding process: problem isomorphs. Cognitive Psychology, 8, 165–190. Van Oostendorp, H., & De Mul, S. (1999). Learning by exploration: thinking aloud while exploring an information system. Instructional Science, 27(3–4), 269–284. Van Oostendorp, H., & Walbeehm, B. J. (1995). Towards modelling exploratory learning in the context of direct manipulation interfaces. Interacting with Computers, 7(1), 3–24. Welsh, M., Cicerello, A., Cuneo, K., & Brennan, M. (1995). Error and temporal patterns in Tower of Hanoi performance—cognitive mechanisms and individual differences. Journal of General Psychology, 122(1), 69–81. Zhang, J. (1991). The interaction of internal and external representations in a problem solving task. In Proceedings of 13th Annual Conference of Cognitive Science Society (pp. 954–958). Chicago, IL: LEA.