The ebb and flow of online learning

Computers in Human Behavior Computers in Human Behavior 21 (2005) 745–771 www.elsevier.com/locate/comphumbeh

The ebb and flow of online learning Jon M. Pearce a

a,*

, Mary Ainley b, Steve Howard

a

Department of Information Systems, The University of Melbourne, Melbourne 3010, Australia b Psychology Department, The University of Melbourne, Melbourne 3010, Australia Available online 14 March 2004

Abstract Past research has suggested that CsikszentmihalyiÕs flow theory describes a state that should be supportive of a studentÕs learning. This paper reports on research that uses the constructs of flow to explore learning in an online environment. An experiment was carried out in which students worked through a learning sequence in the physics domain that had varying degrees of interactivity. Their interactions and flow states were monitored throughout the learning task. The experimental data suggest that flow can be more usefully regarded as a process rather than just an overall state. This process is represented by flow-paths that plot each studentÕs progress through challenge-skill space. Some flow patterns are identified that relate to the learning outcomes of the students. While there is some conflict between this process representation and outcome measures for flow, this flow-path portrayal has provided fresh insights into studentsÕ interactions in online learning environments.
2004 Elsevier Ltd. All rights reserved. Keywords: Flow; Learning; Engagement; Computer-based instruction

1. Introduction One of the challenges of designing online learning tasks is to engage students and keep them engaged through the duration of the task when the rest of the Web is merely a click away. This paper reports on research in which we explored the behaviour of students engaged in such tasks by employing a model of engagement

*

Corresponding author. Fax: +613-9349-4596. E-mail address: [email protected] (J.M. Pearce).

0747-5632/$ - see front matter
2004 Elsevier Ltd. All rights reserved. doi:10.1016/j.chb.2004.02.019

746

J.M. Pearce et al. / Computers in Human Behavior 21 (2005) 745–771

called ÔflowÕ (Csikszentmihalyi, 1975). The concept of flow has been used in many different situations as a theoretical model on which to base such understanding. Research in this area often describes the frequency of flow states over an extended period of time (several days) or experiments in which a measure of flow is established after the event by interview or survey. In the research presented here we have taken a very different approach: we have focussed on the process of flow rather than the state of flow. We have done this in order to understand the complex changes that happen to students during online learning. An approach based on the measurement of the overall-state is of little value in an education setting where one might want to understand how students react to learning materials during a session of relatively short duration. We speculate, on the basis of previous research, that during such sessions students will move in and out of flow according to the nature of the materials presented, their interest in them, and their own ability to cope with the tasks. Yet past research pays little attention to this process nor, in particular, to how the measures of flow vary as the task progresses. We describe a method for representing the frequently changing sequence of states that students exhibit by monitoring their flow states at several times during a challenging online learning task. The Web-based task offered guided instruction in physics enhanced with different degrees of interactive multimedia. We explore the measurement of flow by adopting a process view in which a studentÕs state is monitored throughout the task and contrast it with an aggregate of factors obtained at the end. We are interested in any link between flow and learning as well as any impact the degree of control offered by the software might have on the flow process. From this analysis we can move away from the Ôhow much did you flowÕ question and start to address issues relating to the nature of the interactions that encourage flow and how learning outcomes may or may not be related to them. Through this we can start to appreciate the complex path that students take through a learning exercise as they may ebb in and out of flow states. We can begin to address the vexed issue of designing tasks that maintain studentsÕ engagement in online environments. The next section describes some models of flow and measurement techniques, including the measurement methods used in this research. Section 3 describes the experiment in which an online learning task was presented to undergraduate students. In Section 4, we analyse the results and present a view of flow based on process rather than an overall state judgement. Section 5 discusses these results and Section 6 presents a conclusion to the paper.

2. Flow models and measurement 2.1. Describing flow In 1995, Csikszentmihalyi coined the term ÔflowÕ to refer to Ôoptimal experienceÕ events (Csikszentmihalyi, 1975). The earliest writings on flow have signalled the

J.M. Pearce et al. / Computers in Human Behavior 21 (2005) 745–771

747

expectation that flow is particularly important in an educational setting. More recent research has supported this notion by showing that flow occurred more often during study and schoolwork than other daily activities of Italian teenagers (Massimini & Carli, 1988). Flow describes a state of complete absorption or engagement in an activity. The term was introduced through the study of people involved in common activities such as rock climbing, dancing, chess, etc. (Csikszentmihalyi, 1975). The concept has been studied in a wide range of endeavours spanning the disciplines of HCI, psychology, information systems and education. For example: Web use and navigation (Chen & Nilan, 1998; Chen, Wigand, & Nilan, 1998; Chen, Wigand, & Nilan, 1999; Pace, 2000); Web marketing (Hoffman & Novak, 1996; Novak, Hoffman, & Yung, 2000); in everyday life (Csikszentmihalyi, 1975, 1997); in group work (Ghani, Supnick, & Rooney, 1991); technology use in information systems (Agarwal & Karahanna, 2000; Artz, 1996); in HCI (Webster, Trevino, & Ryan, 1993); and in instructional design (Chan & Ahern, 1999; Konradt & Sulz, 2001; Kronradt, Filip, & Hoffmann, 2003). A Ôflow activityÕ is one in which the mind becomes effortlessly focussed and engaged on an activity, rather than falling prey to distractions. Flow is not an Ôall-or-nothingÕ state, but can be thought of as forming a continuum from no flow to maximum flow. Csikszentmihalyi originally summarised flow experiences as comprising nine elements (Csikszentmihalyi, 1975, 1993). Chen observed that these dimensions can be categorised into three stages: antecedents, experiences and effects (Chen et al., 1999). The antecedents comprise: a clear set of goals; timely and appropriate feedback; and, most importantly, a perception of challenges that are well matched to the personÕs skills. The second stage, experience, comprises: a merging of action and awareness; a sense of control over the activity; and concentration. The final stage describes the individualÕs inner experience: loss of self-consciousness; time distortion; and a feeling that the activity becomes worth doing for its own sake (ÔautotelicÕ). The nature of flow is elegantly put by Massimini: The stream of ordinary experiences, ranging from the faintly pleasant to the boring, and the anxious, is made up of a random collection of discordant notes. Occasionally the notes fall into a harmonious chord – when that happens, information in the consciousness is ordered, and we experience flow.Õ (Massimini & Carli, 1988).

2.2. Models of flow 2.2.1. Overall-state models Csikszentmihalyi, in his early work, described flow as Ôthe holistic sensation that people feel when they can act with total involvementÕ (Csikszentmihalyi, 1975). People were identified as being Ôin flowÕ through interviews and questionnaires about specific activities. The interviewers were looking for evidence of the nine elements of flow mentioned earlier as well as affective attributes indicating a sense of high wellbeing – an Ôoptimal experienceÕ. The flow state can be represented as a ÔchannelÕ on a plot of challenge versus skills, separating the states of anxiety and boredom (Fig. 1). It is a dynamic quality: if the

748

J.M. Pearce et al. / Computers in Human Behavior 21 (2005) 745–771

challenge of a task decreases, it might become boring; if the challenge increases but oneÕs skills do not improve to meet the challenge, then one might get into a state of anxiety. A learning activity might produce a progression up the flow channel as new skills are learnt and greater challenges are sought on which to exercise those skills (Csikszentmihalyi, 1975). The Ôflow channelÕ shows flow ranging from low complexity (ÔmicroflowÕ, at the lower end) to high complexity (ÔmacroflowÕ, at the upper end). This model suggests that only the relative balance of challenge and skill is relevant to flow, not the absolute values. That is, an activity could offer very little challenge, yet could still produce flow if the skills of the person are commensurately low. This model has been refined to include 4, 8 and even 16 different states, usually still referred to as ÔchannelsÕ (Massimini & Carli, 1988). These are discussed later. A complementary approach is to construct a model that represents how particular variables affect flow. Ghani et al. (1991) used a model defining flow as a two-dimensional construct of enjoyment and involvement. They found that two of the predictors for flow were perceived control and perceived challenge. Trevino and Webster (1992) used a four-dimensional model of control, focus of attention, curiosity and cognitive enjoyment. Novak, Hoffman, and Yung (1997) derive a simple conceptualisation of flow based on the measurement of skills and challenges alone. 2.2.2. Limitations to the overall-state approach Models of flow based on the overall-state approach have demonstrated the importance of control, skill and challenge as significant factors for the study of flow. In contrast, we have focussed on the process of flow exploring how flow might change across a learning episode and what factors might be associated with those changes. What we have called a process approach assumes that no single measure will adequately describe the emotional changes experienced by a student during the learning task. Ainley has shown that studentsÕ personal characteristics will influence their initial response to a learning task and then the task itself will

Fig. 1. CsikszentmihalyiÕs original model of flow.

J.M. Pearce et al. / Computers in Human Behavior 21 (2005) 745–771

749

trigger emotions and specific task goals (Ainley, 2002). Her data indicated that students report changing emotions such as boredom, pleasure, interest, challenge, surprise, confusion and frustration, as the task progresses. AinleyÕs research measured studentsÕ emotions at just two times in the learning sequence and showed that some emotions often changed markedly between those times. This suggests that a more fine-grained measurement throughout the task would shed further light on the interaction between the task and the studentÕs engagement with it. By regarding flow as a process or succession of states we have been able to explore changes throughout a task and to understand some of the factors associated with those changes. 2.3. Measuring flow 2.3.1. Techniques Several techniques have been used to measure flow. CsikszentmihalyiÕs original study on flow involved people telling the story of a recent experience, then responding to a survey questionnaire relating to the elements of flow. From this the Experience Sampling Method (ESM) was developed in which respondents were electronically paged maybe 8 times a day for a week to prompt them to respond to a questionnaire (Csikszentmihalyi & Larson, 1987; Csikszentmihalyi, Larson, & Prescott, 1977). A digital implementation of the ESM (Chen & Nilan, 1998) has been used to randomly survey users going about everyday Web activities. This involved an online tool that interrupted users during their work, presenting them with a series of questions. Others have investigated flow by providing a structured activity in a laboratory or workplace, then followed it with a survey (Ghani et al., 1991; Webster et al., 1993). Some have used Web-based surveys to gather retrospective information about flow activities on the Web and in the workplace (Novak et al., 2000). From these studies conceptual models have been developed to describe the flow construct. What appears to be lacking in these studies is a process view of the progression through a particular activity. This is important in the design of online learning activities since one might aim to trigger a flow state to optimise conditions for learning. Yet we know little about how such states change during a learning task nor what conditions influence the change. Without an understanding of this process we cannot use the theory of flow to improve learning exercises. In the research described here we captured data describing the process of working through a specific learning activity. We explored ways of representing a dynamic picture of the perceived challenge and skills during the activity and related these measures to the coarser-grained models of flow such as those of Webster (Webster et al., 1993) and Ghani (Ghani et al., 1991). 2.3.2. Challenge and skills Amongst the various studies researching flow, an ongoing issue has been to find a method for measuring flow independently from the positive states of consciousness (such as enjoyment, concentration, control, lack of self-consciousness, lack of

750

J.M. Pearce et al. / Computers in Human Behavior 21 (2005) 745–771

distraction). One solution has been to use a measure of the balance between the challenge of an activity and the participantÕs perception of their skill to carry out that activity. The perception of these challenges and skills has been described as Ôtheoretically, the most meaningful reference point for the presence or absence of flowÕ (Massimini & Carli, 1988). CsikszentmihalyiÕs early work predicted that flow was a single ÔchannelÕ separating the two states of anxiety and boredom as shown in Fig. 1 (Csikszentmihalyi, 1975). According to this model a challenge–skill ratio of 1:1 should indicate flow. It is now common among researchers to use models with more than 3 channels and to normalise a challenge–skill plot to the individualÕs average challenge–skill value over the extended period of the study. Fig. 2 shows a four-channel model and Fig. 3 an eight-

Fig. 2. Four-channel model of flow.

Fig. 3. Eight-channel model of flow (Massimini & Carli, 1988).

J.M. Pearce et al. / Computers in Human Behavior 21 (2005) 745–771

751

channel model together with the rules that define them. In such models, both challenge and skill should be above the threshold (average value) for flow to take place (Massimini & Carli, 1988). The original flow model of Fig. 1 is not contradicted by these four-channel or eight-channel models. Indeed, in the context of learning activities, it has an important place in that it represents how the process of flow might develop through a single activity. Initially, as a novice, one might have minimal skills and need appropriately low challenges in order to help develop better skills; these ‘‘novice’’ experiences are sometimes referred to as ÔmicroflowÕ (Csikszentmihalyi, 1975). As the skills increase, so must the challenges in order to maintain flow. Hence, the model shows the time progression as one continues to learn a new skill or technique and progresses up the flow channel. If the challenges increase too fast, then there is danger of anxiety; if the skills develop quickly and challenges are not increased there is a danger of boredom occurring. 2.4. A process measure of flow 2.4.1. Measuring challenge and skill We used the variables skill and challenge as primary data for our repeated measures of flow during the learning task. These have been reported to be reliable indicators for measuring flow (Novak & Hoffman, 1997). The process measure was then compared with end-of-task measures based on control, enjoyment and engagement. The various flow models raise an important question in the use of the challenge–skill ratio when studying flow in a short-time educational activity, such as the one-hour learning task used in this research: how does one determine appropriate thresholds of challenge and skill from which to ÔnormaliseÕ the flow diagram? For weeklong experiments that monitor flow throughout the day, an individualÕs average is a clearly appropriate choice. It represents the ÔbackgroundÕ levels for an individual and flow experiences can be expected to rise above such levels. However, there is no such valid measure available for the 60 or 70 min that a student might undertake a learning activity. A session-average value of the challenge–skill ratio would make a student who experienced flow through an entire activity look the same as one who was consistently bored. This study, rather than attempting to attach fine-grained labels to the overall state of an individualÕs performance during an activity, takes the approach of describing the progress of the individual in terms of challenges and skills during the activity. Participants worked through a learning task comprising seven activities lasting about one hour in total. At the end of each of the seven activities a probe, using 5-point Likert scale ratings, was presented to record their perceptions of challenge and skill. The probes were positioned one above the other on the screen as shown in Fig. 4 to increase the likelihood that participants would use consistent referents for both scales. Data from these probes were used to map the movement of each student through the two-dimensional challenge–skills space during their interaction with the activities. This was consistent with CsikszentmihalyiÕs model of a learner moving up the flow channel as they progress through a learning exercise.

752

J.M. Pearce et al. / Computers in Human Behavior 21 (2005) 745–771

Fig. 4. Probe used to measure challenge and skills.

In categorising studentsÕ flow-states from these challenge–skill measurements, CsikszentmihalyiÕs three-channel model of flow was used. This is consistent with his ideas that the flow channel appropriately represents the movement of a learner through an activity. Students were considered to be in flow when their rating of challenge and skill scores were equal on a 5-point Likert scale, to be anxious when challenge was greater than skills, and to be bored when challenge was lower than skills, as represented in Fig. 1. 2.4.2. An overall-state measure of flow An alternative overall-state measure of flow was obtained using an 11-item survey administered at the end of the learning exercise. Ratings of engagement, enjoyment and perceived control using 5-point Likert scales were collected. Previous work has proposed that these indicators can be used to provide an overall impression of flow during learning (Trevino & Webster, 1992). Appendix A lists the 11 survey questions used. Factor analysis was used to identify common factors from these questions. This provided a much coarser–grain view of the userÕs state, but a reflective one more in line with CsikszentmihalyiÕs original work on flow. More details on these measures are given in Section 4.2.

3. Description of experiment 3.1. Aims The experiment explored the relationships between flow, perceived control and learning, for a group of students engaged in an online learning task. By Ôperceived controlÕ we refer to the degree to which students considered themselves to be in control of what they were doing, as opposed to being ÔdrivenÕ by the software. In this paper, we focus on the specific approaches to measuring flow in such an online educational setting and make brief references to learning

J.M. Pearce et al. / Computers in Human Behavior 21 (2005) 745–771

753

outcomes. A more detailed coverage of issues relating to learning will be addressed elsewhere. The particular aims of the experiment were: 1. To explore the changing nature of studentsÕ challenge–skill perceptions through an online learning task. 2. To compare flow measured using challenge–skill ratios to flow measured from post-survey data.

3.2. The online learning exercise The learning exercise involved students working through a sequence of pages presenting physics ideas about velocity and acceleration as shown by the motion of a cart. There were seven activities that began with students observing an animation of a motion and then sketching on paper their prediction of what velocity-time and acceleration-time graphs of the motion would look like. For each of these seven motions the students were presented with a page that guided them to explore the motion and correct, if necessary, their predicted graphs based on what they explored. One aim of the research was to explore the effect of varying the degree of control that students had over their environment. Hence, they were randomly divided into two groups. One group, the simulation group, worked with a true simulation that allowed them to freely manipulate the simulation: drag and release the cart to set it moving; vary the slope of the track; add fans to the cart to produce acceleration; and change the scales of the graphs. The other group, the movie group, worked through the same activities except that the simulation was replaced by three or four video clips that presented animated screen-shots of the simulation showing the cart motions relevant to each particular activity. Participants could play these clips as often as they liked. The movie group was expected to perceive a lower degree of control than the simulation group because they were unable to directly manipulate the simulations. A screen-shot from the exercise, showing the interactive simulation on the left and instructions on the right, is shown in Fig. 5. Csikszentmihalyi comments on the importance to flow of intrinsic motivation, active learning, and feedback that is informational rather than controlling (Csikszentmihalyi, Rathunde, & Whalen, 1997). The activities presented in this experiment provided intrinsic motivation through presenting an engaging and interesting multimedia object – seeing a cart run along a track, graphs being drawn, movies to watch. Active learning was encouraged through the interactive nature of the activities, namely, some fairly limited Web-page navigation, but more importantly the opportunity to play movies and (for some) interact with the simulation. The feedback provided was informational consisting of velocity and acceleration graphs being drawn dynamically as the cartÕs motion progressed. Through the design of this learning exercise we set up an environment that had the potential to deeply engage students in an enjoyable activity, thereby encouraging flow.

754

J.M. Pearce et al. / Computers in Human Behavior 21 (2005) 745–771

Fig. 5. Screen from one of the learning activities.

3.3. Method 3.3.1. Participants The participants for the research were 42 first year Information Systems (IS) students and 17 first year Physics students at a leading Australian university. Fortyfive percent of the participants were male students and 55% female students. All were paid volunteers and spent about one hour working through the set exercise in a computer lab. All the participants had studied physics at some time during the last two years of their secondary schooling; the physics students had also been exposed to the area of physics presented by the experimental materials earlier in the year as part of their university course. 3.3.2. Procedure Participants were randomly allocated to the simulation group or the movie group as they entered the laboratory session. All instructions were on a Web site that guided them through introductory materials, learning activities and post-test. Each participant worked individually. A small group of students was interviewed about their experience immediately after they finished the learning exercise. 3.3.3. Data collection Fig. 6 shows the online information-gathering Web-pages presented to participants before, during and after the learning task. They were first presented with a

J.M. Pearce et al. / Computers in Human Behavior 21 (2005) 745–771

755

Fig. 6. Progress through the activity.

screen requesting general background information about themselves followed by a pre-test to establish their prior knowledge of this area of physics (19 multiplechoice items). After an introductory screen, they worked through the seven physics learning pages each immediately followed by the challenge–skill probe asking them to rate, on a 5-point Likert scale, how challenging they found the last activity and their perceived skills to meet the challenge. This probe can be seen in the screenshot in Fig. 5. Finally, they were presented with a post-survey, which gathered information about affective aspects of their experience (see Appendix A), and a post-test which measured learning gains using the same question items as the pre-test. Each participantsÕ movement through the Web pages and their detailed interactions with the simulation were recorded using Web-logs. Together, these logs recorded page navigation, mouse clicks, movies played, actions within the simulation as well as the time spent directly manipulating the simulation (dragging the cart, in this instance).

4. Analysis of experiment The two sets of analyses reported here relate to the more traditional survey data gathered at the end of the task and the tracking of challenge–skill data throughout the task. The former presents an overall-state perceptive on participantsÕ flow experiences whilst the latter provides a process perspective. 4.1. Mapping challenge and skills onto flow space A question arises in relation to the definition of flow. Should an indication of flow be restricted to a balance of challenge and skill, but only at high levels? Or should a balance indicate flow even if the levels are relatively low? Csikszentmihalyi gives a clue to the answer in his defence of his original 3-channel model compared to a normalised 4-channel model: (It) is a composite diachronic model illustrating how the flow experience proceeds through time, in a single activity, (original emphasis) from enjoyment of small challenges when a

756

J.M. Pearce et al. / Computers in Human Behavior 21 (2005) 745–771 personÕs skills are limited, to an ever-complexifying enjoyment of higher challenges requiring increasingly rare skills (Csikszentmihalyi & Csikszentmihalyi, 1988, p, 261).

We argue that in a learning context it is crucial that a flow model allows for the situation in which a novice with low skills might flow whilst interacting with an appropriate low challenge task. It is thus considered appropriate to use a model that interprets flow as happening whenever challenge and skill are balanced, whether the individual values are high or low. Whether the lower extreme of these values (challenge ¼ 1, skill ¼ 1) should be regarded as flow or apathy is unresolved. The first step in mapping the challenge and skill ratings was to cross tabulate all seven pairs of responses from each of the 59 participants. The resulting patterns are shown in Table 1, which is arranged to make easy comparison with the quadrants of the challenge–skill plot of Fig. 2. Some participants did not complete all ratings. The challenge and skill rating pairs were then classified as indicators of anxiety, flow, or boredom as defined by the 3-channel model of flow (see Fig. 7). As indicated in Fig. 7, flow accounted for about one-quarter of the 399 recorded experiences. There

Table 1 Likert scale scores for 59 students on skill/challenge measures (7 measures each)

Challenge

5 4 3 2 1

20 11 7 1 0 1

4 52 53 18 0 2

0 7 107 29 2 3 Skill

Fig. 7. Frequency of flow experiences.

0 3 21 29 9 4

1 0 1 1 20 5

J.M. Pearce et al. / Computers in Human Behavior 21 (2005) 745–771

757

were also a substantial number of anxiety experiences reported. (more than one third). 4.2. Results 1: the overall-state perspective on flow It was expected on the basis of previous research that the 11 items in the endof-session survey would produce a measure of flow that distinguished factors of control, enjoyment and engagement (see Appendix A for specific items). However, principal components analysis suggested only two factors. There were only two factors with eigen values greater than one and these two factors together accounted for 64.6% of the variance. After varimax rotation two of the items did not load clearly on either of these two factors and were dropped (MSA < 0.5). These two items were: (d) ÔI thought about other thingsÕ, and (h) ÔI was aware of distractionsÕ. These two items were intended to load with item (b), ÔI was absorbed by the activityÕ, to produce the engagement factor. The two factors generated by the principal components analysis were labelled ÔenjoymentÕ, and ÔcontrolÕ. Reliability analysis gave CronbachÕs alpha of 0.86 for the enjoyment factor and 0.79 for the control factor. The psychometric properties of these factors could not be improved by removing any further items. Scores on the two factors ÔenjoymentÕ and ÔcontrolÕ were summed to produce a value for flow from the survey items. This score was labelled Ôflow-finalÕ and represents the overall-state of flow as reported by the participants reflecting on their learning experience. Separate factor scores for both ÔenjoymentÕ and ÔcontrolÕ factors were also used in the analyses. The design of the study allowed for testing how the overall-state of flow was related to learning condition, movie or simulation, and to participantsÕ course of study, Physics or Information Systems. This produced a 2
2 design and analysis of variance was used to test whether learning condition and course of study were significantly associated with Ôflow-finalÕ. Similar analyses were performed for the two component factors of ÔenjoymentÕ and ÔcontrolÕ. There were no significant differences in Ôfinal-flowÕ associated with either learning condition or participantsÕ course of study, and there was no significant interaction between these design variables. When the separate ÔenjoymentÕ and ÔcontrolÕ factor scores were analysed significant differences in ÔcontrolÕ factor scores were found to be associated with participantsÕ course of study. The IS group scored a significantly lower standardised mean factor score of )0.19 compared to the Physics groupÕs mean factor score of 0.46 (F ð1; 56Þ ¼ 5:53; p < 0:05). That Physics students would feel more in control was expected given that they were more familiar with the physics domain than were the IS students. It was also expected that the simulation condition participants would report greater perceptions of control than the movie condition participants. Although the mean values of control for these two groups differed in the predicted direction (simulation: 0.12; movie: )0.13) this difference was not statistically significant. There were no significant differences between groups for the enjoyment factor but the average Likert scale score of 3.61 (s.d. 0.75) for these items indicated that participants enjoyed the experience.

758

J.M. Pearce et al. / Computers in Human Behavior 21 (2005) 745–771

4.3. Results 2: the process perspective on flow 4.3.1. Flow-paths for individual participants Data from the seven challenge–skill probes were used to plot each individualÕs flow-path through the learning exercise. These plots give a record of how an individualÕs flow state changed regardless of the number of channels in the flow model used to label those states. Plots for two participants are shown in Figs. 8 and 9. Each square represents one of the 25 possible states defined by the two 5-point Likert scales as represented in Table 1. A filled diamond represents the participantÕs challenge–skill perception on the first activity of the learning exercise; a smaller filled circle represents the final activity. Each point has had a small random value added to separate them on the plot. The dotted Ôflow-lineÕ represents the ideal flow condition of challenge ¼ skill. The hollow circle represents the Ôcentre-of-gravityÕ of that participantÕs plot. The first plot, Fig. 8, shows the challenge–skill path of a participant who performed very well on the exercise in terms of learning outcomes. It shows us that this participant began the exercise with relatively low skills and perceived the first activity to be relatively challenging. By the second activity she found the activity more challenging but also rated her skills as having increased to meet the challenge. The third activity presented less of a challenge and she still perceived her skills as increasing. The final two activities presented more challenge and this participant felt her skills were becoming increasingly less adequate for what the task required.

Fig. 8. Flow path for a ÔlearnerÕ.

J.M. Pearce et al. / Computers in Human Behavior 21 (2005) 745–771

759

Fig. 9. Flow path for a ÔboredÕ student.

Some of this plot is consistent with a flow model of learning: the participantÕs perceived challenges and skills grew together through the first part of the learning exercise. However, for this participant the final activities were difficult and she ended up in a state of some anxiety. The second plot, Fig. 9, shows the path for a participant who scored a near perfect score on the pre-test but showed a degree of boredom during the process. This participant actually obtained a lower score for his post-test than his pre-test. The plot shows the participant beginning and ending well into the boredom region, with an excursion closer to the flow line for four pages. Inspecting such plots for the cohort of 59 participants does not show any clear pattern linking these flow-paths to any of the post-survey flow measures. For example, using a flow rating value that is calculated from summing the percentage values of the enjoyment and control factors (flow-final), the participant in Fig. 8 had a flow value of 71% and the participant in Fig. 9 a value of 57%. Examples can be found of participants who did spend most of their time on the flow line, yet still returned overall flow scores that were relatively low. Similarly some participants who scored highly on this flow measure spent significant time in the anxiety or boredom regions of the flow-plot. It is an interesting feature of these plots generally that participants show considerable movement throughout the challenge–skill space. Rarely does a participant stay in one cell of the space or stay on the flow-line. This suggests that a single flow measure recorded at the end of the session cannot relate well to the overall experience when viewed as a process. In Section 4.4, we present a new alternative measure which captures better the flow experiences during the session.

760

J.M. Pearce et al. / Computers in Human Behavior 21 (2005) 745–771

4.3.2. Flow paths for groups of students Two other visualisations have been found useful in helping to interpret these flow patterns. After identifying specific groups of participants, we calculated for them an average group flow-path point for each of the seven activities and displayed these in two ways: (i) plots of average flow paths in challenge–skill space, and (ii) plots of challenges and skills versus activity number (Figs. 10 and 11). Both visualisations present the same information, but each offers a slightly different perspective. Three of the groups of participants we identified for this exercise were groups of 10 ÔextremeÕ students based on their pre- and post-test scores: 1. LearnersÕ: those who learnt most by changing the greatest number of incorrect answers to correct answers. 2. ChangersÕ: those who changed their thinking most, but by changing incorrect answers to different incorrect answers. 3. UnlearnersÕ: those who ÔunlearntÕ most by changing correct answers to incorrect answers. A thorough analysis of the educational aspects of these data is not given here, but examining some of the plots gives useful insights to the flow process. Fig. 10 shows the average challenge and skill data for the 10 ÔLearnersÕ. Note that both challenge and skills at first rise in a similar fashion, then skills tend to fall away from challenge in the final activities. The same information is presented in as a flow path Fig. 11, the participants moving up the path from bottom to top. The first section of these plots presents what one might ideally expect from participants who are engaged in the tasks and finding them well suited to their skills. For three

Fig. 10. Challenge and skills for ÔLearnersÕ.

J.M. Pearce et al. / Computers in Human Behavior 21 (2005) 745–771

761

Fig. 11. Flow path for ÔLearnersÕ.

consecutive activity pages their skills improved to match the challenges presented. On the fourth page (p. 5 on the plots), although they did not report the task as being any more challenging, they rated their skills as being less adequate. This moves them towards the ÔanxietyÕ region. From then on the tasks appear to be increasing in challenge and their perceived skills appear to decline. The characteristics of flow that they had in the beginning shift towards anxiety. This is similar to what has consistently been reported in the arousal motivation literature. When arousal is low, increases in arousal are associated with increased performance. However, when arousal reaches the personÕs optimal level further increases in arousal are associated with decline in performance. What caused the change to skill perception on the fourth page? The task on that page introduced a physics situation that many of the participants found confronting, namely, that the cart was moving in one direction while it was accelerating in the opposite direction. This page produced the highest average challenge score for all participants. This physics situation is a counter-intuitive one that often causes physics students difficulty. It appears that it disrupted the movement up the flow line and the participants, on average, moved towards a state of anxiety. We will discuss the effects from this particular page again in the next section. Figs. 12 and 13 show representations of the 10 ÔChangersÕ – those who changed many answers between pre-test and post-test but were still wrong. This group perceived consistently high challenges and relatively low skills resulting in their flowpath being in the ÔanxietyÕ region. Neither the challenge nor skill averages changed by very much. This is echoed by examining their individual flow-paths that shows the majority spending most of their time in the same space on the plot. These participants were clearly challenged beyond their ability. They did not report experiencing increasing challenge nor did they report changes in skill. As a group they had very

762

J.M. Pearce et al. / Computers in Human Behavior 21 (2005) 745–771

Fig. 12. Challenge and skills for ÔChangersÕ.

Fig. 13. Flow path for ÔChangersÕ.

low pre-test scores and generally performed worse after the exercise than before it. It is likely that they had trouble coping with the exercise and were either confused or guessing when it came to answering the post-test questions. Figs. 14 and 15 represent the 10 ÔUnlearnersÕ. When they started they perceived their skills to be higher than the task challenge, a bored pattern. However, as the

J.M. Pearce et al. / Computers in Human Behavior 21 (2005) 745–771

763

activities progressed they reported challenge increasing but no change in their skills. At this point their challenge and skills ratio was well balanced. In spite of ending up in what we label the flow region, they changed many of their correct pre-test responses to incorrect in the post-test. Again, their challenge–skill ratings do not change significantly, unlike that of the ÔLearnersÕ. Their pre-test scores were

Fig. 14. Challenge and skills for ÔUnlearnersÕ.

Fig. 15. Flow path for ÔUnlearnersÕ.

764

J.M. Pearce et al. / Computers in Human Behavior 21 (2005) 745–771

relatively high, two of them outstanding at 17 and 18 out of 19, respectively. Yet even these highly competent students reduced their score during the post-test. They were slightly bored early on in the activity, and gained no benefit from it. It is important to note here that these plots are averages for a group of 10 participants. Although they show steady behaviour, individuals move around the space significantly. For example, the individual flow paths that contribute to Fig. 15 show three participants who spend significant time on the ÔanxietyÕ side of the flow-line, even though the group average shows boredom. These are, of course, balanced by other participants spending more time in the boredom area. These average plots are, however, useful in helping identify trends and suggesting hypotheses to be investigated further. 4.4. Quantifying challenge–skill mappings The challenge–skill plots give a two-dimensional view of the flow state in contrast to the one-dimensional view calculated from the analysis of survey data and most published models. In order to make statistical comparisons between these two measures, we needed to quantify the challenge–skill plots. This was achieved by defining Ôfromflow-distanceÕ as a measure of how far an individual challenge–skill ratio is from the flow-line (challenge/skill ¼ 1). The resulting from-flow-distance measure has been explored in two ways. Firstly, it was used as an unsigned quantity where a value of 0 represents maximum flow and a value of 1 represents the furthest distance from flow, that is, either maximum anxiety (challenge ¼ 5 and skill ¼ 1), or maximum boredom (challenge ¼ 1 and skill ¼ 5). The second form of fromflow-distance involves using signed values such that maximum anxiety is presented by )1 and maximum boredom by +1. Both representations of from-flow-distance give similar results in the analysis described below. This is due to the fact that much of the data in these plots is from the IS students who were generally in the anxiety region. The rest of the analyses reported here use the signed fromflow-distance quantity. The expression for this quantity is derived from the geometry of the 5
5 challenge–skill space and is as follows: from-flow-distance ¼ 0:25
ðskill  challengeÞ: A value of from-flow-distance was calculated for each participant for each of their seven activities as well an average value which was calculated for an individualÕs whole experience. The seven individual activity values for each participant were correlated with their overall-state flow value in order to check for evidence of a recency effect influencing this flow value. The flow value we chose to use was the flow-final value so as to help us understand its relation to the whole task. A correlation value was calculated between the from-flow-distance and the final-flow scores for both cohorts of students for each of the seven pages of the exercise; these values are displayed in Fig. 16. Correlation values for pages 2, 5, 7 and 8 were above the critical value ðrcrit ¼ 0:322; p < 0:02Þ.

J.M. Pearce et al. / Computers in Human Behavior 21 (2005) 745–771

765

Fig. 16. Correlations of from-flow-distance to flow-final for all students.

The graph shows a strong primacy and recency effects on pages 2 and 8, respectively (page 2 contained the first learning activity, page 8 the last activity). This suggests that flow-final (the overall-state) as measured by the survey may be influenced by participantsÕ experiences on the final page of the task. The spike on page 5 is interesting. This is the confronting task discussed in the last section that caused the ÔlearnersÕ to have difficulty and move out of flow. We suggest that this particular challenging activity had a disproportionate influence on participantsÕ survey item judgements. It is interesting to compare this analysis with the definition of flow used by Novak and Hoffman (1997). They define a flow-apathy dimension as Ôskill + challengeÕ and a boredom–anxiety dimension as Ôskill–challengeÕ. Hence their boredom–anxiety dimension is equivalent to the signed from-flow-distance defined above and suggests that the correlations in the above analysis actually relate to boredom rather than flow. A check on the distribution of flow-states suggests that this is not the case. Most of the data points are in the anxiety quadrant of the challenge–skill plots, hence the correlations are telling us how far each point is away from anxiety along the line towards flow, but rarely crossing over into the boredom region. Recalculating these correlations for just those who rate low on the flow-final score (omitting those whose flow-final value is above the mean) gives even stronger correlations. This is the group who show greater anxiety on the boredom–anxiety dimension. That this sub-group has a stronger correlation is consistent with the more anxious participants being the ones who contribute most to the correlation, rather than those few participants in the boredom state. The fact that the signed and unsigned values of flow-final give similar correlations also supports this. A check on using NovakÕs flow definition of Ôchallenge + skillÕ showed no significant correlation between page flow values and the flow-final value. Clearly there are many subtle nuances to these measures that require further investigation.

766

J.M. Pearce et al. / Computers in Human Behavior 21 (2005) 745–771

5. Discussion For this investigation, an online physics learning exercise dealing with issues of velocity and acceleration was presented to Physics and IS students. Two versions of the software were used, a movie simulation and an interactive simulation. All students worked through seven pages of learning activities designed to develop studentsÕ understanding of the physics principles. Two separate measurement techniques were used to record studentsÕ perceptions of the learning exercise in terms of their experience of flow. A survey administered at the end of the learning exercise followed traditional methods of measuring studentsÕ overall-state of flow. The special contribution of this study has been to examine measurement of flow from a process perspective, a sequence of states that potentially might change across the course of any learning exercise. The overall-state of flow measure involves studentsÕ reflective judgements after the learning exercise has been completed. Our new process form of measurement focuses on studentsÕ judgements at different points in the learning sequence. By comparing the flow results generated by each of these measurement forms we have identified important issues that with further investigation will expand our understanding of the role of flow experiences in studentsÕ learning. Two major issues raised by our comparisons of these complementary forms of measurement will be discussed: the consistency of results derived from overall-state and process approaches to the measurement of flow, and individual flow patterns for different learners. 5.1. Consistency of overall-state and process measures of flow The process measure of flow used in the current study involved studentsÕ ratings of their experience seven times during the learning exercise. Following the dominant practice in flow research, we defined flow in terms of balanced challenge and skill ratings. When plotted these ratings frequently indicated balanced challenge–skill ratings. Approximately one-third of all the rating pairs indicated flow. Independently, the responses to the survey questionnaires often indicated high levels of enjoyment and control, also interpreted as an indication that students did experience flow during this learning exercise. Just under one-half of the students reported in the survey that they were unaware of time passing at some stage during the activity. However, while both forms of measurement point to students experiencing flow, at the individual level, there was no clear link between these two different methods for representing flow. We would like to have seen those students who reported high flow in the survey also having their data corroborated by many points of the challenge–skill plots being located on the flow-line. This was not the case. The flow-paths suggest Ôturbulent flowÕ experiences as students progress through the activity, often moving between the three regions of flow, boredom and anxiety. This was true even for students whose post-survey data indicated an extremely high positive level of flow. Clearly these two measures of flow are measuring different aspects of studentsÕ experience.

J.M. Pearce et al. / Computers in Human Behavior 21 (2005) 745–771

767

Although inconsistent with each other, both forms of measurement have the potential for providing insight into studentsÕ learning experiences because their focus is different. There are two considerations to help understand these findings. Firstly, one would not necessarily expect students to flow consistently throughout a learning activity, especially one from outside their discipline area, as was the case for the Information Systems students. Although the activity comprised seven related activities, the activities were sufficiently varied that some might engender a deep fascination and engagement, flow, whilst others might allow attention to wander. The physics teachersÕ holy grail of a student flowing throughout a physics learning activity and achieving deep understanding has not been observed in this study! Secondly, we should consider whether overall-state measures of flow might reflect specific ÔhighÕ points in a learning exercise rather than being an average across the task. Specific points that may have undue influence on overall flow include the first (primacy effects) and the last activities (recency effects). There was some evidence in the current data that a recency effect may have been operating with the overall flow measure. We used the common technique in measuring flow of asking participants to reflect on their experience and to respond retrospectively to questions about affective aspects of that experience. Our results indicated that such reflection was not a considered average of their experience, but was significantly associated with specific points in the learning activities. The flow-distance measure gave us evidence that recency and possibly primacy effects influenced the studentsÕ responses. We know far more about the operation of primacy and recency effects on recall of information than on recall of affect. There was also some indication that a change in the difficulty level (challenge) of the activities may have influenced overall flow ratings. On page 5 of the learning activity the challenge–skill flow value correlated with the overall flow value. The particular physics concepts on this page might explain why students found this page challenging, but more significant is that it appeared to influence their final flow value. This suggests that retrospective measures of flow may also be influenced by one particular activity, a Ôchallenging eventÕ effect, even though several others activities may not encourage flow. Further research is required here to help understand the meaning of an overall reflective judgement of flow and its relationship with experiences localised within the learning task. The apparent operation of a recency effect, a primacy effect, or even the effect of a particularly distinctive activity within a learning exercise is an important result for those measuring flow in an HCI context. If flow is a useful concept to assist with the design and evaluation of a task, be it a learning task or a general computing task, then the granularity of the flow measurement is an important criterion to consider. Too fine a granularity may interrupt the flow state itself, whilst too coarse a granularity may be heavily biased by the most recent experience or by a particularly challenging one. Finally, we agree with Ellis et al. that the label ÔboredomÕ is not a good descriptor for the high-skill/low-challenge state, as boredom is not usually regarded as a pleasant experience (Ellis, Voelkl, & Morris, 1994). Students showed no negative reactions to the experience and generally showed positive enjoyment, in both formal and informal feedback.

768

J.M. Pearce et al. / Computers in Human Behavior 21 (2005) 745–771

5.2. Different flow patterns for different learners By categorising students by their different learning outcomes we identified different patterns of flow behaviour. From the plots of the ÔlearnersÕ, we saw evidence of learning progressing in the manner postulated by Csikszentmihalyi (Csikszentmihalyi & Csikszentmihalyi, 1988) and a clear difference in the plots of the other groups. The plots of the ÔlearnersÕ helped identify a point in the learning materials (page 5) where most students experienced a higher level of difficulty than was in keeping with the activities of the previous pages. Better tailoring of the activities to a reasonable development in the studentsÕ understanding of the concepts being taught should maximize the likelihood that students such as the ÔlearnersÕ group will remain Ôin flowÕ. Situations like this are likely to be the point at which many students ÔbreakdownÕ and either fail to benefit further from the activities or look to their browser button to head off to another Web site altogether. Other groups of students showed no such progression in their flow behaviour and no beneficial learning from the whole exercise. The flow patterns here helped identify students for whom this particular task was not well suited at all. They either maintained a state of mismatch between challenges and skills and hence made no progress in learning physics, or they progressed from a bored state to one closer to flow, but regressed in their post-test scores. These flow patterns highlight that the relation between flow and learning is not a simple one. But then the nature of learning is not a simple process. If we broaden our meaning of ÔlearningÕ to include knowledge beyond the desired physics outcomes of the activity, then the instances of flow identified in this paper might relate to incidental moments of learning. This type of learning would not show up in our tests but might include skills such as: new vocabulary, relating cart motions to graphs, impact of changing graph scales, etc. From the perspective of constructivist learning, some learning is likely to be taking place whenever students are in a state of engagement with appropriately balanced challenges and skills.

5.3. Limitations A significant difficulty in measuring flow using challenge–skill ratios is to have some knowledge of the referent for the ratings made by each student. Do they use the same standards in judging challenge as when judging skill? The issue is one of the reliability of measurement. Do students use the same rating scales consistently across measurement points? More analysis of this type of design based on a sequence of individual ratings is needed before we can be confident of our measurement. In other settings an alternative approach is to use the average from all students so that each student is effectively compared to every other one (Kronradt et al., 2003). However, this is not satisfactory for the measurement of flow since the flow state is very much a personal perception of how challenging the task is and how adequate the learnersÕ skills are. Particularly in learning environments, the background expertise of each individual can be very different.

J.M. Pearce et al. / Computers in Human Behavior 21 (2005) 745–771

769

Finally, we have used a definition of flow based on challenge and skills being balanced to develop our process measure of flow. Our findings have shown that while yielding some important insights into studentsÕ experience of learning, this is not the same as the over-all state measure of flow. Extensions of this investigation are being planned in order to determine how well this process measure of flow models central aspects of studentsÕ experiences within learning.

6. Conclusion By recording challenge and skill ratings throughout a learning task we have gained some useful insights into studentsÕ experience during the task. These reports were compared with perceptions reported after the task was complete. These measures were both designed to be indicators of flow and appear to be complementary approaches although both could be refined further. The notion of flow has been shown to be useful for analysing studentsÕ responses to an online learning exercise in undergraduate physics. As measured in this study, flow was not higher with either the movie or the interactive simulation group. Further research identifying student responses to these forms of simulation may shed more light on the relationship between specific interactive design features and the promotion of flow in student learning. In this research we have demonstrated how challenge-skill flow-paths provide a valuable way of exploring the flow process. We have explored some of these paths and gained insight into the different patterns of learner behaviour they represent. We draw the following three conclusions: • flow can be usefully regarded as a process rather than simply as a destination; • there are both consistencies and inconsistencies between process and outcome measures of flow; • the flow-path representation is a valuable tool in the analysis of the flow process. Representing a studentÕs progression through a learning activity by a single value loses much of the rich information about the variable experiences within a learning session. Examining the paths that students take through the challenge-skill space gives us an exciting new view on their interactions and as a future research tool will help to improve our understanding of online learning environments.

Acknowledgements The authors thank Stephen Ainley and Connor Graham for their help with some of the technical aspects of this project.

Appendix A. Survey questions used to measure control, interest and enjoyment (a) I felt in control of what I was doing (b) I was absorbed intensely by the activity

770

J.M. Pearce et al. / Computers in Human Behavior 21 (2005) 745–771

(c) (d) (e) (f) (g) (h) (i) (j) (k)

I found the activities enjoyable I thought about other things I found the activities interesting I was frustrated by what I was doing The activities bored me I was aware of distractions The activities excited my curiosity I knew the right thing to do It required a lot of effort for me to concentrate on the activities.

References Agarwal, R., & Karahanna, E. (2000). Time flies when youÕre having fun: cognitive absorption and beliefs about information technology usage. MIS Quarterly, 24(4), 665–694. Ainley, M. (2002). Within task challenges: investigating students’ experience as they interact with specific educational tasks. Paper presented at the ICAP 2002, Singapore, 7–12 July. Artz, J. M. (1996). Computers and the quality of life: Assessing flow in information systems. Paper presented at the Proceedings of the Symposium on Computers and the Quality of Life. Chan, T. S., & Ahern, T. C. (1999). Targeting motivation – Adapting flow theory to instructional design. Journal of Educational Computing Research, 21(2), 151–163. Chen, H., & Nilan, M. (1998). An exploration of Web users’ internal experiences: Application of the experience sampling method to the Web environment. Paper presented at the WebNet 98 World Conference, Orlando, Florida. Chen, H., Wigand, R. T., & Nilan, M. (1998). Optimal flow experience in Web navigation. Paper presented at the 9th Information Resources Management Association International Conference., Boston, Massachusetts. Chen, H., Wigand, R. T., & Nilan, M. S. (1999). Optimal experiences of Web activities. Computers in Human Behavior, 15(5), 585–608. Csikszentmihalyi, M. (1975). Beyond boredom and anxiety. San Francisco: Jossey-Bass Publishers. Csikszentmihalyi, M. (1993). The evolving self: A psychology for the third millennium (1st ed.). New York, NY: HarperCollins Publishers. Csikszentmihalyi, M. (1997). Finding flow: the psychology of engagement with everyday life (1st ed.). New York: Basic Books. Csikszentmihalyi, M., & Csikszentmihalyi, I. S. (1988). Optimal experience: Psychological studies of flow in consciousness. New York: Cambridge. Csikszentmihalyi, M., & Larson, R. (1987). Validity and reliability of the experience-sampling method. Journal of Nervous and Mental Disease, 175(9), 526–536. Csikszentmihalyi, M., Larson, R., & Prescott, S. (1977). Ecology of adolescent activity and experience. Journal of Youth and Adolescence, 6(3), 281–294. Csikszentmihalyi, M., Rathunde, K. R., & Whalen, S. (1997). Talented teenagers: The roots of success and failure (1st ed.). Cambridge, England, New York, NY: Cambridge University Press. Ellis, G. D., Voelkl, J. E., & Morris, C. (1994). Measurement and analysis issues with explanation of variance in daily experience using the flow model. Journal of Leisure Research, 26(4), 337–356. Ghani, J. A., Supnick, R., & Rooney, P. (1991). The experience of flow in computer-mediated and in face-toface groups. Paper presented at the Proceedings of the Twelfth International Conference on Information Systems, New York. Hoffman, D. L., & Novak, T. P. (1996). Marketing hypermedia computer-mediated environments: Conceptual foundations. Journal of Marketing, 60(July), 50–68. Konradt, U., & Sulz, K. (2001). The experience of flow in interacting with a hypermedia learning environment. Journal of Educational Multimedia and Hypermedia, 10(1), 69–84.

J.M. Pearce et al. / Computers in Human Behavior 21 (2005) 745–771

771

Kronradt, U., Filip, R., & Hoffmann, S. (2003). Flow experience and positive affect during hypermedia learning. British Journal of Educational Technology, 34(1). Massimini, F., & Carli, M. (1988). The systematic assessment of flow in daily experience. In M. Csikszentmihalyi & I. S. Csikszentmihalyi (Eds.), Optimal experience: Psychological studies of flow in consciousness (pp. 266–287). Cambridge, New York: Cambridge University Press. Novak, T. P., & Hoffman, D. L. (1997). Measuring the flow experience among Web users. Paper presented at the Interval Research Corporation. Novak, T. P., Hoffman, D. L., & Yung, Y. (1997). Modeling the structure of the flow experience among Web users. Retrieved 26/8/01, from http://ecommerce.vanderbilt.edu/papers/flow.abstract.mit.1998/Modeling.the.Structure.of.the.Flow.Experience.html. Novak, T. P., Hoffman, D. L., & Yung, Y. F. (2000). Measuring the flow construct in online environments: A structural modeling approach. Marketing Science, Winter. Pace, S. (2000). Understanding the Flow Experience of Web users. Paper presented at the OzCHI 2000 Doctoral Consortium, Sydney. Trevino, L. K., & Webster, J. (1992). Flow in computer-mediated communication. Communications Research, 19(5), 539–573. Webster, J., Trevino, L. K., & Ryan, L. (1993). The dimensionality and correlates of flow in human– computer interactions. Computers in Human Behavior, 9(4), 411–426. Jon Pearce is a Senior Lecturer in the Department of Information Systems, University of Melbourne. He has a background in physics education and multimedia. He is currently carrying out research into interactivity and online learning. Mary Ainley is a Senior Lecturer in the Department of Psychology, University of Melbourne. She has research interests in: curiosity, engagement and learning; interest; motivation in gifted and talented students; beliefs about aggression; appraisal of conflict. Steve Howard is an Associate professor in the Department of Information Systems, University of Melbourne. He has a background in Psychology, Ergonomics and Human Computer Interaction. His research focus addresses what makes information and communication technology work for its users, and how can IT designers ensure that it does.