An online game approach for improving students’ learning performance in web-based problem-solving activities

Computers & Education 59 (2012) 1246–1256

Contents lists available at SciVerse ScienceDirect

Computers & Education journal homepage: www.elsevier.com/locate/compedu

An online game approach for improving students’ learning performance in web-based problem-solving activities Gwo-Jen Hwang a, *, Po-Han Wu b, Chi-Chang Chen c a

Graduate Institute of Digital Learning and Education, National Taiwan University of Science and Technology, 43, Sec. 4, Keelung Rd., Taipei 106, Taiwan Department of Engineering Science, National Cheng Kung University, No. 1, University Rd., Tainan City 701, Taiwan c Department of Information and Learning Technology, National University of Tainan, No. 33, Sec. 2, Shulin St., Tainan City 70005, Taiwan b

a r t i c l e i n f o

a b s t r a c t

Article history: Received 25 January 2012 Received in revised form 9 May 2012 Accepted 16 May 2012

In this paper, an online game was developed in the form of a competitive board game for conducting web-based problem-solving activities. The participants of the game determined their move by throwing a dice. Each location of the game board corresponds to a gaming task, which could be a web-based information-searching question or a mini-game; the former was used to guide the participants to search for information to answer a series of questions related to the target learning issue, while the latter was used to provide supplementary materials during the gaming process. To evaluate the performance of the proposed approach, an experiment was conducted on an elementary school natural science course. The experimental results showed that the proposed approach not only significantly promoted the flow experience, learning attitudes, learning interest and technology acceptance degree of the students, but also improved their learning achievements in the web-based problem-solving activity. Ó 2012 Elsevier Ltd. All rights reserved.

Keywords: Digital game-based learning Web-based problem solving Online game Interactive learning environments

1. Introduction Researchers have emphasized the importance of fostering students’ web-based problem-solving ability, which refers to the ability to express intentions as keywords, evaluate the correctness and relevance of searched data or information, extract the proper content, and organize the extracted content to answer a series of questions raised for some specific topics (de Vries, van der Meij, & Lazonder, 2008; Dreher, 2002; Tsai, Tsai, & Hwang, 2011). In the past decade, researchers have proposed several tools and strategies to conduct and evaluate web-based problem-solving behaviors. For example, Hwang, Tsai, Tsai, and Tseng (2008) developed a learning environment, MetaAnalyzer, for assisting teachers in analyzing student web-based information solving behaviors; Chen (2010) examined students’ learning behaviors via conducting web-based problem-solving activities in two university music appreciation courses and found that such an approach promoted the students’ learning performance and improved their higher order thinking ability. In the meantime, researchers have reported the difficulties encountered by Internet novice users, especially children, to effectively and efficiently search for information on the web and utilize the collected information (Dias, Gomes, & Correia, 1999; Marchionini, 1995). Moreover, several studies have found that students might get lost or feel frustrated while searching for information to solve complex problems without learning guidance or supports on the Internet (Hargittai, 2006; Li & Kirkup, 2007). As a consequence, researchers have emphasized the importance of providing learning supports for web-based problem-solving activities to enhance the learning performance of students as well as to engage them in an enjoyable learning process (Kauffman, Ge, Xie, & Chen, 2008; Yu, She, & Lee, 2010; Zamani & Shoghlabad, 2010). Games have been perceived as a kind of computer system that engages players in enjoyable activities following a set of rules to accomplish some challenging objectives (Kinzie & Joseph, 2008). In the past decade, researchers have attempted to develop educational computer games for diverse disciplines, such as mathematics (Bos & Shami, 2006; van Eck & Dempsey, 2002), computer science (Cagiltay, 2007; Papastergiou, 2009) and language (Liu & Chu, 2010; Ravenscroft, 2007). van Eck (2007) indicated that a well-designed computer educational game could provide a rich-resource learning environment with challenging learning missions to foster the higher order knowledge and skills of students. Researchers have further pointed out that, via properly integrating the learning content and strategies into

* Corresponding author. Tel.: þ886 915396558. E-mail addresses: [email protected], [email protected] (G.-J. Hwang), [email protected] (P.-H. Wu), [email protected] (C.-C. Chen). 0360-1315/$ – see front matter Ó 2012 Elsevier Ltd. All rights reserved. doi:10.1016/j.compedu.2012.05.009

G.-J. Hwang et al. / Computers & Education 59 (2012) 1246–1256

1247

the game-based learning environment, the students’ learning performance could be improved while maintaining the enjoyable nature of the games (Gros, 2007; Wang & Chen, 2010). To promote students’ learning performance and flow experience, in this study, an online game has been developed for conducting webbased problem-solving activities. It is expected that students can enjoy engaging in the web-based problem-solving process while at the same time improving their learning achievements via participating in the online game. As a consequence, the following research questions are investigated in this study: (1) (2) (3) (4)

Can the proposed online game approach improve students’ learning achievements in web-based problem-solving activities? Can the proposed online game approach promote students’ flow experiences in web-based problem-solving activities? Can the proposed online game approach promote students’ learning attitudes toward science learning? What are the perceptions of the students when learning with the proposed online game approach for web-based problem solving?

2. Literature review 2.1. Web-based problem-solving Problem-solving ability has been recognized as a critical skill for adapting to the living environment of the 21st century (Kuo, Hwang, & Lee, 2012). It consists of six skills, that is, identifying the nature of the problem, determining problem-solving steps, determining problemsolving strategies, choosing appropriate information, allocating proper resources, and monitoring the problem-solving process (Sternberg, 1988). Owing to the popularity of computer and communication technologies, recent studies have shown that students’ problem-solving ability can be fostered by conducting learning activities on the web (Chen, 2010; Hwang et al., 2008; Kim & Hannafin, 2011; Merrill & Gibert, 2008). Such activities have been called web-based problem-solving activities, and entail students being asked to answer a series of questions related to a specified issue via identifying the nature of the questions, determining the keywords, finding the potential web resources, selecting the appropriate web pages, abstracting the related information, and summarizing the information (Hwang & Kuo, 2011). In recent years, various studies have been conducted to investigate the effectiveness of web-based problem-solving for various courses, such as social science (Kuo et al., 2012), physics (Chandra & Watters, 2012; Young-Jin, 2012), mathematics (Rae & Samuels, 2011), biology (Yu et al., 2010), music appreciation (Chen & Hsiao, 2010) and computer studies (Huang et al., 2012). Most of these studies have reported positive effects of conducting web-based problem-solving activities. For example, Chen and Hsiao (2010) conducted a web-based problemsolving activity to examine students’ learning behavior and cognitive change in two music appreciation courses. They found that such a learning approach promoted the students’ learning performance and improved their higher order thinking ability. Moreover, Chandra and Watters (2012) reported that web-based problem-solving instruction had the potential to enhance and sustain the learners’ problemsolving skills over an extended period of time after conducting a physics problem-solving activity. Although conducting web-based problem-solving activities has been recognized as a potentially effective approach for fostering learners’ problem-solving skills, several studies have also indicated that it is easy for students to get lost on the Internet while searching for information to solve complex problems without assistance or scaffolding (Chandra & Watters, 2012; Chen, 2010; Chiou, Hwang, & Tseng et al., 2009; Ferreira & Sanos, 2009; Hargittai, 2006; Ho, Yin, Hwang, Shyu, & Yean, 2009; Kim & Hannafin, 2011; Li & Kirkup, 2007; Merrill & Gibert, 2008). That is, the provision of learning assistance or guidance for web-based problem-solving activities is needed. 2.2. Educational computer games Kinzie and Joseph (2008) indicated that “a game is an immersive, voluntary and enjoyable activity in which a challenging goal is pursued according to agreed-upon rules.” Owing to the rapid advancement and popularity of computer and communication technologies, researchers have predicted that more technology-based learning will occur, and educational computer games could play an important role in education (Prensky, 2001). In the past decade, many studies have been conducted to investigate the effectiveness of educational computer games for various courses, such as visuospatial reasoning (Guven & Kosa, 2008), mathematics (van Eck & Dempsey, 2002; Lowrie & Jorgensen, 2011), software engineering (Cagiltay, 2007; Connolly, Stansfield, & Hainey, 2007), civil engineering (Ebner & Holzinger, 2007), business (Kiili, 2007), computer science (Papastergiou, 2009), social science (Cuenca López & Martín Cáceres, 2010), geography (Tüzün, Yılmaz-Soylu, Karakus, Inal, & Kızılkaya, 2009), language (Liu & Chu, 2010) and decision-science (Chang, Peng, & Chao, 2010). Researchers have indicated the potential of employing educational computer games in helping students improve their learning performance (Brom, Preuss, & Klement, 2011; Huang, Huang, & Tschopp, 2010; Wang & Chen, 2010). For example, some studies have indicated that digital games are an important part of the development of children’s cognition and social processes (Kim, Park, & Baek, 2009; Yien, Hung, Hwang, & Lin, 2011). Some have reported that educational computer games can enhance the learning interest of students (Ebner & Holzinger, 2007; Malone,1980), and further increase their learning motivation (Burguillo, 2010; Dickey, 2010; Harris & Reid, 2005; Miller, Chang, Wang, Beier, & Klisch, 2011). In recent years, owing to the rapid advancement of network technologies, researchers have started to develop and investigate the effectiveness of online educational games (Guillén-Nieto & Aleson-Carbonell, 2012; Rutten, van Joolingen, & van der Veen, 2012). Warren, Dondlinger, McLeod, and Bigenho (2012) integrated game elements, PBL methods and 3-D communication tools in a web-based learning environment for an introductory computing course. From the feedback of the students, several positive findings were reported in terms of the technology skills gained, the understanding of the role that interpersonal communications play in learning and in career success, the sense of the usefulness of accessing resources, and the willingness to explore and experiment in such a learning environment. Yang (2012) investigated the effectiveness of a web-based educational game on students’ problem solving and academic achievement. They found that the web-based gaming strategy was helpful to the students in promoting their problem-solving skills, while no significant improvement was found in terms of learning achievement. On the other hand, some studies have been conducted to investigate the factors that affect the effectiveness of online game-based learning. For example, Hainey, Connolly, Stansfield, and Boyle (2011) investigated the factors affecting learners’ motivations in terms of single player/multiplayer preference and online/offline game participation, and found that challenge was the top ranking motivation for

1248

G.-J. Hwang et al. / Computers & Education 59 (2012) 1246–1256

playing computer games. Moreover, they also reported that multiplayer and online learners ranked competition, cooperation and recognition as significantly more important for playing games than did single players and offline participants. Meluso, Zheng, Spires, and Lester (2012) further conducted a web-based gaming activity to investigate the effects of collaborative and single game player conditions on science content learning and science self-efficacy. They found that students’ science content learning and self-efficacy significantly increased, while no significant differences were found between the two gaming conditions. From the literature, it is found that online educational games have become a widely discussed research issue. Moreover, how to improve the effectiveness of online educational games remains an important and challenging issue. 3. Development of an online game for conducting web-based problem-solving activities In this study, an online game was developed for conducting web-based learning activities. Fig. 1 shows the structure of the game, which is a multiplayer competition board game (Salen & Zimmerman, 2003) consisting of a board game interface, a learning management mechanism, a gaming mechanism, and a link to a search engine (i.e., Google search); moreover, several databases (i.e., a student profile database, a learning portfolio database, a mini-game database, and a learning material database) are established to provide learning supports for individual students. During the gaming process, individual students need to determine their move (i.e., 1–6) by throwing a dice. In each location of the game board, there is a set of gaming tasks. When students arrive at a specific location, the tasks in the corresponding set are triggered in turn. A gaming task can be a multiple-choice question of the web-based problem-solving activity or a mini-game to present the supplementary materials. The questions and the supplementary materials presented to the students are exactly the same as those of the conventional webbased problem-solving activities. The questions are presented in a pre-defined order to the students when they trigger the gaming tasks related to “a barrier to overcome” on the board. On the other hand, when the students move to the locations that are related to “an opportunity to win,” they are asked to play a mini-game related to the supplementary materials. When the students correctly answer a question or play a mini-game in the set time, their personal gaming scores increase. During the gaming process, the system shows the accumulated scores of all of the students on the board, so that they know the gaming status of their peers, as shown in Fig. 2. When a question is correctly answered, the students are asked to throw the dice to move forward. For individual students, the game ends when all of the learning tasks are completed (i.e., all of the questions are correctly answered) or the time is up. When the students fail to correctly answer a question, the learning system guides the students to search for relevant information on the web, as shown in Fig. 3. If the students fail to correctly answer the questions a second time, the learning system will show them the correct answer and the link to access the related supplementary materials, as shown in Fig. 4. In addition to the question-and-answer tasks, when the students move to the locations that are related to “an opportunity to win,” one of the mini-games (e.g., jigsaw puzzle, matching game or shooting game) developed based on the learning content is presented. Fig. 5 shows a jigsaw puzzle that reveals the four growth stages of “Idea leuconoe clar” and a matching game that identifies the features of different butterflies. These games help the students gain a whole picture of the ecology of butterflies. The aim of the board game is to guide the students to complete the web-based problem-solving tasks, while the set of embedded minigames is designed to help the students learn in depth or link what they have found on the web to the learning content. A summary of the aims of the mini-games is given in Table 1.

Fig. 1. Structure of the online game for web-based problem-solving activities.

G.-J. Hwang et al. / Computers & Education 59 (2012) 1246–1256

1249

Fig. 2. User interface of the online game.

4. Experiment design 4.1. Participants The subjects included two classes of fifth and sixth graders of an elementary school in Tainan County in Taiwan. A total of fifty students voluntarily participated in the study. One class was assigned to be the experimental group and the other was the control group. The experimental group, including twenty-nine students, was guided by the educational computer game that used a Graphical Quiz (i.e., a competitive quiz presented with graphical materials) approach to developing the board game-based learning system, while the control

Fig. 3. Interface of searching for information to answer the questions.

1250

G.-J. Hwang et al. / Computers & Education 59 (2012) 1246–1256

Fig. 4. Interface of presenting the answer and the corresponding supplementary materials.

group with twenty-one students was guided by the learning sheets and keyword search on the Internet. All of the students were taught by the same instructor who had taught that natural science course for more than five years. 4.2. Research tools The research tools in this study included the learning achievement tests, and the questionnaire for measuring the students’ “learning attitude,” “learning interest,” “flow experience,” and “technology acceptance.” The test sheets were developed by two experienced teachers. The pre-test was developed to test the students’ prior knowledge of learning the course unit “butterfly ecology.” It consisted of ten yes-or-no items and ten multiple-choice items, with a perfect score of 100. The post-test consisted of ten multiple-choice items, three matching items and five short answer question items for assessing the students’ knowledge of butterfly ecology. Its perfect score was 100. The Cronbach’s alpha values of the pre-test and post-test were 0.982 and 0.978, respectively. The questionnaire of learning attitude was modified from the measure developed by Hwang and Chang (2011). It consists of seven items (e.g., “The natural science course is valuable and worth studying” and “I would like to know more about the learning targets”) with a fivepoint rating scheme. The Cronbach’s alpha value of the questionnaire was 0.82. The questionnaire of learning interest was modified from the measure developed by Chu, Hwang, and Tsai (2010). It consists of thirteen items (e.g., “I have endeavored to follow the learning guidance given by the system during the learning process” and “I would like to learn with this approach in the future”) with a five-point rating scheme. The Cronbach’s alpha value of the questionnaire was 0.93. The questionnaire of flow experience was modified from the measure developed by Wang and Chen (2010). It consists of 11 items in four dimensions; that is, two items for “flow antecedent” (e.g., “I knew clearly what I wanted to do and achieve”), four items for “flow experience”

Fig. 5. Mini-games (jigsaw puzzle and matching game) of the ecology of butterflies.

G.-J. Hwang et al. / Computers & Education 59 (2012) 1246–1256

1251

Table 1 The aim of each mini-game. Name of the game Aim of the game Jigsaw puzzle Matching game Shooting game

Example

Learn a concept, notation, or learning target in depth The features of “Idea Leoconoe” Learn positive relationships between concepts, notations, or learning targets The butterflies and their food plants in different growth stages Learn negative relationships between concepts, notations, or learning targets The butterflies and their natural enemies in different growth stages

(e.g., “I was totally immersed in playing the game”), two items for “intrinsic motivation” (e.g., “I prefer course material that really challenges me so I can learn new things”) and three items for “extrinsic motivation” (e.g., “If I can, I want to get better grades in this class than most of the other students”). The Cronbach’s alpha values of the questionnaire and the four dimensions were 0.91, 0.76, 0.70, 0.91 and 0.80, respectively. The questionnaire of technology acceptance was modified from the measure developed by Wu, Hwang, Milrad, Ke, and Huang (2011). It consists of 13 items in two dimensions; that is, six items for “usefulness” (e.g., “The functions provided by the system are beneficial to my learning achievement”) and seven items for “ease of use” (e.g., “It is easy to operate the interface of this learning system”) with a six-point rating scheme. The Cronbach’s alpha values of the questionnaire and the two dimensions were 0.91, 0.95 and 0.86, respectively. 4.3. Experiment procedures Before the experiment, the two groups of students took a 120 min course unit on the basics of butterfly ecology, which is a part of the existing natural science course. Fig. 6 shows the procedure of the experiment. At the beginning of the learning activity, the students took the pre-test and completed the learning attitude questionnaire. During the learning activity, the students in the experimental group participated in the web-based learning activities with the online game approach. On the other hand, those in the control group learned with the conventional web-based problem-solving approach; that is, they were guided by a series of questions on a learning sheet to search for information on the web for investigating a particular issue with the assistance of the teacher. Moreover, they needed to complete a learning sheet to report what they had found out about the target issue. The teacher then provided feedback on their reports. The time for the students to complete their learning tasks was 150 min. After the learning activity, the students took the post-test and completed the learning attitude, learning reception, flow experience, and technology acceptance questionnaire to measure their learning achievements and any change in their learning attitudes, learning reception, flow experience, and technology acceptance. 5. Results 5.1. Analysis of learning achievement One of the objectives of this study was to examine the effectiveness of the system in terms of improving the learning achievement of the students. ANCOVA was used to exclude the difference between the prior knowledge of the two groups by using the pre-test scores as the covariate and the post-test scores as dependent variables. The homogeneity test result showed that the post-test scores of the two groups were homogeneous (F ¼ 0.29, p ¼ 0.59 > 0.05), implying that ANCOVA could be applied. Table 2 summarizes the ANCOVA results, in which the adjusted mean values of the post-test scores were 80.94 for the experimental group, and 60.09 for the control group; moreover, a significant difference was found between the two groups with F ¼ 57.53 and p < 0.05, implying that the system had significantly positive effects on the learning achievements of the experimental group students for the butterfly ecology course. 5.2. Analysis of learning attitude Another objective of this study was to examine the effectiveness of the online game approach in improving the learning attitudes of the students toward the natural science course after participating in the web-based problem-solving activity.

Fig. 6. Diagram of experiment design.

1252

G.-J. Hwang et al. / Computers & Education 59 (2012) 1246–1256

Table 2 Descriptive data and ANCOVA of the post-test results. Group

N

Mean

S.D.

Adjusted mean

Std. error

F

Experimental group Control group

29 21

80.81 60.29

7.35 10.07

80.94 60.09

1.70 2.12

57.53*

*p < 0.05.

Table 3 t-Test result of learning attitude questionnaire.

Post-test

Group

N

Mean

S.D.

t

Experimental group Control group

29 21

4.58 3.46

0.42 1.30

3.77*

*p < 0.05.

From the experimental results, it was found that, before the learning activity, the mean values and standard deviations of the students’ ratings of the learning attitude questionnaire were 3.48 and 0.48 for the control group, and 3.63 and 0.47 for the experimental group, respectively. The t-test result (t ¼ 1.02, p > 0.05) shows that there was no significant difference between the two groups before participating in the web-based problem-solving activity. After the learning activity, the t-test result (t ¼ 3.77, p < 0.05) showed that the learning attitudes of the students in the experimental group significantly improved after the learning activity in comparison with the control group, as shown in Table 3. Consequently, it can be seen that the online game approach improved not only the students’ learning achievements, but also their attitudes toward the natural science course. Such findings conform to what has been reported by previous studies that situating students in collaborative educational computer games can improve their attitudes toward learning (Chang et al., 2010; Wang & Chen, 2010). 5.3. Analysis of flow experience “Flow” has been defined by researchers as an enjoyable experience that is achieved when individuals engage in an activity with full involvement, concentration and enjoyment (Chen, Wigan, & Niran, 2000). In the flow state, people pay full attention to the activity they are engaging in, and their focus of awareness is narrowed down to only that activity (Csikszentmihalyi, 1975); that is, they are experiencing an intrinsic interest, and hence their sense of time is distorted during the engagement (Choi & Baek, 2011). Table 4 shows the t-test result of the students’ flow experience of the system before and after the learning activity. The mean values and standard deviations of the post-test scores of flow experience were 4.65 and 1.50 for the control group, and 5.85 and 0.86 for the experimental group. The t-test result (t ¼ 3.31, p < 0.05) shows that there was significant difference between the two groups. It is found that the flow experience of the students in the experimental group significantly improved after the learning activity, while the change in the control group students’ flow experience was not significant. Such a finding conforms to what has been reported by several studies, namely that challenge, control and enjoyment are core factors related to flow experience during the online learning process (Chen et al., 2000; Wang & Chen, 2010); moreover, these factors can be introduced by providing instant interactions, explicit objectives and dynamic challenges (Csikszentmihalyi, 1991), which conform to the characteristics of online games. Therefore, it is inferred that the online game approach is able to situate students in the flow state when they are engaging in the web-based problem-solving activity. Furthermore, in terms of the four dimensions of flow experience, the two groups show a significant difference in each dimension; that is, flow antecedent (t ¼ 4.41, p < 0.05), flow experience (t ¼ 2.44, p < 0.05), intrinsic motivation (t ¼ 3.56, p < 0.05), and extrinsic motivation (t ¼ 2.93, p < 0.05), as shown in Table 5. Flow antecedent represents the degree of fitness of the challenge of the game for the skill level of the player (Kiili, 2005). High flow antecedent indicates that the developed game provides clear goals, unambiguous feedback and a good sense of control that meet most of the students’ knowledge levels and computer skills, and hence the students are able to realize the challenges they need to face, focus on the learning objectives, and feel the playability, enjoyment and attraction of the game. Flow experience refers to the state of complete absorption or engagement in an activity (Csikszentmihalyi, 1991). High flow experience indicates that the developed game is able to situate students in a psychological state where they are highly involved with the learning activity and hence nothing else seems to matter (Csikszentmihalyi, 1975). Therefore, the experimental result implies that the developed online game is able to situate students in an enjoyable and focused learning state, and hence time seems to pass quickly when they are trying to search for information to answer the questions. Intrinsic motivation refers to the intention or willingness to learn something and the desire to engage in an activity or to contribute to a task (Csikszentmihalyi & Moneta, 1996; Dev, 1997). Therefore, from the experimental result, it is found that the online game approach has significantly promoted the intention or willingness of the students to complete the web-based problem-solving tasks. On the other hand, extrinsic motivation refers to motives that are outside of and separate from the behaviors they cause (Hoyenga & Hoyenga, 1984). The

Table 4 t-Test result of flow experience of the two groups. Group

N

Mean

S.D.

t

Experimental group Control group

29 21

5.85 4.65

0.86 1.50

3.31*

*p < 0.05.

G.-J. Hwang et al. / Computers & Education 59 (2012) 1246–1256

1253

Table 5 t-Test result of the four aspects of flow experience. Group

Mean

S.D.

t

Flow antecedent

Experimental Control

5.17 3.98

1.30 1.87

2.67*

Flow experience

Experimental Control

5.69 4.69

1.02 1.67

2.44*

Intrinsic motivation

Experimental Control

6.38 4.71

0.92 2.00

3.56*

Extrinsic motivation

Experimental Control

6.15 4.98

0.87 1.67

2.93*

*p < 0.05.

Table 6 t-Test result of learning interest questionnaire. Group

N

Mean

S.D.

t

Experimental group Control group

29 21

4.52 3.59

0.53 1.07

3.65*

*p < 0.05.

Table 7 t-Test result of technology acceptance of the two groups. Group

N

Mean

S.D.

t

Usefulness

Experimental Control

29 21

5.63 4.15

0.56 1.46

4.41*

Easy to use

Experimental Control

29 21

5.40 4.35

0.63 1.58

2.88*

*p < 0.05.

experimental result implies that the competition and challenges of the online game have motivated the students to achieve better performance during the web-based problem-solving activity. 5.4. Analysis of learning interest and technology acceptance In terms of learning interest, the mean values and standard deviations were 3.59 and 1.07 for the control group, and 4.52 and 0.53 for the experimental group. The t-test result (t ¼ 3.65, p < 0.05) shows that there was significant difference between the two groups, as shown in Table 6, implying that the students who learned with the online game approach revealed significantly higher learning interest than those who learned with the traditional web-based learning approach. This finding complies with the experimental results of learning attitudes and flow experience which show that the online game approach has situated the students in a more enjoyable learning context than the traditional approach. Table 7 further shows the t-test result of the students’ technology acceptance of using the system after the learning activity. As shown in Table 7, the t-test results show that the two groups had highly significant differences in both the “usefulness” (t ¼ 4.41, p < 0.05) and the “ease of use” (t ¼ 2.88, p < 0.05) dimensions, implying that the online game approach was highly accepted by the students. Similar findings were obtained from the interviews, indicating that more than 80% of the experimental group students felt that the online game approach was helpful to them in improving their learning effectiveness. 6. Discussion and conclusions In this paper, an online game approach is proposed for conducting web-based problem-solving activities. The experimental results show that the proposed approach not only improved the students’ learning achievement and attitudes, but also situated them in a learning state with full involvement, concentration and enjoyment. The analysis of the students’ learning interest and technology acceptance further showed that most of them enjoyed the learning activity and felt that the online game approach was helpful to them in improving their learning performance in terms of “perceived ease of use” and “perceived usefulness.” While most web-based problem-solving studies have focused on assessing students’ learning achievement in traditional web-based learning activities, one of the major contributions of this study is the proposal of an online game-based learning approach that guides students to complete web-based problem-solving tasks in an effective and enjoyable manner. Most game-based learning researchers pay attention to students’ learning motivation, learning achievement, or flow experience of playing educational computer games for learning the subject materials (Chou & Ting, 2003; Dickey, 2005; Hsu & Lu, 2004; Omale, Hung, Luketkehans, & Cook-Plagwiz, 2009), while this study employed the game-based learning approach for conducting a quite different form of learning activity, that is, web-based problem-solving.

1254

G.-J. Hwang et al. / Computers & Education 59 (2012) 1246–1256

The experimental results show that the proposed approach was successful. In particular, it is interesting to find that even though the learning activity conducted in this study did not provide extrinsic rewards, the students were highly motivated in their web-based problemsolving tasks. Consequently, it is inferred that, in the online game-based learning environment, the students’ intrinsic motivations were promoted and hence they were highly engaged in the tasks (Choi & Baek, 2011). Such a finding is consistent with what has been reported by previous studies, namely that flow experience caused by playing games is able to encourage and support students to face more complicated and greater challenges (Csikszentmihalyi & Csikszentmihalyi, 1988). Moreover, the students’ “perceived ease of use” and “perceived usefulness” further indicate that the difficulty and challenges of the online game conform to the knowledge and skill level of most students, and hence, instead of experiencing anxiety, they were able to engage in the tasks with a feeling of enjoyment (Massimini & Carli, 1988). The approach proposed in this study can be generalized to other applications in which searching for, abstracting and summarizing information for answering a series of questions related to a particular issue is the main concern, such as natural science, ecology and social science courses. The following two-phase guidelines are suggested to those who intend to introduce the proposed approach into their classes. The first phase is related to the preparation of learning materials and the settings of the learning environment before the learning activity: Step 1: Determine the issue to be investigated. Step 2: Prepare a series of questions related to the issue for guiding students to search for information on the web. It should be noted that the teachers or researchers need to examine whether the proposed questions are appropriate by going through the entire web-based problem-solving activity before formally conducting it. Step 3: Prepare the supplementary materials for the learning content that cannot be found or is incomplete on the web. For those materials that state a single conception in detail, use them as the content of the jigsaw puzzle; for those related to relationships between concepts, use them as the content of the matching game. The second phase is carried out in the classroom: Step 1: Give a brief about the learning tasks and the game-based learning environment. This step usually requires 20–30 min. Step 2: Conduct the learning activity. The time needed in this step depends on the number of questions and the supplementary materials prepared for the target issue. It is suggested that at least 1 h is required for students to complete the exploration of an issue. Step 3: Give feedback to the students after the learning activity. Award the students who achieve the pre-defined standard (e.g., top-three scores). It should be noted that the award standard is determined by the teacher or researcher and should be announced to the participants before the learning activity. Although the present approach seems to be satisfactory, it might be difficult to claim that all of the findings are significant since the number of participants is not large and the activity period is short. It is possible that part of the results would not be sustained once the novelty of the online game wears off. Therefore, it is worth conducting extended studies with a large number of participants and over a longer period of time in the future. Furthermore, in the present approach, the students learned individually in a competition mode via network communications. Such an approach has the advantage of stimulating and guiding students to search for and organize information for solving problems on their own. However, for those applications that require collaborative work or peer interactions, the Classroom Multiplayer Presential Game (CMPG) proposed by Villalta et al. (2011) could be a better approach. CMPG enables peers to interact collaboratively. In a CMPG activity, the game is conducted on a screen which is projected at the front of the classroom through which the students interact with the virtual world and amongst themselves in the shared space; moreover, individual players have their own input devices which allow them to control the representative characters within the game. It can be seen that CMPG has the advantages of enabling interactions and collaborations with lower hardware cost; on the other hand, the online competitive game proposed in this study has the advantages of being location free and promoting learning interest, but has the disadvantage of lacking peer interaction and collaboration. Consequently, it is worth trying to extend the idea of CMPG to the development of online competitive games for learners in different locations or classes; that is, employing the CMPG approach to have each group (or class) of students learn collaboratively in the same classroom, while having different groups compete online via network communications. Acknowledgments This study is supported in part by the National Science Council of the Republic of China under contract numbers NSC 99-2511-S-011-011MY3 and NSC 100-2631-S-011-003. References Bos, N., & Shami, N. (2006). Adapting a face-to-face role-playing simulation for online play. Educational Technology Research and Development, 54(5), 493–521. Brom, C., Preuss, M., & Klement, D. (2011). Are educational computer micro-games engaging and effective for knowledge acquisition at high-schools? A quasi-experimental study. Computers & Education, 57(3), 1971–1988. doi:10.1016/j.compedu.2011.04.007. Burguillo, J. C. (2010). Using game theory and competition-based learning to stimulate student motivation and performance. Computers & Education, 55(2), 566–575. Cagiltay, N. E. (2007). Teaching software engineering by means of computer-game development: challenges and opportunities. British Journal of Educational Technology, 38(3), 405–415. Chandra, V., & Watters, J. J. (2012). Re-thinking physics teaching with web-based learning. Computers & Education, 58(1), 631–640. doi:10.1016/j.compedu.2011.09.010. Chang, Y. C., Peng, H. Y., & Chao, H. C. (2010). Examining the effects of learning motivation and of course design in an instructional simulation game. Interactive Learning Environments, 18(4), 319–339. Chen, C. H. (2010). Promoting college students’ knowledge acquisition and ill-structured problem solving: web-based integration and procedure prompts. Computers & Education, 55(1), 292–303. Chen, F. S., & Hsiao, Y. W. (2010). Using Webquest as a creative teaching tool at a science and technology university in Taiwan. World Transaction on Engineering and Technology Education, 8(2).

G.-J. Hwang et al. / Computers & Education 59 (2012) 1246–1256

1255

Chen, H., Wigan, R. T., & Niran, M. S. (2000). Exploring web users’ optimal flow experiences. Information Technology & People, 13(4), 263–277. Chiou, C. K., Hwang, G. J., & Tseng, J. C. R. (2009). An auto-scoring mechanism for evaluating problem-solving ability in a web-based learning environment. Computers & Education, 53(2), 261–272. Choi, B., & Baek, Y. (2011). Exploring factors of media characteristic influencing flow in learning through virtual worlds. Computers & Education, 57(4), 2382–2394. Chou, T. J., & Ting, C. (2003). The role of flow experience in cyber-game addiction. Cyber Psychology and Behavior, 6(6), 663–675. Chu, H. C., Hwang, G. J., & Tsai, C. C. (2010). A knowledge engineering approach to developing mindtools for context-aware ubiquitous learning. Computers & Education, 54(1), 289–297. Connolly, T. M., Stansfield, M., & Hainey, T. (2007). An application of games-based learning within software engineering. British Journal of Educational Technology, 38(3), 416–428. Csikszentmihalyi, M. (1975). Beyond boredom and anxiety. San Francisco, CA: Jossey-Bass. Csikszentmihalyi, M. (1991). Flow: The psychology of optimal experience. New York: Harper Perennial. Csikszentmihalyi, M., & Csikszentmihalyi, I. (1988). Introduction to part IV. In M. Csikszentmihalyi, & I. S. Csikszentmihalyi (Eds.), Optimal experience: Psychological studies of flow in consciousness (pp. 251–265). Cambridge, United Kingdom: Cambridge University Press. Csikszentmihalyi, M., & Moneta, B. G. (1996). The effect or perceived challenges and skills on the quality of subjective experience. Journal of Personality, 64(2), 275–310. Cuenca López, J. M., & Martín Cáceres, M. J. (2010). Virtual games in social science education. Computers & Education, 55(3), 1336–1345. doi:10.1016/j.compedu.2010.05.028. Dev, P. C. (1997). Intrinsic motivation and academic achievement. Remedial and Special Education, 18(10), 12–19. de Vries, B., van der Meij, H., & Lazonder, A. W. (2008). Supporting reflective web searching in elementary schools. Computers in Human Behavior, 24(3), 649–665. Dias, P., Gomes, M. J., & Correia, A. P. (1999). Disorientation in hypermedia environments: mechanisms to support navigation. Journal of Educational Computing Research, 20(2), 93–117. Dickey, M. D. (2005). Three-dimensional virtual worlds and distance learning: two case studies of active worlds as a medium for distance education. British Journal of Educational Technology, 36(5), 439–451. Dickey, M. D. (2010). Murder on Grimm Isle: the impact of game narrative design in an educational game-based learning environment. British Journal of Educational Technology. doi:10.1111/j.1467-8535.2009.01032.x. Dreher, M. J. (2002). Children searching and using information text: a critical part of comprehension. In C. C. Block, & M. Pressley (Eds.), Comprehension instruction: Researchbased best practices (pp. 289–304). New York: Guilford. Ebner, M., & Holzinger, A. (2007). Successful implementation of user-centered game based learning in higher education: an example from civil engineering. Computers & Education, 49(3), 873–890. Ferreira, D. J., & Sanos, L. D. (2009). Scaffolding online discourse in collaborative ill-structured problem-solving for innovation. Informatics in Education, 8(2), 173–190. Gros, B. (2007). Digital games in education: the design of game-based learning environment. Journal of Research on Technology in Education, 40(1), 23–38. Guillén-Nieto, V., & Aleson-Carbonell, M. (2012). Serious games and learning effectiveness: the case of it’s a deal! Computers & Education, 58(1), 435–448. Guven, B., & Kosa, T. (2008). The effect of dynamic geometry software on student mathematics teachers’ spatial visualization skills. Turkish Online Journal of Educational Technology, 7(4), 100–107. Hainey, T., Connolly, T., Stansfield, M., & Boyle, E. (2011). The differences in motivations of online game players and offline game players: a combined analysis of three studies at higher education level. Computers & Education, 57(4), 2197–2211. Hargittai, E. (2006). Hurdles to information seeking: explaining spelling and typographical mistakes in users’ online search behavior. Journal of the Association for Information Systems, 7(1), 52–67. Harris, K., & Reid, D. (2005). The influence of virtual reality play on children’s motivation. Canadian Journal of Occupational Therapy, 72(1), 21–30. Ho, T. F., Yin, P. Y., Hwang, G. J., Shyu, S. J., & Yean, Y. N. (2009). Multi-objective parallel test-sheet composition using enhanced particle swarm optimization. Educational Technology & Society, 12(4), 193–206. Hoyenga, K. B., & Hoyenga, K. T. (1984). Motivational explanations of behavior. Monterey, CA: Brookes/Coles Publishing Company. Hsu, C. L., & Lu, H. P. (2004). Why do people play on-line games? An extended TAM with social influences and flow experience. Information and Management, 41(7), 853–868. Huang, H.-S., Chiou, C.-C., Chiang, H.-K., Lai, S.-H., Huang, C.-Y., & Chou, Y.-Y. (2012). Effects of multidimensional concept maps on fourth graders’ learning in web-based computer course. Computers & Education, 58(3), 863–873. doi:10.1016/j.compedu.2011.10.016. Huang, W. H., Huang, W. Y., & Tschopp, J. (2010). Sustaining iterative game playing processes in DGBL: the relationship between motivational processing and outcome processing. Computers & Education, 55(2), 789–797. Hwang, G. J., & Chang, H. F. (2011). A formative assessment-based mobile learning approach to improving the learning attitudes and achievements of students. Computers & Education, 56(1), 1023–1031. Hwang, G. J., & Kuo, F. R. (2011). An information-summarising instruction strategy for improving web-based problem solving abilities of students. Australasian Journal of Educational Technology, 27(2), 290–306. Hwang, G. J., Tsai, P. S., Tsai, C. C., & Tseng, J. C. R. (2008). A novel approach for assisting teachers in analyzing student web-searching behaviors. Computers & Education, 51(2), 926–938. Kauffman, D. F., Ge, X., Xie, K., & Chen, C. H. (2008). Prompting in web-based environments: supporting self-monitoring and problem solving skills in college students. Journal of Educational Computing Research, 38(2), 115–137. Kiili, K. (2005). On educational game design: Building blocks of flow experience. Tampere, Finland: Tampere University of Technology Press. Kiili, K. (2007). Foundation for problem-based gaming. British Journal of Educational Technology, 38(3), 394–404. Kim, B., Park, H., & Baek, Y. (2009). Not just fun, but serious strategies: using meta-cognitive strategies in game-based learning. Computers & Education, 52(4), 800–810. Kim, M. C., & Hannafin, M. J. (2011). Scaffolding problem solving in technology-enhanced learning environments (TELEs): bridging research and theory with practice. Computers & Education, 56(2), 403–417. Kinzie, M. B., & Joseph, D. R. D. (2008). Gender differences in game activity preferences of middle school children: implications for educational game design. Educational Technology Research & Development, 56, 643–663. Kuo, F. R., Hwang, G. J., & Lee, C. C. (2012). A hybrid approach to promoting students’ web-based problem solving competence and learning attitude. Computers & Education, 58(1), 351–364. Li, N., & Kirkup, G. (2007). Gender and cultural differences in internet use: a study of China and the UK. Computers & Education, 48, 301–317. Liu, T. Y., & Chu, Y. L. (2010). Using ubiquitous games in an English listening and speaking course: impact on learning outcomes and motivation. Computers & Education, 55(2), 630–643. Lowrie, T., & Jorgensen, R. (2011). Gender differences in students’ mathematics game playing. Computers &; Education, 57(4), 2244–2248. doi:10.1016/j.compedu.2011.06.010. Malone, T. W. (1980). What makes things fun to learn? A study of intrinsically motivation computer games. Palo Alto: Xerox. Marchionini, G. (1995). Information seeking in electronic environments. New York: Cambridge University Press. Massimini, F., & Carli, M. (1988). The systematic assessment of flow in daily experience. In M. Csikszentmihalyi, & I. Csikszentmihalyi (Eds.), Optimal experience: Psychological studies of flow in consciousness (pp. 288–306). New York: Cambridge University Press. Meluso, A., Zheng, M., Spires, H. A., & Lester, J. (2012). Enhancing 5th graders’ science content knowledge and self-efficacy through game-based learning. Computers & Education, 59(2), 497–504. Merrill, M. D., & Gibert, C. G. (2008). Effective peer interaction in problem-centered instructional strategy. Distance Education, 29(2), 199–207. Miller, L. M., Chang, C.-I., Wang, S., Beier, M. E., & Klisch, Y. (2011). Learning and motivational impacts of a multimedia science game. Computers & Education, 57(1), 1425–1433. doi:10.1016/j.compedu.2011.01.016. Omale, N., Hung, W. C., Luketkehans, L., & Cook-Plagwiz, J. (2009). Learning in 3-D multiuser virtual environment: exploring the use of unique 3-D attributes for online problem-based learning. British Journal of Educational Technology, 40(3), 480–495. Papastergiou, M. (2009). Digital game-based learning in high school computer science education: impact on educational effectiveness and student motivation. Computers & Education, 52(1), 1–12. Prensky, M. (2001). Digital game-based learning. New York: McGraw Hill. Rae, A., & Samuels, P. (2011). Web-based personalised system of instruction: an effective approach for diverse cohorts with virtual learning environments? Computers & Education, 57(4), 2423–2431. doi:10.1016/j.compedu.2011.06.003. Ravenscroft, A. (2007). Promoting thinking and conceptual change with digital dialogue games. Journal of Computer Assisted Learning, 23(6), 453–465. Rutten, N., van Joolingen, W. R., & van der Veen, J. T. (2012). The learning effects of computer simulations in science education. Computers & Education, 58(1), 136–153. Salen, K., & Zimmerman, E. (2003). Rules of play: Game design fundamentals. The MIT Press.

1256

G.-J. Hwang et al. / Computers & Education 59 (2012) 1246–1256

Sternberg, R. J. (1988). The nature of creativity: Contemporary psychological perspectives. New York: Cambridge University Press. Tsai, P. S., Tsai, C. C., & Hwang, G. J. (2011). The correlates of Taiwan teachers’ epistemological beliefs concerning internet environments, online search strategies, and search outcomes. The Internet and Higher Education, 14(1), 54–63. Tüzün, H., Yılmaz-Soylu, M., Karakus, T., Inal, Y., & Kızılkaya, G. (2009). The effects of computer games on primary school students’ achievement and motivation in geography learning. Computers & Education, 52(1), 68–77. van Eck, R. (2007). Six ideas in search of a discipline. In B. E. Shelton, & D. A. Wiley (Eds.), The design and use of simulation computer games in education (pp. 31–60). Rotterdam: Sense Publishing. van Eck, R., & Dempsey, J. (2002). The effect of competition and contextualized advisement on the transfer of mathematics skills in a computer-based instructional simulation game. Educational Technology Research and Development, 50(3), 23–41. Villalta, M., Gajardo, I., Nussbaum, M., Andreu, J. J., Echeverría, A., & Plass, J. (2011). Design guidelines for classroom multiplayer presential games (CMPG). Computers & Education, 57, 2039–2053. Wang, L. C., & Chen, M. P. (2010). The effects of game strategy and preference-matching on flow experience and programming performance in game-based learning. Innovations in Education and Teaching International, 47(1), 39–52. Warren, S. J., Dondlinger, M. J., McLeod, J., & Bigenho, C. (2012). Opening the door: an evaluation of the efficacy of a problem-based learning game. Computers & Education, 58(1), 397–412. Wu, P. H., Hwang, G. J., Milrad, M., Ke, H. R., & Huang, Y. M. (2011). An innovative concept map approach for improving students’ learning performance with an instant feedback mechanism. British Journal of Educational Technology. doi:10.1111/j.1467-8535.2010.01167.x. Yang, Y.-T. C. (2012). Building virtual cities, inspiring intelligent citizens: digital games for developing students’ problem solving and learning motivation. Computers & Education, 59(2), 365–377. Yien, J. M., Hung, C. M., Hwang, G. J., & Lin, Y. C. (2011). A game-based learning approach to improving students’ learning achievements in a nutrition course. Turkish Online Journal of Educational Technology, 10(2), 1–10. Young-Jin, L. (2012). Developing an efficient computational method that estimates the ability of students in a web-based learning environment. Computers & Education, 58(1), 579–589. doi:10.1016/j.compedu.2011.09.008. Yu, W. F., She, H. C., & Lee, Y. M. (2010). The effects of web-based/non-web-based problem-solving instruction and high/low achievement on students’ problem-solving ability and biology achievement. Innovations in Education and Teaching International, 47(2), 187–199. Zamani, B. E., & Shoghlabad, R. G. (2010). Acceptance of the internet by Iranian business management students. British Journal of Educational Technology, 41(5), E96–E100.