- Email: [email protected]
A Review of Heuristics Evaluation Component for Mobile Educational Games
Available online at www.sciencedirect.com
ScienceDirect ScienceDirect
Procedia Computer Science 00 (2019) 000–000 Procedia Computer Science 00 (2019) 000–000
Available online at www.sciencedirect.com
ScienceDirect
www.elsevier.com/locate/procedia www.elsevier.com/locate/procedia
Procedia Computer Science 161 (2019) 1028–1035
The Fifth Information Systems International Conference 2019 The Fifth Information Systems International Conference 2019
A Review of Heuristics Evaluation Component for Mobile A Review of Heuristics Evaluation Component for Mobile Educational Games Educational Games a a
Nur Marissa Vee Senapa,b, *, Roslina Ibrahima Nur Marissa Vee Senapa,b,*, Roslina Ibrahima
Razak Faculty of Technology and Informatics, Universiti Teknologi Malaysia, 54100 Kuala Lumpur, Malaysia of Computing, UniversitiUniversiti Utara Malaysia, 06010 Sintok 54100 Kedah,Kuala Malaysia Razak FacultybSchool of Technology and Informatics, Teknologi Malaysia, Lumpur, Malaysia b School of Computing, Universiti Utara Malaysia, 06010 Sintok Kedah, Malaysia
Abstract Abstract Mobile educational games research has gained a lot of attention among researchers. This is due to mobile games popularity and Mobile educational research has gained a lot of attention among researchers. is due to game mobileisgames and engagement among games younger generation. However, developing an effective mobileThis educational alwayspopularity a challenge. Heuristics evaluation is one of the most However, popular and cost-effective usabilitymobile inspection method.game Numerous sets of heuristics engagement among younger generation. developing an effective educational is always a challenge. Heuristics evaluation have is onebeen of the most popular and game cost-effective method. Numerous sets of (usability principles) proposed for various productsusability ranging inspection from general to specific. However, theheuristics existing (usability have beencomponents proposed for various game productsofranging generalgame. to specific. However, the existing heuristics principles) lack comprehensive to evaluate the usability mobile from educational This study therefore aims to reviews previous studies that proposed heuristics for evaluation of games, educational gamesgame. and mobile application in aaims bid to heuristics lack comprehensive components to evaluate the usability of mobile educational This study therefore reviews previous studies that proposed heuristics for evaluation of games, educational games and mobile application in a bid to identify the relevant heuristics components for developing heuristics of mobile educational games. identify the relevant heuristics components for developing heuristics of mobile educational games. © 2019 The Authors. Published by Elsevier B.V. © 2019 The Authors. Published by Elsevier B.V. © 2019 The Authors. by Elsevier B.V. This is an open accessPublished article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/) This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/) This is an open access article under CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/) Peer-review under responsibility ofthe thescientific scientific committee The Fifth Information Systems International Conference 2019 Peer-review under responsibility of committee ofofThe Fifth Information Systems International Conference 2019. Peer-review under responsibility of the scientific committee of The Fifth Information Systems International Conference 2019 Keywords: Heuristics Evaluation; Usability Evaluation; Mobile Educational Games; Game Evaluation; Game Design Keywords: Heuristics Evaluation; Usability Evaluation; Mobile Educational Games; Game Evaluation; Game Design
1. Introduction 1. Introduction In this digital age, the use of games application beyond entertainment is popular. It is usually known as Serious In this digital age, the or useGame-Based of games application beyondterms entertainment is popular. It is usually Serious Game, Educational Game Learning. These are often used interchangeably and known withoutas precision. Game, Gamefor or teaching Game-Based Learning. terms are often used interchangeably and precision. SeriousEducational games is a game and learning inThese various disciplines [1]. A serious games that is without used in Education Serious games is a game for teachingGames and learning in various A serious games that is used in Education discipline is known as Educational (EG) [2]. EG is disciplines defined as [1]. game developed purposely for teaching and discipline is known as Educational Games (EG) [2]. EG is defined as game developed purposely for teaching and
* Corresponding author. Tel.: +60-11-271-531-99. address:author. [email protected] * E-mail Corresponding Tel.: +60-11-271-531-99. E-mail address: [email protected] 1877-0509 © 2019 The Authors. Published by Elsevier B.V. This is an open access under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/) 1877-0509 © 2019 Thearticle Authors. Published by Elsevier B.V. Peer-review under responsibility of the scientific committee of The Fifth Information Systems International Conference 2019 This is an open access article under CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/) Peer-review under responsibility of the scientific committee of The Fifth Information Systems International Conference 2019 1877-0509 © 2019 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/) Peer-review under responsibility of the scientific committee of The Fifth Information Systems International Conference 2019. 10.1016/j.procs.2019.11.213
2
Nur Marissa Vee Senap et al. / Procedia Computer Science 161 (2019) 1028–1035 Author name / Procedia Computer Science 00 (2019) 000–000
1029
learning [3] with aims to deliver specific knowledge to the player [4]. While, the use of games as alternative method in teaching and learning is called Game-Based Learning [5]. In this study, the term EG will be used hereinafter. The rise of mobile devices likes smartphones, touch phones or tablets has booming EG for mobile platform which known as Mobile Educational Games (MEG) [6]. This has indirectly generated the global games market growth. According to [7], the global games market revenues is expected to grow from $151.9 billion in 2019 to $180.1 billion in 2021 with biggest market share held by mobile games (59%). MEG is seen as beneficial teaching and learning method as it can improve student’s performance [8, 9]. Its mobility allows education to happen at anytime and anywhere. However, developing MEG is challenging for game designer since mobile devices are popular with impediment in term of interruptions, screen size, input or control interfaces which can hinder usability [10]. Ensuring usability of EG alone is already difficult since it must satisfy both fun and educational requirement [11]. Thus, usability evaluation needs to be conducted to ensure maximal usability of MEG. It is important to evaluate the usability of EG to ensure its success [12]. Usability is defined as “extent to which a system, product or service can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use” [13]. There are various methods can be used to evaluate usability. User testing and inspection method are the most popular usability evaluation method [14, 15]. Heuristics evaluation (HE) is the most frequently used usability inspection method [16, 17]. This method involves usability experts (3 to 5 persons) inspect the interface design against established usability principles (Heuristics) to identify usability problem [18, 19]. It is cost-effective and can be conducted from the earlier design phase [20]. For this reason, many heuristics has been proposed in previous studies to support the usability evaluation of variety game product. Therefore, this paper aims to review the existing heuristics components (evaluation aspect) that are relevant for developing heuristics for MEG. This section explains the background and aim of the research. Section 2 describes usability evaluation of EG. Followed by section 3, discussed the existing heuristics for games, educational games and mobile application and its components. Section 4 discusses the analysis of the discussed component. Finally, the last section will summarize the study and future work. 2. Usability evaluation of educational games In the realm of games, HE and user testing method are popularly used to evaluate usability [12, 21]. HE method involves experts inspect the underdeveloped game (without real user) from the earlier design phase where game prototype is not yet ready. While, user testing or playtesting usually conducted at the later phase with real user by having them playing fully ready game prototype to check any usability problem faced. It is useful to detect any overlooked usability problems by game practitioners as they might not always be able to fully imitate real user’s interaction. However, this method is rather expensive, time and resource consuming. According to [22] in their study on expert review method in game evaluation, it is hard to get real user involvement in the evaluation session and fixing any problem when the game is almost complete are difficult and expensive. As time and budget are crucial in game development project, game practitioner use HE method as alternative to evaluate game [16]. HE is normally conducted to find usability problems in the software interface [14] but for game, the evaluation is beyond that. Game comprises of several components such as game play, story, mechanics and so on [23]. Due to this contrast, games require set of heuristics for their own [24]. Consequently, many heuristics set has been proposed in the previous studies to assess the usability of various game products. However, evaluating specific game using generic heuristics might overlook the usability problems that related to specific aspects of that particular game. Thus, few avenues proposed heuristics concerned specific aspects of game to support the generic one so that more usability problem can be revealed. 3. Related works This section reported existing studies on heuristics for three context of technology which are games, educational games and mobile application. In this study, we focused on heuristics for video games play on computer, mobile devices and online that are published in English. The heuristics components proposed in these studies were derived and explained based on the aforementioned technology context.
Nur Marissa Vee Senap et al. / Procedia Computer Science 161 (2019) 1028–1035 Author name / Procedia Computer Science 00 (2019) 000–000
1030
3
3.1. Usability heuristics component for games When it comes to first heuristics concerning computer video games (CG), it is dates back to [25] in 1980 whom suggested the heuristics for designing instructional CG. Later, [26] proposed heuristics to inspect the usability of fun CG. In 2004, [27] proposed a comprehensive heuristics for playable CG named Heuristic Evaluation for Playability (HEP). Followed by [28] whom proposed generic heuristics for any kind of CG. Later in 2008, [29] and [30] proposed heuristics for CG design. While, [31] introduced heuristics for new CG genres (Real-time strategy, Action adventure & First-person shooters) named Game Playability Principles (PLAY). For mobile games (MG), [32] proposed the first generic heuristics and [33] updated this heuristics by added new heuristics concerning multi-player. Later on, [34] further explored these MG heuristics by proposed new heuristics focused on what has not been covered. For network games (NG), [35] proposed the first heuristics named Networked Game Heuristics (NGH). In the same year, [36] also proposed heuristics for NG but focused on the one played on social media, Facebook. Similarly, [37] also proposed heuristics for NG played on Facebook including Twitter and Myspace. The summary of the proposed heuristics components for these studies is summarized in Table 1. Table 1. Summary of studies on heuristics component for games. Study
Heuristics name
Component(s)
Computer/ Video Games [25]
Heuristic for instructional computer games
Challenge, Fantasy, Curiosity
[26]
Heuristics for video games
Game interface, Game mechanics, Game play
[27]
Heuristics for playable games (HEP)
Game play, Game story, Mechanics, Usability
[28]
Heuristics for video games
Usability, Graphical user interface, Playability
[29]
Heuristics for video games design
Usability
[30]
Heuristics for computer game design
Usability
[31]
Heuristics for new genre video games (PLAY)
Game play, Entertainment, Usability & Game mechanics
Mobile Games [32]
Heuristics for mobile games
Usability, Mobility, Game play
[33]
Heuristics for mobile multi-player games
Playability (multi-player)
[34]
Heuristics for mobile games
Game play, Mobility, Game usability
Network/ Social Games [35]
Heuristics for networked games (NGH)
Usability (multi-player)
[36]
Heuristics for social-network games
Usability
[37]
Heuristics for social-network games
Social
The proposed heuristics components are Challenge, Fantasy, Curiosity, Usability, Interface, Mechanics, Game Play/ Playability, Story, Entertainment, Mobility and Social. Challenge is the series of task player must overcome to achieve game’s goal [25], whilst Fantasy and Curiosity refer to game’s environment that attract player to play more [25]. Usability component concerns on game’s control and interface [32]. The Interface is the element that allow player to interact with game world [26]. While, Mechanics is the rules that determine what operation player can perform [32]. The interaction of player with the game is called Game play [34] and to ensure that game is playable its Playability is usually evaluated. Thus, the term Playability will be used hereinafter. Meanwhile, Story is the sequence of events that happen in game world. Entertainment concerns game’s element that promote emotional connection and immersion to player [31]. For Mobility, this component concerns things that can affect mobility of the game [38]. Lastly, Social component concerns the social aspect of game [37]. The frequencies of the proposed components are presented in Table 2.
4
Nur Marissa Vee Senap et al. / Procedia Computer Science 161 (2019) 1028–1035 Author name / Procedia Computer Science 00 (2019) 000–000
1031
Table 2. Frequencies of games heuristics components across reviewed studies. #
Component
1
Challenge
2
Fantasy
3
Curiosity
4
Interface
5
Mechanics
6
Playability
7
Story
8
Usability
9
Mobility
10
Entertainment
11
Social
[25]
[26]
[27]
[28]
[29]
[30]
[31]
[32]
[33]
[34]
[35]
[36]
[37]
✓
1
✓ ✓
ƒ 1
✓ ✓ ✓
✓
✓
✓
✓
✓
✓
✓
1 ✓
✓
✓
2
✓
✓
✓
✓
✓
✓
✓
3
✓ ✓ ✓
7 ✓
1
✓
9 2 ✓
1 1
3.2. Usability heuristics component for educational games The first heuristics for EG is the Playability Heuristics Evaluation for Educational Computer Games (PHEG) proposed by [39] in 2010. Later, [40] proposed another heuristic for EG named Heuristic Evaluation for Educational Games (HEEG). Meanwhile, [41] proposed an extension to HEP framework by proposed two new heuristics component for digital game based learning. In 2017, [42] suggested heuristics for evaluation of balanced EG. For MEG context, [43] adapted the MG heuristics by [32] and proposed one new educational component to it. Recently, [44] proposed heuristics for designing MEG for older adults. The synopsis of the proposed heuristics components in this context is presented in Table 3. Table 3. Summary of studies on heuristics components for educational games. Study
Heuristics name
Component(s)
Educational Computer/Video Game [39]
Heuristics for educational game (PHEG)
Interface, Pedagogical, Content, Playability, Multimedia
[40]
Heuristic for educational game (HEEG)
Game play, Enjoyment, Usability & Mechanics, Educational design
[41]
Heuristics for digital game-based learning
Characters /Graphic, Pedagogical effectiveness
[42]
Heuristics for educational video game
Playability
Mobile Educational Games [43]
Heuristics for mobile game-based learning
Game play, Mobility, Game usability, Learning content
[44]
Heuristics for mobile serious games
Game Play
In this context, the proposed heuristics components are Interface, Pedagogical/ Educational design/ Pedagogical effectiveness, Content/ Learning content, Playability, Multimedia, Enjoyment, Usability, Mechanics, Character/ Graphic and Mobility. Some of the components are similar to Games context: Interface, Playability, Usability, Mechanics & Entertainment (Enjoyment) because they are essential for all kind of games. For EG, Pedagogical, Content, Multimedia and Characters component are proposed to cater the educational requirements of games. Pedagogical component looks at the element of educational and learning in games. Content component are evaluated to test whether game provide feedback, good content and proper learning materials. While, Multimedia component is to check the representation of text, image, audio, animation and video in games. The Character component refer to the design of player’s role in game world. The frequencies of these proposed heuristics components for EG are depicted in Table 4.
Nur Marissa Vee Senap et al. / Procedia Computer Science 161 (2019) 1028–1035 Author name / Procedia Computer Science 00 (2019) 000–000
1032
5
Table 4. Frequencies of educational games heuristics component across reviewed studies. #
Component
1
Interface
2
Pedagogical
3
Content
4
Playability
5
Multimedia
[39] ✓
6
Entertainment Usability
8
Mechanics
9
Character
10
Mobility
✓
✓
✓
✓
✓
7
[40]
✓
[41]
[42]
[43]
[44]
1
✓
3
✓
✓
✓
✓ ✓
2
✓
5 1 1
✓
✓
ƒ
✓
2 1 1
✓
1
3.3. Usability heuristics component for mobile application The advancement of mobile devices has drove new study on how to design a good usability mobile application. In 2012, [45] proposed heuristics for touchscreen-based mobile devices (TMD). Then, it was further validated and resulted in a proposal of new heuristics [46]. They further updated the heuristic’s definition according to the result of validation [47]. Later on, [48] proposed heuristics for mobile devices. Similarly, [49] suggested heuristics for touchscreen mobile phone and tablet. On the other hand, [50] proposed heuristics for smartphone mobile application named Smartphone Mobile Application heuRisTics (“SMART”). Likewise, [51] refined TMD and fit it to smartphone context and named it as SMArtphone’s uSability Heuristics (SMASH). In 2016, [52] proposed heuristics to evaluate control feature of mobile games. Recently, [18] proposed heuristics for mobile learning application. The summary of heuristics components proposed in these studies is presented in Table 5. Table 5. Summary of studies on heuristics components for mobile application. Study
Heuristics name
Component
[45]
Heuristics for TMD (prelim)
Usability
[46]
Heuristics for TMD (proposal)
Usability
[47]
Heuristics for TMD (update)
Usability
[48]
Heuristics for mobile interface
User Interface
[49]
Heuristics for mobile interface
User Interface
[50]
Heuristics for smartphone mobile app (SMART)
Usability
[51]
Heuristics for smartphones mobile app (SMASH)
Usability
[52]
Heuristics for mobile games
Control
[18]
Heuristics for mobile learning application
Usability
Based on studies reviewed for mobile application, some of the study focused on evaluation of general usability of mobile application, while few studies concerned in evaluating the specific features of the mobile devices like interface and control. Table 6 shows the frequencies of these proposed components across the studies. Table 6. Frequencies of mobile application heuristics component across reviewed studies. #
Components
1
Usability
2
Interface
3
Control
[45]
[46]
[47]
✓
✓
✓
[48] ✓
[49] ✓
[50]
[51]
✓
✓
[52]
[18] ✓
✓
ƒ 6 2 1
6
Nur Marissa Vee Senap et al. / Procedia Computer Science 161 (2019) 1028–1035 Author name / Procedia Computer Science 00 (2019) 000–000
1033
4. Discussion The review of heuristics component for Games in Table 2 has shown that Usability and Playability component stood out the most. The next outstanding heuristics component is Mechanics followed by Interface and Mobility, then Curiosity, Challenge, Fantasy, Story, Entertainment and Social components. In EG context, the review revealed that Playability was the most evaluated heuristics component followed by Pedagogical component as presented in Table 4. Both components are noteworthy when it comes to usability evaluation of EG. Content and Usability components were gained attention too. The least proposed heuristics components for EG were Interface, Mobility, Multimedia, Entertainment, Mechanics and Character component. Meanwhile, Table 6 has shown that Usability component is the most obvious heuristics component for Mobile Application. Interface and Control component also been considered when evaluating usability of mobile application. In result, it was not surprising that Usability and Playability component was dominant in Games context since it was fundamental in any game usability. The Interface and Control component usually integrated with Usability component in some studies because well-designed interface and control will contribute to good usability of games or application. Curiosity, Challenge and Fantasy are game principles suggested by earlier study for designing fun games but were no longer evaluated as independent components in later studies. It usually evaluated in Playability or Entertainment component. Next, Mechanics components only evaluated when it comes to games. For Mobility component, it is only relevant when evaluating usability of game or application designed for mobile devices. While, the Story component is usually relating to Playability and Mechanics component. Social component comes into account only when evaluating social-network games. Lastly, Pedagogical, Content, Multimedia and Character component were significant for EG, however there is none of the reviewed studies on MEG that address these components in detail. Thus, the findings of this study might be valuable for game designer or future research in selecting proper heuristics component for MEG heuristics development. Fig. 1 illustrated the components suggested for MEG context by took into account all relevant components evaluated in games, educational games and mobile application context.
Fig. 1. Consolidation of suggested heuristics components for MEG.
5. Conclusion and future work The pervasiveness of mobile devices has drove new innovations of EG for mobile environment. Ensuring the usability of MEG is challenging due to nature of EG itself and limitations of mobile devices. There are many heuristics available in the literature to evaluate usability of various games product. These heuristics however cannot be applied directly in MEG context. This paper reviews existing heuristics component for Games, Educational Games and Mobile
1034
Nur Marissa Vee Senap et al. / Procedia Computer Science 161 (2019) 1028–1035 Author name / Procedia Computer Science 00 (2019) 000–000
7
Application and represent the consolidation of heuristics components that are relevant for MEG. In future, there is a need to design a set of comprehensive heuristics that cover all of relevant components discussed in this paper for ideal heuristics evaluation of MEG. Moreover, further validation is required to support the findings of this study and to identify if there any new component to develop heuristics for MEG. Acknowledgements This work was supported by the Skim Latihan Akademik IPTA (SLAI), Ministry of Higher Education, Malaysia. References [1] Paiva, Ana C. R, Nuno H Flores, André G Barbosa, and Tânia P. B Ribeiro. (2016). “iLearnTest–framework for Educational Games.” Procedia-Social and Behavioral Sciences 228: 443-448. [2] Backlund, Per and Maurice Hendrix. (2013). “Educational Games-Are They Worth The Effort? A Literature Survey of the Effectiveness of Serious Games” in 5th Int. Conf. on Games and Virtual Worlds for Serious Applications (VS-GAMES), IEEE. pp. 1-8. [3] Ibrahim, Roslina, Khalili Khalil, and Azizah Jaafar. (2011). “Towards educational games acceptance model (EGAM): A Revised Unified Theory of Acceptance and Use of Technology (UTAUT).” Int. Journal of Research and Reviews in Computer Science 2 (3): 839-846. [4] Roungas, Bill and Fabiano Dalpiaz. (2015) “A model-driven framework for educational game design” in Int. Conf. on Games and Learning Alliance, Springer. pp. 1-11. [5] Hainey, Thomas, Thomas M Connolly, Elizabeth A Boyle, Amanda Wilson, and Aisya Razak. (2016) “A Systematic Literature Review of Games-Based Learning Empirical Evidence in Primary Education.” Computers & Education 102: 202-223. [6] Giannakas, Filippos, Georgios Kambourakis, Andreas Papasalouros, and Stefanos Gritzalis. (2018) “A Critical Review of 13 Years of Mobile Game-Based Learning.”Educational Technology Research and Development 66 (2): 341-384. [7] Tom, Wijman. (2019) “Global Games Market Revenues 2018.” Available from: https://newzoo.com/insights/articles/global-games-marketreaches-137-9-billion-in-2018-mobile-games-take-half/. [8] Perry, Gabriela Trindade, Ci Costa Kulpa, ET Pinheiro, and Marcelo Leandro Eichler. (2012) “Lessons from An Educational Game Usability Evaluation.” International Journal of Interactive Mobile Technologies 6 (2): 23-28. [9] Huang, Yu-Lan, Dian-Fu Chang, and Berlin Wu. (2017) “Mobile Game-Based Learning with a Mobile App: Motivational Effects and Learning Performance.”Journal of Advanced Computational Intelligence and Intelligent Informatics 21 (6): 963-970. [10] Hussain, Azham, Abdulkarim Saleh, Abdussalam Taher, Imman Ahmed, and Mohammed Lammasha (2015). “Usability Evaluation Method for Mobile Learning Application Using Agile: A Systematic Review.” Jurnal Teknologi 77 (5): 51-56. [11] Mohamed, Hasiah and Azizah Jaafar. (2010) “Challenges in the Evaluation of Educational Computer Games”, in 2010 International Symposium on Information Technology. IEEE. 1-6. [12] Al Fatta, Hanif, Zulisman Maksom, and M Hafiz Zakaria. (2018) “Systematic Literature Review On Usability Evaluation Model Of Educational Games: Playability, Pedagogy, And Mobility Aspects.” J. Theoretical & Applied Information Technology 96 (14): 4677-4689. [13] ISO (2018). “ISO 9241-11:2018.” Available from: https://www.iso.org/obp/ui/#iso:std:iso:9241:-11:ed-2:v1:en. [14] Jimenez, Cristhy, Pablo Lozada, and Pablo Rosas. (2016) “Usability Heuristics: A Systematic Review” in 2016 IEEE 11th Colombian Computing Conference (CCC). IEEE. pp. 1-8. [15] Dourado, Marcos Antonio Durães and Edna Dias Canedo. (2018) “Usability Heuristics for Mobile Applications - A Systematic Review” in 20th Int. Conf. on Enterprise Information Systems. pp. 483-494. [16] Korhonen, Hannu. (2016) “Evaluating Playability of Mobile Games with The Expert Review Method”, University of Tampere. [17] Mugisha, Alice, Victoria Nankabirwa, Thorkild Tylleskär, and Ankica Babic. (2019) “A Usability Design Checklist for Mobile Electronic Data Capturing Forms: The Validation Process.” BMC Medical Informatics and Decision Making 19 (4): 1-11. [18] Kumar, Bimal Aklesh and Munil Shiva Goundar. (2019) “Usability Heuristics for Mobile Learning Applications.” Education and Information Technologies 24 (2): 1819-1833. [19] Salman, Hasanin Mohammed, Wan Fatimah Wan Ahmad, and Suziah Sulaiman. (2018) “Usability Evaluation of the Smartphone User Interface in Supporting Elderly Users from Experts’ Perspective.” IEEE Access 6: 22578-22591. [20] Fortes Tondello, Gustavo, Dennis Kappen, Elisa Mekler, Marim Ganaba, and Lennart Nacke. (2016) “Heuristic Evaluation for Gameful Design.” CHI PLAY Companion'16. pp. 315-323. [21] Hussain, Azham, Sharaf A. A. Abbas, M. S. Abdulwaheed, Rammah G. Mohammed, and Adil A. Abdulhussein. (2015) “Usability Evaluation of Mobile Game Applications: A Systematic Review.” Int. Journal of Computer and Information Technology 4 (03): 547-551. [22] Korhonen, Hannu, Janne Paavilainen, and Hannamari Saarenpää. (2009) “Expert Review Method In Game Evaluations: Comparison of Two Playability Heuristic Sets”, in MindTrek' 2009. ACM. pp. 74-81. [23] Aleem, Saiqa, Luiz Fernando Capretz, and Faheem Ahmed. (2016) “Game Development Software Engineering Process Life Cycle: A Systematic Review.” Journal of Software Engineering Research and Development 4 (1): 1-30. [24] Al-Azawi, Rula, Aladdin Ayesh, and Mohaned Al Obaidy. (2013) “Generic Evaluation Framework for Games Development Methodology”, in 3rd Int. Conf. on Communications and Information Technology (ICCIT), IEEE. pp. 55-60.
8
Nur Marissa Vee Senap et al. / Procedia Computer Science 161 (2019) 1028–1035 Author name / Procedia Computer Science 00 (2019) 000–000
1035
[25] Malone, Thomas W. (1980) “What Makes Things Fun to Learn? Heuristics for Designing Instructional Computer Games”, in Proceedings of the 3rd ACM SIGSMALL Symposium and the First SIGPC Symposium on Small Systems, ACM. pp. 162-169. [26] Federoff, Melissa A. (2002) “Heuristics and Usability Guidelines for the Creation and Evaluation of Fun in Video Games”, Indiana University. [27] Desurvire, Heather, Martin Caplan, and Jozsef A Toth. (2004) “Using Heuristics to Evaluate The Playability of Games” in CHI'04. pp. 1509-1512. [28] Schaffer, Noah. (2007) “Heuristics for Usability in Games”, in White Paper. pp. 1-30. [29] Pinelle, David, Nelson Wong, and Tadeusz Stach. (2008) “Heuristic Evaluation for Games: Usability Principles for Video Game Design”, in Proc. of the SIGCHI Conference on Human Factors in Computing Systems, ACM. pp. 1453-1462. [30] Brown, Michael. (2008) “Evaluating computer game usability: Developing Heuristics Based On User Experience”, in IHCI conference. pp. 1-7. [31] Desurvire, Heather and Charlotte Wiberg. (2009) “Game usability heuristics (PLAY) for Evaluating and Designing Better Games: The Next Iteration”, in Int. Conf. on Online Communities and Social Computing. Springer. pp. 557-566. [32] Korhonen, Hannu and Elina MI Koivisto. (2006) “Playability Heuristics for Mobile Games”, in Proc. of the 8th Conf. on HCI with Mobile Devices and Services, ACM. pp. 9-16. [33] Korhonen, Hannu and Elina MI Koivisto. (2007) “Playability Heuristics for Mobile Multi-Player Games”, in 2nd Int. Conf. on Digital Interactive Media in Entertainment and Arts, ACM. pp. 28-35. [34] Soomro, Sarmad, Wan Fatimah Wan Ahmad, and Suziah Sulaiman. (2012) “A Preliminary Study on Heuristics for Mobile Games” in 2012 Int. Conf. on Computer & Information Science (ICCIS), IEEE. pp. 1030-1035. [35] Pinelle, David, Nelson Wong, Tadeusz Stach, and Carl Gutwin. (2009) “Usability Heuristics for Networked Multiplayer Games”, in GROUP' 09, ACM. pp. 169-178. [36] Papaloukas, Spyridon, Kiriakos Patriarcheas, and Michalis Xenos. (2009) “Usability Assessment Heuristics in New Genre Videogames”, in 2009 13th Panhellenic Conference on Informatics, IEEE. pp. 202-206. [37] Paavilainen, J. (2010) “Critical Review on Video Game Evaluation Heuristics: Social Games Perspective”, in FuturePlay'10, ACM. pp. 5665. [38] Zaibon, Syamsul Bahrin. (2015) “User Testing on Game Usability, Mobility, Playability, and Learning Content of Mobile Game-Based Learning.” Jurnal Teknologi 77 (29): 131-139. [39] Mohamed, Hasiah and Azizah Jaafar. (2010) “Heuristics Evaluation in Computer Games” in 2010 Int. Conf. on Information Retrieval & Knowledge Management (CAMP), IEEE. pp. 188-193. [40] Barbosa, Marcelo B, Andreza B Rego, and Igor de Medeiros. (2015) “HEEG: Heuristic Evaluation for Educational Games”, in SBGames' 2015. pp. 224-227. [41] Guo, Yan Ru and Dion H-L Goh. (2016) “Heuristic Evaluation of an Information Literacy Game: Poster” in Proc. Assoc. Inf. Sci. Tech. American Society for Information Science 55 (1): 1-4. [42] Ibrahim, Amer. (2017) “Playability Heuristics Evaluation For Educational Video Games.” Int. J. Of Core Eng.& Management 4 (6): 28-39. [43] Zaibon, Syamsul Bahrin and Norshuhada Shiratuddin. (2010) “Heuristics Evaluation Strategy for Mobile Game-Based Learning”, in WMUTE'10, IEEE. pp. 127-131. [44] Machado, Mônica da Consolação, Ronan L Rodrigues Ferreira, and Lucila Ishitani. (2018) “Heuristics and Recommendations for the Design of Mobile Serious Games for Older Adults.” International Journal of Computer Games Technology. [45] Inostroza, Rodolfo, Cristian Rusu, Silvana Roncagliolo, Cristhy Jimenez, and Virginica Rusu. (2012) “Usability Heuristics for TouchscreenBased Mobile Devices”, in 2012 9th Int. Conf. on Information Technology-New Generations. IEEE. pp. 662-667. [46] Inostroza, Rodolfo, Cristian Rusu, Silvana Roncagliolo, Cristhy Jiménez, and Virginica Rusu. (2012) “Usability Heuristics Validation Through Empirical Evidences: A Touchscreen-Based Mobile Devices Proposal”, in SCCC' 12. IEEE. pp. 60-68. [47] Inostroza, Rodolfo, Cristian Rusu, Silvana Roncagliolo, and Virginica Rusu. (2013) “Usability Heuristics For Touchscreen-Based Mobile Devices: Update” in ChileCHI'12, ACM. pp. 24-29. [48] Neto, Olibário Machado and Maria Da Graça Pimentel. (2013) “Heuristics for the Assessment of Interfaces of Mobile Devices”, in WebMedia. ACM. 93-96. [49] Gómez, Rosa Yáñez , Daniel Cascado Caballero, and José-Luis Sevillano. (2014) “Heuristic Evaluation on Mobile Interfaces: A New Checklist.” The Scientific World Journal. pp. 1-19. [50] Joyce, Ger and Mariana Lilley. (2014) “Towards the Development of Usability Heuristics for Native Smartphone Mobile Applications”, in Int. Conf. of Design, User Experience, and Usability, Springer. pp. 465-474. [51] Inostroza, Rodolfo, Cristian Rusu, Silvana Roncagliolo, Virginica Rusu, and César A Collazos. (2016) “Developing SMASH: A Set of Smartphone's Usability Heuristics.” Computer Standards & Interfaces 43: 40-52. [52] Daud, Ezri Hielmi Che, Shamsul Anuar Mokhtar, and Farahwahida Mohd. (2016) “Usability Heuristics in the Context of Control Features on Mobile Games”, in 2016 Int. Conf. on Information and Communication Technology (ICICTM), IEEE. pp. 194-197.
ScienceDirect ScienceDirect
Procedia Computer Science 00 (2019) 000–000 Procedia Computer Science 00 (2019) 000–000
Available online at www.sciencedirect.com
ScienceDirect
www.elsevier.com/locate/procedia www.elsevier.com/locate/procedia
Procedia Computer Science 161 (2019) 1028–1035
The Fifth Information Systems International Conference 2019 The Fifth Information Systems International Conference 2019
A Review of Heuristics Evaluation Component for Mobile A Review of Heuristics Evaluation Component for Mobile Educational Games Educational Games a a
Nur Marissa Vee Senapa,b, *, Roslina Ibrahima Nur Marissa Vee Senapa,b,*, Roslina Ibrahima
Razak Faculty of Technology and Informatics, Universiti Teknologi Malaysia, 54100 Kuala Lumpur, Malaysia of Computing, UniversitiUniversiti Utara Malaysia, 06010 Sintok 54100 Kedah,Kuala Malaysia Razak FacultybSchool of Technology and Informatics, Teknologi Malaysia, Lumpur, Malaysia b School of Computing, Universiti Utara Malaysia, 06010 Sintok Kedah, Malaysia
Abstract Abstract Mobile educational games research has gained a lot of attention among researchers. This is due to mobile games popularity and Mobile educational research has gained a lot of attention among researchers. is due to game mobileisgames and engagement among games younger generation. However, developing an effective mobileThis educational alwayspopularity a challenge. Heuristics evaluation is one of the most However, popular and cost-effective usabilitymobile inspection method.game Numerous sets of heuristics engagement among younger generation. developing an effective educational is always a challenge. Heuristics evaluation have is onebeen of the most popular and game cost-effective method. Numerous sets of (usability principles) proposed for various productsusability ranging inspection from general to specific. However, theheuristics existing (usability have beencomponents proposed for various game productsofranging generalgame. to specific. However, the existing heuristics principles) lack comprehensive to evaluate the usability mobile from educational This study therefore aims to reviews previous studies that proposed heuristics for evaluation of games, educational gamesgame. and mobile application in aaims bid to heuristics lack comprehensive components to evaluate the usability of mobile educational This study therefore reviews previous studies that proposed heuristics for evaluation of games, educational games and mobile application in a bid to identify the relevant heuristics components for developing heuristics of mobile educational games. identify the relevant heuristics components for developing heuristics of mobile educational games. © 2019 The Authors. Published by Elsevier B.V. © 2019 The Authors. Published by Elsevier B.V. © 2019 The Authors. by Elsevier B.V. This is an open accessPublished article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/) This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/) This is an open access article under CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/) Peer-review under responsibility ofthe thescientific scientific committee The Fifth Information Systems International Conference 2019 Peer-review under responsibility of committee ofofThe Fifth Information Systems International Conference 2019. Peer-review under responsibility of the scientific committee of The Fifth Information Systems International Conference 2019 Keywords: Heuristics Evaluation; Usability Evaluation; Mobile Educational Games; Game Evaluation; Game Design Keywords: Heuristics Evaluation; Usability Evaluation; Mobile Educational Games; Game Evaluation; Game Design
1. Introduction 1. Introduction In this digital age, the use of games application beyond entertainment is popular. It is usually known as Serious In this digital age, the or useGame-Based of games application beyondterms entertainment is popular. It is usually Serious Game, Educational Game Learning. These are often used interchangeably and known withoutas precision. Game, Gamefor or teaching Game-Based Learning. terms are often used interchangeably and precision. SeriousEducational games is a game and learning inThese various disciplines [1]. A serious games that is without used in Education Serious games is a game for teachingGames and learning in various A serious games that is used in Education discipline is known as Educational (EG) [2]. EG is disciplines defined as [1]. game developed purposely for teaching and discipline is known as Educational Games (EG) [2]. EG is defined as game developed purposely for teaching and
* Corresponding author. Tel.: +60-11-271-531-99. address:author. [email protected] * E-mail Corresponding Tel.: +60-11-271-531-99. E-mail address: [email protected] 1877-0509 © 2019 The Authors. Published by Elsevier B.V. This is an open access under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/) 1877-0509 © 2019 Thearticle Authors. Published by Elsevier B.V. Peer-review under responsibility of the scientific committee of The Fifth Information Systems International Conference 2019 This is an open access article under CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/) Peer-review under responsibility of the scientific committee of The Fifth Information Systems International Conference 2019 1877-0509 © 2019 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/) Peer-review under responsibility of the scientific committee of The Fifth Information Systems International Conference 2019. 10.1016/j.procs.2019.11.213
2
Nur Marissa Vee Senap et al. / Procedia Computer Science 161 (2019) 1028–1035 Author name / Procedia Computer Science 00 (2019) 000–000
1029
learning [3] with aims to deliver specific knowledge to the player [4]. While, the use of games as alternative method in teaching and learning is called Game-Based Learning [5]. In this study, the term EG will be used hereinafter. The rise of mobile devices likes smartphones, touch phones or tablets has booming EG for mobile platform which known as Mobile Educational Games (MEG) [6]. This has indirectly generated the global games market growth. According to [7], the global games market revenues is expected to grow from $151.9 billion in 2019 to $180.1 billion in 2021 with biggest market share held by mobile games (59%). MEG is seen as beneficial teaching and learning method as it can improve student’s performance [8, 9]. Its mobility allows education to happen at anytime and anywhere. However, developing MEG is challenging for game designer since mobile devices are popular with impediment in term of interruptions, screen size, input or control interfaces which can hinder usability [10]. Ensuring usability of EG alone is already difficult since it must satisfy both fun and educational requirement [11]. Thus, usability evaluation needs to be conducted to ensure maximal usability of MEG. It is important to evaluate the usability of EG to ensure its success [12]. Usability is defined as “extent to which a system, product or service can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use” [13]. There are various methods can be used to evaluate usability. User testing and inspection method are the most popular usability evaluation method [14, 15]. Heuristics evaluation (HE) is the most frequently used usability inspection method [16, 17]. This method involves usability experts (3 to 5 persons) inspect the interface design against established usability principles (Heuristics) to identify usability problem [18, 19]. It is cost-effective and can be conducted from the earlier design phase [20]. For this reason, many heuristics has been proposed in previous studies to support the usability evaluation of variety game product. Therefore, this paper aims to review the existing heuristics components (evaluation aspect) that are relevant for developing heuristics for MEG. This section explains the background and aim of the research. Section 2 describes usability evaluation of EG. Followed by section 3, discussed the existing heuristics for games, educational games and mobile application and its components. Section 4 discusses the analysis of the discussed component. Finally, the last section will summarize the study and future work. 2. Usability evaluation of educational games In the realm of games, HE and user testing method are popularly used to evaluate usability [12, 21]. HE method involves experts inspect the underdeveloped game (without real user) from the earlier design phase where game prototype is not yet ready. While, user testing or playtesting usually conducted at the later phase with real user by having them playing fully ready game prototype to check any usability problem faced. It is useful to detect any overlooked usability problems by game practitioners as they might not always be able to fully imitate real user’s interaction. However, this method is rather expensive, time and resource consuming. According to [22] in their study on expert review method in game evaluation, it is hard to get real user involvement in the evaluation session and fixing any problem when the game is almost complete are difficult and expensive. As time and budget are crucial in game development project, game practitioner use HE method as alternative to evaluate game [16]. HE is normally conducted to find usability problems in the software interface [14] but for game, the evaluation is beyond that. Game comprises of several components such as game play, story, mechanics and so on [23]. Due to this contrast, games require set of heuristics for their own [24]. Consequently, many heuristics set has been proposed in the previous studies to assess the usability of various game products. However, evaluating specific game using generic heuristics might overlook the usability problems that related to specific aspects of that particular game. Thus, few avenues proposed heuristics concerned specific aspects of game to support the generic one so that more usability problem can be revealed. 3. Related works This section reported existing studies on heuristics for three context of technology which are games, educational games and mobile application. In this study, we focused on heuristics for video games play on computer, mobile devices and online that are published in English. The heuristics components proposed in these studies were derived and explained based on the aforementioned technology context.
Nur Marissa Vee Senap et al. / Procedia Computer Science 161 (2019) 1028–1035 Author name / Procedia Computer Science 00 (2019) 000–000
1030
3
3.1. Usability heuristics component for games When it comes to first heuristics concerning computer video games (CG), it is dates back to [25] in 1980 whom suggested the heuristics for designing instructional CG. Later, [26] proposed heuristics to inspect the usability of fun CG. In 2004, [27] proposed a comprehensive heuristics for playable CG named Heuristic Evaluation for Playability (HEP). Followed by [28] whom proposed generic heuristics for any kind of CG. Later in 2008, [29] and [30] proposed heuristics for CG design. While, [31] introduced heuristics for new CG genres (Real-time strategy, Action adventure & First-person shooters) named Game Playability Principles (PLAY). For mobile games (MG), [32] proposed the first generic heuristics and [33] updated this heuristics by added new heuristics concerning multi-player. Later on, [34] further explored these MG heuristics by proposed new heuristics focused on what has not been covered. For network games (NG), [35] proposed the first heuristics named Networked Game Heuristics (NGH). In the same year, [36] also proposed heuristics for NG but focused on the one played on social media, Facebook. Similarly, [37] also proposed heuristics for NG played on Facebook including Twitter and Myspace. The summary of the proposed heuristics components for these studies is summarized in Table 1. Table 1. Summary of studies on heuristics component for games. Study
Heuristics name
Component(s)
Computer/ Video Games [25]
Heuristic for instructional computer games
Challenge, Fantasy, Curiosity
[26]
Heuristics for video games
Game interface, Game mechanics, Game play
[27]
Heuristics for playable games (HEP)
Game play, Game story, Mechanics, Usability
[28]
Heuristics for video games
Usability, Graphical user interface, Playability
[29]
Heuristics for video games design
Usability
[30]
Heuristics for computer game design
Usability
[31]
Heuristics for new genre video games (PLAY)
Game play, Entertainment, Usability & Game mechanics
Mobile Games [32]
Heuristics for mobile games
Usability, Mobility, Game play
[33]
Heuristics for mobile multi-player games
Playability (multi-player)
[34]
Heuristics for mobile games
Game play, Mobility, Game usability
Network/ Social Games [35]
Heuristics for networked games (NGH)
Usability (multi-player)
[36]
Heuristics for social-network games
Usability
[37]
Heuristics for social-network games
Social
The proposed heuristics components are Challenge, Fantasy, Curiosity, Usability, Interface, Mechanics, Game Play/ Playability, Story, Entertainment, Mobility and Social. Challenge is the series of task player must overcome to achieve game’s goal [25], whilst Fantasy and Curiosity refer to game’s environment that attract player to play more [25]. Usability component concerns on game’s control and interface [32]. The Interface is the element that allow player to interact with game world [26]. While, Mechanics is the rules that determine what operation player can perform [32]. The interaction of player with the game is called Game play [34] and to ensure that game is playable its Playability is usually evaluated. Thus, the term Playability will be used hereinafter. Meanwhile, Story is the sequence of events that happen in game world. Entertainment concerns game’s element that promote emotional connection and immersion to player [31]. For Mobility, this component concerns things that can affect mobility of the game [38]. Lastly, Social component concerns the social aspect of game [37]. The frequencies of the proposed components are presented in Table 2.
4
Nur Marissa Vee Senap et al. / Procedia Computer Science 161 (2019) 1028–1035 Author name / Procedia Computer Science 00 (2019) 000–000
1031
Table 2. Frequencies of games heuristics components across reviewed studies. #
Component
1
Challenge
2
Fantasy
3
Curiosity
4
Interface
5
Mechanics
6
Playability
7
Story
8
Usability
9
Mobility
10
Entertainment
11
Social
[25]
[26]
[27]
[28]
[29]
[30]
[31]
[32]
[33]
[34]
[35]
[36]
[37]
✓
1
✓ ✓
ƒ 1
✓ ✓ ✓
✓
✓
✓
✓
✓
✓
✓
1 ✓
✓
✓
2
✓
✓
✓
✓
✓
✓
✓
3
✓ ✓ ✓
7 ✓
1
✓
9 2 ✓
1 1
3.2. Usability heuristics component for educational games The first heuristics for EG is the Playability Heuristics Evaluation for Educational Computer Games (PHEG) proposed by [39] in 2010. Later, [40] proposed another heuristic for EG named Heuristic Evaluation for Educational Games (HEEG). Meanwhile, [41] proposed an extension to HEP framework by proposed two new heuristics component for digital game based learning. In 2017, [42] suggested heuristics for evaluation of balanced EG. For MEG context, [43] adapted the MG heuristics by [32] and proposed one new educational component to it. Recently, [44] proposed heuristics for designing MEG for older adults. The synopsis of the proposed heuristics components in this context is presented in Table 3. Table 3. Summary of studies on heuristics components for educational games. Study
Heuristics name
Component(s)
Educational Computer/Video Game [39]
Heuristics for educational game (PHEG)
Interface, Pedagogical, Content, Playability, Multimedia
[40]
Heuristic for educational game (HEEG)
Game play, Enjoyment, Usability & Mechanics, Educational design
[41]
Heuristics for digital game-based learning
Characters /Graphic, Pedagogical effectiveness
[42]
Heuristics for educational video game
Playability
Mobile Educational Games [43]
Heuristics for mobile game-based learning
Game play, Mobility, Game usability, Learning content
[44]
Heuristics for mobile serious games
Game Play
In this context, the proposed heuristics components are Interface, Pedagogical/ Educational design/ Pedagogical effectiveness, Content/ Learning content, Playability, Multimedia, Enjoyment, Usability, Mechanics, Character/ Graphic and Mobility. Some of the components are similar to Games context: Interface, Playability, Usability, Mechanics & Entertainment (Enjoyment) because they are essential for all kind of games. For EG, Pedagogical, Content, Multimedia and Characters component are proposed to cater the educational requirements of games. Pedagogical component looks at the element of educational and learning in games. Content component are evaluated to test whether game provide feedback, good content and proper learning materials. While, Multimedia component is to check the representation of text, image, audio, animation and video in games. The Character component refer to the design of player’s role in game world. The frequencies of these proposed heuristics components for EG are depicted in Table 4.
Nur Marissa Vee Senap et al. / Procedia Computer Science 161 (2019) 1028–1035 Author name / Procedia Computer Science 00 (2019) 000–000
1032
5
Table 4. Frequencies of educational games heuristics component across reviewed studies. #
Component
1
Interface
2
Pedagogical
3
Content
4
Playability
5
Multimedia
[39] ✓
6
Entertainment Usability
8
Mechanics
9
Character
10
Mobility
✓
✓
✓
✓
✓
7
[40]
✓
[41]
[42]
[43]
[44]
1
✓
3
✓
✓
✓
✓ ✓
2
✓
5 1 1
✓
✓
ƒ
✓
2 1 1
✓
1
3.3. Usability heuristics component for mobile application The advancement of mobile devices has drove new study on how to design a good usability mobile application. In 2012, [45] proposed heuristics for touchscreen-based mobile devices (TMD). Then, it was further validated and resulted in a proposal of new heuristics [46]. They further updated the heuristic’s definition according to the result of validation [47]. Later on, [48] proposed heuristics for mobile devices. Similarly, [49] suggested heuristics for touchscreen mobile phone and tablet. On the other hand, [50] proposed heuristics for smartphone mobile application named Smartphone Mobile Application heuRisTics (“SMART”). Likewise, [51] refined TMD and fit it to smartphone context and named it as SMArtphone’s uSability Heuristics (SMASH). In 2016, [52] proposed heuristics to evaluate control feature of mobile games. Recently, [18] proposed heuristics for mobile learning application. The summary of heuristics components proposed in these studies is presented in Table 5. Table 5. Summary of studies on heuristics components for mobile application. Study
Heuristics name
Component
[45]
Heuristics for TMD (prelim)
Usability
[46]
Heuristics for TMD (proposal)
Usability
[47]
Heuristics for TMD (update)
Usability
[48]
Heuristics for mobile interface
User Interface
[49]
Heuristics for mobile interface
User Interface
[50]
Heuristics for smartphone mobile app (SMART)
Usability
[51]
Heuristics for smartphones mobile app (SMASH)
Usability
[52]
Heuristics for mobile games
Control
[18]
Heuristics for mobile learning application
Usability
Based on studies reviewed for mobile application, some of the study focused on evaluation of general usability of mobile application, while few studies concerned in evaluating the specific features of the mobile devices like interface and control. Table 6 shows the frequencies of these proposed components across the studies. Table 6. Frequencies of mobile application heuristics component across reviewed studies. #
Components
1
Usability
2
Interface
3
Control
[45]
[46]
[47]
✓
✓
✓
[48] ✓
[49] ✓
[50]
[51]
✓
✓
[52]
[18] ✓
✓
ƒ 6 2 1
6
Nur Marissa Vee Senap et al. / Procedia Computer Science 161 (2019) 1028–1035 Author name / Procedia Computer Science 00 (2019) 000–000
1033
4. Discussion The review of heuristics component for Games in Table 2 has shown that Usability and Playability component stood out the most. The next outstanding heuristics component is Mechanics followed by Interface and Mobility, then Curiosity, Challenge, Fantasy, Story, Entertainment and Social components. In EG context, the review revealed that Playability was the most evaluated heuristics component followed by Pedagogical component as presented in Table 4. Both components are noteworthy when it comes to usability evaluation of EG. Content and Usability components were gained attention too. The least proposed heuristics components for EG were Interface, Mobility, Multimedia, Entertainment, Mechanics and Character component. Meanwhile, Table 6 has shown that Usability component is the most obvious heuristics component for Mobile Application. Interface and Control component also been considered when evaluating usability of mobile application. In result, it was not surprising that Usability and Playability component was dominant in Games context since it was fundamental in any game usability. The Interface and Control component usually integrated with Usability component in some studies because well-designed interface and control will contribute to good usability of games or application. Curiosity, Challenge and Fantasy are game principles suggested by earlier study for designing fun games but were no longer evaluated as independent components in later studies. It usually evaluated in Playability or Entertainment component. Next, Mechanics components only evaluated when it comes to games. For Mobility component, it is only relevant when evaluating usability of game or application designed for mobile devices. While, the Story component is usually relating to Playability and Mechanics component. Social component comes into account only when evaluating social-network games. Lastly, Pedagogical, Content, Multimedia and Character component were significant for EG, however there is none of the reviewed studies on MEG that address these components in detail. Thus, the findings of this study might be valuable for game designer or future research in selecting proper heuristics component for MEG heuristics development. Fig. 1 illustrated the components suggested for MEG context by took into account all relevant components evaluated in games, educational games and mobile application context.
Fig. 1. Consolidation of suggested heuristics components for MEG.
5. Conclusion and future work The pervasiveness of mobile devices has drove new innovations of EG for mobile environment. Ensuring the usability of MEG is challenging due to nature of EG itself and limitations of mobile devices. There are many heuristics available in the literature to evaluate usability of various games product. These heuristics however cannot be applied directly in MEG context. This paper reviews existing heuristics component for Games, Educational Games and Mobile
1034
Nur Marissa Vee Senap et al. / Procedia Computer Science 161 (2019) 1028–1035 Author name / Procedia Computer Science 00 (2019) 000–000
7
Application and represent the consolidation of heuristics components that are relevant for MEG. In future, there is a need to design a set of comprehensive heuristics that cover all of relevant components discussed in this paper for ideal heuristics evaluation of MEG. Moreover, further validation is required to support the findings of this study and to identify if there any new component to develop heuristics for MEG. Acknowledgements This work was supported by the Skim Latihan Akademik IPTA (SLAI), Ministry of Higher Education, Malaysia. References [1] Paiva, Ana C. R, Nuno H Flores, André G Barbosa, and Tânia P. B Ribeiro. (2016). “iLearnTest–framework for Educational Games.” Procedia-Social and Behavioral Sciences 228: 443-448. [2] Backlund, Per and Maurice Hendrix. (2013). “Educational Games-Are They Worth The Effort? A Literature Survey of the Effectiveness of Serious Games” in 5th Int. Conf. on Games and Virtual Worlds for Serious Applications (VS-GAMES), IEEE. pp. 1-8. [3] Ibrahim, Roslina, Khalili Khalil, and Azizah Jaafar. (2011). “Towards educational games acceptance model (EGAM): A Revised Unified Theory of Acceptance and Use of Technology (UTAUT).” Int. Journal of Research and Reviews in Computer Science 2 (3): 839-846. [4] Roungas, Bill and Fabiano Dalpiaz. (2015) “A model-driven framework for educational game design” in Int. Conf. on Games and Learning Alliance, Springer. pp. 1-11. [5] Hainey, Thomas, Thomas M Connolly, Elizabeth A Boyle, Amanda Wilson, and Aisya Razak. (2016) “A Systematic Literature Review of Games-Based Learning Empirical Evidence in Primary Education.” Computers & Education 102: 202-223. [6] Giannakas, Filippos, Georgios Kambourakis, Andreas Papasalouros, and Stefanos Gritzalis. (2018) “A Critical Review of 13 Years of Mobile Game-Based Learning.”Educational Technology Research and Development 66 (2): 341-384. [7] Tom, Wijman. (2019) “Global Games Market Revenues 2018.” Available from: https://newzoo.com/insights/articles/global-games-marketreaches-137-9-billion-in-2018-mobile-games-take-half/. [8] Perry, Gabriela Trindade, Ci Costa Kulpa, ET Pinheiro, and Marcelo Leandro Eichler. (2012) “Lessons from An Educational Game Usability Evaluation.” International Journal of Interactive Mobile Technologies 6 (2): 23-28. [9] Huang, Yu-Lan, Dian-Fu Chang, and Berlin Wu. (2017) “Mobile Game-Based Learning with a Mobile App: Motivational Effects and Learning Performance.”Journal of Advanced Computational Intelligence and Intelligent Informatics 21 (6): 963-970. [10] Hussain, Azham, Abdulkarim Saleh, Abdussalam Taher, Imman Ahmed, and Mohammed Lammasha (2015). “Usability Evaluation Method for Mobile Learning Application Using Agile: A Systematic Review.” Jurnal Teknologi 77 (5): 51-56. [11] Mohamed, Hasiah and Azizah Jaafar. (2010) “Challenges in the Evaluation of Educational Computer Games”, in 2010 International Symposium on Information Technology. IEEE. 1-6. [12] Al Fatta, Hanif, Zulisman Maksom, and M Hafiz Zakaria. (2018) “Systematic Literature Review On Usability Evaluation Model Of Educational Games: Playability, Pedagogy, And Mobility Aspects.” J. Theoretical & Applied Information Technology 96 (14): 4677-4689. [13] ISO (2018). “ISO 9241-11:2018.” Available from: https://www.iso.org/obp/ui/#iso:std:iso:9241:-11:ed-2:v1:en. [14] Jimenez, Cristhy, Pablo Lozada, and Pablo Rosas. (2016) “Usability Heuristics: A Systematic Review” in 2016 IEEE 11th Colombian Computing Conference (CCC). IEEE. pp. 1-8. [15] Dourado, Marcos Antonio Durães and Edna Dias Canedo. (2018) “Usability Heuristics for Mobile Applications - A Systematic Review” in 20th Int. Conf. on Enterprise Information Systems. pp. 483-494. [16] Korhonen, Hannu. (2016) “Evaluating Playability of Mobile Games with The Expert Review Method”, University of Tampere. [17] Mugisha, Alice, Victoria Nankabirwa, Thorkild Tylleskär, and Ankica Babic. (2019) “A Usability Design Checklist for Mobile Electronic Data Capturing Forms: The Validation Process.” BMC Medical Informatics and Decision Making 19 (4): 1-11. [18] Kumar, Bimal Aklesh and Munil Shiva Goundar. (2019) “Usability Heuristics for Mobile Learning Applications.” Education and Information Technologies 24 (2): 1819-1833. [19] Salman, Hasanin Mohammed, Wan Fatimah Wan Ahmad, and Suziah Sulaiman. (2018) “Usability Evaluation of the Smartphone User Interface in Supporting Elderly Users from Experts’ Perspective.” IEEE Access 6: 22578-22591. [20] Fortes Tondello, Gustavo, Dennis Kappen, Elisa Mekler, Marim Ganaba, and Lennart Nacke. (2016) “Heuristic Evaluation for Gameful Design.” CHI PLAY Companion'16. pp. 315-323. [21] Hussain, Azham, Sharaf A. A. Abbas, M. S. Abdulwaheed, Rammah G. Mohammed, and Adil A. Abdulhussein. (2015) “Usability Evaluation of Mobile Game Applications: A Systematic Review.” Int. Journal of Computer and Information Technology 4 (03): 547-551. [22] Korhonen, Hannu, Janne Paavilainen, and Hannamari Saarenpää. (2009) “Expert Review Method In Game Evaluations: Comparison of Two Playability Heuristic Sets”, in MindTrek' 2009. ACM. pp. 74-81. [23] Aleem, Saiqa, Luiz Fernando Capretz, and Faheem Ahmed. (2016) “Game Development Software Engineering Process Life Cycle: A Systematic Review.” Journal of Software Engineering Research and Development 4 (1): 1-30. [24] Al-Azawi, Rula, Aladdin Ayesh, and Mohaned Al Obaidy. (2013) “Generic Evaluation Framework for Games Development Methodology”, in 3rd Int. Conf. on Communications and Information Technology (ICCIT), IEEE. pp. 55-60.
8
Nur Marissa Vee Senap et al. / Procedia Computer Science 161 (2019) 1028–1035 Author name / Procedia Computer Science 00 (2019) 000–000
1035
[25] Malone, Thomas W. (1980) “What Makes Things Fun to Learn? Heuristics for Designing Instructional Computer Games”, in Proceedings of the 3rd ACM SIGSMALL Symposium and the First SIGPC Symposium on Small Systems, ACM. pp. 162-169. [26] Federoff, Melissa A. (2002) “Heuristics and Usability Guidelines for the Creation and Evaluation of Fun in Video Games”, Indiana University. [27] Desurvire, Heather, Martin Caplan, and Jozsef A Toth. (2004) “Using Heuristics to Evaluate The Playability of Games” in CHI'04. pp. 1509-1512. [28] Schaffer, Noah. (2007) “Heuristics for Usability in Games”, in White Paper. pp. 1-30. [29] Pinelle, David, Nelson Wong, and Tadeusz Stach. (2008) “Heuristic Evaluation for Games: Usability Principles for Video Game Design”, in Proc. of the SIGCHI Conference on Human Factors in Computing Systems, ACM. pp. 1453-1462. [30] Brown, Michael. (2008) “Evaluating computer game usability: Developing Heuristics Based On User Experience”, in IHCI conference. pp. 1-7. [31] Desurvire, Heather and Charlotte Wiberg. (2009) “Game usability heuristics (PLAY) for Evaluating and Designing Better Games: The Next Iteration”, in Int. Conf. on Online Communities and Social Computing. Springer. pp. 557-566. [32] Korhonen, Hannu and Elina MI Koivisto. (2006) “Playability Heuristics for Mobile Games”, in Proc. of the 8th Conf. on HCI with Mobile Devices and Services, ACM. pp. 9-16. [33] Korhonen, Hannu and Elina MI Koivisto. (2007) “Playability Heuristics for Mobile Multi-Player Games”, in 2nd Int. Conf. on Digital Interactive Media in Entertainment and Arts, ACM. pp. 28-35. [34] Soomro, Sarmad, Wan Fatimah Wan Ahmad, and Suziah Sulaiman. (2012) “A Preliminary Study on Heuristics for Mobile Games” in 2012 Int. Conf. on Computer & Information Science (ICCIS), IEEE. pp. 1030-1035. [35] Pinelle, David, Nelson Wong, Tadeusz Stach, and Carl Gutwin. (2009) “Usability Heuristics for Networked Multiplayer Games”, in GROUP' 09, ACM. pp. 169-178. [36] Papaloukas, Spyridon, Kiriakos Patriarcheas, and Michalis Xenos. (2009) “Usability Assessment Heuristics in New Genre Videogames”, in 2009 13th Panhellenic Conference on Informatics, IEEE. pp. 202-206. [37] Paavilainen, J. (2010) “Critical Review on Video Game Evaluation Heuristics: Social Games Perspective”, in FuturePlay'10, ACM. pp. 5665. [38] Zaibon, Syamsul Bahrin. (2015) “User Testing on Game Usability, Mobility, Playability, and Learning Content of Mobile Game-Based Learning.” Jurnal Teknologi 77 (29): 131-139. [39] Mohamed, Hasiah and Azizah Jaafar. (2010) “Heuristics Evaluation in Computer Games” in 2010 Int. Conf. on Information Retrieval & Knowledge Management (CAMP), IEEE. pp. 188-193. [40] Barbosa, Marcelo B, Andreza B Rego, and Igor de Medeiros. (2015) “HEEG: Heuristic Evaluation for Educational Games”, in SBGames' 2015. pp. 224-227. [41] Guo, Yan Ru and Dion H-L Goh. (2016) “Heuristic Evaluation of an Information Literacy Game: Poster” in Proc. Assoc. Inf. Sci. Tech. American Society for Information Science 55 (1): 1-4. [42] Ibrahim, Amer. (2017) “Playability Heuristics Evaluation For Educational Video Games.” Int. J. Of Core Eng.& Management 4 (6): 28-39. [43] Zaibon, Syamsul Bahrin and Norshuhada Shiratuddin. (2010) “Heuristics Evaluation Strategy for Mobile Game-Based Learning”, in WMUTE'10, IEEE. pp. 127-131. [44] Machado, Mônica da Consolação, Ronan L Rodrigues Ferreira, and Lucila Ishitani. (2018) “Heuristics and Recommendations for the Design of Mobile Serious Games for Older Adults.” International Journal of Computer Games Technology. [45] Inostroza, Rodolfo, Cristian Rusu, Silvana Roncagliolo, Cristhy Jimenez, and Virginica Rusu. (2012) “Usability Heuristics for TouchscreenBased Mobile Devices”, in 2012 9th Int. Conf. on Information Technology-New Generations. IEEE. pp. 662-667. [46] Inostroza, Rodolfo, Cristian Rusu, Silvana Roncagliolo, Cristhy Jiménez, and Virginica Rusu. (2012) “Usability Heuristics Validation Through Empirical Evidences: A Touchscreen-Based Mobile Devices Proposal”, in SCCC' 12. IEEE. pp. 60-68. [47] Inostroza, Rodolfo, Cristian Rusu, Silvana Roncagliolo, and Virginica Rusu. (2013) “Usability Heuristics For Touchscreen-Based Mobile Devices: Update” in ChileCHI'12, ACM. pp. 24-29. [48] Neto, Olibário Machado and Maria Da Graça Pimentel. (2013) “Heuristics for the Assessment of Interfaces of Mobile Devices”, in WebMedia. ACM. 93-96. [49] Gómez, Rosa Yáñez , Daniel Cascado Caballero, and José-Luis Sevillano. (2014) “Heuristic Evaluation on Mobile Interfaces: A New Checklist.” The Scientific World Journal. pp. 1-19. [50] Joyce, Ger and Mariana Lilley. (2014) “Towards the Development of Usability Heuristics for Native Smartphone Mobile Applications”, in Int. Conf. of Design, User Experience, and Usability, Springer. pp. 465-474. [51] Inostroza, Rodolfo, Cristian Rusu, Silvana Roncagliolo, Virginica Rusu, and César A Collazos. (2016) “Developing SMASH: A Set of Smartphone's Usability Heuristics.” Computer Standards & Interfaces 43: 40-52. [52] Daud, Ezri Hielmi Che, Shamsul Anuar Mokhtar, and Farahwahida Mohd. (2016) “Usability Heuristics in the Context of Control Features on Mobile Games”, in 2016 Int. Conf. on Information and Communication Technology (ICICTM), IEEE. pp. 194-197.