- Email: [email protected]
Evaluating the Usability of a Moroccan University Research Management Web Platform
Available online at www.sciencedirect.com Available online at www.sciencedirect.com
ScienceDirect ScienceDirect
Available Availableonline onlineatatwww.sciencedirect.com www.sciencedirect.com Procedia Manufacturing 00 (2018) 000–000 Procedia Manufacturing 00 (2018) 000–000
ScienceDirect ScienceDirect
www.elsevier.com/locate/procedia www.elsevier.com/locate/procedia
Procedia Manufacturing 32 (2019) 1008–1016 Procedia Manufacturing 00 (2017) 000–000 www.elsevier.com/locate/procedia
The The 12th 12th International International Conference Conference Interdisciplinarity Interdisciplinarity in in Engineering Engineering
Evaluating Evaluating the the Usability Usability of of aa Moroccan Moroccan University University Research Research Web Platform Manufacturing EngineeringManagement Society International Conference 2017, MESIC 2017, 28-30 June Management Web Platform 2017, Vigo (Pontevedra), Spain
a, b c d c, Mohamed Larbi Kerkebd and Khaoula Khaoula Benmoussa Benmoussaa, *, *, Majida Majida Laaziri Laazirib,, Samira Samira Khoulji Khoulji , Mohamed Larbi Kerkeb and e Costing models for capacityAbir optimization El Abir El Yamami Yamamie in Industry 4.0: Trade-off 0F
0F
a,b,c a,b,c e e
between used capacity and operational efficiency
Information System Engineering Resarch Group, National School of Applied Sciences, Abdelmalek Essaâdi University, Tetouan, Morocco Information System Engineering Resarch Group, National School of Applied Sciences, Abdelmalek Essaâdi University, Tetouan, Morocco d d Information System Engineering Resarch Group, Faculty of Sciences, Abdelmalek Essaadi University, Tetouan, Morocco Information System Engineering Resarch Group, Faculty of Sciences, Abdelmalek Essaadi University, Tetouan, Morocco
Laboratory Signals, Distributed Systems and Artificial Intelligence, ENSET Mohammedia, Hassan II University, Casablanca, a Artificial Intelligence, a,* b b II University, Casablanca, Laboratory Signals, Distributed Systems and Hassan A. Santana , P. Afonso , A.ENSET ZaninMohammedia, , R. Wernke Morocco Morocco a
University of Minho, 4800-058 Guimarães, Portugal b Unochapecó, 89809-000 Chapecó, SC, Brazil
Abstract Abstract The success of any system or interface depends on the satisfaction of the user and users are only satisfied when they are able to The success of any system or interface depends on the satisfaction of the user and users are only satisfied when they are able to Abstract do their job in a simple and effective way [1]. The purpose of the SIMARECH information system [2], [3] is the piloting of do their job in a simple and effective way [1]. The purpose of the SIMARECH information system [2], [3] is the piloting of research in Moroccan universities, their institutions and structures, to support researchers and enhance their scientific output. researchthe in Moroccan universities, institutions and structures, support researchers enhance their interconnected, scientific output. Under concept of "Industrytheir 4.0", production processes towill be pushed to and be increasingly Ensuring the proper use of the SIMARECH information system has always been a real concern of the Abdelmalek Essaadi Ensuring the based proper on usea of thetime SIMARECH information system hasmore always been a In realthis concern of the Abdelmalek Essaadi information real basis and, necessarily, much efficient. context, capacity optimization University (UAE) in Moroccan universities and national research institutions. The purpose of this study is to investigate usability University (UAE) in Moroccanaim universities and maximization, national researchcontributing institutions. The purpose of this study profitability is to investigate usability goes beyond the traditional of capacity also for organization’s and value. issues and provide recommendations for achieving the system goal within the UAE university institutions. issues andlean provide recommendations for achieving improvement the system goal approaches within the UAE university institutions. Indeed, management and continuous suggest capacity optimization instead of The objective of this research paper is to evaluate the usability of the SIMARECH platform based on a complete test The objective of this research paper is to evaluate the usability of the SIMARECH platform based on a complete test maximization. The study capacityfrom optimization costing models an important research topic that methodology. The study wasofconducted January to and March 2018 based on is Jakob Nielsen methodology [4] and thedeserves methods methodology. The study was conducted from January to March 2018 based on Jakob Nielsen methodology [4] and the methods of data collection Nielsen's Attributes of Usability Questionnaire (NAU) [4]. results showedand a high level ofausability for the contributions from both the practical and theoretical perspectives. ThisThe paper presents discusses mathematical of data collection Nielsen's Attributes of Usability Questionnaire (NAU) [4]. The results showed a high level of usability for the platform, but it shouldmanagement be further improved. It isdifferent found thatcosting the followed approach can effectively assess the usability ofhas scientific model for capacity based on models (ABC and TDABC). A generic model been platform, but it should be further improved. It is found that the followed approach can effectively assess the usability of scientific research information systems. developed and it was used to analyze idle capacity and to design strategies towards the maximization of organization’s research information systems.
value. The trade-off capacity maximization vs operational efficiency is highlighted and it is shown that capacity © 2018 2019 The Published by Ltd. optimization might hide operational inefficiency. © 2018 The Authors. Authors. Published by Elsevier Elsevier Ltd. This is an open access article under the CC BY-NC-ND license (https://creativecommons.org/licenses/by-nc-nd/4.0/) © 2017 The Authors. by Elsevier B.V. This is an open accessPublished article under the CC BY-NC-ND license (https://creativecommons.org/licenses/by-nc-nd/4.0/) Selection and peer-review under responsibility of the 12th International Conference Interdisciplinarity in Engineering. Peer-review responsibility of the scientific of the Manufacturing Engineering Society in International Conference Selection andunder peer-review under responsibility ofcommittee the 12th International Conference Interdisciplinarity Engineering. 2017. Keywords: Usability Evaluation; Usability Testing; Web Platform SIMARECH. Keywords: Usability Evaluation; Usability Testing; Web Platform SIMARECH. Keywords: Cost Models; ABC; TDABC; Capacity Management; Idle Capacity; Operational Efficiency
1. Introduction * Corresponding author. Tel.: +0-000-000-0000 ; fax: +0-000-000-0000 . * The Corresponding author. Tel.: +0-000-000-0000 ; fax: information +0-000-000-0000 cost of idle capacity is a fundamental for. companies and their management of extreme importance E-mail address: [email protected] E-mail address: [email protected]
in modern production systems. In general, it is defined as unused capacity or production potential and can be measured 2351-9789 2018 The Authors. Published by Elsevier Ltd.hours of manufacturing, etc. The management of the idle capacity in several©ways: tons of production, available 2351-9789 © 2018 The Authors. Published by Elsevier Ltd. This is an Afonso. open access under the761; CC BY-NC-ND (https://creativecommons.org/licenses/by-nc-nd/4.0/) * Paulo Tel.:article +351 253 510 +351 253license 604 741 This is an open access article under the CC fax: BY-NC-ND license (https://creativecommons.org/licenses/by-nc-nd/4.0/) Selection and peer-review under responsibility of the 12th International Conference Interdisciplinarity in Engineering. E-mail address: [email protected] Selection and peer-review under responsibility of the 12th International Conference Interdisciplinarity in Engineering.
2351-9789 © 2017 The Authors. Published by Elsevier B.V. Peer-review under of the scientificbycommittee the Manufacturing Engineering Society International Conference 2017. 2351-9789 © 2019responsibility The Authors. Published Elsevier of Ltd. This is an open access article under the CC BY-NC-ND license (https://creativecommons.org/licenses/by-nc-nd/4.0/) Selection and peer-review under responsibility of the 12th International Conference Interdisciplinarity in Engineering. 10.1016/j.promfg.2019.02.315
2
Khaoula Benmoussa et al. / Procedia Manufacturing 32 (2019) 1008–1016 Khaoula Benmoussa et al. / Procedia Manufacturing 00 (2018) 000–000
1009
1. Introduction Today, technological development is evolving very rapidly, forcing government agencies to provide better features to improve the quality of the system to satisfy customers. Products that are used in everyday life and that are not usable, that do not meet the requirements of users, cause frustration of users, and frustrated users no longer want to use these products [5]. Building an appropriate system is crucial, it must have great advantages and its quality can be measured according to its ability to be used by all its users, we talk about system usability. Any information system should provide the necessary functions to its users, that they are defined as people, who take some of their ordinary work and use the information system, so they must be able to understand the features of the used system so that they could accomplish their tasks. However, these features alone are not enough. Goodwin [6] indicated that the determinants of system acceptance are functionality and usability. Therefore, if the system is usable, the user can reach a feature easily. According to the well-known usability expert Jakob Nielsen [7], usability reigns on the web, it is simply stated that if users have difficulty accessing the content of the site, they will not use it. Websites with misuse can have many problems such as information is not hierarchical, no simple navigation, wrong choice of colors and fonts, page loading or file download takes time, not directed towards user requirements, etc. Users can not use these websites when dealing with multiple and repeated problems [8]. According to Dillon [9], users are only satisfied when they can achieve their goals in a successful and effective manner. The website interface must be simple and easy to handle for experienced and inexperienced users who can easily use it and meet their requirements. The acceptance and use of technological devices depend on important and varied factors (appropriate design, user skills, etc.), so usability is the key factor in the success of applications, and the evaluation of usability has methodologies that help to measure usability aspects in system interfaces and to identify problems [10]. Ensuring the proper use of SIMARECH system has always been a real concern of Abdelmalek Essaadi University within Moroccan universities and national research institutions. For this, we need to perform an evaluation of its usability to avoid human errors with the system. With usability assessment, we can understand if the system is efficient and usable and how satisfied users are when they use it. As well as the results of the usability assessment can be used as a reference to improve the system in the future, many usability issues can be taken into account in this usability test, so that it can be achieve the goal of its users [11]. SIMARECH is an IT platform that facilitates the collection and management of data from research activities and results within an organization [3]. It aims to present in a coherent way the establishments, the research units within the university, including the personnel and their scientific activities (publications, patents, events, equipment, etc.), as well as the follow-up of the financial means and the international activities (partnership agreements, congresses, etc.). Researchers, administrators, and evaluators have all the tools they need to monitor research results, improve visibility, and allocate available resources efficiently. Initially, the system was structured to present and include a range of information services (programs, projects, events and products resulting from the activities of the research developed, etc.) for the use of users (researchers , structure managers, deans, administrative staff) according to their role, moreover, at the strategic level, it aims to have a set of indicators and available research statistics. Modular nature of the system and flexibility of its data model allowing different users to access certain information and functionalities throughout the process, and facilitate the processing, organization and transmission of information in accordance with international standards [2], [3]. This research focuses on the evaluation of the usability of the SIMARECH platform within UAE university institutions based on Jakob Nielsen's model [7] and measuring its five attributes: learning, efficiency, memorization, errors, and satisfaction. For this, a survey was conducted among 80 users of the platform (SIMARECH) from January to March 2018. The rest of this paper is organized as follow: section 2 presents a literature revue, section 3 is dedicated to the research methodology, and the results are presented in section 4 before concluding in section 5.
1010
Khaoula Benmoussa et al. / Procedia Manufacturing 32 (2019) 1008–1016 Khaoula Benmoussa et al. / Procedia Manufacturing 00 (2018) 000–000
3
2. Literature review In this section, we present different definitions of usability and identify several usability attributes from different standards and models. 2.1. Usability Definitions Usability plays an important role in the software development process. In the field of Human-Computer Interaction (HMI), the most widely accepted definition of usability is that proposed in ISO 9241-11 [12]: "the extent to which a user can use a product to achieve precise objectives with efficiency, effectiveness and satisfaction in a context of specified use ". From this perspective, usability involves the interaction of users with the software product and can be considered as the ability of the product to meet user expectations. On the other hand, in the field of software engineering (SE), the most widely accepted definition of usability is that proposed in ISO 9126-1 [13]: "the ability of the software product to be understood, learned operated and attractive to the user when used under specified conditions ". From this point of view, usability is considered as a specific feature that affects the quality of a software product, it does not necessarily imply user interaction with the system since it can be measured as "compliance to the specification ". 2.2. Usability models While conducting a literature review of existing work in this area of research, we found many usability models. Among these major models that have influenced usability measures are: • Shackel and Richardson [14]: In academia, the concept of usability was defined for the first time systematically by Shackel and Richardson. They proposed four criteria for measuring usability: efficiency (performance in the performance of tasks), learning (means the degree of learning for the completion of tasks), flexibility (the ability to adapt to the variation of the tasks) and the attitude of the users to satisfy the system. • Nielsen [4]: ''Usability is a quality attribute that evaluates the usability of user interfaces '' [4]. Nielsen has associated five attributes to usability: learning (the system is simple and easy to complete tasks), efficiency (The system is efficient and save users time), memorization (The system is easy to remember), errors (The system has a low error rate) and satisfaction (users are likely to use it and are satisfied with the system). Nielsen's model is more acceptable than the one proposed by Shackel and Richardson because the structure of quality characteristics is described more systematically and the place of use is clearly specified [15]. • Seffah, Donyaee, Kline and Padda [16]: These authors developed the Quality In Use Integrated Measurement (QUIM) model to measure the usability of software. This model includes 10 factors, each of which corresponds to a specific usability facet identified in an existing standard or model. These 10 factors are broken down into 26 sub-factors, which are broken down into 127 specific parameters. The authors state, "More attention to user information and context is also needed to facilitate the selection and customization of many proposed metrics and higher-level criteria". • Oztekin, Nikov and Zaim [17]: These authors have proposed a methodology for usability assessment and design of web-based information systems (UWIS). This methodology includes a new checklist to provide usability indices that are defined by considering the usability sub-features proposed in ISO 9241-11 [12], dialogue principles for user interface design according to ISO 9241-10 [18] and usability heuristics proposed by Nielsen [4]. UWIS is also proposing an application of Structural Equation Modeling (SEM) to establish a quantitative model for usability assessment. 2.3. Nielsen Model To perform the usability assessment of our system, we will use the methodology presented by the Nielsen Model Standard [4] for quantitative results and using data collection methods under the form of Nielsen Attributes of Usability (NAU) questionnaires [4]. As the UWIS methodology [17] and the QUIM model [16] contain several criteria, the thing that risks giving up the test on the part of the users of the platform (teacher-researchers).
4
Khaoula Benmoussa et al. / Procedia Manufacturing 32 (2019) 1008–1016 Khaoula Benmoussa et al. / Procedia Manufacturing 00 (2018) 000–000
1011
Here is a map of the five usability attributes of the Nielsen model in the test method used in this research (Fig.1):
Fig. 1. Map of the five usability attributes of the Nielsen model
3. Research methodology In this research, we evaluate the usability of the SIMARECH platform. To this end, we were carried out a usability test by distributing Nielsen Attributes of Usability (NAU) questionnaire [4]. This test is done during the month of January to March 2018, which includes questions in 5 usability categories according to the Nielsen Model Standard, as well as open-ended questions for users to describe their difficulty and express their needs by proposing the SIMARECH system improvement points (Table 1). We distributed the questionnaire to the teacher-researchers (users of the platform) of the four Moroccan institutions of Abdelmalek Essaadi University (UAE): Faculty of Sciences (FS), Higher Normal School (ENS), Polydisciplinary Faculty (FP), Faculty of Letters and Human Sciences (FLSH). We used the NAU questionnaire for data collection because it can be distributed to a larger number of people for a broad response and is an inexpensive approach [8]. The NAU questionnaire is simple and unambiguous; it has helped users to answer these questions without difficulty. He presents closed questions that allowed them to gain time to complete the questionnaire [8]. It is of a scale of 5 points (1-5) known as the Likert scale so it is easy to interpret and users can choose a neutral center point [8], [19]. Thus, this questionnaire includes two open questions to define new development opportunities or improve the existing system. The purpose of the usability method is to evaluate and propose improvements to SIMARECH. Table 1. Nielsen's Attributes of Usability Questionnaire (NAU) [4] Learning
This system is simple to use. The information provided with this system is easy to understand.
Efficiency
I easily found the information I am looking for. I was able to quickly complete my task with this system.
Memorability
The organization of information in the system screens is clear. The system is easy to remember.
Error
The error messages presented by this system tell me clearly how to solve problems. When I made a mistake using this system, it was easy and quick to correct it.
Satisfaction
The interface of this system is nice This system has all the functions and the potential corresponding to my expectations
Open questions
In your opinion, what are the items, information or services missing from the system? What are the possible improvement points?
Khaoula Benmoussa et al. / Procedia Manufacturing 32 (2019) 1008–1016 Khaoula Benmoussa et al. / Procedia Manufacturing 00 (2018) 000–000
1012
5
4. Result and analysis This evaluation of the usability of the SIMARECH platform [2] is a survey of a sample selected from the teacher-researchers of each UAE university institutions. Nielsen generally recommends testing with 20 users when collecting quantitative usability measures, he clarifies that '' With 20 users, you'll probably have 1 outlier (since 6% of users are outliers), so you'll include data from 19 users in your average. This makes your confidence interval go from 243 to 357 seconds, since the margin of error is +/- 19% for testing 19 users '' [20]. The questionnaire was distributed to the 80 teacher-researchers, and their availability 78 among them who answered the questionnaire, distributed as follows (table 2): Table 2. Number of teacher-researchers who answered the test for each institution UAE university institutions
FS
FLSH
ENS
FP
Number of teacher-researchers
20
20
19
19
We decided to do the survey on four institutions, in order to have a mix of response, consideration of the diversity of teaching specialties is very important. According to Nielsen, enough data can be obtained by testing different user cases [20]. The knowledge of the teachers, their skills or expertise in using the platform differs, so each teacher has his own way of handling and judging the platform. In this analysis, we provide usability measurement as a percentage for each university mentioned. These indications will be useful from two points of view, on the one hand to evaluate the usability and on the other hand to deduce the weaknesses and to determine the improvements proposed by the users of the SIMARECH system. 4.1. Result 4.1.1. Faculty of Sciences The result of the NAU questionnaire of the Faculty of Sciences is summarized as follows (table 3). FS Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10
• • • • •
Table 3. The result of the NAU questionnaire of the Faculty of Sciences 1 strongly disagree 2 disagree 3 Neutral 4 Agree 5 strongly agree 3% 20% 61% 16% 1% 17% 57% 25% 10% 15% 55% 20% 2% 13% 19% 50% 16% 4% 28% 50% 18% 10% 19% 51% 20% 4% 11% 23% 53% 9% 7% 14% 21% 52% 6% 8% 20% 65% 7% 3% 24% 61% 12%
The open answers of the Faculty of Sciences summarized as follows: Q.11: Many indexed journals cannot be found on the system, so consider modifying and updating the list regularly. Lack of online communication support with other teachers in the platform. Q.12: You have to think about looking for an easy method of entering the articles. Ability to import data from different international base (articles, communications, etc.) instead of re-entering information into the platform. Facilitate the task for the teacher to use the system quickly and without wasting time.
6
Khaoula Benmoussa et al. / Procedia Manufacturing 32 (2019) 1008–1016 Khaoula Benmoussa et al. / Procedia Manufacturing 00 (2018) 000–000
1013
4.1.2. Faculty of Letters and Human Sciences The result of the NAU questionnaire of the Faculty of Letters and Human Sciences is summarized as follows (table 4). Table 4. The result of the NAU questionnaire of the Faculty of Letters and Human Sciences FLSH 1 strongly disagree 2 disagree 3 Neutral 4 Agree 5 strongly agree Q1 Q2
36%
5%
21%
42%
32%
2%
9%
20%
50%
19%
Q4
6%
11%
19%
49%
15%
5%
21%
48%
26%
Q6
10%
26%
42%
22%
Q7
11%
16%
20%
48%
5%
Q8
17%
19%
16%
45%
3%
10%
16%
53%
21%
16%
58%
26%
Q9 Q10
•
48%
Q3 Q5
• • • • • • •
16%
The open answers of the Faculty of Letters and Human Sciences summarized as follows: Q.11 Lack of Spanish and German language to use the platform. Lack of research area by discipline. Lack of interactive spaces. Lack of updated journals. Adding a publication takes a lot of time. Lack of guidelines: There were sometimes mistakes to do some tasks in the platform. Search processes sometimes fail (does not give accurate enough information). Q.12 Refine disciplines (disciplinary fields) to adapt them to our realities.
4.1.3. Higher Normal School The result of the NAU questionnaire of the Higher Normal School is summarized as follows (table 5). Table 5. The result of the NAU questionnaire of the Higher Normal School ENS 1 strongly disagree 2 disagree 3 Neutral 4 Agree
5 strongly agree
Q1
6%
15%
56%
23%
Q2
3%
22%
58%
17%
Q3
5%
16%
63%
16%
10%
19%
58%
6%
Q5
10%
17%
43%
30%
Q6
8%
21%
50%
21%
Q4
7%
Q7
9%
20%
21%
42%
8%
Q8
12%
19%
26%
38%
5%
Q9
9%
18%
56%
17%
Q10
4%
11%
61%
24%
The open answers of the Higher Normal School summarized as follows:
Khaoula Benmoussa et al. / Procedia Manufacturing 32 (2019) 1008–1016 Khaoula Benmoussa et al. / Procedia Manufacturing 00 (2018) 000–000
1014
7
Q.11 There were too many steps in the process of adding a publication. Lack of compulsory signs for obligatory fields, which leads us to waste time during a validation of a form. Lack of notification of a new publication added to the system. Unavailability of an online communication space between teacher-researchers. Q.12 • Provide a detailed user manual. • Change the process of adding a publication
• • • •
4.1.4. Polydisciplinary Faculty The result of the NAU questionnaire of the Polydisciplinary Faculty is summarized as follows (table 6). Table 6. The result of the NAU questionnaire of the Polydisciplinary Faculty FP 1 strongly disagree 2 disagree 3 Neutral 4 Agree
5 strongly agree
Q1
14%
14%
58%
14%
Q2
11%
18%
43%
28%
Q3
9%
18%
51%
22%
5%
15%
59%
18%
6%
20%
43%
31%
Q4
3%
Q5 Q6
3%
14%
55%
28%
Q7
6%
16%
28%
46%
4%
Q8
8%
18%
25%
43%
6%
Q9
6%
20%
58%
16%
Q10
3%
15%
53%
29%
The open answers of the Polydisciplinary Faculty summarized as follows: Q.11 • Lack of guidance to add publications to the platform. • Lack of journals in its database presented in the platform Q.12 • Round table, see training by discipline. 4.2. Analysis The results of the questionnaire were generally positive for all UAE university institutions, an explanation of evaluations of all attributes of usability of Nielsen [4] (learning, efficiency, memorization, errors and satisfaction) more positive obtained from the answers to the questionnaire, almost all users agree and validate the usability of the platform. According to the indicators mentioned in Tables 3, 4, 5, 6 we can deduce: • Learning: The platform is easy to understand, the information is clear, the content readable and understandable, which ensures that the platform is simple and easy to accomplish tasks. • Efficiency: The user can access the features of the platform, find the information, but there is a set of users are delaying to perform their tasks. • Memorability: The easy-to-use platform makes it easy for the user to navigate and locate on the platform, which allows memorizing directions of navigation to explore features and content easily. • Error: a set of users find it difficult to fix errors quickly. • Satisfaction: Users have a positive attitude towards platform use. The platform meets the users' desired expectations. The navigation refers to the ease with which the user can cross the interface using the navigation
8
Khaoula Benmoussa et al. / Procedia Manufacturing 32 (2019) 1008–1016 Khaoula Benmoussa et al. / Procedia Manufacturing 00 (2018) 000–000
1015
tools at his disposal (bars, icons, menus, colors / typography, etc.). They usually feel comfortable using the platform, which ensures that users are likely to use it and are satisfied. Tables 3, 4, 5 and 6 of the questionnaire result, especially the indicators in the shaded boxes, show that some users of the system of different UAE university institutions find it difficult to complete their tasks quickly, as well as the error messages sometimes does not help them solve their problem or guide them. Which influences the execution time of a task or block them outright. Users have delays to perform some tasks: the field does not contain a sign of obligation in the case where it is mandatory, it must go through the first task to perform the second task without blocking (fields to perform task 2 is based on the information entered by task 1). Consider the case of adding a publication in the platform that is linked to the choice or the addition of a journal, the user must go through several stages, a lack of information can lead to an error or blockage. At this level, the platform does not present orientations or solutions sufficient enough to ensure an error-free interaction with the user. From the responses to Q11 and Q12, we note that all teacher-researchers at FS, FLSH, ENS and FP have recommended changing or orienting and improving the use of the publication process that is related to the selection or addition of the journal. Based on the result of the questionnaire as a reference to improve the system in the future, so that it can achieve the goal of the teacher-researchers, taking into account their remarks and suggestions, we can draw some points of improvement of the system. 5. Conclusion and future work In general, the use of SIMARECH in the Moroccan university of Abdelmalek Essaadi is rather satisfactory. The results of the SIMARECH platform analysis reveal the good usability that affect the user's acceptance of the platform. It can be deduced that the platform helps users to accomplish what they want to use. They are able to understand how to use it, it is a usable platform because it fulfills the requirements of the users [9]. It can be said that platform quality is effective as important criteria for usability assessment are taken into account in SIMARECH's design and development process [21].The designers and developers of the SIMARECH platform emphasize the importance of clarity, simplicity and consistency in design so that users can perform the desired operations efficiently and effectively and avoid confusion and frustration users. This research produces many recommendations that can be considered, including the development of distance learning to train teacher-researchers, structure managers and administrative staff of Moroccan universities. In order to facilitate the process of learning to use SIMARECH, reduce the administrative costs (due in particular to the travel and commitment of trainers, the reservation of rooms and equipment, etc.), also to save time of training ( meetings, telephone exchanges) and to avoid the problem of the availability of SIMARECH users in a specific time. References [1] Z. Khan and M. Adnan, “Usability evaluation of web-based GIS Applications,” Proc. 11th Int. Conf. Inf. Integr. Web-based Appl. Serv. iiWAS ’09, no. January, p. 557, 2009. [2] “SIMARech 3.” [Online]. Available: http://simarech.uae.ac.ma/. [Accessed: 20-Mar-2018]. [3] K. Benmoussa, M. Laaziri, S. Khoulji, and M. Kerkeb, “SIMARECH 3: A New Application for the Governance of Scientific Research,” First Int. Conf. Affect. Comput. Mach. Learn. Intell. Syst. Sch., vol. 5, pp. 776–784, 2017. [4] J. Nielsen, Usability Engineering, vol. 44, no. 3. 1993. [5] P. W. Jordan, “An Introduction to Usability,” Booksgooglecom, vol. 352, no. 7, pp. 174–203, 1998. [6] N. C. Goodwin, “Functionality and usability,” Commun. ACM, vol. 30, no. 3, pp. 229–233, 1987. [7] J. Nielsen, Designing Web Usability, vol. 3. 2000. [8] C. M. Barnum, Usability Testing and Research, vol. 45, no. 2. 2002. [9] A. Dillon, Designing usable electronic text: ergonomic aspects of human information usage. London: Taylor & Francis, 1994. [10] M. Y. Ivory and M. A. Hearst, “The state of the art in automating usability evaluation of user interfaces,” ACM Comput. Surv., vol. 33, no. 4, pp. 470–516, 2001. [11] H. B. Santoso, A. K. Nisa, and R. Fitriansyah, “Usability evaluation of the Hospital Management Information System: Case study of an emergency installation application of a regional public hospital,” Int. J. Adv. Sci. Eng. Inf. Technol., vol. 7, no. 6, pp. 2294–2301, 2017. [12] ISO, “Iso 9241-11,” Ergon. Requir. Off. Work with Vis. Disp. Termin. -- Part 11 Guid. usability, vol. 1998, p. 22, 1998.
1016
Khaoula Benmoussa et al. / Procedia Manufacturing 32 (2019) 1008–1016 Khaoula Benmoussa et al. / Procedia Manufacturing 00 (2018) 000–000
9
[13] International Organization For Standardization Iso, “ISO/IEC 9126-1,” Software Process: Improvement and Practice, vol. 2, no. 1. pp. 1–25, 2001. [14] B. Shackel and S. J. Richardson, Human factors for informatics usability. Cambridge University Press, 1991. [15] M. Kurosu, Human-Computer Interaction: Design and Evaluation, vol. 9169. Springer International Publishing, 2015. [16] A. Seffah, M. Donyaee, R. B. Kline, and H. K. Padda, “Usability measurement and metrics: A consolidated model,” Softw. Qual. J., vol. 14, no. 2, pp. 159–178, 2006. [17] A. Oztekin, A. Nikov, and S. Zaim, “UWIS: An assessment methodology for usability of web-based information systems,” J. Syst. Softw., vol. 82, no. 12, pp. 2038–2050, 2009. [18] ISO, “Iso 9241,” Ergonomic Requirements for Office Systems with Visual Display Terminals (VDTs). 1997. [19] H. van Laerhoven, H. J. van der Zaag-Loonen, and B. H. F. Derkx, “A comparisonn of Likert scale and visual analogue scales as response options in children’s questionnaires,” Acta Paediatr., vol. 93, pp. 830–835, 2004. [20] J. Nielsen, “Quantitative Studies: How Many Users to Test?,” Nielsen Norman Group, 2006. [Online]. Available: https://www.nngroup.com/articles/quantitative-studies-how-many-users/. [Accessed: 03-May-2018]. [21] R. B. Stone, “Learning and the importance of interactivity information design becomes interaction design,” in Proceedings of the International Conference on Information Visualisation, 2001, vol. 2001–Janua, pp. 624–629.
ScienceDirect ScienceDirect
Available Availableonline onlineatatwww.sciencedirect.com www.sciencedirect.com Procedia Manufacturing 00 (2018) 000–000 Procedia Manufacturing 00 (2018) 000–000
ScienceDirect ScienceDirect
www.elsevier.com/locate/procedia www.elsevier.com/locate/procedia
Procedia Manufacturing 32 (2019) 1008–1016 Procedia Manufacturing 00 (2017) 000–000 www.elsevier.com/locate/procedia
The The 12th 12th International International Conference Conference Interdisciplinarity Interdisciplinarity in in Engineering Engineering
Evaluating Evaluating the the Usability Usability of of aa Moroccan Moroccan University University Research Research Web Platform Manufacturing EngineeringManagement Society International Conference 2017, MESIC 2017, 28-30 June Management Web Platform 2017, Vigo (Pontevedra), Spain
a, b c d c, Mohamed Larbi Kerkebd and Khaoula Khaoula Benmoussa Benmoussaa, *, *, Majida Majida Laaziri Laazirib,, Samira Samira Khoulji Khoulji , Mohamed Larbi Kerkeb and e Costing models for capacityAbir optimization El Abir El Yamami Yamamie in Industry 4.0: Trade-off 0F
0F
a,b,c a,b,c e e
between used capacity and operational efficiency
Information System Engineering Resarch Group, National School of Applied Sciences, Abdelmalek Essaâdi University, Tetouan, Morocco Information System Engineering Resarch Group, National School of Applied Sciences, Abdelmalek Essaâdi University, Tetouan, Morocco d d Information System Engineering Resarch Group, Faculty of Sciences, Abdelmalek Essaadi University, Tetouan, Morocco Information System Engineering Resarch Group, Faculty of Sciences, Abdelmalek Essaadi University, Tetouan, Morocco
Laboratory Signals, Distributed Systems and Artificial Intelligence, ENSET Mohammedia, Hassan II University, Casablanca, a Artificial Intelligence, a,* b b II University, Casablanca, Laboratory Signals, Distributed Systems and Hassan A. Santana , P. Afonso , A.ENSET ZaninMohammedia, , R. Wernke Morocco Morocco a
University of Minho, 4800-058 Guimarães, Portugal b Unochapecó, 89809-000 Chapecó, SC, Brazil
Abstract Abstract The success of any system or interface depends on the satisfaction of the user and users are only satisfied when they are able to The success of any system or interface depends on the satisfaction of the user and users are only satisfied when they are able to Abstract do their job in a simple and effective way [1]. The purpose of the SIMARECH information system [2], [3] is the piloting of do their job in a simple and effective way [1]. The purpose of the SIMARECH information system [2], [3] is the piloting of research in Moroccan universities, their institutions and structures, to support researchers and enhance their scientific output. researchthe in Moroccan universities, institutions and structures, support researchers enhance their interconnected, scientific output. Under concept of "Industrytheir 4.0", production processes towill be pushed to and be increasingly Ensuring the proper use of the SIMARECH information system has always been a real concern of the Abdelmalek Essaadi Ensuring the based proper on usea of thetime SIMARECH information system hasmore always been a In realthis concern of the Abdelmalek Essaadi information real basis and, necessarily, much efficient. context, capacity optimization University (UAE) in Moroccan universities and national research institutions. The purpose of this study is to investigate usability University (UAE) in Moroccanaim universities and maximization, national researchcontributing institutions. The purpose of this study profitability is to investigate usability goes beyond the traditional of capacity also for organization’s and value. issues and provide recommendations for achieving the system goal within the UAE university institutions. issues andlean provide recommendations for achieving improvement the system goal approaches within the UAE university institutions. Indeed, management and continuous suggest capacity optimization instead of The objective of this research paper is to evaluate the usability of the SIMARECH platform based on a complete test The objective of this research paper is to evaluate the usability of the SIMARECH platform based on a complete test maximization. The study capacityfrom optimization costing models an important research topic that methodology. The study wasofconducted January to and March 2018 based on is Jakob Nielsen methodology [4] and thedeserves methods methodology. The study was conducted from January to March 2018 based on Jakob Nielsen methodology [4] and the methods of data collection Nielsen's Attributes of Usability Questionnaire (NAU) [4]. results showedand a high level ofausability for the contributions from both the practical and theoretical perspectives. ThisThe paper presents discusses mathematical of data collection Nielsen's Attributes of Usability Questionnaire (NAU) [4]. The results showed a high level of usability for the platform, but it shouldmanagement be further improved. It isdifferent found thatcosting the followed approach can effectively assess the usability ofhas scientific model for capacity based on models (ABC and TDABC). A generic model been platform, but it should be further improved. It is found that the followed approach can effectively assess the usability of scientific research information systems. developed and it was used to analyze idle capacity and to design strategies towards the maximization of organization’s research information systems.
value. The trade-off capacity maximization vs operational efficiency is highlighted and it is shown that capacity © 2018 2019 The Published by Ltd. optimization might hide operational inefficiency. © 2018 The Authors. Authors. Published by Elsevier Elsevier Ltd. This is an open access article under the CC BY-NC-ND license (https://creativecommons.org/licenses/by-nc-nd/4.0/) © 2017 The Authors. by Elsevier B.V. This is an open accessPublished article under the CC BY-NC-ND license (https://creativecommons.org/licenses/by-nc-nd/4.0/) Selection and peer-review under responsibility of the 12th International Conference Interdisciplinarity in Engineering. Peer-review responsibility of the scientific of the Manufacturing Engineering Society in International Conference Selection andunder peer-review under responsibility ofcommittee the 12th International Conference Interdisciplinarity Engineering. 2017. Keywords: Usability Evaluation; Usability Testing; Web Platform SIMARECH. Keywords: Usability Evaluation; Usability Testing; Web Platform SIMARECH. Keywords: Cost Models; ABC; TDABC; Capacity Management; Idle Capacity; Operational Efficiency
1. Introduction * Corresponding author. Tel.: +0-000-000-0000 ; fax: +0-000-000-0000 . * The Corresponding author. Tel.: +0-000-000-0000 ; fax: information +0-000-000-0000 cost of idle capacity is a fundamental for. companies and their management of extreme importance E-mail address: [email protected] E-mail address: [email protected]
in modern production systems. In general, it is defined as unused capacity or production potential and can be measured 2351-9789 2018 The Authors. Published by Elsevier Ltd.hours of manufacturing, etc. The management of the idle capacity in several©ways: tons of production, available 2351-9789 © 2018 The Authors. Published by Elsevier Ltd. This is an Afonso. open access under the761; CC BY-NC-ND (https://creativecommons.org/licenses/by-nc-nd/4.0/) * Paulo Tel.:article +351 253 510 +351 253license 604 741 This is an open access article under the CC fax: BY-NC-ND license (https://creativecommons.org/licenses/by-nc-nd/4.0/) Selection and peer-review under responsibility of the 12th International Conference Interdisciplinarity in Engineering. E-mail address: [email protected] Selection and peer-review under responsibility of the 12th International Conference Interdisciplinarity in Engineering.
2351-9789 © 2017 The Authors. Published by Elsevier B.V. Peer-review under of the scientificbycommittee the Manufacturing Engineering Society International Conference 2017. 2351-9789 © 2019responsibility The Authors. Published Elsevier of Ltd. This is an open access article under the CC BY-NC-ND license (https://creativecommons.org/licenses/by-nc-nd/4.0/) Selection and peer-review under responsibility of the 12th International Conference Interdisciplinarity in Engineering. 10.1016/j.promfg.2019.02.315
2
Khaoula Benmoussa et al. / Procedia Manufacturing 32 (2019) 1008–1016 Khaoula Benmoussa et al. / Procedia Manufacturing 00 (2018) 000–000
1009
1. Introduction Today, technological development is evolving very rapidly, forcing government agencies to provide better features to improve the quality of the system to satisfy customers. Products that are used in everyday life and that are not usable, that do not meet the requirements of users, cause frustration of users, and frustrated users no longer want to use these products [5]. Building an appropriate system is crucial, it must have great advantages and its quality can be measured according to its ability to be used by all its users, we talk about system usability. Any information system should provide the necessary functions to its users, that they are defined as people, who take some of their ordinary work and use the information system, so they must be able to understand the features of the used system so that they could accomplish their tasks. However, these features alone are not enough. Goodwin [6] indicated that the determinants of system acceptance are functionality and usability. Therefore, if the system is usable, the user can reach a feature easily. According to the well-known usability expert Jakob Nielsen [7], usability reigns on the web, it is simply stated that if users have difficulty accessing the content of the site, they will not use it. Websites with misuse can have many problems such as information is not hierarchical, no simple navigation, wrong choice of colors and fonts, page loading or file download takes time, not directed towards user requirements, etc. Users can not use these websites when dealing with multiple and repeated problems [8]. According to Dillon [9], users are only satisfied when they can achieve their goals in a successful and effective manner. The website interface must be simple and easy to handle for experienced and inexperienced users who can easily use it and meet their requirements. The acceptance and use of technological devices depend on important and varied factors (appropriate design, user skills, etc.), so usability is the key factor in the success of applications, and the evaluation of usability has methodologies that help to measure usability aspects in system interfaces and to identify problems [10]. Ensuring the proper use of SIMARECH system has always been a real concern of Abdelmalek Essaadi University within Moroccan universities and national research institutions. For this, we need to perform an evaluation of its usability to avoid human errors with the system. With usability assessment, we can understand if the system is efficient and usable and how satisfied users are when they use it. As well as the results of the usability assessment can be used as a reference to improve the system in the future, many usability issues can be taken into account in this usability test, so that it can be achieve the goal of its users [11]. SIMARECH is an IT platform that facilitates the collection and management of data from research activities and results within an organization [3]. It aims to present in a coherent way the establishments, the research units within the university, including the personnel and their scientific activities (publications, patents, events, equipment, etc.), as well as the follow-up of the financial means and the international activities (partnership agreements, congresses, etc.). Researchers, administrators, and evaluators have all the tools they need to monitor research results, improve visibility, and allocate available resources efficiently. Initially, the system was structured to present and include a range of information services (programs, projects, events and products resulting from the activities of the research developed, etc.) for the use of users (researchers , structure managers, deans, administrative staff) according to their role, moreover, at the strategic level, it aims to have a set of indicators and available research statistics. Modular nature of the system and flexibility of its data model allowing different users to access certain information and functionalities throughout the process, and facilitate the processing, organization and transmission of information in accordance with international standards [2], [3]. This research focuses on the evaluation of the usability of the SIMARECH platform within UAE university institutions based on Jakob Nielsen's model [7] and measuring its five attributes: learning, efficiency, memorization, errors, and satisfaction. For this, a survey was conducted among 80 users of the platform (SIMARECH) from January to March 2018. The rest of this paper is organized as follow: section 2 presents a literature revue, section 3 is dedicated to the research methodology, and the results are presented in section 4 before concluding in section 5.
1010
Khaoula Benmoussa et al. / Procedia Manufacturing 32 (2019) 1008–1016 Khaoula Benmoussa et al. / Procedia Manufacturing 00 (2018) 000–000
3
2. Literature review In this section, we present different definitions of usability and identify several usability attributes from different standards and models. 2.1. Usability Definitions Usability plays an important role in the software development process. In the field of Human-Computer Interaction (HMI), the most widely accepted definition of usability is that proposed in ISO 9241-11 [12]: "the extent to which a user can use a product to achieve precise objectives with efficiency, effectiveness and satisfaction in a context of specified use ". From this perspective, usability involves the interaction of users with the software product and can be considered as the ability of the product to meet user expectations. On the other hand, in the field of software engineering (SE), the most widely accepted definition of usability is that proposed in ISO 9126-1 [13]: "the ability of the software product to be understood, learned operated and attractive to the user when used under specified conditions ". From this point of view, usability is considered as a specific feature that affects the quality of a software product, it does not necessarily imply user interaction with the system since it can be measured as "compliance to the specification ". 2.2. Usability models While conducting a literature review of existing work in this area of research, we found many usability models. Among these major models that have influenced usability measures are: • Shackel and Richardson [14]: In academia, the concept of usability was defined for the first time systematically by Shackel and Richardson. They proposed four criteria for measuring usability: efficiency (performance in the performance of tasks), learning (means the degree of learning for the completion of tasks), flexibility (the ability to adapt to the variation of the tasks) and the attitude of the users to satisfy the system. • Nielsen [4]: ''Usability is a quality attribute that evaluates the usability of user interfaces '' [4]. Nielsen has associated five attributes to usability: learning (the system is simple and easy to complete tasks), efficiency (The system is efficient and save users time), memorization (The system is easy to remember), errors (The system has a low error rate) and satisfaction (users are likely to use it and are satisfied with the system). Nielsen's model is more acceptable than the one proposed by Shackel and Richardson because the structure of quality characteristics is described more systematically and the place of use is clearly specified [15]. • Seffah, Donyaee, Kline and Padda [16]: These authors developed the Quality In Use Integrated Measurement (QUIM) model to measure the usability of software. This model includes 10 factors, each of which corresponds to a specific usability facet identified in an existing standard or model. These 10 factors are broken down into 26 sub-factors, which are broken down into 127 specific parameters. The authors state, "More attention to user information and context is also needed to facilitate the selection and customization of many proposed metrics and higher-level criteria". • Oztekin, Nikov and Zaim [17]: These authors have proposed a methodology for usability assessment and design of web-based information systems (UWIS). This methodology includes a new checklist to provide usability indices that are defined by considering the usability sub-features proposed in ISO 9241-11 [12], dialogue principles for user interface design according to ISO 9241-10 [18] and usability heuristics proposed by Nielsen [4]. UWIS is also proposing an application of Structural Equation Modeling (SEM) to establish a quantitative model for usability assessment. 2.3. Nielsen Model To perform the usability assessment of our system, we will use the methodology presented by the Nielsen Model Standard [4] for quantitative results and using data collection methods under the form of Nielsen Attributes of Usability (NAU) questionnaires [4]. As the UWIS methodology [17] and the QUIM model [16] contain several criteria, the thing that risks giving up the test on the part of the users of the platform (teacher-researchers).
4
Khaoula Benmoussa et al. / Procedia Manufacturing 32 (2019) 1008–1016 Khaoula Benmoussa et al. / Procedia Manufacturing 00 (2018) 000–000
1011
Here is a map of the five usability attributes of the Nielsen model in the test method used in this research (Fig.1):
Fig. 1. Map of the five usability attributes of the Nielsen model
3. Research methodology In this research, we evaluate the usability of the SIMARECH platform. To this end, we were carried out a usability test by distributing Nielsen Attributes of Usability (NAU) questionnaire [4]. This test is done during the month of January to March 2018, which includes questions in 5 usability categories according to the Nielsen Model Standard, as well as open-ended questions for users to describe their difficulty and express their needs by proposing the SIMARECH system improvement points (Table 1). We distributed the questionnaire to the teacher-researchers (users of the platform) of the four Moroccan institutions of Abdelmalek Essaadi University (UAE): Faculty of Sciences (FS), Higher Normal School (ENS), Polydisciplinary Faculty (FP), Faculty of Letters and Human Sciences (FLSH). We used the NAU questionnaire for data collection because it can be distributed to a larger number of people for a broad response and is an inexpensive approach [8]. The NAU questionnaire is simple and unambiguous; it has helped users to answer these questions without difficulty. He presents closed questions that allowed them to gain time to complete the questionnaire [8]. It is of a scale of 5 points (1-5) known as the Likert scale so it is easy to interpret and users can choose a neutral center point [8], [19]. Thus, this questionnaire includes two open questions to define new development opportunities or improve the existing system. The purpose of the usability method is to evaluate and propose improvements to SIMARECH. Table 1. Nielsen's Attributes of Usability Questionnaire (NAU) [4] Learning
This system is simple to use. The information provided with this system is easy to understand.
Efficiency
I easily found the information I am looking for. I was able to quickly complete my task with this system.
Memorability
The organization of information in the system screens is clear. The system is easy to remember.
Error
The error messages presented by this system tell me clearly how to solve problems. When I made a mistake using this system, it was easy and quick to correct it.
Satisfaction
The interface of this system is nice This system has all the functions and the potential corresponding to my expectations
Open questions
In your opinion, what are the items, information or services missing from the system? What are the possible improvement points?
Khaoula Benmoussa et al. / Procedia Manufacturing 32 (2019) 1008–1016 Khaoula Benmoussa et al. / Procedia Manufacturing 00 (2018) 000–000
1012
5
4. Result and analysis This evaluation of the usability of the SIMARECH platform [2] is a survey of a sample selected from the teacher-researchers of each UAE university institutions. Nielsen generally recommends testing with 20 users when collecting quantitative usability measures, he clarifies that '' With 20 users, you'll probably have 1 outlier (since 6% of users are outliers), so you'll include data from 19 users in your average. This makes your confidence interval go from 243 to 357 seconds, since the margin of error is +/- 19% for testing 19 users '' [20]. The questionnaire was distributed to the 80 teacher-researchers, and their availability 78 among them who answered the questionnaire, distributed as follows (table 2): Table 2. Number of teacher-researchers who answered the test for each institution UAE university institutions
FS
FLSH
ENS
FP
Number of teacher-researchers
20
20
19
19
We decided to do the survey on four institutions, in order to have a mix of response, consideration of the diversity of teaching specialties is very important. According to Nielsen, enough data can be obtained by testing different user cases [20]. The knowledge of the teachers, their skills or expertise in using the platform differs, so each teacher has his own way of handling and judging the platform. In this analysis, we provide usability measurement as a percentage for each university mentioned. These indications will be useful from two points of view, on the one hand to evaluate the usability and on the other hand to deduce the weaknesses and to determine the improvements proposed by the users of the SIMARECH system. 4.1. Result 4.1.1. Faculty of Sciences The result of the NAU questionnaire of the Faculty of Sciences is summarized as follows (table 3). FS Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10
• • • • •
Table 3. The result of the NAU questionnaire of the Faculty of Sciences 1 strongly disagree 2 disagree 3 Neutral 4 Agree 5 strongly agree 3% 20% 61% 16% 1% 17% 57% 25% 10% 15% 55% 20% 2% 13% 19% 50% 16% 4% 28% 50% 18% 10% 19% 51% 20% 4% 11% 23% 53% 9% 7% 14% 21% 52% 6% 8% 20% 65% 7% 3% 24% 61% 12%
The open answers of the Faculty of Sciences summarized as follows: Q.11: Many indexed journals cannot be found on the system, so consider modifying and updating the list regularly. Lack of online communication support with other teachers in the platform. Q.12: You have to think about looking for an easy method of entering the articles. Ability to import data from different international base (articles, communications, etc.) instead of re-entering information into the platform. Facilitate the task for the teacher to use the system quickly and without wasting time.
6
Khaoula Benmoussa et al. / Procedia Manufacturing 32 (2019) 1008–1016 Khaoula Benmoussa et al. / Procedia Manufacturing 00 (2018) 000–000
1013
4.1.2. Faculty of Letters and Human Sciences The result of the NAU questionnaire of the Faculty of Letters and Human Sciences is summarized as follows (table 4). Table 4. The result of the NAU questionnaire of the Faculty of Letters and Human Sciences FLSH 1 strongly disagree 2 disagree 3 Neutral 4 Agree 5 strongly agree Q1 Q2
36%
5%
21%
42%
32%
2%
9%
20%
50%
19%
Q4
6%
11%
19%
49%
15%
5%
21%
48%
26%
Q6
10%
26%
42%
22%
Q7
11%
16%
20%
48%
5%
Q8
17%
19%
16%
45%
3%
10%
16%
53%
21%
16%
58%
26%
Q9 Q10
•
48%
Q3 Q5
• • • • • • •
16%
The open answers of the Faculty of Letters and Human Sciences summarized as follows: Q.11 Lack of Spanish and German language to use the platform. Lack of research area by discipline. Lack of interactive spaces. Lack of updated journals. Adding a publication takes a lot of time. Lack of guidelines: There were sometimes mistakes to do some tasks in the platform. Search processes sometimes fail (does not give accurate enough information). Q.12 Refine disciplines (disciplinary fields) to adapt them to our realities.
4.1.3. Higher Normal School The result of the NAU questionnaire of the Higher Normal School is summarized as follows (table 5). Table 5. The result of the NAU questionnaire of the Higher Normal School ENS 1 strongly disagree 2 disagree 3 Neutral 4 Agree
5 strongly agree
Q1
6%
15%
56%
23%
Q2
3%
22%
58%
17%
Q3
5%
16%
63%
16%
10%
19%
58%
6%
Q5
10%
17%
43%
30%
Q6
8%
21%
50%
21%
Q4
7%
Q7
9%
20%
21%
42%
8%
Q8
12%
19%
26%
38%
5%
Q9
9%
18%
56%
17%
Q10
4%
11%
61%
24%
The open answers of the Higher Normal School summarized as follows:
Khaoula Benmoussa et al. / Procedia Manufacturing 32 (2019) 1008–1016 Khaoula Benmoussa et al. / Procedia Manufacturing 00 (2018) 000–000
1014
7
Q.11 There were too many steps in the process of adding a publication. Lack of compulsory signs for obligatory fields, which leads us to waste time during a validation of a form. Lack of notification of a new publication added to the system. Unavailability of an online communication space between teacher-researchers. Q.12 • Provide a detailed user manual. • Change the process of adding a publication
• • • •
4.1.4. Polydisciplinary Faculty The result of the NAU questionnaire of the Polydisciplinary Faculty is summarized as follows (table 6). Table 6. The result of the NAU questionnaire of the Polydisciplinary Faculty FP 1 strongly disagree 2 disagree 3 Neutral 4 Agree
5 strongly agree
Q1
14%
14%
58%
14%
Q2
11%
18%
43%
28%
Q3
9%
18%
51%
22%
5%
15%
59%
18%
6%
20%
43%
31%
Q4
3%
Q5 Q6
3%
14%
55%
28%
Q7
6%
16%
28%
46%
4%
Q8
8%
18%
25%
43%
6%
Q9
6%
20%
58%
16%
Q10
3%
15%
53%
29%
The open answers of the Polydisciplinary Faculty summarized as follows: Q.11 • Lack of guidance to add publications to the platform. • Lack of journals in its database presented in the platform Q.12 • Round table, see training by discipline. 4.2. Analysis The results of the questionnaire were generally positive for all UAE university institutions, an explanation of evaluations of all attributes of usability of Nielsen [4] (learning, efficiency, memorization, errors and satisfaction) more positive obtained from the answers to the questionnaire, almost all users agree and validate the usability of the platform. According to the indicators mentioned in Tables 3, 4, 5, 6 we can deduce: • Learning: The platform is easy to understand, the information is clear, the content readable and understandable, which ensures that the platform is simple and easy to accomplish tasks. • Efficiency: The user can access the features of the platform, find the information, but there is a set of users are delaying to perform their tasks. • Memorability: The easy-to-use platform makes it easy for the user to navigate and locate on the platform, which allows memorizing directions of navigation to explore features and content easily. • Error: a set of users find it difficult to fix errors quickly. • Satisfaction: Users have a positive attitude towards platform use. The platform meets the users' desired expectations. The navigation refers to the ease with which the user can cross the interface using the navigation
8
Khaoula Benmoussa et al. / Procedia Manufacturing 32 (2019) 1008–1016 Khaoula Benmoussa et al. / Procedia Manufacturing 00 (2018) 000–000
1015
tools at his disposal (bars, icons, menus, colors / typography, etc.). They usually feel comfortable using the platform, which ensures that users are likely to use it and are satisfied. Tables 3, 4, 5 and 6 of the questionnaire result, especially the indicators in the shaded boxes, show that some users of the system of different UAE university institutions find it difficult to complete their tasks quickly, as well as the error messages sometimes does not help them solve their problem or guide them. Which influences the execution time of a task or block them outright. Users have delays to perform some tasks: the field does not contain a sign of obligation in the case where it is mandatory, it must go through the first task to perform the second task without blocking (fields to perform task 2 is based on the information entered by task 1). Consider the case of adding a publication in the platform that is linked to the choice or the addition of a journal, the user must go through several stages, a lack of information can lead to an error or blockage. At this level, the platform does not present orientations or solutions sufficient enough to ensure an error-free interaction with the user. From the responses to Q11 and Q12, we note that all teacher-researchers at FS, FLSH, ENS and FP have recommended changing or orienting and improving the use of the publication process that is related to the selection or addition of the journal. Based on the result of the questionnaire as a reference to improve the system in the future, so that it can achieve the goal of the teacher-researchers, taking into account their remarks and suggestions, we can draw some points of improvement of the system. 5. Conclusion and future work In general, the use of SIMARECH in the Moroccan university of Abdelmalek Essaadi is rather satisfactory. The results of the SIMARECH platform analysis reveal the good usability that affect the user's acceptance of the platform. It can be deduced that the platform helps users to accomplish what they want to use. They are able to understand how to use it, it is a usable platform because it fulfills the requirements of the users [9]. It can be said that platform quality is effective as important criteria for usability assessment are taken into account in SIMARECH's design and development process [21].The designers and developers of the SIMARECH platform emphasize the importance of clarity, simplicity and consistency in design so that users can perform the desired operations efficiently and effectively and avoid confusion and frustration users. This research produces many recommendations that can be considered, including the development of distance learning to train teacher-researchers, structure managers and administrative staff of Moroccan universities. In order to facilitate the process of learning to use SIMARECH, reduce the administrative costs (due in particular to the travel and commitment of trainers, the reservation of rooms and equipment, etc.), also to save time of training ( meetings, telephone exchanges) and to avoid the problem of the availability of SIMARECH users in a specific time. References [1] Z. Khan and M. Adnan, “Usability evaluation of web-based GIS Applications,” Proc. 11th Int. Conf. Inf. Integr. Web-based Appl. Serv. iiWAS ’09, no. January, p. 557, 2009. [2] “SIMARech 3.” [Online]. Available: http://simarech.uae.ac.ma/. [Accessed: 20-Mar-2018]. [3] K. Benmoussa, M. Laaziri, S. Khoulji, and M. Kerkeb, “SIMARECH 3: A New Application for the Governance of Scientific Research,” First Int. Conf. Affect. Comput. Mach. Learn. Intell. Syst. Sch., vol. 5, pp. 776–784, 2017. [4] J. Nielsen, Usability Engineering, vol. 44, no. 3. 1993. [5] P. W. Jordan, “An Introduction to Usability,” Booksgooglecom, vol. 352, no. 7, pp. 174–203, 1998. [6] N. C. Goodwin, “Functionality and usability,” Commun. ACM, vol. 30, no. 3, pp. 229–233, 1987. [7] J. Nielsen, Designing Web Usability, vol. 3. 2000. [8] C. M. Barnum, Usability Testing and Research, vol. 45, no. 2. 2002. [9] A. Dillon, Designing usable electronic text: ergonomic aspects of human information usage. London: Taylor & Francis, 1994. [10] M. Y. Ivory and M. A. Hearst, “The state of the art in automating usability evaluation of user interfaces,” ACM Comput. Surv., vol. 33, no. 4, pp. 470–516, 2001. [11] H. B. Santoso, A. K. Nisa, and R. Fitriansyah, “Usability evaluation of the Hospital Management Information System: Case study of an emergency installation application of a regional public hospital,” Int. J. Adv. Sci. Eng. Inf. Technol., vol. 7, no. 6, pp. 2294–2301, 2017. [12] ISO, “Iso 9241-11,” Ergon. Requir. Off. Work with Vis. Disp. Termin. -- Part 11 Guid. usability, vol. 1998, p. 22, 1998.
1016
Khaoula Benmoussa et al. / Procedia Manufacturing 32 (2019) 1008–1016 Khaoula Benmoussa et al. / Procedia Manufacturing 00 (2018) 000–000
9
[13] International Organization For Standardization Iso, “ISO/IEC 9126-1,” Software Process: Improvement and Practice, vol. 2, no. 1. pp. 1–25, 2001. [14] B. Shackel and S. J. Richardson, Human factors for informatics usability. Cambridge University Press, 1991. [15] M. Kurosu, Human-Computer Interaction: Design and Evaluation, vol. 9169. Springer International Publishing, 2015. [16] A. Seffah, M. Donyaee, R. B. Kline, and H. K. Padda, “Usability measurement and metrics: A consolidated model,” Softw. Qual. J., vol. 14, no. 2, pp. 159–178, 2006. [17] A. Oztekin, A. Nikov, and S. Zaim, “UWIS: An assessment methodology for usability of web-based information systems,” J. Syst. Softw., vol. 82, no. 12, pp. 2038–2050, 2009. [18] ISO, “Iso 9241,” Ergonomic Requirements for Office Systems with Visual Display Terminals (VDTs). 1997. [19] H. van Laerhoven, H. J. van der Zaag-Loonen, and B. H. F. Derkx, “A comparisonn of Likert scale and visual analogue scales as response options in children’s questionnaires,” Acta Paediatr., vol. 93, pp. 830–835, 2004. [20] J. Nielsen, “Quantitative Studies: How Many Users to Test?,” Nielsen Norman Group, 2006. [Online]. Available: https://www.nngroup.com/articles/quantitative-studies-how-many-users/. [Accessed: 03-May-2018]. [21] R. B. Stone, “Learning and the importance of interactivity information design becomes interaction design,” in Proceedings of the International Conference on Information Visualisation, 2001, vol. 2001–Janua, pp. 624–629.