A study on the usability of E-books and APP in engineering courses: A case study on mechanical drawing

Accepted Manuscript A study on the Usability of E-books and APP in Engineering Courses: A Case Study on Mechanical Drawing Min Jou, Robert D. Tennyson, Jingying Wang, Szu-Ying Huang PII:

S0360-1315(15)30053-1

DOI:

10.1016/j.compedu.2015.10.004

Reference:

CAE 2919

To appear in:

Computers & Education

Received Date: 16 July 2015 Revised Date:

1 October 2015

Accepted Date: 5 October 2015

Please cite this article as: Jou M., Tennyson R.D., Wang J. & Huang S.-Y., A study on the Usability of Ebooks and APP in Engineering Courses: A Case Study on Mechanical Drawing, Computers & Education (2015), doi: 10.1016/j.compedu.2015.10.004. This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

ACCEPTED MANUSCRIPT

A study on the Usability of E-books and APP in Engineering Courses: A Case Study on Mechanical Drawing

RI PT

Min Jou* [email protected] * 162, Section 1, He-Ping East Road, Taipei, Taiwan Corresponding author at: National Taiwan Normal University

Robert D. Tennyson [email protected] Department of Educational Psychology, University of Minnesota

M AN U

Szu-Ying Huang

SC

Jingying Wang [email protected] Capital Normal University, Beijing 100048, China

Department of Industrial Education, National Taiwan Normal University

AC C

EP

TE D

[email protected]

1

ACCEPTED MANUSCRIPT

A study on the Usability of E-books and APP in Engineering Courses: A Case Study on Mechanical Drawing

TE D

M AN U

SC

RI PT

Developments in cloud technology has made it possible for users to use the Internet and smart handheld devices for performing various tasks. Many types of E-books and APP have been made as a result. However, few investigations have been carried out to determine whether E-books and APP are useful for college-level engineering courses in the classroom settings. Hence, this study has selected a mechanical drawing course where E-books and APP could be used in theoretical and practical teaching to perform an empirical investigation. E-books were used to study the principles of mechanical drawing, while the APP was used for creating mechanical drawings and learning the processes involved in actual mechanical drawing processes. The investigation then analyzed the relationship between learning styles and the usability of the E-books and the APP. Results showed that students gave higher ratings for the aesthetics, convenience, intention to use, user satisfaction, and task-technology fit for E-book usability. For APP, students gave good reviews for 7 aspects, namely intention to use, completeness, consistency and functionality, course management, perceived usefulness, aesthetics, and user satisfaction. Further findings showed that the use of E-books and APP in the learning of CAD was closely related to learning styles and that students with different learning styles have their respective preferences. Student performance in the course after using E-books and APP in the learning was closely related to learning styles. Theorists achieved the best scores in the mechanical drawing course with E-books and the APP, followed by the pragmatist, the activist, the reflector, and finally the mixed learner. To develop E-books that are practical in the field of education, this study recommends converting abstract concepts into actual examples, providing challenging learning activities, offering multi-sensory experiences, establishing logical connections between knowledge points, and giving learners with problem solutions, means of searching for additional information, operational interfaces, and self-learning systems as well. This study offered a direction in the development of usable learning materials (E-books and APP) that would be more helpful and compatible with actual lessons in class. Keywords: Engineering courses, E-books, APP, usability, learning styles

AC C

EP

1. Introduction Mobile technology has ushered in mobile learning as a brand new learning model. Mobile learning is a new learning model that includes personalized, flexible, and context-based teaching and learning while providing interactivity, mobility, timeliness, incorporation of multimedia elements as well as the ability to overcome multiple temporal and spatial barriers as it can be used anytime and anywhere. Mobile learning allows educators to generate digital learning solutions for learners that can be used anytime and anywhere as required in order to achieve results that cannot be achieved with other current educational models. The Horizon Report 2012 released by the New Media Consortium pointed out that the use of mobile applications (APPs) and tablet computers will be mainstream within the following year. Major focuses for the next five years shall include the means of creating seamless and comprehensive learning experiences and environments by exploring the means of applying new technologies such as context awareness, radio frequency identification (RFID), and augmented reality (AR) as well as how to create corresponding learning strategies and tools in order to help learners adapt to the complex learning environment mentioned above and make mobile learning a reality. Forhberg et al. (2009) identified 102 mobile learning projects by using the stringent mobile learners' objective model proposed by Sharples and Taylor which analyzed each project based upon the 6 dimensions of context, tools, control, interactions, objective goals, and subjective goals, introducing archetypal cases for each of the dimensions mentioned. The investigation proposed a creative idea that mobile learning is capable of supporting learning under various contexts. However, the learner must be capable of employing knowledge under such contexts instead of simply acquiring knowledge. New learners' often encounter difficulty in applying acquired 1

ACCEPTED MANUSCRIPT

SC

RI PT

knowledge. Hence, mobile learning should target advanced learners in order to realize its maximum potential as a tool for inspiring deep reflection, interaction, and collaboration. Electronic books (E-books) and APPs are some of the mainstream mobile learning tools that are currently available. Such tools are also becoming increasingly popular amongst learners as well. E-books, as a form of network publishing, are receiving greater attention from more and more individuals and readers in the industry. The theory of remedial media believes that new media emerge as remedial solutions for existing media formats. The following lists the remedial solutions offered by E-books for traditional printed books: E-books satisfy the need for accessing digital content for readers in the Internet age. Secondly, E-books offer breakthrough solutions for the monotonous presentation of printed books. Thirdly, E-books provides a technical foundation for achieving two-way interactivity. Finally, E-books presents a publishing format that allows writers to skip procedures typically required in traditional publishing. Many definitions have been offered for E-books. Rapid technological developments have made it difficult to offer a single definition for this publishing format. E-books can be defined as “any piece of electronic text regardless of size or composition, excluding journal publications, made available electronically or optically, for any device, hand-held or desk-bound that includes a screen” (Dinkelman & Stacy-Bates,

TE D

M AN U

2007)。 For precise definitions, E-books may be divided into 2 separate concepts. In the first of these concepts, E-books may be regarded as a digital version of a traditional, printed text or digital books published on the Internet that have never been released in print. In the second concept, E-books may also be a specialized reader hardware. The first concept shall be employed for the purpose of this study. Many previous studies have employed mobile technologies to engage student in various forms of learning activities, such as experimental procedure training, skill training, creativities thinking and problem solving. They report the trend of mobile technology-enhanced learning. Tsai and Hwang (2013) believes that technology-based learning in Asia is specifically focused upon the proliferation and popularity of smartphones and mobile devices which had helped to encourage student learning. Hwang, Yang, Tsai, & Yang, Stephen (2009) believe that for several decades, researchers have been committed to studying the effects and results of technology-enhanced learning. These researchers investigated the values and functions of context-based u-computing from technological, educational, and benefit aspects, using experimental research in order to qualify their hypotheses. Technology-based learning contexts such as computers, network, and mobile technologies are heavily correlated

EP

with the students' personal background (Tsai, Tsai, & Hwang, 2012). Only technology-based mobile learning media designed to adapt to different modes of student learning will be able to effectively improve student learning results. Hwang and Wu (2014) investigated 2008–2012 publications in seven well-recognized Social Science Citation Index (SSCI) journals of technology-enhanced learning to investigate the applications and impacts of mobile technology-enhanced learning. Results revealed that mobile learning improved student learning performance and exerted a positive influence over the students' learning motivation and interest. Amongst the

AC C

diverse selection of computer-based learning media, students also demonstrated enthusiasm with learning with E-books. Multiple researches showed that E-books promote professional learning amongst students that help to improve non-intellectual factors of student learning such as learning style and learning motivation (Abd, Azelin, Hezlina, Razol and Zullina, 2012; Malathi and Rohani, 2010). The Technology Acceptance Model (TAM) was first developed by Davis (1989) in order to investigate user acceptance of information technology (IT) systems. The original purpose of the TAM was to understand the factors that determined the general acceptance of computers. The TAM proposed two important determinants: (1) perceived usefulness which reflects a person's perception that using the specified system would help improve work performance and (2) perceived ease of use which reflects a person's perception on the ease of using the system. The TAM believes that system use is determined by behavioral intention which is then determined by attitudes toward using as well as perceived usefulness. The attitude toward using would be determined jointly by the perceived usefulness and ease of use; perceived usefulness would be determined jointly by the perceived ease of use and other external factors; and perceived ease of use would be determined by external factors. These external factors may include system design 2

ACCEPTED MANUSCRIPT

F Task-technology fit A Convenience I Aesthetics

Usage attitude

Perceived ease of use

AC C

H Performance outcome expectation

J User satisfaction

Perceived usefulness

EP

C Perceived usefulness

TE D

M AN U

SC

RI PT

features, user characteristics (include form of perception and other characteristics), task characteristics, development or implementation processes, and influences from public policies or organizational framework. External factors would also be linked to internal beliefs, attitudes, intentions, differences between individuals, environment constraints, and controllable interference factors (Venkatesh & Bala, 2008). Other researchers have carried out additional investigations and made subsequent changes to the original TAM. Taylor and Todd (1995a) believed that although large amounts of empirical research support the use of TAM in predicting user behavioral intention and practical usage behaviors for new technologies, the model did not include social factors and control factors. Social and control factors would be key variables in the theory of planned behavior (TPB), and numerous studies have shown that these factors exert significant influence on users' actual utilization of new technologies. Taylor and Todd thus integrated TAM with TPB (1995b), adding the two variables of objective regulations and perceived behavioral control to the TAM model to form the C-TAM-TPB. This new integrative model was then used in an empirical research targeting student behavior in the use of a computer resource center. In many previous researches (Taylor& Todd, 1995c; Karahanna & Straub, 1999; Venkatesh & Davis, 2000), subjective norm has been identified as a key determinant for technology acceptance as it directly influences personal belief, thereby influencing the intention to use specific technologies. By referring to motivation theories and the amended TAM (Legris & Ingham, 2003), Jou and Wang (2013) believed that the perceived usefulness and ease of use in TAM lacked considerations for motivation as an intermediary factor. However, studies have shown that students would only change their learning attitude after gaining learning motivation. Another research on TAM conducted by Srite and Karahanna (2006) focused only upon the intention of use, and did not focus on practical usage issues in the later phases of TAM. Hence, the intention of use stated in this paper focused on the intermediary efficiency of intention of use on learning results in order to understand how the use of computer-assisted technology help students evaluate the adequacy of the digital learning materials as well as the relationships between learning style and E-books learning results. This section describes the importance of TAM in this investigation, where intention of use is used as an intermediary efficiency for learning results. Results of this study showed that student academic performance is closely correlated to their learning style while past investigations on TAM only focused on intention of use and failed to consider the connections between intermediaries and their linkages with student academic results.

K Intention to use

Learning style

Behavior intention

G Task-technology effectiveness

Usage behavior

D Time management

Learning results

E Self-evaluation

B Compatibility

Figure 1 - TAM model for E-books in this study

3

ACCEPTED MANUSCRIPT

M Perceived usefulness

I Memorability

N User satisfaction

Perceived usefulness

L Reducing redundancy

Usage attitude

C Convenience

Learning style

Behavior intention

K Aesthetics B Visibility J Completeness

Perceived ease of use

O Intention to E Interactivity use feedback and help

Learning results

D Course management

H Assessment strategy

SC

F Accessibility

Usage behavior

RI PT

A Error prevention

G Consistency

M AN U

Figure 2 - TAM model for APP in this study

AC C

EP

TE D

The American researcher Kolb (1976) defined learning style as an individual's preferred perception and approach on information processing. A more integrative view described learning style as a specific requirement and preferred learning strategy used regardless of the learning task involved, and a blueprint for information processing. Although there has yet to be a commonly accepted definition for learning styles, three common properties could be identified from various definitions. Firstly, learning style emphasizes a student's preference or commonly used learning strategy and inclination. Secondly, learning style emphasizes stability and consistency, and would change very little regardless contents, location, and other factors. Finally, each learning style should be unique from each other. Kolb has divided learning process into information perception and information processing steps. Any preference of inclination exhibited by learners in these two steps would be described as learning style. Under such learning environments, every learner must respond to information according to their own data perception. Given that psychological differences exist between individual learners, the rate of information acquisition, information perception, and information response would differ as well. Kolb believed that learning was a cyclical process where a learner constantly increases, perceives, and processes information. Kolb (1984) also analyzed learning cycles and pointed out learning cycles were composed of four interlocking segments, namely concrete experience, reflective observation, abstract conceptualization, and active experimentation. Under such theories, Kolb divided learning cycles into 4 phases of experimentation, reflection, theorization, and planning. Together, these four phases would form a comprehensive learning process. This study would adopt this definition. The research conducted by Honey and Mumford (1992) was based upon Kolb's theories. They believed that differences exist in the acquisition and processing of information between different individuals. However, a certain rule could be observed in these distributions. Honey and Mumford thus proposed 4 categories of learning methods: theorist, pragmatist, activist, and reflector. Usually, learners would adopt one or two learning methods that they are comfortable with, and tend to avoid methods where they feel less at ease. Therefore, learners would tend to start and stop in a phase (or two) in the learning cycle they incline towards instead of trying to complete the entire learning cycle. However, according to Kolb's 4-phase model of learning, learners must adopt all four methods to complete the learning cycle. Any missing learning method would be a barrier against effective learning. However, some learners may also adopt two or more learning styles at the same time. This study describes such approaches as the mixed learning style. Additionally, Honey and Mumford also developed a Learning Styles Questionnaire (LSQ) based upon the learning style inventory to help learners identify their learning method. By understanding their learning methods, learners would be able to adopt a most suitable learning approach and make adjustments to learning approaches that they are not familiar with. 4

ACCEPTED MANUSCRIPT

2. Research method and tools

M AN U

SC

RI PT

This study made use the Technology Acceptance Model (TAM) to evaluate the usability of E-books and APP in a mechanical drawing course and analyze whether there were correlations between student learning results and learning styles as well as E-books and APP usability. Usability of the E-books and APP was mainly analyzed using the three TAM phases of perception, intent, and usage. Each phase would include various aspects of E-books and APP usability. This study has reviewed the descriptions provided in other studies such as Barnard et al. (2008), Lai and Ulhas (2012), Lin & Huang (2006) and Oztekin et al. (2010) to compile an initial draft on the assessment indicators and contents for E-books and APP usability for a mechanical drawing course. Experts and professionals were then invited to review and revise this initial draft in order to achieve expert validation. The final draft on E-books usability included 11 aspects, namely: (A) convenience, (B) compatibility, (C) perceived usefulness, (D) time management, (E) self-evaluation, (F) task-technology fit, (G) task-technology effectiveness, (H) performance outcome expectation, (I) aesthetics, (J) user satisfaction, and (K) intention to use. Table 1 shows the corresponding contents / indicators of E-books usability assessment for the learning of mechanical drawing. A total of 15 aspects were investigated for APP usability, namely (A) error prevention, (B) visibility, (C) convenience, (D) course management, (E) interactivity feedback and help, (F) accessibility, (G) consistency and functionality, (H) assessment strategy, (I) memorability, (J) completeness, (K) aesthetics, (L) reducing redundancy, (M) perceived usefulness, (N) user satisfaction, and (O) intention to use. Table 2 shows the corresponding indicators of APP usability assessment for the learning of mechanical drawing. Before formally implementing the test, the assessment questionnaire for the usability of the E-books and APP for the learning of mechanical drawing was pre-tested for 3 consecutive years which involved 76, 72, and 74 students. The reliability test values of Cronbach's Alpha for the 40 questions on E-book usability for each of the 3 years were 0.874, 0.891, and 0.902 respectively for an overall score of 0.892. Cronbach's Alpha scores for the 46 questions on APP usability were 0.872, 0.889, and 0.898 respectively for an overall score of 0.896. This analysis showed that the internal consistency of the questionnaire was excellent and that the test results for all 3 years could be regarded as highly reliable.

EP

TE D

The aforementioned research (Abd, Azelin, Hezlina, Razol and Zullina, 2012; Malathi and Rohani, 2010) showed that mobile learning solutions such as E-books and APPs exert a direct influence over non-intellectual factors such as learning style, thereby influencing the process of E-books and APP usage. To study and determine a learner's learning style, this study made use of the Learning Style Questionnaire (LSQ) developed by Honey and Mumford. Further investigations were carried out to analyze the relationship between student learning styles and learning performance using the E-books and APP and determine whether there were correlations between E-books and APP usability with learning performance.

Aspect

AC C

Table 1 shows the E-books usability assessment indicators and corresponding question number on the questionnaire form

A: Convenience

B: Compatibility

Content / indicator

Question

Question number

A-1 Freedom of

A-2 Ease of

A-3 Electronic

time

acquiring

applications

A-4 Time saving

4

1~4

B-4 Requirements satisfied

4

5~8

C-4 Learning support

4

9~12

3

13~15

information

B-1 Suitable

B-2 Emotional

B-3 Learning

method

preference

ease

C: Perceived

C-1 Grades

C-2 Learning

C-3

usefulness

improvement

results

Professional knowledge

D: Time management

D-1 Time

D-2 Fixed

D-3 Time savings

5

ACCEPTED MANUSCRIPT

scheduling

schedules

E: Self-evaluation

E-1 Content

E-2 Self

E-3 Student

review

questioning

discussion

F: Task-technology fit

F-1 Practical

F-2 Suitability of

F-3

nature of the

the technology

Effectiveness of

4

16~19

F-4 Compatibility of the technology

4

20~23

the technology

G: Task-technology

G-1 Technical

G-2 Technical

G-3 Easiness of

effectiveness

support

contents

the task

H: Performance

H-1

H-2 Learning

outcome expectation

Organization

efficacy

G-4 Personal requirements

4

24~27

H-3 Learning

H-4 Learning

5

28~32

time

quality

2

33~34

3

35~37

3

38~40

skills I-1 Color and

I-2 Interface comfort

icons

K: Intention to use

J-1 Information

J-2 Exceeding

use

of expectations

K-1 Continued

K-2 Emotional

use

improvement

J-3 Expected results

performance

K-3 Recommend to others

M AN U

J: User satisfaction

H-5 Learning

SC

I: Aesthetics

RI PT

technology

E-4 Self evaluation

Table 2 APP usability assessment indicators and corresponding question number in the questionnaire Indicator category

A: Error prevention

A-1 Multiple operations

A-2 Cancellation of function

B: visibility

B-1 Reasonable

B-2 Clear functions

arrangement

TE D

Indicator aspect

C: Convenience

C-1 Loading speed

D: Course

D-1 Links to information

management

Question

Question

A-3 Hints and warnings

3

1~3

B-3 Effective layout

3

4~6

2

7~8

4

9~12

E-3 Keeping track of progress

3

13~15

number

C-2 Personalized environment D-2 Personalized resources

D-4 Download and views

E-1 Improved

feedback and help

communication

F: Accessibility

F-1 Personal preferences

F-2 Instantaneous support

F-3 Usage approach

3

16~18

G: Consistency and

G-1 Similar formats

G-2 Clear functions

G-3 Convenient interface

3

19~21

H-1 Degree of preparation

H-2 Assessment

H-3 Consistent objectives

3

22~24

4

25~28

3

29~31

2

32~33

3

34~36

4

37~40

3

41~43

strategies I: Memorability

AC C

H: Assessment

EP

E: Interactivity

functionality

E-2 Regular feedback

D-3 Ease of uploads

performance

I-1 Question help

I-2 Options program

I-3 Hint windows

J: Completeness

J-1 Indicating links

J-2 Browsing interface

J-3 Overall structure

K: Aesthetics

K-1 Color and icons

K-2 Interface comfort

L-1 Amending the errors

L-2 Flexible interface

L-3 Reading materials

M-1 Grades improvement

M-2 Learning efficacy

M-3 Improved

M-4 Facilitating

knowledge

learning

N-1 Information use

N-2 Exceeding of

N-3 Expected results

L: Reducing

I-3 Clear interface

redundancy M: Perceived usefulness N: User satisfaction

expectations 6

ACCEPTED MANUSCRIPT

O: Intention to use

O-1 Continued use

O-2 Emotional improvement

O-3 Recommend to others

3

44~46

SC

RI PT

The first stage on quantification included assessing the usability of E-books and APP as well as analyzing whether usability of these two learning tools differed according to educational background and gender of the student. Correlation analysis was also performed on the general status of previous use of IT products with the usability of the E-books and APP as well. The second stage of quantification would include the analysis of the relationship between learning styles and learning results. Correlation statistics were also implemented between the learning styles of high scoring and low scoring students. Qualitative analysis was mainly carried out through semi-structured interviews with the students. Results of the interviews were analyzed using the following process: responses for the semi-structured interview on E-books and APP usability were first compiled and then given open coding. Representative student responses were selected to analyze the category of the interview results. The list of 5 types of student learning styles were then used to identify keywords and key phrases in the six categories of students with similar learning styles. Frequency and percentage of these keywords and phrases were then statistically compiled. Results were then summarized and discussed.

M AN U

Students from the department of mechanical engineering were selected as the subjects of this study. The experimental course was on the topic of mechanical drawing and lasted 3 semesters. All students were taught by the same professor and utilized E-books and mobile APP as learning materials for the course.

3. Results and discussions

AC C

EP

TE D

3.1 Quantification and assessment of E-books usability After analysis, it was found that aesthetics was the dimension of E-books usability that received the highest score (Figure 3) and was obviously higher compared to other indicators. This was followed by convenience, intention to use, user satisfaction, and task-technology fit. The lowest scoring indicators were compatibility, indicating that the E-book compatibility with student learning styles was given a lower rating compared to other indicators. However, the score was 3.5, or 70% on a percentage score, may be considered high enough in some cases. The lowest scoring item of compatibility was the one on emotional preference (Table 3).

Figure 3. A comparison of average scores on the various dimensions for E-books usability in the mechanical drawing course. Table 3. A list of average scores on the various dimensions for E-books usability in the mechanical drawing course. Aspect A: Convenience 3.85

Indicator category A-1 Freedom of time 3.93

A-2 Ease of acquiring information 3.96

A-3 Electronic applications 3.68

A-4 Time saving 3.82

7

ACCEPTED MANUSCRIPT

B: Compatibility

B-2 Emotional preference 3.35

B-1 Suitable method

B-3 Learning ease

B-4 Requirements satisfied

3.56 C: Perceived usefulness 3.67 D: Time management

3.59 C-1 Grades improvement 3.70

3.64

D-1 Time scheduling

D-2 Fixed schedules

3.69 E: Self-evaluation

3.92

3.58

E-1 Content review

E-2 Self questioning

3.69 F: Task-technology fit

3.54 F-1 Practical nature of the technology 3.80

3.65 F-2 Suitability of the technology 3.78

G-1 Technical support

G-2 Technical contents

G-3 Easiness of the task

3.82

3.68

3.72

3.78 F-4 Compatibility of the technology 3.73 G-4 Personal requirements 3.66

H-1 Organization skills

H-2 Learning efficacy

H-3 Learning time

H-4 Learning quality

3.58

3.65

3.76

3.69

I-1 Color and icons

I-2 Interface comfort

3.85 J: User satisfaction

4.05

3.78 K: Intention to use

3.97

3.85 J-2 Exceeding of expectations 3.59 K-2 Emotional improvement 3.77

C-2 Learning results

K-1 Continued use

3.82

C-4 Learning support

3.70

3.78

D-3 Time savings 3.50 E-4 Self evaluation

RI PT

E-3 Student discussion 3.78 F-3 Effectiveness of the technology 3.72

H-5 Learning performance 3.72

J-3 Expected results 3.78

M AN U

J-1 Information use

3.65

C-3 Professional knowledge

SC

3.76 G: Task-technology effectiveness 3.72 H: Performance outcome expectation 3.68 I: Aesthetics

3.64

3.84

K-3 Recommend to others

3.84

3.2 Quantification and assessment of APP usability

AC C

EP

TE D

For APP usability indicator dimensions, students gave higher scores for user satisfaction, completeness, consistency and functionality, course management, perceived usefulness, aesthetics, and user satisfaction. In other words, students felt that these 7 dimensions of the APP had the highest usability for learning purposes (Figure 4). From Table 6, it could be seen that many students expressed the willingness to recommend the APP to other people in the aspect of intention to use. For the aspect of course management, students also felt that uploading and downloading of data was quite convenient. In perceived usefulness, the APP provided significant learning support, and for user satisfaction, the highest score was given to the practicality of the APP. Figure 4 shows that aspects that students gave lower scores for were error prevention, interactivity feedback and help, convenience, and accessibility. The cancel function item of error prevention had the lowest score, while the communication item may need work for the aspect of interactivity feedback and help. Loading speed received lower review scores in the dimension of convenience, while personal preference had a low rating in the aspect of accessibility (Table 4). Students felt that these APP usability dimensions could be improved and further used as a guideline for developing APPs for teaching purposes.

3.99

4.00

3.94

3.93

3.94

3.90

3.82

3.80

3.82

3.76

3.92

3.90

3.86

3.76 3.72

3.69

3.70

3.84

3.93

3.60

d

A

an

db ac k

Co ns ist en cy

fe e

he lp cc es s ib an ili d ty fu nc A tio ss na es lit sm y en ts tra te M gy em or ab ili Co ty m pl et en es s Re A e du s th ci e ng tic s re Pe du rc nd ei a nc ve y d us ef ul U ne se ss rs at isf a ct In io te n nt io n to us e

en t

ce

em

ie n

an ag

er ac tiv

ity

rs em

ve n

bi isi V

Co n

Co u In t

Er ro r

pr ev en t

io

lit y

n

3.50

8 Figure 4 Statistical diagram on the average value of APP usability indicators

ACCEPTED MANUSCRIPT

Table 4. Average scores of APP usability aspects and items

A-1 Multiple operations

3.69

3.69 B-1 Reasonable arrangement 3.76

B visibility 3.82 C Convenience

C-1 Loading speed

3.76

3.61

D Course management

D-1 Links to information

3.93

3.85 C-2 Personalized environment 3.86 D-2 Personalized resources 3.91

B-3 Effective layout 3.85

D-3 Ease uploads

of

3.96 E-3 Keeping track of progress 3.82

M AN U

3.88

B-2 Clear functions

RI PT

A Error prevention

Indicator category A-2 A-3 Hints and Cancellation warnings of function 3.64 3.76

SC

Aspect

E Interactivity feedback and help

E-1 Improved communication

E-2 Regular feedback

3.62

3.73 I-1 Question help 3.67 J-1 Indicating links 3.89 K-1 Color and icons 3.91 L-1 Amending the errors 3.73

3.72 F-2 Instantaneous support 3.82 G-2 Clear functions 3.99 H-2 Assessment performance 3.85 I-2 Options program 3.92 J-2 Browsing interface 3.97 K-2 Interface comfort 3.93 L-2 Flexible interface 3.85

M Perceived usefulness 3.93

M-1 Grades improvement 3.84

M-2 Learning results 3.89

N User satisfaction 3.90

N-1 Information use 4.01

N-2 Exceeding of expectations 3.81

N-3 Expected results 3.88

O Intention to use 3.99

O-1 Continued use 3.91

O-2 Emotional improvement 4.00

O-3 Recommend to others 4.07

3.72 F Accessibility

F-1 Personal preferences

3.76

H Assessment strategies 3.82 I Memorability 3.84

3.94

EP

J Completeness

K Aesthetics

3.72 G-1 Similar formats 3.84

and

AC C

3.92 L Reducing redundancy 3.86

H-1 Degree of preparation

TE D

G Consistency functionality 3.94

D-4 Download and views 3.97

F-3 Usage approach

3.74 G-3 Convenient interface 3.99 H-3 Consistent objectives 3.86 I-3 Hint windows 3.91 J-3 Overall structure 3.95

L-3 Reading materials 4.00 M-3 Professional knowledge 3.99

I-4 Clear interface 3.85

M4 Learning support 4.01

Correlation testing was performed between 15 aspects of APP usability and IT devices owned by the students, online experience, Internet usage frequency, Internet usage time, and previous experience of APP usage (Table 5). 9

ACCEPTED MANUSCRIPT

RI PT

Results of this correlation analysis showed that there were no correlations between APP usability dimensions and IT devices owned, online experience, Internet usage frequency, and Internet usage time. Previous experience in APP usage, however, was correlated with a total of 3 APP usability dimensions, namely course management, user satisfaction, and intention to use. This result was consistent with that of the E-books usability where there were no correlations with IT equipment owned, online experience, Internet usage frequency, and Internet usage time. However, the number of items with significant correlations with APP usability was less than that for E-books usability. Table 5. Correlation between student status and dimensions of APP usability

*.

Percei Consistency Error Interactivity, Mem User Intenti Accessib Visibi Conven Course Assessment Reducing ved Completenes feedback, and preve orabil Aesthetics satisfac on to lity ience management ility redundancy useful strategies s and help functionality ntion tion use ity ness

.12

.10

.13

.17

.11

.04

.19

-.01

.1

.19

.20

.16

.07

-.17

.02

.05

-.02

.04

-.02

-.01

.01

.12

.07

.02

-.12

.05

.12

-.09

-.09

.00

-.03

-.14

-.04

.13

-.03

-.13

-.14

-.05

.03

-.11

.15

.21

.11

.24*

.21

.08

SC

.07

indicates correlation at 0.05 level of significance

.09

M AN U

Pearson' s correlati on coefficie nt Informat ion Equipme nt Online experien ce Internet usage frequenc y Internet usage time Previous APP usage

.09

-.14

.11

.07

.05

-.11

-.03

.01

-.02

-.14

-.09

-.19

-.09

-.01

-.07

-.00

-.04

-.09

-.01

.07

.16

.21

.12

.11

.12

.25*

.28*

TE D

3.3 Relationship between learning style and learning performance for the course using E-books and APP

AC C

EP

To further determine if there were correlations between learning performance and student learning styles in the mechanical drawing course using cloud-based E-books and APP learning resources, student learning styles were coded using serial numbers and taught by the same instructor. Scores from the final examination were used as learning performance and compared. The correlation coefficient between the final examination score and learning style was -0.285, indicating that there were significant correlations between the two. Learning styles were coded in the following: (1) theorist, (2) pragmatist, (3) activist, (4) reflector, and (5) mixed. Results showed that mechanical drawing course performance of students learning using cloud-based resources was intimately correlated to their learning style. Theorists scored the highest, followed by pragmatists, activists, reflectors, and mixed learners. This analysis indicates that students with theoretical learning styles were more compatible with the use of E-books in learning. Further analysis was carried out to identify patterns in the learning styles of high scorers and low scorers in the course. Hence, another correlation study was performed between students ranking by score as well as their corresponding learning styles. Results indicate that there was a negative correlation in student learning style in the mechanical drawing course using E-books as learning materials and student score ranking. In other words, students with higher scores tend to be theoretical learners, while students with lower scores tend to be mixed learners. This showed that students without a single learning style tend to have lower learning performance when using E-books to conduct their studies. 3.4 Analysis of E-books and APP usability and properties of the learning styles A total of 222 students underwent the qualitative analysis. Statistical analysis of the open-ended Learning Style Questionnaire (LSQ) results revealed that there were 36 theoretical learners, 42 practical learners, 12 active learners, 66 reflective learners, and 66 mixed learners. Results from previous quantitative studies showed that theorists had the best 10

ACCEPTED MANUSCRIPT

RI PT

learning styles in terms of learning performance for the mechanical drawing course using cloud-based learning resources of E-books and APP. However, a larger proportion of the students seemed to be reflectors and mixed learners. The focus of subsequent research would be to improve learning performance for these types of learners. Reflectors and mixed learners have different characteristics. For example, reflectors tend to be more imaginative and capable of brainstorming and creativity. These learning style features would not be very compatible with learning using E-books and APP. However, results of the open-ended questionnaire revealed that reflectors accepted learning approaches using E-books and APP (Table 6). Further investigations, however, would be needed in order to study the influence between instruction method and their degree of acceptance. Table 6. Comparison between learning styles and qualitative key term analysis

Theorists

User

Reading

experience

method

30(83%)

(36 individuals)

Usage focus

Usable contents

Degree of usability

E-books

and

APP

usage

E-books（75%

Problem solving

Electronics

）

（75%）

electrical

and

Convenience and fast

Professional

（92%）

techniques（83%）

SC

Learning style

engineering （ 67% ） 24(57%)

(42 individuals) Activist

Printed books

Data

reference

Theoretical

（57%）

（64%）

research（71%）

）

Best of the two（64%

M AN U

Pragmatists

Learning by doing（ 86%）

Printed books

Operational

Cannot be verified

Written

(12 individuals)

（75%）

difficulty （ 75%

（75%）

100%）

）

Reflectors

Having two at

Self-learning and

Literature and art

Written records（45%

Peer discussion（77%

a time（68%） revision（68%）

（50%）

）

）

Having two at

Cannot be verified

Together with the

Familiarity with the

（59%）

books（59%）

contents（73%）

6(50%)

） 30(45%)

(66 individuals) Mixed

36(55%)

(66 individuals)

Written

a time（73%） reference （ 73%

records （

Case analysis（100%

TE D

）

AC C

EP

Statistical results from the open-ended questionnaire revealed that previous experience in using E-books and APP was closely correlated with student learning performance from the quantitative study. In this section, 83% of the theorists had previous usage experience which would be the highest when compared to pragmatists, activists, reflectors, and mixed learners, of which 57%, 50%, 45%, and 55% had similar experiences respectively (Table 8). This finding showed that most theorists had usage experiences with E-books and APP, which meant that these learners would be able to effectively transfer their experiences and adapt to course learning. Pragmatists, given their practical learning styles, would have less experience in using E-books and APP compared to theorists. However, over half of the pragmatists as well as the three other types of learners have previous experiences in using E-books and APP. Reflectors, which made up a substantial proportion of the students, had relatively less experience in E-books and APP usage at only 45%. This may be the reason why they performed less well in learning performance of the course. Future efforts would focus on improving their interests in using E-books and APP and enhancing the usability of E-books and APP in actual classroom settings. 3.5 Usability and path analysis of E-books and APP in the mechanical drawing course Results of TAM analysis for E-books usability in the mechanical drawing course showed that hypothetical relationships in the overall sample model were indeed significant. As shown from Figure 5, usage attitude was directly influenced by perceived usefulness and perceived ease of use with path coefficient of being 0.27 and 0.56 respectively. Perceived usefulness itself would be indirectly influenced by perceived ease of use with an indirect path coefficient of 0.21. Behavioral intention was mainly under the direct influences of usage attitude and perceived behavioral controls with path coefficients of 0.51 and 0.44 respectively. Usage behavior was mainly under the direct influences of usage intention and perceived behavioral 11

ACCEPTED MANUSCRIPT

RI PT

controls with a path coefficient of 0.66 and 0.14 respectively. Learning results was under the direct influence of usage behavior with a path coefficient of 0.09. Finally, statistical analysis showed that learning style did not have significant influence over behavioral intention, usage behavior, and learning results. According to the R-square analysis, the R-square value of perceived usefulness was 0.62, meaning that perceived ease of use provided 61.7% interpretability to perceived usefulness; perceived usefulness and perceived ease of use provided 61.4% interpretability to usage attitude; usage attitude and perceived behavioral controls provided 75.5% interpretability to behavioral intention; while behavioral intention and perceived behavioral controls provided slightly less interpretability to usage behavior at 58.1%.

Learning styles Perceived usefulness

0.56

Usage attitude

Perceived ease of use

0.51

Behavioral 0.66 intention

0.09

Learning results

0.14

M AN U

0.44

Usage behavior

SC

0.79

0.27

Perceived behavioral controls

Figure 5. TAM path analysis for E-books usability in the mechanical drawing course.

AC C

EP

TE D

TAM analysis results for APP usability in the course of mechanical drawing showed that with the exception of causal relationship with learning styles, significant causal relationships were present for each assumed relationship in the overall sample model. As shown from Figure 6, the perceived ease of use directly influences perceived usefulness with a path coefficient of 0.82; usage attitude was directly influenced by perceived usefulness and perceived ease of use with path coefficient of being 0.44 and 0.42 respectively. Perceived usefulness itself would be indirectly influenced by perceived ease of use with an indirect path coefficient of 0.360. Behavioral intention was mainly under the direct influences of usage attitude and perceived behavioral controls with path coefficients of 0.63 and 0.29 respectively. Usage behavior was mainly under the direct influences of usage intention and perceived behavioral controls with a path coefficient of 0.33 and 0.46 respectively. Learning results was under the direct influence of usage behavior with a path coefficient of 0.14. Finally, statistical analysis showed that learning style did not exert significant influence over usage intention, usage behavior, and learning results. Although there were no direct correlations between these factors, results of correlation tests still indicated significance which meant that indirect relationships were probably present. According to the R-square analysis, the R-square value of perceived usefulness was 0.666, meaning that perceived ease of use provided 66.6% interpretability to perceived usefulness; perceived usefulness and perceived ease of use provided 66% interpretability to usage attitude; usage attitude and perceived behavioral controls provided 71.4% interpretability to behavioral intention; while behavioral intention and perceived behavioral controls provided slightly less interpretability to usage behavior at 51.8%.

12

ACCEPTED MANUSCRIPT

Learning styles

0.44 Usage attitude

0.82 Perceived ease of use

0.63

Behavioral 0.33 intention 0.29

0.42

0.14

Usage behavior

0.46

Perceived behavioral

Learning results

RI PT

Perceived usefulness

SC

Figure 6. TAM path analysis for APP usability in the mechanical drawing course.

Table 7. Cross comparison between learning styles and test results high scoring groups 5(42%) 9(64%) 4(100%) 9(41%) 10(45%) 37

group 1(8%) 2(14%) 0 5(23%) 11(50%) 19

High scoring

M AN U

Low scoring

Theorist Pragmatist Activist Reflector Mixed Total

group 6(50%) 3(22%) 0 8(36%) 1(5%) 18

Total 12 14 4 22 22 74

TE D

Lea rni ng styl e

Between low and

AC C

EP

Path analysis indicated that learning styles had no significant influences on behavioral intention, usage behaviors, and learning results. However, correlation analysis revealed that there were correlations between learning styles and learning results. We thus performed qualitative analysis on the five types of student learning styles and statistically analyzed the number and percentage of high scorers, low scorers, and those in between in each learning style. The proportion of high scorers among the theorists was the highest at 50% (Table 7), with 42% in between, and only 8% were low scorers. In other learning styles, reflectors, which had the highest proportion of students, also had a large number of high scorers at 36%. Low scorers and those in between amongst the reflectors were 23% and 41% respectively. The learning style with the third highest proportion of high scorers were the pragmatists, with 22% being high scorers, 64% being low scorers, and 14% being in between. Activists and mixed learners had the lowest number of high scorers. All 4 of the activists scored between the high and low scores, while only 5% of mixed learners were high scorers, 50% were low scorers, and 45% were in between (Table 7). Hence, we can see that different students with the same learning style would score differently as well, and could be used to explain why path analysis failed to reveal any significant effects. At the same time, it was evident that a large proportion of theorists were high scorers in learning the mechanical drawing course using E-books and APP. Reflectors had the next highest proportion of high scorers, but a larger number of students were considered reflectors as well. Hence, when learning the mechanical drawing course using E-books and APP, it would be critical to identify the means to improve scoring effectiveness amongst low scorers with the reflective or mixed learning styles.

4. Conclusion： ： This study considered behavioral usage intention as a factor for intermediary efficiency in assessing student review on E-books and APP usability after using these materials in the learning of a mechanical drawing course. After the 13

ACCEPTED MANUSCRIPT

SC

RI PT

study, it was found that IT equipment owned by the students as well as their Internet experience, frequency of Internet use, and daily time spent using the Internet did not correlate with E-books and APP usability. However, students' previous experience in E-books did influence E-books and APP usability. In the usability assessment for E-books and APP, it was found that the aspects of aesthetics and intention to use were given higher scores. Aspects that students scored lowly for also provided a reference for future usage of E-books and APP for educational purposes. Correlation tests found that learning performance was not correlated with usability, but strongly correlated with student learning styles. Path analysis found that the E-books and APP usability and assumed causal relationships in the TAM model exist and were significant. With the exception of low path coefficient for learning styles, most of the causal relationships were significantly related to the diversity of learning styles. This study found that the learning performance in the mechanical drawing course using E-books and APP was significantly correlated with the students' learning styles. Theorists had the best performance, followed by pragmatists, activists, reflectors, and mixed learners. Theorists were hence more compatible with learning using E-books and APP. High and low scoring population analysis found that most of the high scorers tend to be theorists. The qualitative investigation on usability and learning styles interview showed that most students tend to be reflectors and mixed learners. Subsequent research in courses using E-books and APP would need to focus on improving learning performance for reflectors and mixed learners.

M AN U

By reviewing the results of this study, it would be recommended to design more usable E-books and APP learning materials to cater to students with different learning styles. Below lists some of these suggestions: 1. For theorists: Provide theorists with well-structured and tightly organized E-books and APP and provide effective control over the depths and scope of information provided in the hyperlinks. Learning materials should emphasize theoretical or logical concepts, formulas, and principles. Logical relationships between different knowledge may be illustrated using concept diagrams, mind maps and other tools. E-books should try to include information analysis and systems for acquiring learning support.

TE D

2. For pragmatists: When providing pragmatists with learning contents via E-books and APP, the emphasis should be on actual links between learning objectives and actual problems. Practical activities relevant to the learning contents should be arranged. For example, virtual labs could be provided so that pragmatists could learn from these demonstrations. When using E-books and APP, students should be encouraged to reflect upon experiments and practical sessions in order to improve their ability to theorize and reflect upon issues and strengthen learning performance. 3. For activists: E-books and APP contents should be diverse and challenging so that activists could attempt these activities and acquire experience from them. Information exchange tools (network learning groups, course forums, chat rooms and other tools) should be provided to allow course and learning discussion.

AC C

EP

4. For reflectors: E-books and APP contents should provide adequate materials and background information. Learning materials must make full use of diagrams, figures, animations, and other visual materials in addition to textual data in order to encourage reflectors to use their observation skills. These learners should also be provided with search tools to satisfy their desire to search for and record data. Reference

Abd Mutalib Embong, Azelin Mohamed Noor, Hezlina M Hashim, Razol Mahari Ali and Zullina H Shaari (2012) E-books as textbooks in the classroom. Conference proceedings of the International Conference of Education Research, Cyprus. Procedia-Social and Behavioral Sciences. Barnard, L., Paton, V. O., & Lan, W. Y. (2008). Online self-regulatory learning behaviors as a mediator in the relationship between online course perceptions with achievement. International Review of Research in Open and Distance Learning, 9(2), 1−11 Davis, F.D. (1989). Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology. MIS Quarterly, 13(3), 319-340. Dinkelman, A. & Stacy-Bates, K. (2007). Accessing e-Books through academic library web Sites. College and 14

ACCEPTED MANUSCRIPT

M AN U

SC

RI PT

Research Libraries, 68(1), 45-57. Frohberg,D.,Göth,C.&Schwabe,G.(2009). Mobile Learning Projects:A Critical Analysis of the State of the Art[J].Journal of Computer Assisted Learning,(4):307-331. Honey, P. and Mumford, A. (1992) Manual of Learning Styles. Maidenhead: Honey Publications. Hwang, G. J., Yang, T. C., Tsai, C. C., & Yang, Stephen J. H. (2009). A context-aware ubiquitous learning environment for conducting complex science experiments. Computers & Education, 53(2), 402-413. Hwang, G. J., & Wu, P. H. (2014). Applications, impacts and trends of mobile learning - a review of 2008-2012 publications in selected journals. International Journal of Mobile Learning and Organisation, 8(2), 83-95. Jou, M. & Wang, J.Y. (2013). Observations of achievement and motivation in using cloud computing driven CAD: Comparison of college students with high school and vocational high school backgrounds. Computers in Human Behavior 29(2): 364-369 Karahanna, E., Straub, D. W., and Chervany, N. L. (1999). Information Technology Adoption Across Time: A Cross-Sectional Comparison of Pre-Adoption and Post-Adoption Beliefs,” MIS Quarterly, 23(2): 183-213. Kolb, D. A. (1976). The Learning Style Inventory: Technical Manual, Boston, Ma.: McBer. Kolb D. A. (1984). Experiential Learning experience as a source of learning and development, New Jersey: Prentice Hall. Lai, J.Y. & Ulhas, K.R. (2012). Understanding Acceptance of Dedicated E-textbook Applications for Learning: Involving Taiwanese University Students. The Electronic Library, 30, 3, 321-338. Legris, P., Inghamb, J., and Collerette, P. “Why do People Use Information Technology? A Critical Review of the Technology Acceptance Model,” Information and Management (40:3), 2003, pp. 191-204. Lin, T. C. & Huang, C. C. (2010). Withholding effort in knowledge contribution: The role of social exchange and social cognitive on project teams. Information & Management, 47(3): 188-196.

TE D

Malathi L. and Rohani A. T (2010) Utilization of e-book among university mathematics students. Proceeding in the International Conference on Mathematics Education Research (ICMER).Procedia Social and Behavioral Sciences 8 pp 580-587.

AC C

EP

Oztekin, A., Kong, Z.J. & Uysal, O. (2010). UseLearn: A novel checklist and usability evaluation method for eLearning systems by criticality metric analysis, International Journal of Industrial Ergonomics. 40(4): 455-469. Srite, M. & Karahanna, E. (2006). The role of espoused national cultural values in technology acceptance. MIS Quarterly, 30(3), 679-704. Taylor, S., & Todd, P. A. (1995a).Assessing IT Usage: The Role of Prior Experience,” MIS Quarterly, 19(4): 561-570. Taylor, S., & Todd, P. A. (1995b). Understanding Information Technology Usage: A Test of Competing Models, Information Systems Research, 6(2): 144-176. Taylor, S., & Todd, P. A. (1995c). Decomposition and Crossover Effects in the Theory of Planned Behaviour: A Study of Consumer Adoption Intentions,” International Journal of Research in Marketing, 12(2): 137-155. Tsai, C. C., & Hwang, G. J. (2013). Issues and challenges of educational technology research in Asia. The Asia Pacific Education Researcher, 22(2), 215-216. Tsai, P. S., Tsai, C. C., & Hwang, G. J. (2012). Developing a survey for assessing preferences in constructivist context-aware ubiquitous learning environments. Journal of Computer-Assisted Learning, 28(3), 250-264. Venkatesh, V. (2000). Determinants of Perceived Ease of Use: Integrating Control, Intrinsic Motivation, and Emotion into the Technology Acceptance Model, Information Systems Research, 11(4): 342-365. Venkatesh, V. & Bala, H. (2008). Technology acceptance model and a research agenda on interventions. Decision Sciences, 39( 2): 273-315.

15

ACCEPTED MANUSCRIPT

AC C

EP

TE D

M AN U

SC

RI PT

> Usability of E-books and APP. > A direction in the development of usable learning materials (E-books and APP). > Helpful and compatible with actual lessons in class.

ACCEPTED MANUSCRIPT

Acknowledgement The author hereby expresses sincere gratitude for the support provided by the Ministry of Science and technology under the Grant No. MOST

AC C

EP

TE D

M AN U

SC

helped to broaden the scope and depth of this paper.

RI PT

100-2511-S-003-054-MY3 that allowed the establishment of fundamental research of this paper as well as overseas short-term research (MOST 104-2918-I-003-010) that