Unpacking teachers' acceptance of technology: Tests of measurement invariance and latent mean differences

Computers & Education 75 (2014) 127–135

Contents lists available at ScienceDirect

Computers & Education journal homepage: www.elsevier.com/locate/compedu

Unpacking teachers’ acceptance of technology: Tests of measurement invariance and latent mean differences Timothy Teo* Faculty of Education, University of Macau, Macau SAR, China

a r t i c l e i n f o

a b s t r a c t

Article history: Received 14 September 2013 Received in revised form 29 January 2014 Accepted 31 January 2014 Available online 27 February 2014

This study examines the factors that explain teachers’ technology acceptance. A sample of 673 primary and secondary school teachers gave their responses to a 16-item technology acceptance measure for preservice teachers (TAMPST). Results of this study showed teachers have a generally positive level of technology acceptance and that the TAMPST is a valid tool to be applied to teachers although it was originally developed to test pre-service teachers. Tests for measurement invariance and latent mean differences on the five factors in the TAMPST provided support for full and partial configural, metric, and partial scalar invariance by gender, length of service in teaching, and teaching level. The tests of latent mean differences found significant differences by gender for perceived ease of use, with male teachers rating higher than their female counterparts. Between teachers with shorter and longer years of teaching service, statistical significance was found in the mean differences for perceived ease of use and attitude towards technology use. No significant mean differences in each of the five factors were found between the primary and secondary teachers. Ó 2014 Elsevier Ltd. All rights reserved.

Keywords: Human–computer interface Pedagogical issues Country-specific developments

1. Introduction Since the 1970s, technology acceptance has been a key area of interest among researchers in business and information systems disciplines. Among the research themes has been a focus on identifying the conditions or factors that drive technology integration among business users (Legris, Ingham, & Collerette, 2003). Arising from these efforts, several theories and models have been proposed as frameworks to enable researchers to identify significant variables that both explain and predict technology acceptance at individual and organizational levels. In their review of 99 studies on information technology acceptance/adoption, Jeyaraj, Rottman, and Lacity (2006) identified at least 10 theories that had been proposed between 1983 and 2003. In addition, these authors reported 135 independent and 8 dependent variables in their review. In recent years, researchers have adapted some of these theories to investigate their capability in understanding technology acceptance of users in education (e.g., Hammond, 2011; Teo, 2009; Teo, Koh, & Lee, 2011). Of these theories, the technology acceptance model (TAM) (Davis, 1989), the theory of planned behaviour (TPB) (Ajzen, 1991), and the unified theory of acceptance and use of technology (UTAUT) (Venkatesh Morris, Davis, & Davis, 2003) have received much attention and been widely applied and tested in educational contexts with students and teachers as user groups. In any school, teachers play a key role in the effective integration of technology for teaching and learning. Teachers decide on the type, frequency, and quantity of technology tools they use in their curriculum design and lesson delivery. Although, it may appear that technology integration is part of their job requirements, teachers exercise complete volition over their intention and actual usage of technology within the professional space. With rapid advancements in technologies, there is greater pressure on teachers to engage various types of tools in conceptualising, preparing, and delivering their lessons. In addition, with covert expectations from their increasingly technologically savvy students, teachers may feel that engaging technology in the instructional process as an option that they cannot exercise. Despite the significant role of technology in effective instruction, there is evidence to suggest that teachers have lacklustre responses towards using technology for teaching and learning in many parts of the world (Zhao & Cziko, 2001). For example, Becker (2001) found that

* Av. Padre Tomás Pereira, Taipa, Macau SAR, China. E-mail address: [email protected]. http://dx.doi.org/10.1016/j.compedu.2014.01.014 0360-1315/Ó 2014 Elsevier Ltd. All rights reserved.

128

T. Teo / Computers & Education 75 (2014) 127–135

teachers in the United States were not regular users of the computer for teaching and, when they did, the computers were used for low-level purposes such as games and drills in the classroom. In the United Kingdom, BECTA (2004) found that teachers cited a lack of technical support, their own lack of confidence, and a lack of belief in the advantages of using technology for instruction as some of barriers they faced in achieving technology integration in the classroom. In Australia, Birch and Burnett (2009) cited a lack of clear institutional direction concerning course design and delivery time as major issues teachers have to cope with in the development of e-learning environments. The extent to which technology has been effectively employed for teaching and learning depends largely on the level of teachers’ acceptance. An individual’s acceptance of technology is referred to as the level at which he or she is willing to use the technology for which it was designed (Teo, 2010a). When teachers do not use technology in the way it was designed to serve, the affordances of technology cannot be maximised for effective teaching and learning to take place. The literature suggests that many acceptance studies focused on the identification of factors that affect users’ technology acceptance. These included personal factors such as: attitudes towards using computers (Teo, 2009); perceived enjoyment thereof (Teo & Noyes, 2011); and emotional attachment (Read, Robertson, & McQuilken, 2011), technical factors such as technological complexity (Thong, Hong, & Tam, 2004); and environmental factors such as facilitating conditions (Venkatesh, Brown, Maruping, & Bala, 2008). 1.1. Theoretical background From the literature, the theory of reasoned action (TRA) (Fishbein & Ajzen, 1975), the theory of planned behaviour (TPB) (Ajzen, 1991), the technology acceptance model (TAM) (Davis, 1989), and the unified theory of acceptance and use of technology (UTAUT) (Venkatesh, Morris, Davis, & Davis, 2003) have been widely reported to be effective in predicting acceptance among users in educational settings. In the TRA, behaviour is posited to be determined by an individual’s intention to perform the behaviour and intention is a function of that person’s attitude toward the behaviour and his or her ‘subjective norm’ (Ajzen & Fishbein, 1980). While attitude toward behaviour refers to the amount of pleasure a person derives from performing a behaviour, subjective norm is defined as the extent to which an individual is motivated to comply with the views others hold about the behaviour. The TPB is an extension of the TRA, which includes perceived behavioural control, defined as what factors influence an individual’s decision through that person’s perception of how easy or difficult it would be to perform a behaviour (Ajzen, 1991). The TAM was proposed by Davis (1989) with an expressed desire to explain a user’s level of technology acceptance. In the TAM, actual technology use is determined by one’s behavioural intention to use a particular technology. Behavioural intention is affected by attitude toward usage, and by the direct and indirect influences of perceived usefulness and perceived ease of use. Both perceived usefulness and perceived ease of use jointly affect attitude toward usage, whereas perceived ease of use has a direct impact on perceived usefulness (Davis, 1989). Having reviewed the above and five other models of technology adoption, Venkatesh et al. (2003) proposed the UTAUT to explain users’ intentions to use technology and subsequent usage behaviour. This theory relies on four key constructs (performance expectancy, effort expectancy, social influence, and facilitating conditions) to predict both usage intention and behaviour. Considering the evidence drawn from empirical studies that employed the above theories and models, Teo (2010a) developed a model depicting technology acceptance as a multidimensional construct comprising five factors: perceived usefulness; perceived ease of use; attitude towards technology use; subjective norm; and facilitating conditions (Fig. 1) 1.2. Measurement invariance When examining user acceptance, researchers were also interested in making comparisons across groups using the data obtained from the same instrument or measure. These groups may be across gender, age, types of technology, or educational levels. Methodologically, such comparisons are predicated on the equivalence of the responses although evidence to support this assumption is rarely reported in research on technology acceptance (e.g., Teo, Ursavas, & Bahcekapili, 2012; Wong & Teo, 2009; Wong, Teo, & Russo, 2013). In not doing so, researchers have implicitly assumed that their data were sufficiently invariant to allow comparison across groups (e.g., males and females) without first establishing measurement invariance. Borsboom (2006) defined measurement invariance as the same attribute relating to the same set of observations in the same way in each group. This means that the mathematical function that relates latent variables to the observations must be the same in each of the groups involved for meaningful comparison between groups or persons to be made and failure to do so may result in biased estimates leading to erroneous interpretations of the findings based on scores that were obtained due to chance or stained by error. To avoid this situation, measurement invariance is often tested before between-group comparisons are made. 1.3. Steps in establishing invariance After an extensive review of the literature, Vandenberg and Lance (2000) proposed several steps to establishing measurement invariance in increasingly restrictive stages. In addition to configural invariance, they suggested examining whether (1) the rating scales are used similarly in different groups (metric invariance) and (2) the quantifiable meanings of the scale are the same across groups (scalar invariance). Prior to further tests of invariance and substantive analysis being performed, metric invariance should be established (Steenkamp & Baumgartner, 1998) and, if we wish to compare the mean differences of constructs across groups, scalar invariance is required (Meredith, 1993). Although it is possible to test for other forms of invariance, such as equality of error variances and covariances across groups, these tests are considered to be excessively rigorous (Byrne, 2010). Before testing for measurement invariance across groups, the one-sample models are tested separately (configural invariance). This provides an overview of how consistent the model results are. This will be followed by testing for metric and scalar invariance. Configural invariance acts as the baseline and is satisfied if the pattern of fixed and non-fixed parameters in the model is invariant across groups. Because the configural invariance model provides the basis for comparison with all subsequent models in the invariance hierarchy, if the data do not support configural invariance, they will not support the more restrictive metric and scalar models either (Bollen, 1989). The metric invariance is more restrictive than the baseline model and is conducted by constraining the factor pattern coefficients (loadings), which reflect the relationship between latent scores and observed scores, to be equal across groups. When metric invariance is supported, it

T. Teo / Computers & Education 75 (2014) 127–135

129

Fig. 1. Confirmatory factor model.

means that different groups had responded to the items in the same way so that the differences in the item scores can be meaningfully compared across groups: that is, the observed item differences are indicative of group differences in the underlying latent construct (Steenkamp & Baumgartner, 1998). Scalar invariance indicates that the mean of a construct has the same meaning between the different groups being compared. Practically, this means that a score of 4.0 in one group would be taken to be the same in any comparison group. Scalar invariance is tested by constraining the intercepts of items to be the same across groups and a failure to satisfy the scalar invariance condition indicates that the data may be suffering from measurement bias. 1.4. Aim of this study This study aims to assess teachers’ technology acceptance and to test the mean differences of their acceptance by gender, length of service, and level of teaching. Below are the research questions: 1. 2. 3. 4.

What is the overall level of technology acceptance among teachers? To what extent is the hypothesized model technology acceptance valid in explaining teachers’ acceptance of technology? Is the hypothesized model technology acceptance invariant by gender, length of service, and teaching level? Are there significant latent mean differences in each dimension of teachers’ technology acceptance by gender, length of service, and teaching level?

2. Method 2.1. Participants The participants were 673 South-east Asian school teachers who responded to an invitation issued by this author through their school principals. A total of 60 schools (30 primary and 30 secondary) were invited and 36 schools (17 primary and 19 secondary) agreed to participate, making a participation rate of 60%. Among the participants, 516 (76.7%) were females and the overall mean age was 35.69 (SD ¼ 9.02) years. The mean year of teaching experience among the participants was 9.67 (SD ¼ 8.31). Nearly all the participants own a computer at home (97.3%) and the mean year of computer usage was 14.48 (SD ¼ 5.10).

130

T. Teo / Computers & Education 75 (2014) 127–135

2.2. Procedure Participants were given an URL to access the online survey questionnaire used in this study. Implicit consent was given by the participants who completed the questionnaire. Statements on the purpose of this study and participant’s rights not to participate in the study and, the option for them to withdraw from the study during or after they had completed the questionnaire were presented in the front of the survey items. Participation in this study was voluntary and no reward either in monies or in kind was given. On average, each participant took 20 min to complete the questionnaire. 2.3. Measures The instrument used in this study was the technology acceptance measure for pre-service teachers (TAMPST) (Teo (2010a). The TAMPST measures five factors that explain technology acceptance of pre-service teachers: perceived usefulness (PU); perceived ease of use (PEU); attitude towards technology use (ATTU); subjective norm (SN); and facilitating conditions (FC). Ranging from two to four items per factor, the TAMPST was originally developed and validated with three separate samples of pre-service teachers in Singapore. The reliability index for each factor was computed using Cronbach’s alpha (.87 for PU, .77 for PEU, .81; .82 for ATTU; for SN, and .79 for FC) and these were considered high for scale development purposes (Henson, 2001). Since its development, the TAMPST has been validated and used in other studies with pre-service teachers and health professionals as participants. Teo (2010b) administered the TAMPST to 193 pre-service teachers in Malaysia and obtained alpha values ranging from .79 to .86 for the five factors indicating high reliability while Kuilema (2012) found the TAMPST to be a similarly reliable instrument on a sample of 282 health professionals in the United States with an alpha of .89. Although the items in Teo (2010a) were measured on five-point scale, the items in this study were measured on a 7-point Likert scale, with 1 ¼ strongly disagree to 7 ¼ strongly agree. There was no impact on the internal consistency of the TAMPST in this study despite the change in the scale category from 5 to 7 points. 2.4. Data analysis Data will be screened for univariate normality by examining the skewness and kurtosis values, together with the descriptive statistics (mean and standard deviation). Following a confirmatory factor analysis would be conducted to assess the factorial validity and item reliability of measure used in this study. Proceeding on, tests of measurement invariance (configural, metric, and scalar) across three groups (gender, length of service, teaching level) will be performed on each item in the scale. Finally, tests of latent mean differences will be conducted for all factors whose items have met the criteria for full or partial scalar invariance. 3. Results 3.1. Descriptive statistics The means, standard deviations, skewness, and kurtosis values for each of the 16 items in the TAMPST were computed. The mean values were above the mid-point of 4.0, ranging from 4.56 to 6.01 and standard deviations ranged from 1.00 to 1.42, indicating a fairly positive response to the items by the participants and spread of scores around the mean. The values of the skewness and kurtosis for the items were between 1.78 and .35, and .43 and 5.08, respectively. These values were within the recommended cutoffs of j3.0j and j8.0j for skewness and kurtosis, respectively, and univariate normality in the data was assumed (Kline, 2010). 3.2. Confirmatory factor analysis A confirmatory factor analysis (CFA) was conducted with AMOS 21.0 using maximum likelihood estimation (MLE). However, MLE is known to produce distorted results when the normality assumption is violated (Curran, West, & Finch, 1996); multivariate normality was assessed using the Mardia measure of multivariate kurtosis (Mardia, 1970). The Mardia’s coefficient for the data in this study was 180.81, which is lower than the value of 288 computed based on the formula p(pþ2) where p equals the number of observed variables in the model (Raykov & Marcoulides, 2008). From this, multivariate normality of the data in this study was assumed. The overall model fit was assessed using the c2 test and, because it is highly sensitive to sample size, the ratio of c2 to its degree of freedom was also computed (c2/df), with a value of not more than 3.0 being indicative of an acceptable fit between the hypothetical model and the sample data (Carmines & McIver, 1981). In addition, other fit indices were also considered: the Tucker–Lewis index (TLI); the comparative fit index (CFI); root mean square error of approximation (RMSEA); and standardized root mean square residual (SRMR). Hu and Bentler (1999) proposed that TLI and CFI statistics greater than .95 represent a good model fit and those for RMSEA and SRMR, values with less than .06 and .08, respectively, are good. Figure 2 shows the measurement model that has been tested to reveal that it has a good fit to the sample data (c2 ¼ 255.525; c2/df ¼ 2.748; TLI ¼ .980; CFI ¼ .985; RMSEA ¼ .051; SRMR ¼ .028). The reliability and validity of the items purported to measure each variable were measured using the composite reliability (CR), average variance extracted (AVE). Cronbach’s alpha was not reported in this study because it was prone to violate key assumptions when used with a multidimensional and multi-item scale such as the TAMPST (Teo & Fan, 2013). In assessing the validity of the items, the direction, magnitude, and statistical significant of each parameter (t-value) were examined (Schumacker & Lomax, 2010). An item explains its latent variable well if its standardized estimate was greater than .50 (Hair, Black, Babin, & Anderson, 2010). Using a more conservative indicator of validity, AVE for each construct, which measures the amount of variance captured by the construct in relation to the amount of variance attributable to measurement error, was computed. Both the CR and AVE are judged to be adequate when they equal or exceed .50 (i.e., when the amount of variance captured by the construct exceeds the variance due to measurement error) (Fornell & Larcker, 1981). As shown in Table 1, the t-values, standardized estimates, CR, and AVE of all items and variables meet the recommended guidelines.

T. Teo / Computers & Education 75 (2014) 127–135

131

Table 1 Results of the confirmatory factor analysis.

Perceived Usefulness (PU)

Perceived Ease of Use (PEU)

Attitude Towards Technology Use (ATTU)

Subjective Norm (SN) Facilitating Conditions (FC)

Item

UE

t-value*

SE

CRc

AVEd

PU1 PU2 PU3 PU4 PEU1 PEU2 PEU3 ATTU1 ATTU2 ATTU3 ATTU4 SN1 SN2 FC1 FC2 FC3

1.189 1.217 1.223 1.000 .984 1.008 1.000 .953 1.051 1.142 1.000 1.000 1.000 1.151 1.153 1.000

33.560 35.254 37.415 –a 46.439 41.808 –a 23.971 33.501 43.067 –a –b –a 21.758 21.719 –a

.915 .937 .964 .852 .935 .905 .945 .730 .866 .967 .902 .902 .928 .901 .896 .717

.96

.84

.95

.86

.93

.76

.91

.83

.88

.71

Notes: *p < .01. UE: Unstandardized Estimate; SE: Standardised Estimate. a This value was fixed at 1.00 for model identification purposes. b This item was constrained to equal as a measure to deal with the improper solution (negative residual variance) as a result of having two items for this factor. P P P c l)2 / ( l)2 þ ( (1 – l2)) CR ¼ ( P . P P d ( l2) / ( l2) þ ( (1 – l2)) AVE ¼ .

3.3. Tests of measurement invariance Various multi-group invariance analyses were performed for this study: gender (male versus female); length of service (short versus long); teaching level (primary versus secondary). Using the median year (7.0) of service as a guide, teachers below the median were classified as ‘short’ and those above, ‘long’. Estimation for each analysis was performed with the maximum likelihood procedure using variance–covariance matrices as input data. The tests of measurement invariance proceeded in a hierarchical order by testing for configural invariance, followed by metric invariance, and scalar invariance using several model fit indices. First, a baseline chi-square value is derived by computing model fit for the configural invariance model. Then constraints were added to various model parameters to be equal across groups and the model is fitted, yielding a chi-square value for the constrained model. This is followed by a chi-square difference test to see if the difference between the constrained-equal and unconstrained models is significant. If it is not significant, it is concluded that the constrained-equal model is the same as the unconstrained multi-group model, leading to the conclusion that the model does apply across groups and does display measurement invariance. However, the use of Dc2 has been criticized because of its sensitivity to sample size (Chen, 2007). On the basis of extensive simulations, Cheung and Rensvold (2002) recommended using a DCFI value higher than .01 to be indicative of a significant drop in fit. Supplemental to the Dc2, Chen recommended using the DCFI, DRMSEA, and DSRMR to assess for evidence of invariance. The criteria for invariance will be DCFI  .01, DRMSEA  .015, and DSRMR  .03 for tests of metric invariance and DCFI  .01, DRMSEA  .015, and DSRMR  .01 for test of scalar invariance. On establishing scalar invariance, the test of latent means differences will be performed. 3.4. Results of the tests of measurement invariance Result of the test of configural invariance revealed an acceptable fit for all three comparison groups: gender (c2 ¼ 499.689; c2/df ¼ 2.658; TLI ¼ .963; CFI ¼ .971; RMSEA ¼ .050; SRMR ¼ .047); length of teaching service (c2 ¼ 470.102; c2/df ¼ 2.501; TLI ¼ .966; CFI ¼ .973; RMSEA ¼ .047; SRMR ¼ .033); teaching level (c2 ¼ 473.895; c2/df ¼ 2.521; TLI ¼ .966; CFI ¼ .973; RMSEA ¼ .048; SRMR ¼ .029). This provided support that the pattern of fixed and non-fixed parameters in the TAMPST is identical by gender, length of teaching service, and teaching level groups. Tables 2 and 3 shows the results of the tests of measurement invariance. 3.4.1. Gender When the factor pattern coefficients were constrained to be equal, there was an increase in the c2 value of 17.584 with a gain in 11 degrees of freedom (df). As this was not statistically significant at p < .05, metric invariance was supported (Table 3: Model 1.2). By Table 2 CFA results of the pooled and separate samples by gender, length of service, and teaching level. Group

Sample

N

c2

c2/df

TLI

CFI

RMSEA (CI)

SRMR

Gender

Pooled Male Female Pooled Short Long Pooled Primary Secondary

673 157 516 673 353 320 673 432 241

499.689 196.811 228.092 470.102 185.253 220.529 473.895 201.610 206.292

2.658 2.116 2.453 2.501 1.992 2.371 2.521 2.168 2.218

.963 .949 .978 .966 .978 .968 .966 .921 .957

.971 .960 .983 .973 .983 .975 .973 .985 .967

.05 .08 .05 .05 .05 .07 .05 .05 .07

.05 .04 .03 .03 .03 .04 .03 .03 .03

Length of service in teaching

Teaching level

132

T. Teo / Computers & Education 75 (2014) 127–135

Table 3 Results of the tests of measurement invariance by gender,a length of service,b and teaching level.c Model

c2

df

CFI

RMSEA

SRMR

Dc2

Ddf

DCFI

DRMSEA

DSRMR

1.1 1.2 1.3 2.1 2.2 2.2.1 2.3 2.3.1 3.1 3.2 3.2.1 3.3

499.689 517.273 531.463 470.102 492.651 485.922 519.103 495.328 473.895 500.748 489.888 498.697

188 199 210 188 199 198 209 206 188 199 198 209

.971 .970 .970 .973 .972 .973 .971 .973 .973 .972 .973 .973

.050 .049 .048 .047 .047 .047 .047 .046 .048 .048 .047 .045

.047 .048 .048 .033 .030 .031 .031 .031 .029 .030 .029 .030

– 17.584 14.19 – 22.549* 15.82 33.181* 9.406 – 26.853* 15.993 8.809

– 11 11 – 11 10 11 8 – 11 10 11

– .001 .001 – .001 .001 .002 .002 – .001 .001 .000

– .001 .000 – .000 .000 .000 .001 – .000 .001 .002

– .001 .000 – .003 .001 .000 .000 – .001 .001 .001

Notes:*p < .05. 1.1: gender configural invariance; 1.2: gender metric invariance; 1.3 gender scalar invariance. 2.1: length of service configural invariance; 2.2: length of service metric invariance; 2.2.1: length of service partial metric invariance; 2.3: length of teaching scalar invariance; 2.3.1: length of service partial scalar invariance. 3.1: teaching level configural invariance; 3.2: teaching level metric invariance; 3.2.1: teaching level partial metric invariance 3.3: teaching level scalar invariance. a 157 males, 516 females. b 353 short, 320 long. c 432 primary, 241 secondary.

constraining the intercepts to be equal, the c2 value was increased by 14.19 with a gain in 11 df. Scalar invariance was also supported as this was not statistically significant at p < .05 level (Model 1.3). 3.4.2. Length of service When constraining the factor coefficients to be equal, the c2 value increased by 22.549. With a gain in 11 df, there was significant drop in model fit at the p < .05 level and metric invariance was not supported (Model 2.2). On the recommendation by Steenkamp and Baumgartner (1998) that full metric or scalar invariance is not necessary for further tests of invariance and substantive analysis, a partial scalar invariance test was conducted by removing the constraint on the intercept of each item in turn in order to identify the non-invariant item(s) (Byrne, 2010). This process revealed that the factor coefficients of item FC1 (“Specialized instruction concerning technology is available to me”) was non-invariant were non-invariant and when removed, there was a c2 difference of 15.82 with a gain in 10 df, which was not statistically significant hence supporting a partial metric invariance (Model 2.2.1). Scalar invariance was tested by constraining the intercepts of the item to be equal between the ‘short’ and ‘long’ groups. The result showed that full scalar invariance was not supported by the data as a c2 difference of 31.181 with a gain in 11 df was statistically significant at the p < .05 level (Model 2.3). Following the process of establishing a partial invariance, three items: PEU1 (“My interaction with technology does not require much effort”); PEU2 (“I find it easy to use technology to do what I want to do”); and ATTU3 (“I like working with technology”) were found to be non-invariant and removed. The resulting difference in the c2 value of 9.406, with a gain in 8 df, was not statistically significant and partial scalar invariance was supported (Model 2.3.1). 3.4.3. Teaching level Full metric invariance was not supported since an increase of c2 value of 26.853 and a gain of 11 df was statistically significant at the p < .05 level. Having identified and removed the non-invariant item, FC1 (“Specialized instruction concerning technology is available to me”), there was a non-significant drop in model fit at the p < .05 level with a c2 difference of 15.993 and a gain of 10 df, thus supporting a partial metric invariance (Mode1 2.2.1). When the intercepts of the items were constrained, the c2 value increased by 8.809 and gain in 10 df. Given that these differences were not statistically significant at p < .05 level, full scalar invariance was supported. 3.5. Test of latent mean differences Where the responses (i.e., survey items) are likely to be causally influenced by the underlying constructs and that these are latent (unobserved) constructs, such as the TAMPST used in this study, the test of latent mean differences is used to compare the responses between subgroups (e.g., male and female). An advantage of the test of latent mean differences over the traditional t-test or ANOVA is that it provides error-free measures of the latent variables (e.g., constructs, factors, subscales) by accounting for the random error of measurement for the observed variables (e.g., questionnaire items) associated with each latent variable (Aiken, Stein, & Bentler, 1994). In testing for latent mean differences, the means have to be constrained to zero in one group to get the model identified. The group whose mean was constrained to zero serves as the reference group against which the estimated mean of the comparison group will be compared. That is, the estimated mean of one group will be compared to zero, representing the other group. In the present study, the male, short, and primary groups were used as the reference groups for gender, length of service, and teaching level, respectively. However, such a comparison does not allow for the absolute mean in each group to be estimated, but rather the mean difference in the latent variables (i.e., PU, PEU, ATTU, SN, and FC) between the groups. Assessment of latent mean differences was based on the critical ratio (CR) index where a CR greater or equal to 1.96 indicates statistically significant differences in the means. Furthermore, a positive CR value suggests that the comparison group has higher latent mean values than the reference group. Because there is no true zero in the scoring a self-reported measure such as the TAMPST, it was difficult to interpret the mean differences in terms of their magnitude even if the differences were statistically significant. Researchers suggested using the effect size index such as the Cohen’s d, which expresses a group mean difference as a proportion of the pooled withingroup standard deviation on a variable. According to Cohen, d values around .20 indicate “small” differences, about .50 “medium”

T. Teo / Computers & Education 75 (2014) 127–135

133

Table 4 Tests of latent mean differences for gender, length of teaching service, and teaching level.

Gender

Length of teaching service

Teaching level

Construct

c2

c2/df

TLI

CFI

RMSEA

SRMR

MD

CR (t)

PU PEU ATTU SN FC PU PEU ATTU SN FC PU PEU ATTU SN FC

518.541

2.529

.965

.970

.048

.048

.045 .297 .048 .044 .048 .073 .597 .223 .014 .036 .002 .080 .045 .233 .010

.552 2.848* .505 .302 .568 1.072 6.087* 2.778* .104 .477 .026 .855 .537 1.632 .155

492.822

493.341

2.428

2.418

.968

.968

.973

.973

.046

.046

.031

.029

d .22

.52 .22

*p < .05; MD: mean difference. PU: perceived usefulness; PEU: perceived ease of use; ATTU: attitude towards technology use; SN: subjective norm; FC: facilitating conditions.

differences, and greater than .80, “large” differences. According to these criteria, the ES of the mean differences between males and females in this study could be regarded as small. Table 4 shows the results of the tests of latent mean differences for each factor in the TAMPST. By gender, male teachers gave higher scores than their female counterparts for all five factors in the TAMPST although only those for PEU were significantly different (t ¼ 2.848, p < .05). However, the effect size for this mean difference was regarded as small (d ¼ .22). Considering their length of service in teaching, those with seven years and below (short) gave higher ratings for all factors except SN. However, only the mean differences in PEU (t ¼ 6.087, p < .05) and ATTU (t ¼ 2.778, p < .05) were statistically significant, with the difference for PEU having a medium effect (d ¼ .52) and that of ATTU having a small effect (d ¼ .22). Between the primary and secondary teachers, the former gave higher ratings for all five factors in the TAMPST although all mean differences were not statistically significant. 4. Discussion This aims of this study were to assess the overall level of technology acceptance among teachers and to test the validity of the 5-factor hypothesized model of technology acceptance to explain teachers’ acceptance of technology. Further, this study attempted to assess the measurement invariance of the technology acceptance measure for pre-service teachers (TAMPST) among the study sample by gender, length of service, teaching level. Finally, several tests of latent mean differences in each factor of teachers’ technology acceptance (PU, PEU, AATU, SN, and FC) by gender, length of service, and teaching level were assessed for statistical significance. Results showed that the 673 teachers in this study had a generally positive acceptance of technology with a mean rating between 4.56 and 6.01 on a 7-point scale. This is in tandem with the worldwide drive among governments in the past few decades to employ technology in teaching and learning and in association with the infrastructural building, there has been training to equip teachers to use technology and resources have been allocated at all levels of schooling to ensure that students have sufficient experience with and engagement with technology for learning and in their daily lives. On the validity of the TAMPST, the results from the confirmatory factor analysis suggest that the 5-factor conception of technology acceptance can be applied to explain teachers’ acceptance of technology. This is consistent with, and supports existing research that found the TAMPST to be a valid and parsimonious instrument to measure individual acceptance of technology among users in education (Teo, 2010b) and beyond (Kuilema, 2012). In relation to measurement invariance, the results of the current study indicated support for configural invariance (pattern structure), metric invariance (factor loadings), and scalar invariance (item intercepts) by gender, length of service, and teaching level. In cases where partial metric or scalar invariance had to be established, these pertained to items for PEU, ATTU, and FC. While only one item from ATTU and FC was non-invariant by length of service and teaching level, two items in PEU were found to be non-invariant by length of service. Researchers have suggested that metric non-invariance could be due to differences in the conceptual meaning or understanding of the items across groups or that particular items are more applicable for one group than another (Chen, 2008). In case of scalar non-invariance, it could be that a particular group was displaying a propensity to respond more strongly to an item despite having the same latent trait or factor mean or having different reference points when making statements about themselves. In other words, different groups may have different thresholds (or distribution cut points) on the item distributions. For example, the distance between 3 and 4 on a likert scale may be perceived to be shorter or longer for one group than the other, depending on how each group respond to a particular item. However, although the excluded items were not invariant across groups, their validity and reliability were not in question as these have been established statistically earlier. On examining the latent mean differences, it appears that the reference groups (i.e., groups with their means constrained to zero) had higher mean values than the comparison groups for all five factors. This means that male teachers, teachers with a shorter length of service, and primary school teachers had rated themselves higher than their counterparts for all factors, suggesting that they had a higher level of technology acceptance. In particular, the mean difference for PEU was significantly different by gender indicating that this factor affected more male than female teachers. However, the effect size of this difference appeared to be small and the debate around the impact of gender on technology acceptance is hardly conclusive (e.g., Anderson, Lankhear, Timms, & Courtney, 2008; Broos, 2005). For example, although there is some evidence to suggest that male teachers tend to make more use of technology in their work than do female teachers, it was

134

T. Teo / Computers & Education 75 (2014) 127–135

possible this imbalance may be explained by contextual factors such as the school level where there are disproportionately more female than male teachers (BECTA, 2004). Among those with different lengths of service in teaching, significant mean differences were found for PEU and ATTU, with higher ratings given by teachers with seven or less years of service. It is possible that teachers with fewer years in service were still adjusting to the complexities of the teaching profession such as managing their time, setting up routines within their classrooms, planning, and learning about their schools and students (Clausen, 2007). In order for them to actively engage with technology for instructional and administrative purposes, they would need to perceive the use of technology to be easy or relatively free from effort. At the same time, these teachers were more driven by their feelings towards using technology than their colleagues who have taught for a longer period of time. This is consistent with research that found that teachers with more years of service tend to possess greater familiarity with educational practices relating to technology-based teaching and learning than their less experienced counterparts (Liu, Jones, & Sadera, 2010). In the case of attitudes towards using technology, it has been found to be a direct and significant driver of users’ intention to use and actual usage of technology (e.g., Davies, 1989; Ertmer, Ottenbreit-Leftwich, Sadik, Sendurur, & Sendurur, 2012; Teo, 2009; Venkatesh et al., 2003). No significant mean differences were found between primary and secondary teachers’ acceptance of technology. The reasons for this could be a combination of personal factors where technology has become an integral part of their daily living, or professional factors where school leaders and society at large expect teaching and learning to be heavily technology-enabled and mediated. Translating this into their professional activities, teachers are expected to harness technology for planning, lesson delivery, assessment, and administration. Finally, teachers in the primary and secondary sectors operate in multiple learning environments and modalities of teaching thus explaining a lack of significance in their mean differences in all five factors of the TAMPST. 4.1. Contributions of this study The findings of this study contribute in the area of technology acceptance by providing evidence for the viability of several model-driven constructs on a sample of educational users such as teachers. Methodologically, by using multi-group confirmatory factor analysis (MGCFA) to establish measurement invariance and a test for latent mean differences in technology acceptance across groups of teachers, this study demonstrated that advanced statistical techniques are more superior and allow for more in-depth data analysis over the more traditional approaches, such as exploratory factor analysis and t-tests. The results of this study can inform policy makers who manage the extent and rate of technology adoption at professional workplaces. Teacher educators could use the findings of this study to guide their curriculum design in working towards fostering a positive mindset and encouraging use of technology for their current learning and future professional lives. 4.2. Limitations of the study Several limitations exist in this study. Although the TAMPST has a good model fit, it is possible that other constructs could be considered to enhance our understanding teachers’ technology acceptance. Future research could include more tests of measurement invariance across samples and populations of the measure in response to increasing complexity in the teaching profession and rapid changes in the learning environment, with a view to achieve greater precision in measurement and validity. Secondly, the use of self-reported data in this study could be susceptible to common method variance, leading to inflation in the relationships among constructs and, subsequently, measurement bias. Finally, it was not possible to distinguish the technology tools that teachers were thinking of when they responded to the items in the TAMPST and it was possible that different tools may evoke different reactions to the items hence contributing to several instances of measurement non-invariance in this study. Further study could elicit responses on the acceptance of specific technology tools that are used for various purposes (e.g., IWB for instruction, Facebook for social communication) from teachers and pre-service teachers. Finally, the responses in this study were collected from only one country and this limits the generalizability of the findings. Future research may include an assessment of the TAMPST across other cultural groups, user groups, and technological tools to enhance its validity and usability. 5. Conclusion Technology has relatively recently become ubiquitous and it has greatly influenced all aspects of a teacher’s professional life, from the training stage to the professional development process. Any adoption and integration of technology in teaching and learning requires considerable and sustained commitment on the part of the teacher who has to work within the constraints imposed by the workplace, colleagues, and students to optimise technological resources for effective teaching. One way to gauge teacher’s willingness to use technology for what it is designed to do is through examining their levels of technology acceptance and this study offers a glimpse of how these might be assessed. Specifically, this study suggests that we should examine the extent to which teachers’ perceived usefulness (“How it will add to their productivity?”), perceived ease of use (“How much effort is required to use technology?”), their attitudes towards technology use (“How much do I like to use technology”), subjective norm (“Do people who are important to me think I should use technology?”), and perceived support (“Is help readily available when I need it?”) to obtain a holistic picture of teacher’s technology acceptance. References Aiken, L. S., Stein, J. A., & Bentler, P. M. (1994). Structural equation analyses of clinical subpopulation differences and comparative treatment outcomes: characterizing the daily lives of drug addicts. Journal of Consulting and Clinical Psychology, 62, 488–499. Ajzen, I. (1991). The theory of planned behavior. Organizational Behavior and Human Decision Processes, 50(2), 179–211. Ajzen, I., & Fishbein, M. (1980). Understanding attitudes and predicting social behavior. Englewood Cliffs, NJ: Prentice-Hall. Anderson, N., Lankhear, C., Timms, C., & Courtney, L. (2008). Because it’s boring, irrelevant and I don’t like computers: why high school girls avoid professionally-oriented ICT subjects. Computers & Education, 50, 1304–1318. Becker, H. J. (2001 April). How are teachers using computers in instruction?. Paper delivered at the meetings of the American Educational Research Association. BECTA. (2004). A review of the research literature on barriers to the uptake of ICT by teachers. Retrieved from http://dera.ioe.ac.uk/1603/1/becta_2004_barrierstouptake_litrev. pdf.

T. Teo / Computers & Education 75 (2014) 127–135

135

Birch, D., & Burnett, B. (2009). Bringing academics on board: encouraging institution-wide diffusion of e-learning environments. Australasian Journal of Educational Technology, 25(1), 117–134. Bollen, K. A. (1989). Structural equations with latent variables. New York, NY: John Wiley & Sons. Borsboom, D. (2006). When does measurement invariance matter? Medical Care, 44, 176–181. Broos, A. (2005). Gender and information and communication technologies (IT) anxiety: male self-assurance and female hesitation. CyberPsychology & Behaviour, 8(1), 21–31. Byrne, B. M. (2010). Structural equation modeling with Amos: Basic concepts applications, and programming (2nd ed.). Mahwah, NJ: Erlbaum. Carmines, E. G., & McIver, J. P. (1981). Analyzing models with unobserved variables. In G. W. Bohrnstedt, & E. F. Borgatta (Eds.), Social measurement: Current issues. Beverly Hills, CA: Sage. Chen, F. F. (2007). Sensitivity of goodness of fit indices to lack of measurement invariance. Structural Equation Modeling, 14, 464–504. Chen, F. F. (2008). What happens if we compare chopsticks with forks? The impact of making inappropriate comparisons in cross-cultural research. Journal of Personality and Social Psychology, 95, 1005–1018. Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing MI. Structural Equation Modeling, 9, 235–255. Clausen, J. M. (2007). Beginning teachers’ technology use: first-year teacher development and the institutional context’s affect on new teachers’ instructional technology use with students. Journal of Research on Technology in Education, 35(3), 245–261. Curran, P. J., West, S. G., & Finch, J. F. (1996). The robustness of test statistics to nonnormality and specification error in confirmatory factor analysis. Psychological Methods, 1, 16–29. Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13(3), 319–340. Ertmer, P. A., Ottenbreit-Leftwich, A. T., Sadik, O., Sendurur, E., & Sendurur, P. (2012). Teacher beliefs and technology integration practices: a critical relationship. Computers & Education, 59(2), 423–435. Fishbein, M., & Ajzen, I. (1975). Belief, attitude, intention, and behavior: An introduction to theory and research. Reading, MA.; Don Mills, Ontario: Addison-Wesley. Fornell, C., & Larcker, D. F. (1981). Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research, 18(1), 39–50. Hair, J. F., Jr., Black, W. C., Babin, B. J., & Anderson, R. E. (2010). Multivariate data analysis (7th ed.). Upper Saddle River, NJ: Prentice Hall. Hammond, M. (2011). Beliefs and ICT: what can we learn from experienced educators? Technology, Pedagogy and Education, 20(3), 289–300. Henson, R. K. (2001). Understanding internal consistency reliability estimates: a conceptual primer on coefficient alpha. Measurement and Evaluation in Counseling and Development, 34, 177–188. Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1–55. Jeyaraj, A., Rottman, J. W., & Lacity, M. C. (2006). A review of the predictors, linkages, and biases in IT innovation adoption research. Journal of Information Technology, 21(1), 1–23. Kline, R. B. (2010). Principles and practice of structural equation modeling (3rd ed.). New York, NY: Guilford Press. Kuilema, J. (2012). Social workers and broadband advocacy: social justice and information communication technologies. Social Science Computer Review, 31(3), 291–305. Legris, P., Ingham, J., & Collerette, P. (2003). Why do people use information technology? A critical review of the technology acceptance model. Information & Management, 40, 191–204. Liu, L., Jones, P. E., & Sadera, W. A. (2010). An investigation on experienced teachers’ knowledge and perceptions of instructional theories and practices. Computers in the Schools, 27(1), 20–34. Mardia, K. V. (1970). Measures of multivariate skewness and kurtosis with applications. Biometrika, 57, 519–530. Meredith, W. (1993). MI, factor analysis and factorial invariance. Psychometrika, 58, 525–543. Raykov, T., & Marcoulides, G. A. (2008). An introduction to applied multivariate analysis. New York, NY: Taylor and Francis. Read, W., Robertson, N., & McQuilken, L. (2011). A novel romance: the technology acceptance model with emotional attachment. Australasian Marketing Journal, 19, 223–229. Schumacker, R. E., & Lomax, R. G. (2010). A beginners’ guide to structural equation modeling. New York, NY: Routledge. Steenkamp, J. E., & Baumgartner, H. (1998). Assessing MI in cross-national consumer research. Journal of Consumer Research, 25, 78–90. Teo, T. (2009). Modelling technology acceptance in education: a study of pre-service teachers. Computers & Education, 52(1), 302–312. Teo, T. (2010a). The development, validation, and analysis of measurement invariance of the technology acceptance measure for pre-service teachers (TAMPST). Educational and Psychological Measurement, 70(6), 990–1006. Teo, T. (2010b). Validation of the technology acceptance measure for pre-service teachers (TAMPST) on a Malaysian sample: a cross-cultural study. Multicultural Education & Technology Journal, 4(3), 163–172. Teo, T., & Fan, X. (2013). Coefficient alpha and beyond: issues and alternatives for educational research. The Asia-Pacific Education Researcher, 22(2), 209–213. Teo, T., Koh, N. K., & Lee, C. B. (2011). Teachers’ intention to teach financial literacy in Singapore: a path analysis of an extended theory of planned behaviour (TPB). The AsiaPacific Education Researcher, 20(2), 412–421. Teo, T., & Noyes, J. (2011). An assessment of the influence of attitude and perceived enjoyment on the intention to use technology among pre-service teachers: a structural equation modelling approach. Computers & Education, 57(2), 1645–1653. Teo, T., Ursavas, O. F., & Bahcekapili, E. (2012). An assessment of pre-service teachers’ technology acceptance in Turkey: a structural equation modeling approach. The AsiaPacific Education Researcher, 21(1), 199–210. Thong, J. Y. L., Hong, W., & Tam, K. Y. (2004). What leads to user acceptance of digital libraries? Communications of the ACM, 47(11), 79–83. Vandenberg, R. J., & Lance, C. E. (2000). A review and synthesis of the MI literature: suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3, 4–69. Venkatesh, V., Brown, S. A., Maruping, L. M., & Bala, H. (2008). Predicting different conceptualizations of systems use: the competing roles of behavioural intention, facilitating conditions and behavioural expectation. MIS Quarterly, 32(3), 483–502. Venkatesh, V., Morris, M. G., Davis, G. B., & Davis, F. D. (2003). User acceptance of information technology: toward a unified view. MIS Quarterly, 27(3), 425–478. Wong, S. L., & Teo, T. (2009). Investigating the technology acceptance among student teachers in Malaysia: an application of the Technology Acceptance Model (TAM). The Asia-Pacific Education Researcher, 18(2), 261–272. Wong, K. T., Teo, T., & Russo, S. (2013). Interactive whiteboard acceptance: applicability of the UTAUT Model among student teachers. The Asia Pacific Education Researcher, 22(1), 1–10. Zhao, Y., & Cziko, G. A. (2001). Teacher adoption of technology: a perceptual control theory perspective. Journal of Technology and Teacher Education, 9(1), 5–30.