- Email: [email protected]
Learning new uses of technology: Situational goal orientation matters
ARTICLE IN PRESS
Int. J. Human-Computer Studies 67 (2009) 50–61 www.elsevier.com/locate/ijhcs
Learning new uses of technology: Situational goal orientation matters Tina Loraasa,, Michelle Chandler Diazb,1 a
School of Accountancy, Auburn University, 415 W. Magnolia Avenue, AL 36849, USA Department of Accounting, E.J. Ourso College of Business, 3111C Patrick F Taylor Hall, Baton Rouge, LA 70803, USA
b
Received 12 April 2007; received in revised form 2 July 2008; accepted 24 August 2008 Communicated by P. Zhang Available online 29 August 2008
Abstract We study the decision to learn a new use of technology within a post-adoption context. This particular nuance of technology adoption is interesting because while the technology has been adopted at some level by both users and organizations, expanding technology use relies on users adopting additional tools and features within a given system on their own accord. This study addresses how situational goal orientation moderates the effects of ease of learning perceptions within the post-adoption context. We find that when a potential user has a situational learning goal orientation, they indicate intent to learn a new use of technology regardless of whether the technology is perceived to be easy or difficult to learn. However, potential users with a situational performance goal orientation indicate intent to learn the new system feature depending on ease of learning. These results have implications for future research using traditional technology acceptance parameters in the post-adoption context, and provide evidence that situational goal orientation is an effective managerial intervention for use in organizational training. r 2008 Elsevier Ltd. All rights reserved. Keywords: Goal orientation; Ease of use; Human/computer interaction; Technology management
1. Introduction Businesses invest large sums of money in technology (up to 50% of firms’ capital budgets) (Rockart et al., 1996), yet do not see comparable gains in productivity due to lack of full implementation by employees (Devaraj and Kohli, 2003). We study this phenomenon, specifically the decision to learn a new tool or feature within a technology that has already been adopted and used at some level. This context is interesting because while the organizational adoption decision has already been made, individual users have the ability to increase the use to a level that would benefit them and their organizations (Jasperson et al., 2005). We answer a call in the literature by studying the impact of a potential Corresponding author. Tel.: +1 334 844 6203; fax: +1 334 844 5875.
E-mail addresses: [email protected] (T. Loraas), [email protected] (M.C. Diaz). 1 Tel.: +1 225 578 6216. 1071-5819/$ - see front matter r 2008 Elsevier Ltd. All rights reserved. doi:10.1016/j.ijhcs.2008.08.005
managerial intervention on the decision to learn a new use of technology to increase efficiencies on the job (Jasperson et al., 2005). The managerial intervention of interest in this study is situational goal orientation. Specifically, we study whether potential users are more likely to switch from a known means of completing a task to learn a new systems approach based on their situational goal orientation and ease of learning perceptions. The interaction of these constructs is interesting because depending on the situational goal orientation of the potential user, ease of learning perceptions may affect intent to learn a new use of technology contrary to what the Technology Acceptance Model (TAM) would predict (if only the main effects of ease of learning were present). We attribute this interaction to the potential user’s reaction to the possibility of failure. In the context of post-adoption behavior, potential users are faced with the possibility of attempting to learn, but failing instead. This potential failure may result in lost time
ARTICLE IN PRESS T. Loraas, M.C. Diaz / Int. J. Human-Computer Studies 67 (2009) 50–61
and productivity, which are valuable commodities. Consideration of failure in a voluntary, post-adoption environment is important, because potential failure is not likely as a steep barrier in initial adoption or mandatory use phases as ‘‘failure time’’ is likely built into the implementation budget. Within the context of post-adoption behaviors, ‘‘failure’’ time would be costly to the user. Thus, this is a dimension of ease of use that should be considered. When considering the impact potential failure has on the decision to learn a new use of technology, we believe that situational goal orientation will be an effective intervention. Goal orientation theory describes how the type of goals pursued by an individual affects decision making (Nicholls, 1984; Dweck, 1986). The two distinct classes of goals that have been identified are learning goals and performance goals. A learning goal is defined by eagerness to learn for the sake of self-improvement, whereas a performance goal is defined by wanting to appear better (or at least no worse than) one’s peers. The facet of goal orientation that deals with the reaction to failure is what we find to be interesting in the postadoption context. With learning goals, failure is deemed a part of the learning process and, as such, is not feared. However, with performance goals, anything that might jeopardize performance is considered a threat, so failure is feared (Button et al., 1996). In the context of this study, if a potential system user has a situational learning orientation, they should view learning a system or a system feature as a positive self-improvement opportunity and intention to learn will not be as heavily influenced by ease of learning perceptions because fear of failure will be diminished. On the other hand, if potential users have a performance orientation, fear of failure will be heightened, which means that intention to learn will be influenced by ease of learning perceptions as traditionally shown in acceptance and/or adoption studies. This study investigates the efficacy of situational goal orientation as a practical managerial intervention that can motivate users to expand their use of technology to increase effectiveness and efficiencies on the job, even when the technology is deemed difficult to use. 2. Background and hypotheses This research investigates the decision to learn and use a new feature of an existing technology to replace a less efficient means of completing a routine task. This has been described as finding the equilibrium between exploring new possibilities (within a system) and exploiting old certainties (within a system; March, 1991). Most users of technology use a specific subset of available features, and seldom elect to extend this subset on their own (Jasperson et al., 2005). While determining how to encourage more encompassing use of technology on the job is of practical concern (Bowen 1986; Nambisan et al., 1999; Mahmood et al., 2001), there are a limited number of experiments studying this particular nuance of technology adoption.
51
Bhattacherjee (1998) performed an experiment where participants had to complete a task for which they had been trained, but they were given the option to complete the task using a more efficient/effective software solution that they had to learn in order to implement. The participants were more inclined to learn and use the new tool when the reward for doing so was significant. In a more recent study, Loraas and Wolfe (2006) performed an experiment where participants read a vignette that described a situation whereby the hypothetical character was faced with a routine task and had the option of learning a new software to complete that task, thereby also increasing future efficiencies. When the participants had sufficient motivation to follow their referents’ preferences (staff accountants early in their careers) subjective norms dominated the decision of when to learn the new software to complete the task. Our goal is to extend these findings by more closely examining the characteristics of the post-adoption context and to explore the efficacy of situational goal orientation as a managerial intervention to promote system users to learn new uses of technology. 2.1. The post-adoption context The traditional technology adoption context typically focuses on new technologies, whereas the post-adoption context is concerned with users learning new features or functions within a system that has already been adopted by the user at some level (Jasperson et al., 2005). While traditional technology acceptance studies (whether using the TAM, or such more recent adaptations) rely in part on perceptions regarding ease of use to inform a potential user’s intent to use a technology, we believe perceptions regarding ease of learning to be a more appropriate construct for the post-adoption context. Research on the discriminant validity of ease of use and ease of learning has concluded that the two constructs are highly correlated (.79), and are in essence, congruent (Roberts and Moran, 1983; Whiteside et al., 1985). However, we believe these constructs differ in one specific area, and that is the consideration of potential failure. This subtle difference is important because an assumption that is inherent in most technology adoption studies is that once intent to use a technology is formed, there are no impediments to use (Loraas and Wolfe, 2006). However, in the post-adoption context, after forming intent to use, the potential user still has to attempt to learn the technology, and that attempt carries with it the possibility of failure. This potential failure is inversely related to perceptions regarding ease of learning. If ease of learning is perceived to be high, the perceived likelihood that the potential user will fail, if he or she tries to learn, will be low. On the other hand, if perceptions regarding ease of learning are low, the perceived likelihood the potential user will fail, if he or she tries to learn, is high. This potential for failure is especially salient in a voluntary environment, where the potential
ARTICLE IN PRESS 52
T. Loraas, M.C. Diaz / Int. J. Human-Computer Studies 67 (2009) 50–61
user can choose to ‘‘not learn’’ and maintain the status quo at no cost. We believe this aspect of ease of learning to be important in the post-adoption decision to learn a new use of technology. We look to situational goal orientation as a potential managerial intervention that would promote learning of new uses of technology to increase efficiencies on the job because of the aspect of ease of learning that deals with potential failure. 2.2. Goal orientation Goal orientation describes the objective typically undertaken when a person completes a task (Nicholls, 1984; Dweck, 1986). Goal orientation theory defines two predominant orientations: learning and performance (Nicholls, 1984; Dweck, 1986). A learning orientation focuses on the importance of gaining knowledge, whereas a performance orientation focuses on completing tasks correctly (Dweck, 1986). Goal orientation has become an important explicator of behavior in the educational and training literatures (Colquitt and Simmering, 1998; Fisher and Ford, 1998; Chen et al., 2000). Further, a learning goal orientation has been shown to influence perceptions regarding ease of use via self-efficacy in the technology acceptance setting (Yi and Hwang, 2003). While interesting, instead of looking at the a priori affects of dispositional goal orientation, we focus on goal orientation as a managerial intervention to promote technology use after ease of learning perceptions has been formed. While originally conceptualized as a dispositional characteristic, goal orientation has shown efficacy as a situational influence as well. Situational goal orientations are also categorized as learning and performance, and inducing these orientations is commonly done by normative instruction. A learning orientation is manipulated by indicating that the focus of the task is to obtain knowledge, and a performance orientation is manipulated by telling subjects that successful task performance is the overriding objective (Ames and Archer, 1988; Winters and Latham, 1996; Stevens and Gist, 1997; Kozlowski et al., 2001). Compared to performance manipulations, learning manipulations have resulted in higher effort, enhanced challenge seeking, and a predisposition for learning strategies in complex tasks (Ames and Archer, 1988; Winters and Latham, 1996). Further, learning manipulations have been found to contribute to the development of knowledge structure, where performance manipulations did not affect knowledge acquisition (Kozlowski et al., 2001). In the modern era, employees are expected to adapt to new tools and situations (Smith et al., 1997; Kozlowski, 1998). Active, voluntary learning of systems represents an important element of that adaptation. Prior research has shown that a learning orientation is associated with positive effects in terms of motivation to learn and effort expended to learn, while a performance orientation has a negative influence on these elements (Colquitt and Simmering,
1998; Fisher and Ford, 1998). For this reason, all things equal, we would expect that in our post-adoption context, potential users of technology with a situational learning orientation will be more willing to attempt learning a new use of technology than those with a situational performance orientation. In addition to the obvious, there is a component of situational goal orientation that we believe lends itself to the post-adoption context. This is the piece of situational goal orientation that describes how persons react to potential failure. A potential user with a learning goal orientation has adaptive responses to new and/or challenging situations (Kozlowski et al., 2001). Specifically, individuals displaying this orientation treat new and/or challenging situations as opportunities for self-improvement through learning. Failure is viewed as a normal obstacle necessary to the learning process, and is not feared. Alternatively, a potential user with a situational performance orientation is maladaptive, as these potential users will look for situations that ensure success (Kozlowski et al., 2001). These persons seek positive evaluations of their capabilities, and therefore, avoid new and/or challenging situations, due to the possibility of failure. We believe that how potential users react to the possibility of failure drives the interaction between situational goal orientation and perceived ease of learning. 2.3. Interaction of ease of learning perceptions and situational goal orientation The interaction of ease of learning perceptions with situational goal orientation is of particular interest because it offers a way to promote post-adoption use of technology, even when the technology is deemed difficult to learn. Situational goal orientation as a managerial intervention is compelling because it could influence potential nonlearners to attempt learning by diminishing the impact of ease of learning on the intent decision. Considering ease of learning in isolation, if a new feature is deemed difficult to learn, the likelihood that an attempt at learning will result in failure is increased. In a voluntary environment, the potential user will consider this possibility of failure and will be deterred from learning (Loraas and Wolfe, 2006). However, by introducing a situational goal orientation that is focused on learning, the potential users’ view of failure goes from that of a detriment to a normal facet of learning new skills. As a result, fear of failing is diminished, reducing the effect of ease of learning on a potential user’s choice to learn a new tool to complete a routine task. However, if a situational performance orientation that triggers failure avoidance is introduced, ease of learning will be predictive in the choice to learn a new use of technology. This is because the potential user will want to avoid failure by not choosing to learn when the technology is perceived to be difficult. In summary, we study how a potential user’s situational goal orientation moderates the influence of ease of learning
ARTICLE IN PRESS T. Loraas, M.C. Diaz / Int. J. Human-Computer Studies 67 (2009) 50–61
53
Panel A: Complete Model Hypothesis Situational Goal Orientation
Ease of Learning
Intent to Learn
Covariates Risk Preferences Computer Confidence Panel B: Hypothesis Situational Learning Orientation
Intent to Learn
Ease of Learning
Situational Performance Orientation
Intent to Learn
Ease of Learning
Indicates no effect Fig. 1. Research model. Panel A: complete model. Panel B: hypothesis.
perceptions in the decision to learn a new use of technology (see Fig. 1). Specifically, Hypothesis 1a. When perceived ease of learning is low, potential users with a more situational learning orientation will indicate higher intention to learn a new system feature to complete a routine task than potential users with a situational performance orientation. Hypothesis 1b. When perceived ease of learning is high, potential users will indicate higher intention to learn a new system feature, regardless of their situational goal orientation. Situational goal orientation is important for one overriding reason: it offers a managerial intervention that can promote intent to learn even when the system is deemed to be difficult.
3. Methodology We used a laboratory experiment to study whether situational goal orientation differentially influences a potential user to learn a new use of technology depending on ease of learning perceptions. Our participants began the experiment by performing a simple decoding task. Once the participants understood the task and had completed it for pay, they were informed that a feature within a software program known to them could automate the work. Participants were told that the feature was more efficient, and offered higher payoffs (to simulate utility gained from future efficiencies not available in an experimental context), but were also told that they must learn this new feature in a timely manner, and attempting to learn it and failing was costly. This experimental design was rooted in an experimental economics methodology. Experimental economics relies on game-like tasks to produce behavior that parallels behavior in richer real-world settings (Friedman and Sunder, 1994).
ARTICLE IN PRESS 54
T. Loraas, M.C. Diaz / Int. J. Human-Computer Studies 67 (2009) 50–61
An important aspect of this methodology was that our participants completely understood the task, which they had been paid to do, when they made the decision of whether to maintain the status quo or learn a system-based approach to the task. Given that our participants used their native utility functions for decisions in this context, then they should behave similarly in other system adoption decisions of this type, regardless of the task (Chow, 1983). 3.1. Participants Participants consisted of junior-level students enrolled in a 5-year professional program in accountancy at a large university in the southwestern United States. Participants were enrolled in a seminar series when recruited for the experiment, and they were paid and given class credit for participating. Formal discussions with the department chairman and professional program chair indicated that most of these participants (approximately 90%) would take positions with large, international accounting firms. Additionally, this homogeneous pool had taken classes focusing on how to use end-user software; therefore they had experienced learning systems and system elements. Our initial sample consisted of 195 participants. The experimental parameters required the participants to indicate their intent to learn a new use of technology to complete the task; however they did not actually have to learn the new technology during the experiment due to time constraints. To control for contamination across our participant pool we conducted four sessions of the experiment back to back, and had an administrator ensure that during the changeover there was no speaking between participants. Further, we analyzed the findings across the four sessions with no evidence of contamination. 3.2. Experimental design The experimental task environment consisted of decoding numbers into letters in a Microsoft Excel application using paper decoding sheets. Participants began the experiment by performing two short, paid decoding trials. These practice trials allowed participants to become familiar with the decoding task and to experience being paid in the experimental currency, which was redeemed for cash at the conclusion of the experiment.2 After the practice trials, participants were told that there was a final decoding task and that they had 15 min to complete it. Pay for the final decoding task was at its maximum if it was completed in 10 min, a reduced pay was offered for completion in more than 10 min but less than 15 min, and there was no pay for completion of the decoding task after 15 min. Pilot tests had indicated that the final decoding task 2
We used an experimental currency to inflate the amounts paid for successful completion so that comparisons between learning and not learning had greater magnitude.
could be completed in 10 min, and all but four participants did the final decoding task in 10 min or less. (These four participants were dropped from our analyses.3) At this point participants were told that the final decoding task had to be completed five more times. Further, they were told that there was an Excel procedure that could automate the task, but the feature was new to them and they would have to learn it before they could use it.4 Participants were given three options for completing the final five decoding tasks. First, they could simply follow the manual method that they knew and had successfully used; second, they could use 5 min of the first decoding period to try to learn the Excel procedure; or third, they could use the entire 15 min of the first decoding period to try to learn the Excel procedure. Learning the Excel procedure was valuable, and if it was successfully learned and used, the payoff was higher than simply using manual decoding (to simulate current and future efficiencies). On the other hand, it was uncertain that the Excel procedure could be learned in either 5 or 15 min and trying and failing was costly. Participants made choices ‘‘as if’’ they were going to decode the final five decoding tasks—no one actually decoded for five 15 min periods. In essence, participants were supplied with contextualized lotteries that offered a choice among a certainty equivalent (decoding by hand) and two risky options (learning the new procedure). The probabilities (of successfully learning) and payoffs were calculated so that we could determine whether the choices were simply being made via an expected value calculation. Table 1 contains a summary of odds and payoffs for the choice options used in this experiment. We manipulated the payoffs and probabilities to determine whether participants were simply maximizing their expected value by their choices. If so, they should always choose to indicate that they would learn at the highest level (highest expected value). Appendix A contains an example of the wording used. 3.3. Variables 3.3.1. Ease of learning Ease of learning was manipulated by changing the subjective probabilities associated with successfully learning and using the system. Participants were asked their intention to learn if the probability of learning the Excel 3 Within our experimental economics environment it was necessary to drop these participants because they did not complete the task within the necessary time and therefore, for these individuals, completion at the status quo could not be assumed. Therefore, these participants would be making decisions between two uncertainties. 4 Participants were queried as to whether they had a spreadsheet procedure in mind when they made their decisions of whether or not to learn the ‘‘unknown’’ spreadsheet procedure. Two participants listed Vlookup (the tool we had in mind for automating the task), and were dropped from the analysis, as by listing the tool, it was likely the participants were choosing between two certainties.
ARTICLE IN PRESS T. Loraas, M.C. Diaz / Int. J. Human-Computer Studies 67 (2009) 50–61
55
Table 1 Experimental choice options, payoffs, and odds Choice options Success francs
Success odds
Failure francs
Failure odds
Expected value
High ease of learning condition Manual method 3000 5/15 min Learning 8000 15/15 min Learning 7600
1.00 .33 .70
0 2400 1600
0 .67 .30
3000 4248 5800
Low ease of learning condition Manual method 3000 5/15 min Learning 8000 15/15 min Learning 7600
1.00 .20 .40
0 2400 1600
0 .80 .60
3000 3520 4000
Note: Francs were the experimental currency used (exchange rate: 10 Francs ¼ $.05).
procedure in 5 min was 33% (20%) and the probability of learning the Excel procedure in 15 min was 70% (40%), to represent high (low) ease of learning.5 In addition, we gave participants ease of learning manipulations that consisted of either numerical probabilities or textual expressions, e.g., 40% versus ‘‘slightly less than average’’. We observed no statistically significant differences on the dependent variable based on whether the ease of learning intervention was numerical probabilities or textual expressions. Therefore, we combine these treatments for analysis purposes. Further, we had the participants indicate on a scale from one to 11 how they interpreted either the probabilities or textual expressions we provided to them. All participants ranked the ease of use manipulations as we intended. 3.3.2. Situational goal orientation The experiment contained a relatively mild statement designed to induce either a more situational learning orientation or a more situational performance orientation. The participants read one of the following passages before indicating intention to learn: (Learning) In most work environments, knowledge level defines the value of human capital, and that makes learning new things critical. Learning new things leads to self-improvement. So, note in your analysis the importance of learning. Learning more efficient ways to accomplish a task is important. (Performance) Most work environments are time constrained and making the budget is critical. Performance within budget is an essential facet of judging and comparing individuals. So, note in your analysis the importance of making the budget. Performing at budget level is important. 5 Our original intent was for ease of learning to be a within-subject variable. Participants answered the intent question at three levels of ease of learning: high, moderate, and low. Due to order effects, only the first condition viewed by a participant was used in the analysis.
In addition, we had a control group that received neither passage. We adapted a goal commitment instrument from Klein et al. (2001) and used a six-item index to determine participants’ commitment to either a situational goal of learning the new use of technology or performing within budget. We found the situational goal of the participants who received the normative learning to be significantly more learning oriented than the participants who received the normative performance (p-value o.05). 3.3.3. Covariates To help control for potential alternative explanations, we gathered several variables to use as covariates and/or to ensure our participants were appropriate for the task. First, we gathered a measure of computer confidence, which is described as the self-assessed ability to use technology, because it has been found to be predictive of both intent to use and actual use of technology (Loyd and Loyd, 1985; Compeau and Higgins, 1995; Venkatesh et al., 2003).6 In addition to controlling for perceived ability, we also gathered measures to control for actual ability. These measures consisted of participant reported cumulative grade point average, systems classes grade point average, and number of systems classes taken (see Table 2 for all demographics). As the context of this study is post-adoption behavior, it implies that the potential user has experience with the system in question, in this case, Excel. To ensure that the participants had Excel knowledge and could make a postadoption decision, all participants took Excel spreadsheet quizzes. On average, the participants correctly answered 75% of the questions. This indicates that the decision can be considered to be post-adoption for the participants in our study. Finally, because some participants encountered ease of learning point estimates that were based on 6 The authors acknowledge that collecting the dispositional goal orientations of the participants would have strengthened the findings. However, Yi and Hwang (2003) find that self-efficacy controls, at least in part, for dispositional learning orientations.
ARTICLE IN PRESS T. Loraas, M.C. Diaz / Int. J. Human-Computer Studies 67 (2009) 50–61
56
4. Results
Table 2 Descriptive statistics Normative situational goala Variables
Learning Control Performance F-stat n ¼ 65 n ¼ 61 n ¼ 63
Age Class Genderb Computer courses Computer course GPA Overall GPA Excel quiz score (8 max.) Situational goal orientationc Computer confidencee Risk preferencesf
20.63 3.05 .36 1.42 3.74 3.58 6.58 21.86 34.24 14.98
20.69 3.11 .44 1.53 3.64 3.59 6.62 22.85 32.64 15.82
20.69 3.10 .37 1.56 3.80 3.60 6.67 23.80 33.00 15.43
.133 .374 .594 .544 .152 .892 .902 .271d 2.358 .912
a Each of these cells has participants who answered either high or low ease of use intent questions. Means did not differ between high and low ease cells. Thus for parsimony we collapse the cells for presentation of the descriptive statistics. b Scored as a dummy variable, where 1 ¼ male, 0 ¼ female; 39% of our participants were male. c Sum score of 6-item, 7-point Likert scale. Higher scores indicate a more performance orientation. d A contrast was conducted to determine significant differences between participants given a situational goal of learning versus performance. The contrast was significant (t-statistic 1.674, p-value o.05 (one tailed)). e Sum score of 10-item, 5-point Likert scale. Higher scores indicate more confidence. f Sum score of 6-item, 5-point Likert scale. Higher scores indicate more risk averse.
probabilities, we include a measure for risk preferences to control for choices made as a ‘‘gambler’’. 3.3.4. Intent Our dependent variable, intent, is both categorical and ordered in that the options represent an increasing intention to learn the feature. (Manual decoding represents no intent to learn, spending 5 of 15 min on learning is the mid-level intention, and spending 15 of 15 min on learning indicates the highest intention to learn.) Therefore, we use an ordered logit to estimate model relationships (Long, 1997). The underlying functional model used to test our hypothesis takes the following form: Intent ¼ f ðease of learning; situational goal orientation; computer confidence; risk preferences, ease of learning by situational goal orientationÞ
(1)
Ease of learning is a dummy variable with the value of one assigned to high ease of learning (zero is low ease of learning). Situational goal orientation is a sum score, with higher (lower) scores indicating a higher performance (learning) situational goal orientation. Computer confidence and risk preferences are covariates. Although we collected other variables, due to lack of variability, those measures added no value to the final model and were therefore excluded.
All the items from the Willingness to Take Risks scale (Keil, 1999), Computer Confidence subscale of the Computer Attitude scale (Loyd and Loyd, 1985; Al-Jabri and Al-Khaldi, 1997), and Goal Commitment scale (Klein et al., 2001) were pooled and factor analyzed using principal axis factoring and varimax rotation using SPSS 15.0 for Windows. Two computer confidence questions produced loadings below .35 and were dropped. The correlation between the computer confidence factor and the risk preference factor is .218. Computer confidence has a Cronbach’s alpha of .854, risk preferences has an alpha of .748, and goal commitment has an alpha of .844. Our analyses are supportive of good discriminant and convergent validity for the computer confidence, risk preference, and goal commitment constructs. Sum scores are used in all analyses. Factor analysis results, including cross-loadings, are shown in Appendix B. Our major premise is that situational goal orientation moderates ease of learning perceptions. Table 3 reports the results, and Fig. 2 graphically represents the same. As predicted, we find an interaction between ease of learning and situational goal orientation. Strictly interpreting the interaction coefficient (.102, p-value .022), when ease of learning was high, the more performance oriented our participants were, the more likely they were to choose a higher level of commitment to learn the technology. However, the odds ratio of 1.10 indicates the effect size is minimal, as depicted graphically in Fig. 2.7 The coefficient of the main effect of goal orientation (.078, p-value .014) indicates that when ease of learning was low, participants with a learning orientation were more likely to commit to a higher level of learning the new use of technology than participants with a performance orientation. Simply put, we find that ease of learning matters when a potential user has a situational performance orientation. In this situation, ease of learning impacts the commitment to learning the new use of technology just as the TAM would predict; the harder the technology is perceived to be to learn, the less intent the potential user has to learn it. However, when a potential user has a situational learning orientation, ease of learning does not seem to sway the decision, and the potential user indicates intention to learn and use the new use of technology at a fairly constant level (see Fig. 2). This means that when a task is complex, a situational learning orientation can promote leaving the status quo to learn a more efficient way to complete a task even when the more efficient way is deemed difficult. Interestingly, we also find that when a task is easy to learn, a performance orientation can promote learning at least as effectively as a learning orientation.
7
In a logistical analysis, the odds ratio is an indicator of effect size, with an odds ratio of 1.00 indicating no change in likelihood based on the independent variable (Garson, 2008).
ARTICLE IN PRESS T. Loraas, M.C. Diaz / Int. J. Human-Computer Studies 67 (2009) 50–61
We studied a specific class of technology acceptance defined as the choice to voluntarily learn a new system feature to complete a task routinely completed in a different manner. While highly stylized, our scenario is both important and common in that it defines the learning and use of many system features in a post-adoption environment. End-user software, such as the standard suite of office products, contains features that are often self-taught, and done so only when they are needed. Financial reporting system users can choose to learn and use the report writing tools that they know exist in accounting and ERP applications, or they can follow a simple, laborious process of downloading the data to a spreadsheet. 5.1. Implications for research The technology acceptance paradigm is rich and well developed; however, we contend that the majority of that research stream focuses on the initial adoption context. We extend the literature by investigating the traditional technology acceptance parameter, ease of use, in the post-adoption context. We believe that this context has barriers to adoption that are not as salient in the initial adoption setting, which then impacts how this traditional predictor interacts with situational goal orientation, which, in turn, influences the adoption/acceptance decision. While we recognize that TAM parameters as traditionally studied have offered great insight as to the technology acceptance decision, by narrowing the focus of these general studies to more specific contexts, we can begin to more fully understand behavior with respect to learning and using technology (Sun and Zhang, 2006). While we focus on how the potential for failure acts as an additional facet of ease of use that contributes to postadoption behavior, we acknowledge that different dimensions of constructs within the technology acceptance paradigm may also be contributing and are worthy of additional study. For example, future research might consider perceptions regarding perceived usefulness. While perceived usefulness has traditionally focused on the utility gained from using the technology, usefulness may also incorporate other cultural or organizational factors. As in our scenario, a message from a supervisor that learning is important may work together with actual usefulness of the technology. Not only will the potential user gain utility from the technology itself, but he/she may gain utility from following a supervisor suggestion that learning the technology is important. On the other hand, when faced with a situational performance-oriented supervisor, the usefulness of learning the technology may be discounted. In this case, perceived usefulness of the technology may be reduced sufficiently to deter learning. Subjective norms may also be a construct worthy of additional study. Recent research provides evidence that
staff accountants are likely to follow supervisor suggestions due to their motivation to comply with said referents’ beliefs. While we do not find that our participants acted totally in line with the situational goal orientation presented to them, the finding that the situational goal orientation was influential does provide support that subjective norms can impact the decision to learn a new use of technology in a voluntary setting, in direct contrast with earlier work focused on subjective norms (Legris et al., 2003; Venkatesh et al., 2003; Schepers and Wetzels, 2007). However, our finding could be an artifact of the situational goal presented; in other words, the message of the supervisor may cloud the potential user’s perception of
Table 3 Ordered logistic regression Variables
Coefficient
Wald
p-valuea
Ease of learningb Risk preferencesc Computer confidenced Situational goal orientatione Ease Situational goal orientation Pseudo R2 Sample size
2.076 .009 .008 .078
3.897 .038 .054 6.017
.048 .846 .816 .014
.102
5.217
.022
Odds ratio .125 .991 .992 .925 1.11
.04 189
The dependent variables were defined as follows: 1—manual method; 2—5/15 min learning; 3—15/15 min learning. a Two-tailed p-values. b Ease of learning is a between-subjects variable coded one (1) when the system feature is easy to learn and zero (0) when it is harder to learn. c Sum score of 6-item, 5-point Likert scale. Higher scores indicate more risk aversion. d Sum score of 10-item, 5-point Likert scale. Higher scores indicate more confidence. e Sum score of 6-item, 7-point Likert scale. Higher scores indicate more performance orientation.
3 2.8 2.6 Intention to Learn
5. Discussion
57
Learning Performance
2.4 2.2 2 1.8 1.6 1.4 1.2 1 Low Ease
High Ease
Fig. 2. Interaction of ease of learning and situational goal orientation.
ARTICLE IN PRESS 58
T. Loraas, M.C. Diaz / Int. J. Human-Computer Studies 67 (2009) 50–61
voluntariness. It could be that a supervisor suggestion that learning is important leaves the potential user with the perception that learning the technology is, in essence, mandatory. 5.2. Implications for practice We find that situational goal orientation matters. Specifically, participants who are more committed to a learning goal are more likely to learn a new use of technology, even when the new technology is determined to be difficult to learn. Further, this situational goal orientation can be influenced via a simple supervisor instruction. This presents management with a simple and inexpensive intervention to influence employees to learn new technologies, even when deemed difficult to learn. It should be noted that participants committed to a more performance-oriented goal did not intend to learn when ease of learning was low. Performance orientation is described in two dimensions, wanting to appear better than peers do and not appearing incompetent (Elliott and Dweck, 1988). Research has indicated that the performance orientation can be split based on these dimensions into performance approach orientation (look better) and performance avoidance orientation (not incompetent; Middleton and Midgley, 1997). Our results support the notion of a performance approach orientation. Individuals with a performance approach orientation will try to learn, as long as the technology is deemed easy to use. Although learning is not their goal (as it is with those with a learning orientation), they intend to learn in order to meet their goal of looking better than others. This suggests that an organizational culture that creates competition between employees may deter learning new uses of technology when failure may be likely, as employees will be unwilling to look bad in comparison to others. We found participants willing to give up large incremental expected payoffs from learning and using a new use of technology to return to the security of a known task completion method. In the end, this decision is costly to the employee and to the firm. Our findings suggest that managers who universally espouse performance can modify an employee’s situational goal orientation and as such may unwittingly curtail the learning of difficult, yet beneficial new software solutions by their employees. On the other hand, managers who can nurture and/or promote learning orientations should observe increased productivity over the long term. 6. Conclusion We study a mechanism that promotes users of technology to learn new uses of an established system. We investigated the efficacy of situational goal orientation as a managerial intervention that could either promote or dissuade potential users from learning a new use of technology to complete a routine task. We found that
individuals primed for learning choose to learn without regard to perceived ease of learning. However, potential users primed with a performance orientation indicated intention to learn depending on ease of learning as would be expected, as ease of learning increased (decreased), intent to learn increased (decreased). It should be noted that the impact of our normative guidance on situational goals, although statistically significant, did not appear to be very strong. That being said, we do find that the situational goal of the potential user influences intent to learn when ease of learning was low. This suggests that looking for more effective ways to modify situational goals may be an effective avenue for future research. However, in all likelihood, in a work environment, a message from a supervisor is likely to have a much greater effect on situational goals than a simple paragraph in an experiment. Another possible explanation for the seemingly small effect size is that the scale used to capture the situational goal of the participant likely also captured elements of the participants’ goal disposition. We believe that there is evidence of this occurring, as the mean of the scale was the scale midpoint with little variance. (If disposition were not captured, we would expect a larger dispersion in scores.) Since we did not measure the participants’ dispositional goal orientation prior to the experiment we cannot say conclusively. However, we did randomize, so a priori dispositional goal orientation effects should be equal across treatments. As in all work of this type, our study has limitations. Our results are dependent on the parameters of the experimental task used. We used point estimates for both system value and ease of learning. Different estimates could lead to different decision outcomes. However, we did find that using contextual expressions in lieu of point estimates did not change behavior. Additionally, participants were required to make a decision in a single period, and did not have the opportunity to ‘‘change their mind’’. As such, we test a single period decision model. Finally, our experimental task setting is not as rich as a real-world decision context, and actual system usage was never required or observed in the experiment. However, there are a number of studies that indicate intention is a predictor of behavior, which suggests that our measure of intent is meaningful (Dholakia and Bagozzi, 2002).
Acknowledgements The authors wish to thank Chris Wolfe, Elaine Mauldin, Tammy Kowalczyk, Dan Stone, Brad Tuttle, Patrick Wheeler, and Ed O’Donnell for comments on earlier drafts of this research. This paper has also benefited from comments made by workshop participants at the 2003 AAA Annual Meeting, 2003 Accounting Information Systems Research Symposium, and the Management Information Systems Department at Texas A&M University. We also want to thank Don Warren, Kun Wang,
ARTICLE IN PRESS T. Loraas, M.C. Diaz / Int. J. Human-Computer Studies 67 (2009) 50–61
Davit Adut, and Ryan Huston for help administering this experiment. Appendix A. Instrument for collection of subject choices The instrument shown below is for ease of use order highto-low and for high payoffs. Assume the same parameters as the last task (15 min task period) and your budget is still set at 175 correctly decoded items. The pay for reaching the budget is 400 francs; and for completing the budget with 5 min to spare (out of 15 min), there is a leisure pay of 200 francs. In total, 600 francs can be earned by decoding 175 items in 10 min or less, or 400 francs can be earned by decoding 175 items in 15 min or less. However, if in 15 min, 175 items are not decoded, the pay is zero. The payoffs are summarized in the table below: Items decoded
Time (min)
Pay
175 175 Less than 175
Under 10 Over 10 and under 15 Over 15
600 400 0
Assume that you have five more paid decoding sessions (i.e., five more 15 min decoding sessions). However, it has come to your attention that there is a set of Excel procedures that can automate the decoding task, but the Excel procedures are new to you and must be learned. Payoffs for the different choices are as follows. Decode by hand You can choose not to try to learn the Excel procedures, manually decode in each session, and earn 600 francs per session by decoding 175 items in 10 min. Five minute learning option If the Excel procedures are learned in 5 min, you will lose your leisure pay in Session one. However in the remaining four sessions your leisure pay will increase to 1500 francs per period and your budget pay will remain at 400 francs per period (i.e., the last four sessions are worth 1900 francs each). Since you are using only 5 min to learn in Session one you will have enough time left over to decode the budgeted 175 items. Thus you will still earn 400 francs in Session one. If you try to learn for 5 min and do not learn the Excel procedures, you will receive no pay for Session one, but you will be able to resort to manual decoding and earn 600 francs for each of the remaining four sessions by decoding 175 items in 10 min.
59
Fifteen minute learning option If the Excel procedures are learned in 15 min, you will not earn any pay in Session one. In the remaining four sessions however your leisure pay will increase to 1500 francs per period and your budget pay will remain at 400 francs per period (i.e., the last four sessions are worth 1900 francs each). If you try for 15 min and do not learn the Excel procedures, you will receive no pay for Session one, and you will not receive any leisure pay for the remaining four sessions regardless of how quickly you can decode by hand (i.e., the last four sessions are worth 400 francs each). The table given above was followed by three queries for participant choices, only one of which is shown (low ease of learning) due to space limitations. Francs converted into US dollars at the rate of 10 Francs ¼ $0.05. Assume the following payoffs are in effect for each of the three options across the last five sessions: Decoded by handItems decoded
Speed(min)
Total pay (all five periods)
175 175
Under 10 Over 10 under 15 Over 15
3000 2000
Less than 175
0
Spend 5 min learning Excel procedures
Total pay (all five periods)
Succeed in learning them Do not learn them
8000 2400
Spend 15 min learning Excel procedures
Total pay (all five periods)
Succeed in learning them Do not learn them
7600 1600
Using the payoffs described, which option would you select? Assume that after looking at the Excel procedures you assess the probability of learning the Excel procedures in 5 min at 20% and the probability of learning the Excel procedures in 15 min at 40%? Would you
a. attempt to learn in 5 min? b. attempt to learn in 15 min? c. decode by hand?
ARTICLE IN PRESS T. Loraas, M.C. Diaz / Int. J. Human-Computer Studies 67 (2009) 50–61
60
Appendix B. Factor analysis and construct validity (indicators are bolded) Item
Interpretation
Risk preferences
Computer confidence
Goal commitment
Riska_Q1 Risk_Q2 Risk_Q3 Risk_Q4 Risk_Q5 Risk_Q6 Confid.b_Q1 Confid._Q2 Confid._Q3 Confid._Q4 Confid._Q5 Confid._Q6 Confid._Q7 Confid._Q8 Confid._Q9 Confid._Q10 Goalc Q1 Goal Q2 Goal Q3 Goal Q4 Goal Q5 Goal Q6
Take risks choosing a new job I prefer a high security job Trade unknown problems for high reward Avoid risky situations at all costs Play it safe even if opportunity lost I am cautious and generally avoid risk No good with computers. Feel OK trying a new computer Would do advanced computer work I could do work with computers Not the type to do well with computers Could learn a computer language Computer work would be hard for me Could get good grades in computer courses Cannot handle a computer course I have self-confidence for computer work Important to shoot fory Most important not to abandony Unrealistic to take perspective ofy Strongly committed toy Did not matter to me if Iy Effort best used toy
.637 .497 .714 .658 .450 .466 .051 .067 .131 .028 .135 .227 .094 Dropped Dropped .097 .080 .046 .101 .055 .026 .118
.058 .286 .064 .047 .232 .307 .705 .771 .743 .624 .541 .463 .461 Dropped Dropped .771 .168 .020 .062 .005 .022 .068
.080 .046 .101 .055 .026 .118 .002 .034 .057 .017 .046 .109 .088 Dropped Dropped .047 .790 .555 .533 .833 .639 .763
a
Willingness to Take Risks scale (5-point scale, anchored with ‘‘Strongly Agree’’ and ‘‘Strongly Disagree’’). Computer Confidence scale (5-point scale anchored with ‘‘Strongly Agree’’ and ‘‘Strongly Disagree’’). c Goal Commitment scale (7-point scale anchored with ‘‘Learning New Things’’ and ‘‘Performing within Budget’’). b
Reliability and discriminant validity constructs
Risk Preferences Computer Confidence Goal Commitment a
Cronbach’s a
Risk Preferences
Computer Confidence
.748a .854b .844b
.218a .179b
.119a
Square root of the shared variance of the elements measuring a factor. Correlation of sum scores used in analysis.
b
References Al-Jabri, I.M., Al-Khaldi, M.A., 1997. Effects of user characteristics on computer attitudes among undergraduate business students. Journal of End User Computing, 16–22. Ames, C., Archer, J., 1988. Achievement goals in the classroom: students’ learning strategies and motivation processes. Journal of Educational Psychology September, 260–267. Bhattacherjee, A., 1998. Managerial influences on intraorganizational information technology use: a principal-agent model. Decision Sciences 29, 139–162. Bowen, D., 1986. The puny payoff from office computers. Fortune, 22–26. Button, S., Mathieu, J., Zajac, D., 1996. Goal orientation in organizational research: a conceptual and empirical foundation. Organizational Behavior and Human Decision Processes 67, 26–48. Chen, G., Gully, S., Whiteman, J., Kilcullen, R., 2000. Examination of relationships among trait-like individual differences, state-like individual differences, and learning performance. Journal of Applied Psychology, 835–847.
Chow, C., 1983. The effects of job standard tightness and compensation scheme on performance: an exploration of linkages. The Accounting Review, 667–685. Colquitt, J., Simmering, M., 1998. Conscientiousness, goal orientation, and motivation to learn during the learning process: a longitudinal study. Journal of Applied Psychology, 654–665. Compeau, D.R., Higgins, C.A., 1995. Computer self-efficacy-development of a measure and initial test. MIS Quarterly 19, 189–211. Devaraj, S., Kohli, R., 2003. Performance impacts of information technology: is actual usage the missing link? Management Science 49, 273–289. Dholakia, U.M., Bagozzi, R.P., 2002. Mustering motivation to enact decisions: how decision process characteristics influence goal realization. Journal of Behavioral Decision Making 15, 167–188. Dweck, C., 1986. Motivational processes affecting learning. American Psychologist 41, 1040–1048. Elliott, E.S., Dweck, C.S., 1988. Goals: an approach to motivation and achievement. Journal of Personality and Social Psychology, 5–12. Fisher, S., Ford, J., 1998. Differential effects of learner effort and goal orientation on two learning outcomes. Personnel Psychology, 397–420.
ARTICLE IN PRESS T. Loraas, M.C. Diaz / Int. J. Human-Computer Studies 67 (2009) 50–61 Friedman, D., Sunder, S., 1994. Experimental Methods: A Primer for Economists. Cambridge University Press, New York. Garson, G.D., 2008. Logistic regression. In: Statnotes: Topics in Multivariate Analysis. Retrieved February 20, 2008, from /http://www2. chass.ncsu.edu/garson/pa765/statnote.htmS. Jasperson, J.S., Carter, P.E., Zmud, R.W., 2005. A comprehensive conceptualization of post-adoptive behaviors associated with information technology enabled work systems. MIS Quarterly 29, 525–557. Keil, J.W., 1999. Compensation risk preferences: self-reported information vs. experimental evidence. ACA Journal, 67–71. Klein, H.J., Wesson, M.J., Hollenbeck, J.R., Wright, P.M., DeShon, R.P., 2001. The assessment of goal commitment: a measurement model meta-analysis. Organizational Behavior and Human Decision Processes 85, 32–55. Kozlowski, S., 1998. Training and developing adaptive teams: theory, principles, and research. In: Cannon-Bowers, J.A., Salas, E. (Eds.), Decision Making Under Stress: Implications for Training and Simulation. APA Books, Washington, DC. Kozlowski, S., Gully, S., Brown, K., Salas, E., Smith, E., Nason, E., 2001. Effects of training goals and goal orientation traits on multidimensional training outcomes and performance adaptability. Organizational Behavior and Human Decision Processes 85, 1–31. Legris, P., Ingham, J., Collerette, P., 2003. Why do people use information technology? A critical review of the technology acceptance model. Information and Management 40, 191–204. Long, S., 1997. Regression Models for Categorical and Limited Dependent Variables. Sage Publications, Thousand Oaks, CA. Loraas, T., Wolfe, C., 2006. Why wait? Modeling factors that influence the decision of when to learn a new use of technology. Journal of Information Systems 20, 1–24. Loyd, B.H., Loyd, D.E., 1985. The reliability and validity of an instrument for the assessment of computer attitudes. Educational and Psychological Measurement, 903–908. Mahmood, M.A., Hall, L., Swanberg, D.L., 2001. Factors affecting information technology usage: a meta-analysis of the empirical literature. Journal of Organizational Computing and Electronic Commerce 11, 107–130. March, J.G., 1991. Exploration and exploitation in organizational learning. Organization Science 2, 71–87.
61
Middleton, M., Midgley, C., 1997. Avoiding the demonstration of lack of ability: an under-explored aspect of goal theory. Journal of Educational Psychology, 514–523. Nambisan, S., Agarwal, R., Tanniru, M., 1999. Organizational mechanisms for enhancing user innovation in information technology. MIS Quarterly 23, 365–395. Nicholls, J.G., 1984. Achievement motivation: conceptions of ability, subjective experience, task choice, and performance. Psychological Review, 328–346. Roberts, T.L., Moran, T.P. (Eds.), 1983. The evaluation of text editors: methodology and empirical results, Communications of the ACM 26, 265–283. Rockart, J.F., Earl, M.J., Ross, J.W., 1996. Eight imperatives for the new IT organization. MIT Sloan Management Review 38, 43–55. Schepers, J., Wetzels, M., 2007. A meta-analysis of the technology acceptance model: investigating subjective norm and moderation effects. Information and Management 44, 90–103. Smith, E.M., Ford, J.K., Kozlowski, S.W., 1997. Building adaptive expertise: implications for training design. In: Quinones, M.A., Dudda, A. (Eds.), Training for the 21st Century Technology: Applications of Psychological Research. APA Books, Washington, DC. Stevens, C., Gist, M., 1997. Effects of self-efficacy and goal-orientation training on negotiation skill maintenance: what are the mechanisms? Personnel Psychology 50, 955–978. Sun, H., Zhang, P., 2006. The role of moderating factors in user technology acceptance. International Journal of Human–Computer Studies 64, 53–78. Venkatesh, V., Morris, M.G., Davis, F.D., Davis, G.B., 2003. User acceptance of information technology: toward a unified view. MIS Quarterly 27, 425–478. Whiteside, J., Jones, S., Levy, P.S., Wixon, D., 1985. User performance with command, menu and iconic interfaces. In: Proceedings of CHI ’85, ACM, San Fransisco. Winters, D., Latham, G., 1996. The effect of learning versus outcome goal on a simple versus complex task. Group and Organization Management, 236–250. Yi, M.Y., Hwang, Y., 2003. Predicting the use of web-based information systems: self-efficacy, enjoyment, learning goal orientation and the technology acceptance model. International Journal of Human– Computer Studies 59, 431–449.
Int. J. Human-Computer Studies 67 (2009) 50–61 www.elsevier.com/locate/ijhcs
Learning new uses of technology: Situational goal orientation matters Tina Loraasa,, Michelle Chandler Diazb,1 a
School of Accountancy, Auburn University, 415 W. Magnolia Avenue, AL 36849, USA Department of Accounting, E.J. Ourso College of Business, 3111C Patrick F Taylor Hall, Baton Rouge, LA 70803, USA
b
Received 12 April 2007; received in revised form 2 July 2008; accepted 24 August 2008 Communicated by P. Zhang Available online 29 August 2008
Abstract We study the decision to learn a new use of technology within a post-adoption context. This particular nuance of technology adoption is interesting because while the technology has been adopted at some level by both users and organizations, expanding technology use relies on users adopting additional tools and features within a given system on their own accord. This study addresses how situational goal orientation moderates the effects of ease of learning perceptions within the post-adoption context. We find that when a potential user has a situational learning goal orientation, they indicate intent to learn a new use of technology regardless of whether the technology is perceived to be easy or difficult to learn. However, potential users with a situational performance goal orientation indicate intent to learn the new system feature depending on ease of learning. These results have implications for future research using traditional technology acceptance parameters in the post-adoption context, and provide evidence that situational goal orientation is an effective managerial intervention for use in organizational training. r 2008 Elsevier Ltd. All rights reserved. Keywords: Goal orientation; Ease of use; Human/computer interaction; Technology management
1. Introduction Businesses invest large sums of money in technology (up to 50% of firms’ capital budgets) (Rockart et al., 1996), yet do not see comparable gains in productivity due to lack of full implementation by employees (Devaraj and Kohli, 2003). We study this phenomenon, specifically the decision to learn a new tool or feature within a technology that has already been adopted and used at some level. This context is interesting because while the organizational adoption decision has already been made, individual users have the ability to increase the use to a level that would benefit them and their organizations (Jasperson et al., 2005). We answer a call in the literature by studying the impact of a potential Corresponding author. Tel.: +1 334 844 6203; fax: +1 334 844 5875.
E-mail addresses: [email protected] (T. Loraas), [email protected] (M.C. Diaz). 1 Tel.: +1 225 578 6216. 1071-5819/$ - see front matter r 2008 Elsevier Ltd. All rights reserved. doi:10.1016/j.ijhcs.2008.08.005
managerial intervention on the decision to learn a new use of technology to increase efficiencies on the job (Jasperson et al., 2005). The managerial intervention of interest in this study is situational goal orientation. Specifically, we study whether potential users are more likely to switch from a known means of completing a task to learn a new systems approach based on their situational goal orientation and ease of learning perceptions. The interaction of these constructs is interesting because depending on the situational goal orientation of the potential user, ease of learning perceptions may affect intent to learn a new use of technology contrary to what the Technology Acceptance Model (TAM) would predict (if only the main effects of ease of learning were present). We attribute this interaction to the potential user’s reaction to the possibility of failure. In the context of post-adoption behavior, potential users are faced with the possibility of attempting to learn, but failing instead. This potential failure may result in lost time
ARTICLE IN PRESS T. Loraas, M.C. Diaz / Int. J. Human-Computer Studies 67 (2009) 50–61
and productivity, which are valuable commodities. Consideration of failure in a voluntary, post-adoption environment is important, because potential failure is not likely as a steep barrier in initial adoption or mandatory use phases as ‘‘failure time’’ is likely built into the implementation budget. Within the context of post-adoption behaviors, ‘‘failure’’ time would be costly to the user. Thus, this is a dimension of ease of use that should be considered. When considering the impact potential failure has on the decision to learn a new use of technology, we believe that situational goal orientation will be an effective intervention. Goal orientation theory describes how the type of goals pursued by an individual affects decision making (Nicholls, 1984; Dweck, 1986). The two distinct classes of goals that have been identified are learning goals and performance goals. A learning goal is defined by eagerness to learn for the sake of self-improvement, whereas a performance goal is defined by wanting to appear better (or at least no worse than) one’s peers. The facet of goal orientation that deals with the reaction to failure is what we find to be interesting in the postadoption context. With learning goals, failure is deemed a part of the learning process and, as such, is not feared. However, with performance goals, anything that might jeopardize performance is considered a threat, so failure is feared (Button et al., 1996). In the context of this study, if a potential system user has a situational learning orientation, they should view learning a system or a system feature as a positive self-improvement opportunity and intention to learn will not be as heavily influenced by ease of learning perceptions because fear of failure will be diminished. On the other hand, if potential users have a performance orientation, fear of failure will be heightened, which means that intention to learn will be influenced by ease of learning perceptions as traditionally shown in acceptance and/or adoption studies. This study investigates the efficacy of situational goal orientation as a practical managerial intervention that can motivate users to expand their use of technology to increase effectiveness and efficiencies on the job, even when the technology is deemed difficult to use. 2. Background and hypotheses This research investigates the decision to learn and use a new feature of an existing technology to replace a less efficient means of completing a routine task. This has been described as finding the equilibrium between exploring new possibilities (within a system) and exploiting old certainties (within a system; March, 1991). Most users of technology use a specific subset of available features, and seldom elect to extend this subset on their own (Jasperson et al., 2005). While determining how to encourage more encompassing use of technology on the job is of practical concern (Bowen 1986; Nambisan et al., 1999; Mahmood et al., 2001), there are a limited number of experiments studying this particular nuance of technology adoption.
51
Bhattacherjee (1998) performed an experiment where participants had to complete a task for which they had been trained, but they were given the option to complete the task using a more efficient/effective software solution that they had to learn in order to implement. The participants were more inclined to learn and use the new tool when the reward for doing so was significant. In a more recent study, Loraas and Wolfe (2006) performed an experiment where participants read a vignette that described a situation whereby the hypothetical character was faced with a routine task and had the option of learning a new software to complete that task, thereby also increasing future efficiencies. When the participants had sufficient motivation to follow their referents’ preferences (staff accountants early in their careers) subjective norms dominated the decision of when to learn the new software to complete the task. Our goal is to extend these findings by more closely examining the characteristics of the post-adoption context and to explore the efficacy of situational goal orientation as a managerial intervention to promote system users to learn new uses of technology. 2.1. The post-adoption context The traditional technology adoption context typically focuses on new technologies, whereas the post-adoption context is concerned with users learning new features or functions within a system that has already been adopted by the user at some level (Jasperson et al., 2005). While traditional technology acceptance studies (whether using the TAM, or such more recent adaptations) rely in part on perceptions regarding ease of use to inform a potential user’s intent to use a technology, we believe perceptions regarding ease of learning to be a more appropriate construct for the post-adoption context. Research on the discriminant validity of ease of use and ease of learning has concluded that the two constructs are highly correlated (.79), and are in essence, congruent (Roberts and Moran, 1983; Whiteside et al., 1985). However, we believe these constructs differ in one specific area, and that is the consideration of potential failure. This subtle difference is important because an assumption that is inherent in most technology adoption studies is that once intent to use a technology is formed, there are no impediments to use (Loraas and Wolfe, 2006). However, in the post-adoption context, after forming intent to use, the potential user still has to attempt to learn the technology, and that attempt carries with it the possibility of failure. This potential failure is inversely related to perceptions regarding ease of learning. If ease of learning is perceived to be high, the perceived likelihood that the potential user will fail, if he or she tries to learn, will be low. On the other hand, if perceptions regarding ease of learning are low, the perceived likelihood the potential user will fail, if he or she tries to learn, is high. This potential for failure is especially salient in a voluntary environment, where the potential
ARTICLE IN PRESS 52
T. Loraas, M.C. Diaz / Int. J. Human-Computer Studies 67 (2009) 50–61
user can choose to ‘‘not learn’’ and maintain the status quo at no cost. We believe this aspect of ease of learning to be important in the post-adoption decision to learn a new use of technology. We look to situational goal orientation as a potential managerial intervention that would promote learning of new uses of technology to increase efficiencies on the job because of the aspect of ease of learning that deals with potential failure. 2.2. Goal orientation Goal orientation describes the objective typically undertaken when a person completes a task (Nicholls, 1984; Dweck, 1986). Goal orientation theory defines two predominant orientations: learning and performance (Nicholls, 1984; Dweck, 1986). A learning orientation focuses on the importance of gaining knowledge, whereas a performance orientation focuses on completing tasks correctly (Dweck, 1986). Goal orientation has become an important explicator of behavior in the educational and training literatures (Colquitt and Simmering, 1998; Fisher and Ford, 1998; Chen et al., 2000). Further, a learning goal orientation has been shown to influence perceptions regarding ease of use via self-efficacy in the technology acceptance setting (Yi and Hwang, 2003). While interesting, instead of looking at the a priori affects of dispositional goal orientation, we focus on goal orientation as a managerial intervention to promote technology use after ease of learning perceptions has been formed. While originally conceptualized as a dispositional characteristic, goal orientation has shown efficacy as a situational influence as well. Situational goal orientations are also categorized as learning and performance, and inducing these orientations is commonly done by normative instruction. A learning orientation is manipulated by indicating that the focus of the task is to obtain knowledge, and a performance orientation is manipulated by telling subjects that successful task performance is the overriding objective (Ames and Archer, 1988; Winters and Latham, 1996; Stevens and Gist, 1997; Kozlowski et al., 2001). Compared to performance manipulations, learning manipulations have resulted in higher effort, enhanced challenge seeking, and a predisposition for learning strategies in complex tasks (Ames and Archer, 1988; Winters and Latham, 1996). Further, learning manipulations have been found to contribute to the development of knowledge structure, where performance manipulations did not affect knowledge acquisition (Kozlowski et al., 2001). In the modern era, employees are expected to adapt to new tools and situations (Smith et al., 1997; Kozlowski, 1998). Active, voluntary learning of systems represents an important element of that adaptation. Prior research has shown that a learning orientation is associated with positive effects in terms of motivation to learn and effort expended to learn, while a performance orientation has a negative influence on these elements (Colquitt and Simmering,
1998; Fisher and Ford, 1998). For this reason, all things equal, we would expect that in our post-adoption context, potential users of technology with a situational learning orientation will be more willing to attempt learning a new use of technology than those with a situational performance orientation. In addition to the obvious, there is a component of situational goal orientation that we believe lends itself to the post-adoption context. This is the piece of situational goal orientation that describes how persons react to potential failure. A potential user with a learning goal orientation has adaptive responses to new and/or challenging situations (Kozlowski et al., 2001). Specifically, individuals displaying this orientation treat new and/or challenging situations as opportunities for self-improvement through learning. Failure is viewed as a normal obstacle necessary to the learning process, and is not feared. Alternatively, a potential user with a situational performance orientation is maladaptive, as these potential users will look for situations that ensure success (Kozlowski et al., 2001). These persons seek positive evaluations of their capabilities, and therefore, avoid new and/or challenging situations, due to the possibility of failure. We believe that how potential users react to the possibility of failure drives the interaction between situational goal orientation and perceived ease of learning. 2.3. Interaction of ease of learning perceptions and situational goal orientation The interaction of ease of learning perceptions with situational goal orientation is of particular interest because it offers a way to promote post-adoption use of technology, even when the technology is deemed difficult to learn. Situational goal orientation as a managerial intervention is compelling because it could influence potential nonlearners to attempt learning by diminishing the impact of ease of learning on the intent decision. Considering ease of learning in isolation, if a new feature is deemed difficult to learn, the likelihood that an attempt at learning will result in failure is increased. In a voluntary environment, the potential user will consider this possibility of failure and will be deterred from learning (Loraas and Wolfe, 2006). However, by introducing a situational goal orientation that is focused on learning, the potential users’ view of failure goes from that of a detriment to a normal facet of learning new skills. As a result, fear of failing is diminished, reducing the effect of ease of learning on a potential user’s choice to learn a new tool to complete a routine task. However, if a situational performance orientation that triggers failure avoidance is introduced, ease of learning will be predictive in the choice to learn a new use of technology. This is because the potential user will want to avoid failure by not choosing to learn when the technology is perceived to be difficult. In summary, we study how a potential user’s situational goal orientation moderates the influence of ease of learning
ARTICLE IN PRESS T. Loraas, M.C. Diaz / Int. J. Human-Computer Studies 67 (2009) 50–61
53
Panel A: Complete Model Hypothesis Situational Goal Orientation
Ease of Learning
Intent to Learn
Covariates Risk Preferences Computer Confidence Panel B: Hypothesis Situational Learning Orientation
Intent to Learn
Ease of Learning
Situational Performance Orientation
Intent to Learn
Ease of Learning
Indicates no effect Fig. 1. Research model. Panel A: complete model. Panel B: hypothesis.
perceptions in the decision to learn a new use of technology (see Fig. 1). Specifically, Hypothesis 1a. When perceived ease of learning is low, potential users with a more situational learning orientation will indicate higher intention to learn a new system feature to complete a routine task than potential users with a situational performance orientation. Hypothesis 1b. When perceived ease of learning is high, potential users will indicate higher intention to learn a new system feature, regardless of their situational goal orientation. Situational goal orientation is important for one overriding reason: it offers a managerial intervention that can promote intent to learn even when the system is deemed to be difficult.
3. Methodology We used a laboratory experiment to study whether situational goal orientation differentially influences a potential user to learn a new use of technology depending on ease of learning perceptions. Our participants began the experiment by performing a simple decoding task. Once the participants understood the task and had completed it for pay, they were informed that a feature within a software program known to them could automate the work. Participants were told that the feature was more efficient, and offered higher payoffs (to simulate utility gained from future efficiencies not available in an experimental context), but were also told that they must learn this new feature in a timely manner, and attempting to learn it and failing was costly. This experimental design was rooted in an experimental economics methodology. Experimental economics relies on game-like tasks to produce behavior that parallels behavior in richer real-world settings (Friedman and Sunder, 1994).
ARTICLE IN PRESS 54
T. Loraas, M.C. Diaz / Int. J. Human-Computer Studies 67 (2009) 50–61
An important aspect of this methodology was that our participants completely understood the task, which they had been paid to do, when they made the decision of whether to maintain the status quo or learn a system-based approach to the task. Given that our participants used their native utility functions for decisions in this context, then they should behave similarly in other system adoption decisions of this type, regardless of the task (Chow, 1983). 3.1. Participants Participants consisted of junior-level students enrolled in a 5-year professional program in accountancy at a large university in the southwestern United States. Participants were enrolled in a seminar series when recruited for the experiment, and they were paid and given class credit for participating. Formal discussions with the department chairman and professional program chair indicated that most of these participants (approximately 90%) would take positions with large, international accounting firms. Additionally, this homogeneous pool had taken classes focusing on how to use end-user software; therefore they had experienced learning systems and system elements. Our initial sample consisted of 195 participants. The experimental parameters required the participants to indicate their intent to learn a new use of technology to complete the task; however they did not actually have to learn the new technology during the experiment due to time constraints. To control for contamination across our participant pool we conducted four sessions of the experiment back to back, and had an administrator ensure that during the changeover there was no speaking between participants. Further, we analyzed the findings across the four sessions with no evidence of contamination. 3.2. Experimental design The experimental task environment consisted of decoding numbers into letters in a Microsoft Excel application using paper decoding sheets. Participants began the experiment by performing two short, paid decoding trials. These practice trials allowed participants to become familiar with the decoding task and to experience being paid in the experimental currency, which was redeemed for cash at the conclusion of the experiment.2 After the practice trials, participants were told that there was a final decoding task and that they had 15 min to complete it. Pay for the final decoding task was at its maximum if it was completed in 10 min, a reduced pay was offered for completion in more than 10 min but less than 15 min, and there was no pay for completion of the decoding task after 15 min. Pilot tests had indicated that the final decoding task 2
We used an experimental currency to inflate the amounts paid for successful completion so that comparisons between learning and not learning had greater magnitude.
could be completed in 10 min, and all but four participants did the final decoding task in 10 min or less. (These four participants were dropped from our analyses.3) At this point participants were told that the final decoding task had to be completed five more times. Further, they were told that there was an Excel procedure that could automate the task, but the feature was new to them and they would have to learn it before they could use it.4 Participants were given three options for completing the final five decoding tasks. First, they could simply follow the manual method that they knew and had successfully used; second, they could use 5 min of the first decoding period to try to learn the Excel procedure; or third, they could use the entire 15 min of the first decoding period to try to learn the Excel procedure. Learning the Excel procedure was valuable, and if it was successfully learned and used, the payoff was higher than simply using manual decoding (to simulate current and future efficiencies). On the other hand, it was uncertain that the Excel procedure could be learned in either 5 or 15 min and trying and failing was costly. Participants made choices ‘‘as if’’ they were going to decode the final five decoding tasks—no one actually decoded for five 15 min periods. In essence, participants were supplied with contextualized lotteries that offered a choice among a certainty equivalent (decoding by hand) and two risky options (learning the new procedure). The probabilities (of successfully learning) and payoffs were calculated so that we could determine whether the choices were simply being made via an expected value calculation. Table 1 contains a summary of odds and payoffs for the choice options used in this experiment. We manipulated the payoffs and probabilities to determine whether participants were simply maximizing their expected value by their choices. If so, they should always choose to indicate that they would learn at the highest level (highest expected value). Appendix A contains an example of the wording used. 3.3. Variables 3.3.1. Ease of learning Ease of learning was manipulated by changing the subjective probabilities associated with successfully learning and using the system. Participants were asked their intention to learn if the probability of learning the Excel 3 Within our experimental economics environment it was necessary to drop these participants because they did not complete the task within the necessary time and therefore, for these individuals, completion at the status quo could not be assumed. Therefore, these participants would be making decisions between two uncertainties. 4 Participants were queried as to whether they had a spreadsheet procedure in mind when they made their decisions of whether or not to learn the ‘‘unknown’’ spreadsheet procedure. Two participants listed Vlookup (the tool we had in mind for automating the task), and were dropped from the analysis, as by listing the tool, it was likely the participants were choosing between two certainties.
ARTICLE IN PRESS T. Loraas, M.C. Diaz / Int. J. Human-Computer Studies 67 (2009) 50–61
55
Table 1 Experimental choice options, payoffs, and odds Choice options Success francs
Success odds
Failure francs
Failure odds
Expected value
High ease of learning condition Manual method 3000 5/15 min Learning 8000 15/15 min Learning 7600
1.00 .33 .70
0 2400 1600
0 .67 .30
3000 4248 5800
Low ease of learning condition Manual method 3000 5/15 min Learning 8000 15/15 min Learning 7600
1.00 .20 .40
0 2400 1600
0 .80 .60
3000 3520 4000
Note: Francs were the experimental currency used (exchange rate: 10 Francs ¼ $.05).
procedure in 5 min was 33% (20%) and the probability of learning the Excel procedure in 15 min was 70% (40%), to represent high (low) ease of learning.5 In addition, we gave participants ease of learning manipulations that consisted of either numerical probabilities or textual expressions, e.g., 40% versus ‘‘slightly less than average’’. We observed no statistically significant differences on the dependent variable based on whether the ease of learning intervention was numerical probabilities or textual expressions. Therefore, we combine these treatments for analysis purposes. Further, we had the participants indicate on a scale from one to 11 how they interpreted either the probabilities or textual expressions we provided to them. All participants ranked the ease of use manipulations as we intended. 3.3.2. Situational goal orientation The experiment contained a relatively mild statement designed to induce either a more situational learning orientation or a more situational performance orientation. The participants read one of the following passages before indicating intention to learn: (Learning) In most work environments, knowledge level defines the value of human capital, and that makes learning new things critical. Learning new things leads to self-improvement. So, note in your analysis the importance of learning. Learning more efficient ways to accomplish a task is important. (Performance) Most work environments are time constrained and making the budget is critical. Performance within budget is an essential facet of judging and comparing individuals. So, note in your analysis the importance of making the budget. Performing at budget level is important. 5 Our original intent was for ease of learning to be a within-subject variable. Participants answered the intent question at three levels of ease of learning: high, moderate, and low. Due to order effects, only the first condition viewed by a participant was used in the analysis.
In addition, we had a control group that received neither passage. We adapted a goal commitment instrument from Klein et al. (2001) and used a six-item index to determine participants’ commitment to either a situational goal of learning the new use of technology or performing within budget. We found the situational goal of the participants who received the normative learning to be significantly more learning oriented than the participants who received the normative performance (p-value o.05). 3.3.3. Covariates To help control for potential alternative explanations, we gathered several variables to use as covariates and/or to ensure our participants were appropriate for the task. First, we gathered a measure of computer confidence, which is described as the self-assessed ability to use technology, because it has been found to be predictive of both intent to use and actual use of technology (Loyd and Loyd, 1985; Compeau and Higgins, 1995; Venkatesh et al., 2003).6 In addition to controlling for perceived ability, we also gathered measures to control for actual ability. These measures consisted of participant reported cumulative grade point average, systems classes grade point average, and number of systems classes taken (see Table 2 for all demographics). As the context of this study is post-adoption behavior, it implies that the potential user has experience with the system in question, in this case, Excel. To ensure that the participants had Excel knowledge and could make a postadoption decision, all participants took Excel spreadsheet quizzes. On average, the participants correctly answered 75% of the questions. This indicates that the decision can be considered to be post-adoption for the participants in our study. Finally, because some participants encountered ease of learning point estimates that were based on 6 The authors acknowledge that collecting the dispositional goal orientations of the participants would have strengthened the findings. However, Yi and Hwang (2003) find that self-efficacy controls, at least in part, for dispositional learning orientations.
ARTICLE IN PRESS T. Loraas, M.C. Diaz / Int. J. Human-Computer Studies 67 (2009) 50–61
56
4. Results
Table 2 Descriptive statistics Normative situational goala Variables
Learning Control Performance F-stat n ¼ 65 n ¼ 61 n ¼ 63
Age Class Genderb Computer courses Computer course GPA Overall GPA Excel quiz score (8 max.) Situational goal orientationc Computer confidencee Risk preferencesf
20.63 3.05 .36 1.42 3.74 3.58 6.58 21.86 34.24 14.98
20.69 3.11 .44 1.53 3.64 3.59 6.62 22.85 32.64 15.82
20.69 3.10 .37 1.56 3.80 3.60 6.67 23.80 33.00 15.43
.133 .374 .594 .544 .152 .892 .902 .271d 2.358 .912
a Each of these cells has participants who answered either high or low ease of use intent questions. Means did not differ between high and low ease cells. Thus for parsimony we collapse the cells for presentation of the descriptive statistics. b Scored as a dummy variable, where 1 ¼ male, 0 ¼ female; 39% of our participants were male. c Sum score of 6-item, 7-point Likert scale. Higher scores indicate a more performance orientation. d A contrast was conducted to determine significant differences between participants given a situational goal of learning versus performance. The contrast was significant (t-statistic 1.674, p-value o.05 (one tailed)). e Sum score of 10-item, 5-point Likert scale. Higher scores indicate more confidence. f Sum score of 6-item, 5-point Likert scale. Higher scores indicate more risk averse.
probabilities, we include a measure for risk preferences to control for choices made as a ‘‘gambler’’. 3.3.4. Intent Our dependent variable, intent, is both categorical and ordered in that the options represent an increasing intention to learn the feature. (Manual decoding represents no intent to learn, spending 5 of 15 min on learning is the mid-level intention, and spending 15 of 15 min on learning indicates the highest intention to learn.) Therefore, we use an ordered logit to estimate model relationships (Long, 1997). The underlying functional model used to test our hypothesis takes the following form: Intent ¼ f ðease of learning; situational goal orientation; computer confidence; risk preferences, ease of learning by situational goal orientationÞ
(1)
Ease of learning is a dummy variable with the value of one assigned to high ease of learning (zero is low ease of learning). Situational goal orientation is a sum score, with higher (lower) scores indicating a higher performance (learning) situational goal orientation. Computer confidence and risk preferences are covariates. Although we collected other variables, due to lack of variability, those measures added no value to the final model and were therefore excluded.
All the items from the Willingness to Take Risks scale (Keil, 1999), Computer Confidence subscale of the Computer Attitude scale (Loyd and Loyd, 1985; Al-Jabri and Al-Khaldi, 1997), and Goal Commitment scale (Klein et al., 2001) were pooled and factor analyzed using principal axis factoring and varimax rotation using SPSS 15.0 for Windows. Two computer confidence questions produced loadings below .35 and were dropped. The correlation between the computer confidence factor and the risk preference factor is .218. Computer confidence has a Cronbach’s alpha of .854, risk preferences has an alpha of .748, and goal commitment has an alpha of .844. Our analyses are supportive of good discriminant and convergent validity for the computer confidence, risk preference, and goal commitment constructs. Sum scores are used in all analyses. Factor analysis results, including cross-loadings, are shown in Appendix B. Our major premise is that situational goal orientation moderates ease of learning perceptions. Table 3 reports the results, and Fig. 2 graphically represents the same. As predicted, we find an interaction between ease of learning and situational goal orientation. Strictly interpreting the interaction coefficient (.102, p-value .022), when ease of learning was high, the more performance oriented our participants were, the more likely they were to choose a higher level of commitment to learn the technology. However, the odds ratio of 1.10 indicates the effect size is minimal, as depicted graphically in Fig. 2.7 The coefficient of the main effect of goal orientation (.078, p-value .014) indicates that when ease of learning was low, participants with a learning orientation were more likely to commit to a higher level of learning the new use of technology than participants with a performance orientation. Simply put, we find that ease of learning matters when a potential user has a situational performance orientation. In this situation, ease of learning impacts the commitment to learning the new use of technology just as the TAM would predict; the harder the technology is perceived to be to learn, the less intent the potential user has to learn it. However, when a potential user has a situational learning orientation, ease of learning does not seem to sway the decision, and the potential user indicates intention to learn and use the new use of technology at a fairly constant level (see Fig. 2). This means that when a task is complex, a situational learning orientation can promote leaving the status quo to learn a more efficient way to complete a task even when the more efficient way is deemed difficult. Interestingly, we also find that when a task is easy to learn, a performance orientation can promote learning at least as effectively as a learning orientation.
7
In a logistical analysis, the odds ratio is an indicator of effect size, with an odds ratio of 1.00 indicating no change in likelihood based on the independent variable (Garson, 2008).
ARTICLE IN PRESS T. Loraas, M.C. Diaz / Int. J. Human-Computer Studies 67 (2009) 50–61
We studied a specific class of technology acceptance defined as the choice to voluntarily learn a new system feature to complete a task routinely completed in a different manner. While highly stylized, our scenario is both important and common in that it defines the learning and use of many system features in a post-adoption environment. End-user software, such as the standard suite of office products, contains features that are often self-taught, and done so only when they are needed. Financial reporting system users can choose to learn and use the report writing tools that they know exist in accounting and ERP applications, or they can follow a simple, laborious process of downloading the data to a spreadsheet. 5.1. Implications for research The technology acceptance paradigm is rich and well developed; however, we contend that the majority of that research stream focuses on the initial adoption context. We extend the literature by investigating the traditional technology acceptance parameter, ease of use, in the post-adoption context. We believe that this context has barriers to adoption that are not as salient in the initial adoption setting, which then impacts how this traditional predictor interacts with situational goal orientation, which, in turn, influences the adoption/acceptance decision. While we recognize that TAM parameters as traditionally studied have offered great insight as to the technology acceptance decision, by narrowing the focus of these general studies to more specific contexts, we can begin to more fully understand behavior with respect to learning and using technology (Sun and Zhang, 2006). While we focus on how the potential for failure acts as an additional facet of ease of use that contributes to postadoption behavior, we acknowledge that different dimensions of constructs within the technology acceptance paradigm may also be contributing and are worthy of additional study. For example, future research might consider perceptions regarding perceived usefulness. While perceived usefulness has traditionally focused on the utility gained from using the technology, usefulness may also incorporate other cultural or organizational factors. As in our scenario, a message from a supervisor that learning is important may work together with actual usefulness of the technology. Not only will the potential user gain utility from the technology itself, but he/she may gain utility from following a supervisor suggestion that learning the technology is important. On the other hand, when faced with a situational performance-oriented supervisor, the usefulness of learning the technology may be discounted. In this case, perceived usefulness of the technology may be reduced sufficiently to deter learning. Subjective norms may also be a construct worthy of additional study. Recent research provides evidence that
staff accountants are likely to follow supervisor suggestions due to their motivation to comply with said referents’ beliefs. While we do not find that our participants acted totally in line with the situational goal orientation presented to them, the finding that the situational goal orientation was influential does provide support that subjective norms can impact the decision to learn a new use of technology in a voluntary setting, in direct contrast with earlier work focused on subjective norms (Legris et al., 2003; Venkatesh et al., 2003; Schepers and Wetzels, 2007). However, our finding could be an artifact of the situational goal presented; in other words, the message of the supervisor may cloud the potential user’s perception of
Table 3 Ordered logistic regression Variables
Coefficient
Wald
p-valuea
Ease of learningb Risk preferencesc Computer confidenced Situational goal orientatione Ease Situational goal orientation Pseudo R2 Sample size
2.076 .009 .008 .078
3.897 .038 .054 6.017
.048 .846 .816 .014
.102
5.217
.022
Odds ratio .125 .991 .992 .925 1.11
.04 189
The dependent variables were defined as follows: 1—manual method; 2—5/15 min learning; 3—15/15 min learning. a Two-tailed p-values. b Ease of learning is a between-subjects variable coded one (1) when the system feature is easy to learn and zero (0) when it is harder to learn. c Sum score of 6-item, 5-point Likert scale. Higher scores indicate more risk aversion. d Sum score of 10-item, 5-point Likert scale. Higher scores indicate more confidence. e Sum score of 6-item, 7-point Likert scale. Higher scores indicate more performance orientation.
3 2.8 2.6 Intention to Learn
5. Discussion
57
Learning Performance
2.4 2.2 2 1.8 1.6 1.4 1.2 1 Low Ease
High Ease
Fig. 2. Interaction of ease of learning and situational goal orientation.
ARTICLE IN PRESS 58
T. Loraas, M.C. Diaz / Int. J. Human-Computer Studies 67 (2009) 50–61
voluntariness. It could be that a supervisor suggestion that learning is important leaves the potential user with the perception that learning the technology is, in essence, mandatory. 5.2. Implications for practice We find that situational goal orientation matters. Specifically, participants who are more committed to a learning goal are more likely to learn a new use of technology, even when the new technology is determined to be difficult to learn. Further, this situational goal orientation can be influenced via a simple supervisor instruction. This presents management with a simple and inexpensive intervention to influence employees to learn new technologies, even when deemed difficult to learn. It should be noted that participants committed to a more performance-oriented goal did not intend to learn when ease of learning was low. Performance orientation is described in two dimensions, wanting to appear better than peers do and not appearing incompetent (Elliott and Dweck, 1988). Research has indicated that the performance orientation can be split based on these dimensions into performance approach orientation (look better) and performance avoidance orientation (not incompetent; Middleton and Midgley, 1997). Our results support the notion of a performance approach orientation. Individuals with a performance approach orientation will try to learn, as long as the technology is deemed easy to use. Although learning is not their goal (as it is with those with a learning orientation), they intend to learn in order to meet their goal of looking better than others. This suggests that an organizational culture that creates competition between employees may deter learning new uses of technology when failure may be likely, as employees will be unwilling to look bad in comparison to others. We found participants willing to give up large incremental expected payoffs from learning and using a new use of technology to return to the security of a known task completion method. In the end, this decision is costly to the employee and to the firm. Our findings suggest that managers who universally espouse performance can modify an employee’s situational goal orientation and as such may unwittingly curtail the learning of difficult, yet beneficial new software solutions by their employees. On the other hand, managers who can nurture and/or promote learning orientations should observe increased productivity over the long term. 6. Conclusion We study a mechanism that promotes users of technology to learn new uses of an established system. We investigated the efficacy of situational goal orientation as a managerial intervention that could either promote or dissuade potential users from learning a new use of technology to complete a routine task. We found that
individuals primed for learning choose to learn without regard to perceived ease of learning. However, potential users primed with a performance orientation indicated intention to learn depending on ease of learning as would be expected, as ease of learning increased (decreased), intent to learn increased (decreased). It should be noted that the impact of our normative guidance on situational goals, although statistically significant, did not appear to be very strong. That being said, we do find that the situational goal of the potential user influences intent to learn when ease of learning was low. This suggests that looking for more effective ways to modify situational goals may be an effective avenue for future research. However, in all likelihood, in a work environment, a message from a supervisor is likely to have a much greater effect on situational goals than a simple paragraph in an experiment. Another possible explanation for the seemingly small effect size is that the scale used to capture the situational goal of the participant likely also captured elements of the participants’ goal disposition. We believe that there is evidence of this occurring, as the mean of the scale was the scale midpoint with little variance. (If disposition were not captured, we would expect a larger dispersion in scores.) Since we did not measure the participants’ dispositional goal orientation prior to the experiment we cannot say conclusively. However, we did randomize, so a priori dispositional goal orientation effects should be equal across treatments. As in all work of this type, our study has limitations. Our results are dependent on the parameters of the experimental task used. We used point estimates for both system value and ease of learning. Different estimates could lead to different decision outcomes. However, we did find that using contextual expressions in lieu of point estimates did not change behavior. Additionally, participants were required to make a decision in a single period, and did not have the opportunity to ‘‘change their mind’’. As such, we test a single period decision model. Finally, our experimental task setting is not as rich as a real-world decision context, and actual system usage was never required or observed in the experiment. However, there are a number of studies that indicate intention is a predictor of behavior, which suggests that our measure of intent is meaningful (Dholakia and Bagozzi, 2002).
Acknowledgements The authors wish to thank Chris Wolfe, Elaine Mauldin, Tammy Kowalczyk, Dan Stone, Brad Tuttle, Patrick Wheeler, and Ed O’Donnell for comments on earlier drafts of this research. This paper has also benefited from comments made by workshop participants at the 2003 AAA Annual Meeting, 2003 Accounting Information Systems Research Symposium, and the Management Information Systems Department at Texas A&M University. We also want to thank Don Warren, Kun Wang,
ARTICLE IN PRESS T. Loraas, M.C. Diaz / Int. J. Human-Computer Studies 67 (2009) 50–61
Davit Adut, and Ryan Huston for help administering this experiment. Appendix A. Instrument for collection of subject choices The instrument shown below is for ease of use order highto-low and for high payoffs. Assume the same parameters as the last task (15 min task period) and your budget is still set at 175 correctly decoded items. The pay for reaching the budget is 400 francs; and for completing the budget with 5 min to spare (out of 15 min), there is a leisure pay of 200 francs. In total, 600 francs can be earned by decoding 175 items in 10 min or less, or 400 francs can be earned by decoding 175 items in 15 min or less. However, if in 15 min, 175 items are not decoded, the pay is zero. The payoffs are summarized in the table below: Items decoded
Time (min)
Pay
175 175 Less than 175
Under 10 Over 10 and under 15 Over 15
600 400 0
Assume that you have five more paid decoding sessions (i.e., five more 15 min decoding sessions). However, it has come to your attention that there is a set of Excel procedures that can automate the decoding task, but the Excel procedures are new to you and must be learned. Payoffs for the different choices are as follows. Decode by hand You can choose not to try to learn the Excel procedures, manually decode in each session, and earn 600 francs per session by decoding 175 items in 10 min. Five minute learning option If the Excel procedures are learned in 5 min, you will lose your leisure pay in Session one. However in the remaining four sessions your leisure pay will increase to 1500 francs per period and your budget pay will remain at 400 francs per period (i.e., the last four sessions are worth 1900 francs each). Since you are using only 5 min to learn in Session one you will have enough time left over to decode the budgeted 175 items. Thus you will still earn 400 francs in Session one. If you try to learn for 5 min and do not learn the Excel procedures, you will receive no pay for Session one, but you will be able to resort to manual decoding and earn 600 francs for each of the remaining four sessions by decoding 175 items in 10 min.
59
Fifteen minute learning option If the Excel procedures are learned in 15 min, you will not earn any pay in Session one. In the remaining four sessions however your leisure pay will increase to 1500 francs per period and your budget pay will remain at 400 francs per period (i.e., the last four sessions are worth 1900 francs each). If you try for 15 min and do not learn the Excel procedures, you will receive no pay for Session one, and you will not receive any leisure pay for the remaining four sessions regardless of how quickly you can decode by hand (i.e., the last four sessions are worth 400 francs each). The table given above was followed by three queries for participant choices, only one of which is shown (low ease of learning) due to space limitations. Francs converted into US dollars at the rate of 10 Francs ¼ $0.05. Assume the following payoffs are in effect for each of the three options across the last five sessions: Decoded by handItems decoded
Speed(min)
Total pay (all five periods)
175 175
Under 10 Over 10 under 15 Over 15
3000 2000
Less than 175
0
Spend 5 min learning Excel procedures
Total pay (all five periods)
Succeed in learning them Do not learn them
8000 2400
Spend 15 min learning Excel procedures
Total pay (all five periods)
Succeed in learning them Do not learn them
7600 1600
Using the payoffs described, which option would you select? Assume that after looking at the Excel procedures you assess the probability of learning the Excel procedures in 5 min at 20% and the probability of learning the Excel procedures in 15 min at 40%? Would you
a. attempt to learn in 5 min? b. attempt to learn in 15 min? c. decode by hand?
ARTICLE IN PRESS T. Loraas, M.C. Diaz / Int. J. Human-Computer Studies 67 (2009) 50–61
60
Appendix B. Factor analysis and construct validity (indicators are bolded) Item
Interpretation
Risk preferences
Computer confidence
Goal commitment
Riska_Q1 Risk_Q2 Risk_Q3 Risk_Q4 Risk_Q5 Risk_Q6 Confid.b_Q1 Confid._Q2 Confid._Q3 Confid._Q4 Confid._Q5 Confid._Q6 Confid._Q7 Confid._Q8 Confid._Q9 Confid._Q10 Goalc Q1 Goal Q2 Goal Q3 Goal Q4 Goal Q5 Goal Q6
Take risks choosing a new job I prefer a high security job Trade unknown problems for high reward Avoid risky situations at all costs Play it safe even if opportunity lost I am cautious and generally avoid risk No good with computers. Feel OK trying a new computer Would do advanced computer work I could do work with computers Not the type to do well with computers Could learn a computer language Computer work would be hard for me Could get good grades in computer courses Cannot handle a computer course I have self-confidence for computer work Important to shoot fory Most important not to abandony Unrealistic to take perspective ofy Strongly committed toy Did not matter to me if Iy Effort best used toy
.637 .497 .714 .658 .450 .466 .051 .067 .131 .028 .135 .227 .094 Dropped Dropped .097 .080 .046 .101 .055 .026 .118
.058 .286 .064 .047 .232 .307 .705 .771 .743 .624 .541 .463 .461 Dropped Dropped .771 .168 .020 .062 .005 .022 .068
.080 .046 .101 .055 .026 .118 .002 .034 .057 .017 .046 .109 .088 Dropped Dropped .047 .790 .555 .533 .833 .639 .763
a
Willingness to Take Risks scale (5-point scale, anchored with ‘‘Strongly Agree’’ and ‘‘Strongly Disagree’’). Computer Confidence scale (5-point scale anchored with ‘‘Strongly Agree’’ and ‘‘Strongly Disagree’’). c Goal Commitment scale (7-point scale anchored with ‘‘Learning New Things’’ and ‘‘Performing within Budget’’). b
Reliability and discriminant validity constructs
Risk Preferences Computer Confidence Goal Commitment a
Cronbach’s a
Risk Preferences
Computer Confidence
.748a .854b .844b
.218a .179b
.119a
Square root of the shared variance of the elements measuring a factor. Correlation of sum scores used in analysis.
b
References Al-Jabri, I.M., Al-Khaldi, M.A., 1997. Effects of user characteristics on computer attitudes among undergraduate business students. Journal of End User Computing, 16–22. Ames, C., Archer, J., 1988. Achievement goals in the classroom: students’ learning strategies and motivation processes. Journal of Educational Psychology September, 260–267. Bhattacherjee, A., 1998. Managerial influences on intraorganizational information technology use: a principal-agent model. Decision Sciences 29, 139–162. Bowen, D., 1986. The puny payoff from office computers. Fortune, 22–26. Button, S., Mathieu, J., Zajac, D., 1996. Goal orientation in organizational research: a conceptual and empirical foundation. Organizational Behavior and Human Decision Processes 67, 26–48. Chen, G., Gully, S., Whiteman, J., Kilcullen, R., 2000. Examination of relationships among trait-like individual differences, state-like individual differences, and learning performance. Journal of Applied Psychology, 835–847.
Chow, C., 1983. The effects of job standard tightness and compensation scheme on performance: an exploration of linkages. The Accounting Review, 667–685. Colquitt, J., Simmering, M., 1998. Conscientiousness, goal orientation, and motivation to learn during the learning process: a longitudinal study. Journal of Applied Psychology, 654–665. Compeau, D.R., Higgins, C.A., 1995. Computer self-efficacy-development of a measure and initial test. MIS Quarterly 19, 189–211. Devaraj, S., Kohli, R., 2003. Performance impacts of information technology: is actual usage the missing link? Management Science 49, 273–289. Dholakia, U.M., Bagozzi, R.P., 2002. Mustering motivation to enact decisions: how decision process characteristics influence goal realization. Journal of Behavioral Decision Making 15, 167–188. Dweck, C., 1986. Motivational processes affecting learning. American Psychologist 41, 1040–1048. Elliott, E.S., Dweck, C.S., 1988. Goals: an approach to motivation and achievement. Journal of Personality and Social Psychology, 5–12. Fisher, S., Ford, J., 1998. Differential effects of learner effort and goal orientation on two learning outcomes. Personnel Psychology, 397–420.
ARTICLE IN PRESS T. Loraas, M.C. Diaz / Int. J. Human-Computer Studies 67 (2009) 50–61 Friedman, D., Sunder, S., 1994. Experimental Methods: A Primer for Economists. Cambridge University Press, New York. Garson, G.D., 2008. Logistic regression. In: Statnotes: Topics in Multivariate Analysis. Retrieved February 20, 2008, from /http://www2. chass.ncsu.edu/garson/pa765/statnote.htmS. Jasperson, J.S., Carter, P.E., Zmud, R.W., 2005. A comprehensive conceptualization of post-adoptive behaviors associated with information technology enabled work systems. MIS Quarterly 29, 525–557. Keil, J.W., 1999. Compensation risk preferences: self-reported information vs. experimental evidence. ACA Journal, 67–71. Klein, H.J., Wesson, M.J., Hollenbeck, J.R., Wright, P.M., DeShon, R.P., 2001. The assessment of goal commitment: a measurement model meta-analysis. Organizational Behavior and Human Decision Processes 85, 32–55. Kozlowski, S., 1998. Training and developing adaptive teams: theory, principles, and research. In: Cannon-Bowers, J.A., Salas, E. (Eds.), Decision Making Under Stress: Implications for Training and Simulation. APA Books, Washington, DC. Kozlowski, S., Gully, S., Brown, K., Salas, E., Smith, E., Nason, E., 2001. Effects of training goals and goal orientation traits on multidimensional training outcomes and performance adaptability. Organizational Behavior and Human Decision Processes 85, 1–31. Legris, P., Ingham, J., Collerette, P., 2003. Why do people use information technology? A critical review of the technology acceptance model. Information and Management 40, 191–204. Long, S., 1997. Regression Models for Categorical and Limited Dependent Variables. Sage Publications, Thousand Oaks, CA. Loraas, T., Wolfe, C., 2006. Why wait? Modeling factors that influence the decision of when to learn a new use of technology. Journal of Information Systems 20, 1–24. Loyd, B.H., Loyd, D.E., 1985. The reliability and validity of an instrument for the assessment of computer attitudes. Educational and Psychological Measurement, 903–908. Mahmood, M.A., Hall, L., Swanberg, D.L., 2001. Factors affecting information technology usage: a meta-analysis of the empirical literature. Journal of Organizational Computing and Electronic Commerce 11, 107–130. March, J.G., 1991. Exploration and exploitation in organizational learning. Organization Science 2, 71–87.
61
Middleton, M., Midgley, C., 1997. Avoiding the demonstration of lack of ability: an under-explored aspect of goal theory. Journal of Educational Psychology, 514–523. Nambisan, S., Agarwal, R., Tanniru, M., 1999. Organizational mechanisms for enhancing user innovation in information technology. MIS Quarterly 23, 365–395. Nicholls, J.G., 1984. Achievement motivation: conceptions of ability, subjective experience, task choice, and performance. Psychological Review, 328–346. Roberts, T.L., Moran, T.P. (Eds.), 1983. The evaluation of text editors: methodology and empirical results, Communications of the ACM 26, 265–283. Rockart, J.F., Earl, M.J., Ross, J.W., 1996. Eight imperatives for the new IT organization. MIT Sloan Management Review 38, 43–55. Schepers, J., Wetzels, M., 2007. A meta-analysis of the technology acceptance model: investigating subjective norm and moderation effects. Information and Management 44, 90–103. Smith, E.M., Ford, J.K., Kozlowski, S.W., 1997. Building adaptive expertise: implications for training design. In: Quinones, M.A., Dudda, A. (Eds.), Training for the 21st Century Technology: Applications of Psychological Research. APA Books, Washington, DC. Stevens, C., Gist, M., 1997. Effects of self-efficacy and goal-orientation training on negotiation skill maintenance: what are the mechanisms? Personnel Psychology 50, 955–978. Sun, H., Zhang, P., 2006. The role of moderating factors in user technology acceptance. International Journal of Human–Computer Studies 64, 53–78. Venkatesh, V., Morris, M.G., Davis, F.D., Davis, G.B., 2003. User acceptance of information technology: toward a unified view. MIS Quarterly 27, 425–478. Whiteside, J., Jones, S., Levy, P.S., Wixon, D., 1985. User performance with command, menu and iconic interfaces. In: Proceedings of CHI ’85, ACM, San Fransisco. Winters, D., Latham, G., 1996. The effect of learning versus outcome goal on a simple versus complex task. Group and Organization Management, 236–250. Yi, M.Y., Hwang, Y., 2003. Predicting the use of web-based information systems: self-efficacy, enjoyment, learning goal orientation and the technology acceptance model. International Journal of Human– Computer Studies 59, 431–449.