- Email: [email protected]
Influences of cognitive style and training method on training effectiveness
Computers & Education 37 (2001) 11–25 www.elsevier.com/locate/compedu
Influences of cognitive style and training method on training effectiveness Huey-Wen Chou * Department of Information Management, National Central University, 38, Wu-Chuan Li, Chungli, Taiwan 32054, People’s Republic of China Received 1 April 2000; accepted 1 February 2001
Abstract The present study compares the relative effects of cognitive style and training method on learners’ computer self-efficacy and learning performance by a field experiment. The purpose was to determine which training method could be best utilized in computer-related training while taking trainees’ cognitive style into account. The significant two- and three-way interactions indicate the critical roles that personal characteristics and situation factors play as joint determinants of behavior. Gender did significantly moderate the effects of training method on performance and self-efficacy. The cognitive by training method effects were most significant for female participants. This finding suggests that to assist individuals taking IT-related training courses, the contingency effects of gender, cognitive style, training approach, and training objective should be taken into account. # 2001 Elsevier Science Ltd. All rights reserved. Keywords: Country-specific developments; Gender studies; Improving classroom teaching; Secondary education
As information technology (IT) becomes more critical to business operation, computer literacy becomes a requisite for employees. Although training has been widely recognized as a critical contributor to successful systems implementation and end user computing (Grover & Teng, 1994), the broad diversity of individual differences among potential trainees should be taken into account when developing the training program (Chou & Wang, 1999; Liu & Reed, 1994). In addition, while many approaches for developing computer skills are available, little research has systematically examined the relative merits of various training programs. The current study addresses these concerns. This paper focuses on three objectives: (1) to develop a conceptual model to evaluate how training method and cognitive style influence learning performance and computer self-efficacy; (2) * Corresponding author. Tel.: +886-3-426-7256; fax: +886-3-425-4604. E-mail address: [email protected] (H.-W. Chou). 0360-1315/01/$ - see front matter # 2001 Elsevier Science Ltd. All rights reserved. PII: S0360-1315(01)00028-8
12
H.-W. Chou / Computers & Education 37 (2001) 11–25
to investigate whether individuals with different cognitive styles perform differently in two training conditions, that is, assessing the feasibility of a contingency approach to training on learning performance and on computer self-efficacy; and (3) to investigate the gender-moderating effects on the causal relationship between cognitive style and/or training method, and learning performance and computer self-efficacy. The remainder of this paper is organized as follows. Section 1 reviews the literature on the variables included in the proposed model, followed by the theoretical framework and hypotheses for the model. Section 2 presents the research methodology including participants, experimental design, procedure, and training intervention. Section 3 describes the data analysis and corresponding results. Section 4 provides summaries of the study findings, study limitations, and recommendations for future research.
1. Literature review 1.1. Self-efficacy Self-efficacy emanates from the Social Cognitive Theory (SCT; Bandura, 1986) and is defined as the beliefs about one’s ability to perform a particular behavior, which affects choices about which behaviors to undertake, and the effort and persistence exerted in the face of obstacles to the performance of those behaviors. Self-efficacy perceptions influence an individual’s actual ability to perform the behavior. Gist, Schwoerer, and Rosen (1989) and Gist, Stevens, and Bavetta (1991) confirmed a positive relationship between self-efficacy and achievement. They also suggested that initial computer self-efficacy moderated the effect of training method on training outcome. Self-efficacy was treated as both a dependent variable and an independent variable in the computer-training literature. 1.1.1. Self-efficacy as an independent variable Igbaria and Iivari’s (1995) study found that self-efficacy had both direct and indirect effects on computer usage. Compeau, Higgins, and Huff (1999) conducted a longitudinal study to test an SCT-based model, which indicated a significant relationship between self-efficacy and computer usage. Coffin and MacIntyre (1999) found that self-efficacy had a significant effect on learning performance in a programming course. Compeau and Higgins (1995) found that computer selfefficacy exerted a strong influence on performance in learning Lotus 1-2-3 and WordPerfect. Christoph, Schoenfeld, and Tansky’s (1998) study on the influence of self-efficacy on multimediabased training receptiveness indicated that training effectiveness was determined partly by the trainees’ self-efficacy. 1.1.2. Self-efficacy as a dependent variable Self-efficacy was also treated as a dependent variable in the literature. Martocchio (1994) found age and outcome expectation significantly affect self-efficacy. Torkzadeh and Koufteros (1994) found a significant training effect on self-efficacy. In Compeau and Higgins’ (1995) study, they confirmed that the behavior-modeling approach was more effective for training in Lotus 1-2-3 and resulted in a higher computer self-efficacy.
H.-W. Chou / Computers & Education 37 (2001) 11–25
13
The main purpose of the present study was to determine whether different training methods and/or participants’ cognitive style would impact their computer learning performance and computer self-efficacy. Self-efficacy was therefore employed as a dependent variable. 1.2. Training methods Many training formats are reviewed in the computer training literature, such as lecture, exploration-based, tutorial, and computer-assisted instruction (CAI). Nevertheless, little research has systematically examined the relative merits of various training programs. Studies suggested that the choice of training method has consequences for the degree of learning (Webster & Martocchio, 1993) as well as for self-efficacy (Compeau & Higgins, 1995; Gist et al., 1989). In the present study, the instruction-based method was chosen because it represents a more traditional approach and is appropriate for almost all training needs. Behavior modeling was included because of the work of Baldwin (1992), Compeau and Higgins (1995), Gist et al. (1989), Olfman and Mandviwalla (1994), Simon and Werner (1996), and Simon, Grover, Teng, and Whitcomb (1996) on computer training. The instruction-based method offers a traditional approach that is appropriate for almost all training. This approach teaches primarily by lecture and follows a deductive way to learning, where learners proceed from general rules to specific examples (Chou & Wang, 2001). Overall, the literature suggests that the instructional technique is superior for retention of information. Behavior modeling is a training process developed in the 1970s for building an individual’s skill in the context of managerial skill training. This task-focused method involves a visual observation of the behaviors of a model performing a task. Learners then imitate and extend the model’s behavior in practice and experimentation to master the task. The behavior-modeling method employs an inductive approach that teaches first by hands-on demonstrations, followed by complementary lectures. Bandura (1986) suggested that by observing someone performing the target behavior, the participants’ self-efficacy would be raised. In a field experiment, Gist et al. (1989) compared alternative training methods on self-efficacy and computer software learning. In the behavior-modeling group, video modeling served as the principal means of instruction, whereas in the tutorial training group, a one-on-one interactive tutorial instruction on the computer monitor was the primary means of instruction. After each step was demonstrated, the trainees in the behavior-modeling group were asked to execute the step. On the other hand, the participants in the tutorial condition were only told what to do; they were not asked to execute specific tasks. The results indicate the behavior-modeling approach relative to a computer-aided instructional approach yielded higher self-efficacy and performance scores in learning computer software. Compeau and Higgins (1995) examined the relative effectiveness of behavior-modeling and traditional lecture-based programs on the learners’ performance and self-efficacy. They found that the behavior-modeling approach was more effective than the lecture-based approach in learning Lotus 1-2-3. Chou and Wang (2001) confirmed the superiority of the behavior-modeling approach to the instruction-based approach on computer self-efficacy and learning performance. Simon et al. (1996) and Simon and Werner (1996) compared three training techniques — instruction, self-paced, and behavior modelin — with a no-training control group. Findings of
14
H.-W. Chou / Computers & Education 37 (2001) 11–25
both studies indicate that the behavior-modeling approach generates the highest scores in cognitive learning, skill demonstration, and satisfaction with the computer system. H1. Participants in the behavior modeling group will score significantly higher on learning
performance and computer self- efficacy measures than participants in the instruction-based group. 1.3. Cognitive style Cognitive styles define learner traits. The cognitive style refers to the way individuals collect, analyze, evaluate, and interpret data (Harrison & Rainer, 1992). The study of individual differences in cognition and their impact on learning and instruction has long been a focus in cognitive psychology and education. Witkin’s (1964) research on how people separate one factor from the total visual field has become the field-independence/dependence theory, which is also one of the most extensively researched cognitive styles. Witkin, Moore, Goodenough, and Cox (1977) defined field independence as ‘‘the extent to which a person perceives part of a field as discrete from the surrounding field as a whole, rather than embedded in the field; or . . . the extent to which the person perceives analytically’’ (p. 7). According to Witkin and Goodenough (1981), field-independent (FI) people are more likely to do well with numbers, science, and problem-solving tasks. They tend to analytically approach a problem and perceive a particular and relevant item in a field of distracting items. On the other hand, field-dependent (FD) people tend to be better at recalling such social information as conversations and relationships. They prefer to approach a problem in a more global way and are capable of perceiving the total picture in a situation. The practical implications of FI/FD cognitive style for education have indicated that the individuals’ different cognitive styles bear direct impact upon their achievement performance (Tinajero & Pa´ramo, 1997; Wieseman, Portis, & Simpson, 1992). Witkin and Goodenough (1981) suggested that FI relates to academic progress in some specialized courses. Several studies have suggested that more FI students tend to academically outperform FD students. For instance, Pa´ramo and Tinajero’s (1990) study showed that FI people tend to outperform FD people in overall school performance. Moore and Dwyer (1992) conducted a study that examined the effect of B&W- and color-coding of information. They found that FI students, on the whole, scored significantly higher than FD students in both visually and verbally oriented tests. In addition, significant performance differences were found between FI and FD students in the B&W-coded treatment but not in the color-coded treatment. The results also showed that external support could be important for FD people in processing information. Liu and Reed (1994) found that participants of different cognitive styles employed different learning strategies in accomplishing the same task in a hypermedia environment. H2. Participants with FI cognitive style will score significantly higher on learning performance
and computer self-efficacy measures than participants with FD cognitive style. In addition to the direct effects of cognitive style on achievement performance, the relationship between cognitive style and training approach has been studied. Studies that have attempted to
H.-W. Chou / Computers & Education 37 (2001) 11–25
15
match situational factors to learning styles have only inconsistently found any benefits for learning (Knight, Halpin, & Halpin, 1992; Snow & Swanson, 1992). However, the search for such aptitude-treatment interaction goes on, and a few studies have found positive effects for programs that adapt instruction to an individual’s learning style (Chou & Wang, 1999; Lipsky, 1989). Lipsky (1989) found FD participants performed better with the outlining technique, and FI students performed better with the mapping technique. The above results confirmed that one instructional method suitable for FI learners might not necessarily be beneficial for FD learners. Learners with different cognitive styles pursue quite different ways of learning. Snow (1991) proposed that learners differ profoundly in what they do in learning and in their success in any particular learning situation. The present study employs the interactional psychology perspective that emphasizes the importance of considering both the person characteristics (cognitive style) and the situation factor (training method) as joint eterminants of behavior (Lee, Kim, & Lee, 1995). H3. FD participants will score significantly higher on learning performance and computer
self-efficacy with the behavior-modeling method, whereas FI participants will score significantly higher on learning performance and computer self-efficacy with the instruction-based approach. 1.4. Gender Harrison and Rainer (1992) found that male gender is associated with higher computer skill. In a meta-analysis, Whiteley (1997) found men and boys exhibited higher computer self-efficacy than did women and girls. The author claimed gender differences appear in attitude towards computers, knowledge about computers, and skills to work with computers. Gender was introduced in this study because it may contribute to the understanding of self-efficacy exerted in improving the training benefits of computer skills (Rattanapian & Gibbs, 1995, p. 60). Research in instructional psychology has demonstrated that individual characteristics could call for one or the other method of instruction. The moderating variable is a kind of independent variable that has a significant contributory or contingent effect on the independent-dependent relationship. In the present study, gender is proposed to be a moderating variable that moderates the effects of cognitive style and/or training method on learning performance and computer self-efficacy. In addition, the interaction effect of cognitive style and training method was proposed. H4. FI cognitive style will have stronger positive effect on learning performance and computer
self-efficacy for male participants than for female participants. H5. The behavior-modeling method has a stronger positive effect on learning performance and
computer self-efficacy for male students than for female students. H6. FI cognitive style will have stronger positive effect on learning performance and computer
self-efficacy for male participants in the behavior-modeling group than for male participants in the instruction-based group, or female participants in either training condition.
16
H.-W. Chou / Computers & Education 37 (2001) 11–25
1.5. Conceptual research model Fig. 1 presents the research model. Four constructs are included in the model. Cognitive style and training method each are posited to directly impact self-efficacy and learning performance. Gender is posited to moderate the causal effects of cognitive style and/or training method on selfefficacy and learning performance. Two training methods, instruction-based and behavior-modeling, are developed. Cognitive style is classified as field dependent and field independent. 2. Research methodology 2.1. Participants To obtain permission for conducting the field experiment, the researcher contacted local senior high schools in a northern city of Taiwan. All of the school principals expressed their willingness to participate. One school that had a computer classroom equipped with 55 Pentium-level networked PCs was chosen for the experiment. Two classes of 10th graders were randomly chosen and assigned to one of the two training conditions. The number of participants in each class was 53 and 55, respectively. Twenty-four participants either did not complete the entire training process or had missing data and were therefore dropped from the final statistical analysis. The two groups were comparable in terms of gender distribution (approximately 57 and 63% male, and 43% and 37% female for each class, respectively) and past performance (P=0.790 and P=0.821 for last semester’s math and Chinese grades, respectively). 2.2. Experimental design Due to practical limitations, a field experiment, instead of a laboratory experiment, was conducted to test the hypotheses. The experiment is a factorial design with training method and
Fig. 1. Conceptual research model.
H.-W. Chou / Computers & Education 37 (2001) 11–25
17
cognitive style as independent variables, and learning performance and computer self-efficacy as dependent variables. The present study proposes that the training method and cognitive style will have differential effects on self-efficacy and learning performance. That is, the research employs the interactional psychology perspective that emphasizes the importance of considering both the personal characteristics (cognitive style) and the situation factor (training method) as joint determinants of behavior (Lee et al., 1995). 2.3. Training material Three segments of training materials were developed based on a commercialized reference book for WWW homepage design (Horton, Taylor, Ignacio, & Hoft, 1996). The first segment included introduction of Netscape Composer, primary attributes, and document background. The second segment contained paragraph definition and image insertion. The final segment included hyperlink and table manipulation. Each segment was designed for a 1-h training program. 2.4. Procedure The three-session workshop was held on three consequent weeks, with 2 h in each session. At the beginning of the first session, all participants were given a background questionnaire, a computer self-efficacy scale (CSE; Murphy, Coover, & Owen, 1989), and a cognitive style inventory, the Group Embedded Figures Test (GEFT; Witkin, Oltman, Raskin, & Karp, 1971). A lecture on the basics of computing concepts, introduction of Netscape Composer, primary attributes, and document background was delivered thereafter. In the remaining two sessions, the manipulation was introduced. Each of the sessions began with 60 min of instruction in a specific training approach, followed by 30 min of practice, and concluding with a 30-min performance test. All training was conducted using a single instructor, providing continuity throughout all training sessions. The instructor followed a fixed outline, but questions were answered as they arose. Participants were told that their performance on TEST1 and TEST2 would contribute to their course final grades. Following the manipulation, a CSE test was given again after the third session. 2.5. Treatment intervention Uniform content of information was provided to both training groups in the same computer room. The training intervention was implemented through the following procedure. The two training approaches differ in the presentation sequence, instructional philosophy, and instructional media. The instruction-based training method employs a deductive approach in the lecture format. At each session, the instructor began by briefly outlining the key learning points, followed by descriptions on general knowledge, which included concepts, general rules, and command features. Next, procedural knowledge and the accompanying examples were presented step-by-step using computer-driven transparencies as the primary instructional media. In the behavior-modeling condition, the instructor started with specific examples and proceeded to general rules, which is termed an inductive approach. From their computer monitors,
18
H.-W. Chou / Computers & Education 37 (2001) 11–25
participants observed how the instructor (model) demonstrated examples and executed corresponding step-by-step procedures on the computer. Next, the instructor gave a lecture on general knowledge, followed by a summary on key learning points. The computer-driven demonstration was the principal instructional media. 2.6. Measures Cognitive style and computer self-efficacy were assessed by two inventories: the Group Embedded Figures Test (GEFT; Witkin et al., 1971) and the Computer Self-efficacy scale (CSE, Murphy et al., 1989). Two self-developed tests, TEST1 and TEST2, including hands-on and objective questions to test procedural and general knowledge, were self-developed to evaluate learning performance. 2.6.1. GEFT The GEFT, comprised of 18 complex figures, measures the cognitive style ‘‘field independence’’. Within each complex figure is embedded a previously seen simple figure. The participants’ task was to locate the embedded figures that were hidden with each of the complex figures. People who were successful at disembedding the hidden figures were considered to be more fieldindependent. A score was obtained by counting the number of correctly traced embedded figures. Scores range from 0 to 18, with higher scores indicating more field-independence. The GEFT grand mean score was employed as the cutting point for FD/FI group classification. 2.6.2. Computer self-efficacy The CSE self-reported scale was translated and slightly modified to include questions associated with working in a networked environment. This five-point Likert-type scale inventory, including 32 items, was administered twice to the participants, before and after the experiment. CSE.net, defined as the difference between the pre- and post-experiment CSE scores (CSE.pre and CSE.post), is the change in expectation for performance due to the treatment effects. The range of possible total scores for CSE.pre and CSE.post is between 32 and 160, with higher scores indicating more computer self-efficacy. Positive CSE.net scores indicate higher computer self-efficacy due to the experiment. 2.6.3. Learning performance Two learning performance tests, TEST1 and TEST2, were administered during the last 30 min of the second and third sessions, respectively. Both tests were self-developed and included objective questions for testing general knowledge and skill-based tasks for testing procedural knowledge. The range of possible total scores for each test is between 0 and 100, with higher scores indicating better learning performance. Examples of the objective questions and skill-based tasks for TEST1 and TEST2 include the following: 1.TEST1: Name the basic edit functions of FrontPage Express, explain the meaning of ‘‘attributes’’ (objective questions), and set the text format as directed (skill-based task). 2. TEST2: Explain the functions of hyperlink (objective question), and insert figures and tables as requested (skill-based task).
19
H.-W. Chou / Computers & Education 37 (2001) 11–25
2.7. Statistical techniques t Tests, correlation analysis, and the analysis of covariance (ANCOVA) procedure were used to analyze data. The ANCOVA procedure was used to remove potential sources of bias from the experiment so that unbiased estimates of treatment effects could be obtained. In the present study, subjects’ pre-experiment computer self-efficacy, chosen as the covariate, represents one of the potential sources that must be partitioned out to purify the effects of the experimental variables, the training method and learning style. The moderating variable, gender, is included because it may have a significant contributory or contingent effect on the independent-dependent relationship. In the present study, it is posited that training method and/or cognitive style will have different effects on learning performance and computer self-efficacy, depending on the subject’s gender.
3. Results Reliability measures for GEFT, CSE.pre, CSE.post, CSE.net, TEST1, and TEST2 were assessed. The Cronbach a coefficients are 0.75, 0.95, 0.93, 0.85, 0.81, and 0.82, respectively, with all >0.70. Table 1 provides a correlation matrix for all variables. The matrix displays the strength of association for the entire group as well as for the individual treatment group. 3.1. Correlation analysis The significant correlations between TEST1 and TEST2, and among CSE.pre, CSE.post, and CSE.net proved the validity of those measures. Compared with TEST1, TEST2 has stronger association with CSE.pre and CSE.post, which implies that people who performed better in Table 1 Correlation matrix among studied variablesa Variable
1.
2.
1. TEST1 2. TEST2
– 0.587**
–
3.
4.
5.
0.614**(0.603**)
3. CSE.pre 4. CSE.post 5. CSE.net
0.066
0.161
0.031(0.109)
0.118(0.102)
0.004
0.188
0.782**
008(0.011)
0.049(0.181)
708**(0.800**)
0.086 0.029( 0.141)
6. GEFT
0.080 0.020(0.173)
a
–
0.072 0.085(0.149)
0.137 0.044(0.3066
*)
–
0.163
488**
0.333**( 0.144)
0.430**(0.478**)
–
0.280**
0.190
0.091
0.299**(0.3346*)
0.198(0.276*)
0.118( 0.034)
The first row is for the entire group, whereas the second row is for instruction group and behavior modeling group (in parentheses). *P< 0.05. **P< 0.01.
20
H.-W. Chou / Computers & Education 37 (2001) 11–25
TEST2 would have higher perception on their computer efficacy. CSE.pre and CSE.post are strongly correlated with GEFT, which suggests that people who are more field-independent tend to develop higher computer self-efficacy. An opposite correlation pattern was found between the two training methods in the following variable sets. 3.1.1. CSE.net and TEST2 The directions of the correlation between the CSE.net and TEST2 in two treatment groups were opposite (r=0.149 and =0.085 for behavior-modeling and instruction-based, respectively), although neither was significant. This suggested that in the behavior-modeling group, students who performed better in TEST2 also scored higher on CSE.net, whereas the instructionbased group students who performed better in TEST2 showed smaller gains in CSE.net. 3.1.2. GEFT and TEST1, TEST2 GEFT has a higher association with TEST1 and TEST2 in the behavior-modeling group (r=0.173 and r=0.306) but is much weaker in the instruction group (r= 0.020 and r=0.044), which implies that in the behavior-modeling group, the more field-independent participants performed better in both tests. On the other hand, no significant difference was found between different learning style participants in the instruction-based group. 3.2. Main effects study The three-way ANCOVA technique was employed to examine whether different training methods and personal traits would generate significant differences in learning performance and computer self-efficacy change. The effects of gender on TEST1 [F (1,75)=4.29, MSE=890.1, P<0.05], and of training method on TEST2 [F (1,75)=5.51, MSEs=609.7, P<0.05) and CSE.net (F (1, 75)=3.29, MSE=647.8, P<0.05] were significant. These partially supported the propositions displayed in the previous section. Since some main effects were found to be significant, t-tests on the outcome measures by training condition, gender, and cognitive style were conducted. Table 2 displays gender by training method subgroup means in TEST1, TEST2, and CSE.net. Participants with GEFT scores greater than 12.32 (grand means) were assigned to the FI group, whereas the remaining participants were assigned to the FD group. Overall, the behavior-modeling method results in better learning outcomes and FI participants performed better than FD participants. Male participants obtained higher scores in TEST2, CSE.pre, and CSE.post, and female participants performed better in TEST1 and CSE.net. Table 2 Gender by treatment subgroup means Male
Behavior modeling Instruction-based
Female
CSE.net
TEST1
TEST2
CSE.net
TEST1
TEST2
12.03 1.82
58.55 67.50
55.97 56.36
9.71 11.88
80.29 67.50
68.71 37.13
H.-W. Chou / Computers & Education 37 (2001) 11–25
21
3.2.1. Training method effects Participants in the behavior-modeling group performed consistently better in all of the studied variables. The differences in TEST2 (t (1,82)= 2.24, P<0.05) and CSE.post were significant [t (1,82)= 3.34, P<0.00]. The significant differences in CSE.pre [t (1,82)= 2.35, P<0.05] indicated pre-existing class differences in computer self-efficacy. Nevertheless, the significant difference in CSE.net [t (1,82)= 2.19, P<0.05] implies that participants in the behavior-modeling group obtained a much higher computer self-efficacy score change than participants in the instruction-based group. The superiority of behavior modeling in computer learning performance and computer self-efficacy partially supported H1. 3.2.2. Gender effects Gender effects were found in all studied variables. Male participants scored significantly higher on CSE.pre [t (1,82)=4.66, P< 0.00] and CSE.post [t (1,82)=3.12, {P <0.00] whereas female participants scored higher on CSE.net and TEST1. This suggests that the female participants learned better due to the experiment, whereas male participants in general rated significantly higher in computer self-efficacy. 3.2.3. Cognitive style differences The difference was significant on CSE.pre [t (1, 82)= 2.53, P<0.01] and on CSE.post [t (1, 82)= 1.98, P<0.05]. This finding is consistent with the literature that suggested people with FI cognitive style in general tend to academically outperform FD people (Tinajero & Pa´ramo, 1997; Wieseman et al., 1992). Thus, H2 was partially supported. 3.3. Interaction effects study 3.3.1. Two-way interaction Cognitive style by training method interaction effect was found insignificant on any of the three dependent variables. Therefore, H3 was not supported. The hypothesized moderating effects of gender were significant on the relationships between training method and TEST1 [F (1,75)=3.35, P<0.05], TEST2 [F (1,75)=9.04, P<0.00], and CSE.net [F (1,75)=3.30, P<0.05]. Participants of different gender would favor different training methods, depending on the training objective. H5 then was supported. Concerning TEST1 and TEST2, male participants preferred instruction-based, whereas female participants favored behavior-modeling conditions. On the other hand, when self-efficacy is the training objective, instruction-based is more appropriate for female participants, and behavior modeling is the better approach for male learners. ANCOVA results showed that the other gender-moderating effects on the causal relation between cognitive style and dependent variables were not significant. Therefore, H4 was not supported. 3.3.2. Three-way interaction The significant three-way interaction effects of gender by training method by cognitive style on TEST1 [F (1,75)=3.966, P<0.05] partially supported H6. Table 3 provides subgroup means.
22
H.-W. Chou / Computers & Education 37 (2001) 11–25
Table 3 Gender by treatment by cognitive style subgroup meansa Male
Female
FD
FI
FD
FI
Instruction-based
n=8 0.88 72.5(61.38)
n=14 3.36 64.64(53.50)
n=8 6.75 55.0(33.50)
n=8 17.0 80.0(40.75)
Behavior-modeling
n=13 12.69 51.92(50.54)
n=16 11.50 63.94(60.37)
n=8 14.13 82.5(68.75)
n=9 5.78 78.33(68.67)
a The second row represents group means of CSE.net; the third row represents group means of TEST1 and TEST2 (in parentheses).
Regarding CSE.net, FI male participants preferred the behavior-modeling approach, whereas their female counterparts favored the instruction-based condition. On the other hand, FD male participants performed better on TEST1 and TEST2 in the instruction-based condition, whereas their female counterparts preferred the behavior-modeling method. In addition, the cognitive by training method interaction effects were most evident for female participants in CSE.net. Female FI participants preferred the instruction-based condition, whereas FD participants favored the behavior-modeling approach.
4. Conclusions The study results showed that while the behavior-modeling training method is superior with respect to learning performance and computer self-efficacy, the significant two- and three-way interactions indicate the critical roles that personal characteristics and situation factors play as joint determinants of behavior. This finding confirms the contingency effects of gender, cognitive style, training approach, and training objectives on behavior. Gender did significantly moderate the effects of training method on performance and selfefficacy, as hypothesized. Concerning TEST1 and TEST2, male participants benefited more from the instruction-based condition, and female participants learned better in the behaviormodeling condition. Nevertheless, the combined effects of gender and training method on computer self-efficacy were reversed, that is, female participants preferred the instruction-based condition, whereas male participants were more suited to the behavior-modeling condition. Cognitive style by training method interaction effect was not significant on any of the three dependent variables. Regarding the three-way interaction, gender did significantly moderate the effects of training method on TEST1. The three-way interaction indicates that, regarding CSE.net, FI male participants preferred the behavior-modeling approach, whereas their female counterparts favored the instruction-based condition. On the other hand, FD male participants performed better on TEST1 and TEST2 in the instruction-based condition, whereas their female counterparts
H.-W. Chou / Computers & Education 37 (2001) 11–25
23
preferred the behavior-modeling method. The cognitive by training method effects were most significant for female participants in CSE.net. Female FI participants preferred the instruction-based condition, whereas female FD participants favored the behavior-modeling approach. 4.1. Limitations Although the results of this research were clear-cut, the study is not without limitations. Specific weaknesses include the artificial nature of a laboratory setting, the use of students instead of employed individuals, and a relatively small sample. Although some evidence exists for the external validity of laboratory research and participant populations, caution is warranted when applying these results in different settings (Goldstein & Musicante, 1986). 4.2. Recommendations for future research Recommendations for future research come from the present findings. First, the moderating influence of software differences on learning outcomes is worth exploring. Another avenue of research is to understand some aspects of the model that were not supported. Finally, the effects of different types of cognitive style, as well as other individual differences, on training outcomes is worth examining. As with any scientific finding, replication is needed in different settings and with diverse populations. In particular, it would be helpful to test the hypotheses in an organizational setting to increase the external validity.
References Baldwin, T. T. (1992). Effects of alternative modeling strategies on outcomes of interpersonal-skills training. Journal of Applied Psychology, 77(2), 147–154. Bandura, A. (1986). Social foundations of thought and action: a social cognitive theory. Englewood Cliffs, NJ: PrenticeHall. Chou, H. W., & Wang, Y. F. (1999). The effects of learning style and training method on computer attitude and performance in WWW page design training. Journal of Educational Computing Research, 21(3), 323–342. Chou, H. W., & Wang, T. B. (2001). The influence of learning style and training method on self-efficacy and learning performance in WWW homepage design training. International Journal of Information Management, in press. Christoph, R. T., Schoenfeld Jr., G. A., & Tansky, J. W. (1998). Overcoming barriers to training utilizing technology: the influence of self-efficacy factors on multimedia-based training receptiveness. Human Resource Development Quarterly, 9(1), 25–38. Coffin, R. J., & MacIntyre, P. D. (1999). Motivational influences on computer-related affect states. Computers in Human Behavior, 15, 549–569. Compeau, D. R., & Higgins, C. A. (1995). Application of social cognitive theory to training for computer skills. Information Systems Research, 6(2), 118–143. Compeau, D. R., Higgins, C. A., & Huff, S. (1999). Social cognitive theory and individual reactions to computing technology: A longitudinal study. MIS Quarterly, 23(2), 145–158. Gist, M. E., Schwoerer, C., & Rosen, B. (1989). Effects of alternative training methods on self-efficacy and performance in computer software training. Journal of Applied Psychology, 74, 884–891. Gist, M. E., Stevens, C. K., & Bavetta, A. G. (1991). Effects of self-efficacy and post training intervention on the acquisition and maintenance of complex interpersonal skills. Personnel Psychology, 44, 837–861.
24
H.-W. Chou / Computers & Education 37 (2001) 11–25
Goldstein, I. L., & Musicante, G. R. (1986). The applicability of a training transfer model to issues concerning rater training. In E. A. Locke, Generalizing from laboratory to field setting (pp. 83–98). Lexington, MA: Lexington. Grover, V., & Teng, T. C. (1994). Facilitating the implementation of customer-based inter-organizational systems: an empirical analysis of innovation and support factors. Information Systems Journal, 4, 61–89. Harrison, A. W., & Rainer, R. K. Jr. (1992). The influence of individual differences on skill in end-user computing. Journal of Management Information Systems, 9(1), 93–111. Horton, W. L., Taylor, A. L., Ignacio, A. & Hoft, N. L. (1996). The web page design cookbook (pp. 31–50). New York: John Wiley & Sons. Igbaria, M., & Iivari, J. (1995). The effects of self-efficacy on computer usage. Omega, 23(6), 587–605. Knight, C. B., Halpin, G. & Halpin, G. (1997). The effects of learning environment accommodations on the achievement of second graders. Research in the Schools, 3(2), 9–14. Lee, S. M., Kim, Y. R., & Lee, J. (1995). An empirical study of the relationships among end-user information systems acceptance, training, and effectiveness. Journal of Management Information Systems, 12(2), 189–202. Lipsky, S. A. (1989). Effect of field independence/dependence on two textbook notetaking techniques. (ERIC Document Reproduction Service No. ED 311 983). Liu, M., & Reed, W. M. (1994). The relationship between the learning strategies and learning styles in a hypermedia environment. Computers in Human Behavior, 10(4), 419–434. Martocchio, J. J. (1994). Effects of conceptions of ability on anxiety, self-efficacy, and learning in training. Journal of Applied Psychology, 79(6), 819–825. Moore, D. M. & Dwyer, F. M. (1992). Effects of color coding on cognitive style. Paper presented at the annual meeting of the Eastern Education Research Association, Hilton Head, SC. Murphy, C. A., Coover, D., & Owen, S. V. (1989). Development and validation of the computer self-efficacy scale. Educational and Psychological Measurement, 49, 893–899. Olfman, L., & Mandviwalla, M. (1994). Conceptual versus procedural software training for graphical user interfaces: a longitudinal field experiment. Management Information Systems Quarterly, 18, 405–426. Pa´ramo, M. F., & Tinajero, C. (1990). Field dependence/independence and performance in school: an argument against neutrality of cognitive style. Perceptual and Motor Skills, 70, 1079–1087. Rattanapian, V., & Gibbs, W. (1995). Computerized drill and practice: design options and learner characteristics. International Journal of Instructional Media, 22(1), 59–77. Simon, S. J., Grover, V., Teng, J. T. C., & Whitcomb, K. (1996). The relationship of information system training methods and cognitive ability to end-user satisfaction, comprehension, and skill transfer: a longitudinal field study. Information Systems Research, 7(4), 466–490. Simon, S. J., & Werner, J. M. (1996). Computer training through behavior modeling, self-paced, and instructional approaches: a field experiment. Journal of Applied Psychology, 81(6), 648–659. Snow, R. E. (1991). Aptitude-treatment interaction as a framework for research on individual differences in psychotherapy. Journal of Consulting and Clinical Psychology, 59(2), 205–216. Snow, R. E., & Swanson, J. (1992). Instructional psychology: aptitude, adaptation, and assessment. Annual Review of Psychology, 43, 583–626. Tinajero, C., & Pa´ramo, M. F. (1997). Field dependence-independence and academic achievement: a re-examination of their relationship. British Journal of Educational Psychology, 67, 199–212. Torkzadeh, G., & Koufteros, X. (1994). Factorial validity of a computer self-efficacy scale and the impact of computer training. Educational & Psychological Measurement, 54(3), 813–821. Webster, J., & Martocchio, J. J. (1993). Turning work into play: implications for microcomputer software training. Journal of Management, 19(1), 127–146. Wieseman, R. A., Portis, S. C., & Simpson, F. M. (1992). An analysis of the relationship between cognitive styles and grades: new perspectives on success or failure of preservice education majors. College Student Journal, 26(4), 512–517. Witkin, H. A. (1964). Origins of cognitive style. In C. Scheerer, Cognition: theory, research, promise (pp. 172–205). New York: Harper and Row. Witkin, H. A., & Goodenough, D. R. (1981). Cognitive style: essence and origins. New York: International Universities Press.
H.-W. Chou / Computers & Education 37 (2001) 11–25
25
Witkin, H. A., Moore, C. A., Goodenough, D. R., & Cox, P. W. (1977). Field-dependent and field-independent cognitive styles and their educational implications. Review of Educational Research, 47, 1–64. Witkin, H. A., Oltman, P., Raskin, E., & Karp, S. (1971). A manual for the embedded figures test. Palo Alto, CA: Consulting Psychologist Press. Whiteley Jr., B. E. (1997). Gender differences in computer-related attitudes and behavior: a meta-analysis. Computers in Human Behavior, 13(1), 1–22.
Influences of cognitive style and training method on training effectiveness Huey-Wen Chou * Department of Information Management, National Central University, 38, Wu-Chuan Li, Chungli, Taiwan 32054, People’s Republic of China Received 1 April 2000; accepted 1 February 2001
Abstract The present study compares the relative effects of cognitive style and training method on learners’ computer self-efficacy and learning performance by a field experiment. The purpose was to determine which training method could be best utilized in computer-related training while taking trainees’ cognitive style into account. The significant two- and three-way interactions indicate the critical roles that personal characteristics and situation factors play as joint determinants of behavior. Gender did significantly moderate the effects of training method on performance and self-efficacy. The cognitive by training method effects were most significant for female participants. This finding suggests that to assist individuals taking IT-related training courses, the contingency effects of gender, cognitive style, training approach, and training objective should be taken into account. # 2001 Elsevier Science Ltd. All rights reserved. Keywords: Country-specific developments; Gender studies; Improving classroom teaching; Secondary education
As information technology (IT) becomes more critical to business operation, computer literacy becomes a requisite for employees. Although training has been widely recognized as a critical contributor to successful systems implementation and end user computing (Grover & Teng, 1994), the broad diversity of individual differences among potential trainees should be taken into account when developing the training program (Chou & Wang, 1999; Liu & Reed, 1994). In addition, while many approaches for developing computer skills are available, little research has systematically examined the relative merits of various training programs. The current study addresses these concerns. This paper focuses on three objectives: (1) to develop a conceptual model to evaluate how training method and cognitive style influence learning performance and computer self-efficacy; (2) * Corresponding author. Tel.: +886-3-426-7256; fax: +886-3-425-4604. E-mail address: [email protected] (H.-W. Chou). 0360-1315/01/$ - see front matter # 2001 Elsevier Science Ltd. All rights reserved. PII: S0360-1315(01)00028-8
12
H.-W. Chou / Computers & Education 37 (2001) 11–25
to investigate whether individuals with different cognitive styles perform differently in two training conditions, that is, assessing the feasibility of a contingency approach to training on learning performance and on computer self-efficacy; and (3) to investigate the gender-moderating effects on the causal relationship between cognitive style and/or training method, and learning performance and computer self-efficacy. The remainder of this paper is organized as follows. Section 1 reviews the literature on the variables included in the proposed model, followed by the theoretical framework and hypotheses for the model. Section 2 presents the research methodology including participants, experimental design, procedure, and training intervention. Section 3 describes the data analysis and corresponding results. Section 4 provides summaries of the study findings, study limitations, and recommendations for future research.
1. Literature review 1.1. Self-efficacy Self-efficacy emanates from the Social Cognitive Theory (SCT; Bandura, 1986) and is defined as the beliefs about one’s ability to perform a particular behavior, which affects choices about which behaviors to undertake, and the effort and persistence exerted in the face of obstacles to the performance of those behaviors. Self-efficacy perceptions influence an individual’s actual ability to perform the behavior. Gist, Schwoerer, and Rosen (1989) and Gist, Stevens, and Bavetta (1991) confirmed a positive relationship between self-efficacy and achievement. They also suggested that initial computer self-efficacy moderated the effect of training method on training outcome. Self-efficacy was treated as both a dependent variable and an independent variable in the computer-training literature. 1.1.1. Self-efficacy as an independent variable Igbaria and Iivari’s (1995) study found that self-efficacy had both direct and indirect effects on computer usage. Compeau, Higgins, and Huff (1999) conducted a longitudinal study to test an SCT-based model, which indicated a significant relationship between self-efficacy and computer usage. Coffin and MacIntyre (1999) found that self-efficacy had a significant effect on learning performance in a programming course. Compeau and Higgins (1995) found that computer selfefficacy exerted a strong influence on performance in learning Lotus 1-2-3 and WordPerfect. Christoph, Schoenfeld, and Tansky’s (1998) study on the influence of self-efficacy on multimediabased training receptiveness indicated that training effectiveness was determined partly by the trainees’ self-efficacy. 1.1.2. Self-efficacy as a dependent variable Self-efficacy was also treated as a dependent variable in the literature. Martocchio (1994) found age and outcome expectation significantly affect self-efficacy. Torkzadeh and Koufteros (1994) found a significant training effect on self-efficacy. In Compeau and Higgins’ (1995) study, they confirmed that the behavior-modeling approach was more effective for training in Lotus 1-2-3 and resulted in a higher computer self-efficacy.
H.-W. Chou / Computers & Education 37 (2001) 11–25
13
The main purpose of the present study was to determine whether different training methods and/or participants’ cognitive style would impact their computer learning performance and computer self-efficacy. Self-efficacy was therefore employed as a dependent variable. 1.2. Training methods Many training formats are reviewed in the computer training literature, such as lecture, exploration-based, tutorial, and computer-assisted instruction (CAI). Nevertheless, little research has systematically examined the relative merits of various training programs. Studies suggested that the choice of training method has consequences for the degree of learning (Webster & Martocchio, 1993) as well as for self-efficacy (Compeau & Higgins, 1995; Gist et al., 1989). In the present study, the instruction-based method was chosen because it represents a more traditional approach and is appropriate for almost all training needs. Behavior modeling was included because of the work of Baldwin (1992), Compeau and Higgins (1995), Gist et al. (1989), Olfman and Mandviwalla (1994), Simon and Werner (1996), and Simon, Grover, Teng, and Whitcomb (1996) on computer training. The instruction-based method offers a traditional approach that is appropriate for almost all training. This approach teaches primarily by lecture and follows a deductive way to learning, where learners proceed from general rules to specific examples (Chou & Wang, 2001). Overall, the literature suggests that the instructional technique is superior for retention of information. Behavior modeling is a training process developed in the 1970s for building an individual’s skill in the context of managerial skill training. This task-focused method involves a visual observation of the behaviors of a model performing a task. Learners then imitate and extend the model’s behavior in practice and experimentation to master the task. The behavior-modeling method employs an inductive approach that teaches first by hands-on demonstrations, followed by complementary lectures. Bandura (1986) suggested that by observing someone performing the target behavior, the participants’ self-efficacy would be raised. In a field experiment, Gist et al. (1989) compared alternative training methods on self-efficacy and computer software learning. In the behavior-modeling group, video modeling served as the principal means of instruction, whereas in the tutorial training group, a one-on-one interactive tutorial instruction on the computer monitor was the primary means of instruction. After each step was demonstrated, the trainees in the behavior-modeling group were asked to execute the step. On the other hand, the participants in the tutorial condition were only told what to do; they were not asked to execute specific tasks. The results indicate the behavior-modeling approach relative to a computer-aided instructional approach yielded higher self-efficacy and performance scores in learning computer software. Compeau and Higgins (1995) examined the relative effectiveness of behavior-modeling and traditional lecture-based programs on the learners’ performance and self-efficacy. They found that the behavior-modeling approach was more effective than the lecture-based approach in learning Lotus 1-2-3. Chou and Wang (2001) confirmed the superiority of the behavior-modeling approach to the instruction-based approach on computer self-efficacy and learning performance. Simon et al. (1996) and Simon and Werner (1996) compared three training techniques — instruction, self-paced, and behavior modelin — with a no-training control group. Findings of
14
H.-W. Chou / Computers & Education 37 (2001) 11–25
both studies indicate that the behavior-modeling approach generates the highest scores in cognitive learning, skill demonstration, and satisfaction with the computer system. H1. Participants in the behavior modeling group will score significantly higher on learning
performance and computer self- efficacy measures than participants in the instruction-based group. 1.3. Cognitive style Cognitive styles define learner traits. The cognitive style refers to the way individuals collect, analyze, evaluate, and interpret data (Harrison & Rainer, 1992). The study of individual differences in cognition and their impact on learning and instruction has long been a focus in cognitive psychology and education. Witkin’s (1964) research on how people separate one factor from the total visual field has become the field-independence/dependence theory, which is also one of the most extensively researched cognitive styles. Witkin, Moore, Goodenough, and Cox (1977) defined field independence as ‘‘the extent to which a person perceives part of a field as discrete from the surrounding field as a whole, rather than embedded in the field; or . . . the extent to which the person perceives analytically’’ (p. 7). According to Witkin and Goodenough (1981), field-independent (FI) people are more likely to do well with numbers, science, and problem-solving tasks. They tend to analytically approach a problem and perceive a particular and relevant item in a field of distracting items. On the other hand, field-dependent (FD) people tend to be better at recalling such social information as conversations and relationships. They prefer to approach a problem in a more global way and are capable of perceiving the total picture in a situation. The practical implications of FI/FD cognitive style for education have indicated that the individuals’ different cognitive styles bear direct impact upon their achievement performance (Tinajero & Pa´ramo, 1997; Wieseman, Portis, & Simpson, 1992). Witkin and Goodenough (1981) suggested that FI relates to academic progress in some specialized courses. Several studies have suggested that more FI students tend to academically outperform FD students. For instance, Pa´ramo and Tinajero’s (1990) study showed that FI people tend to outperform FD people in overall school performance. Moore and Dwyer (1992) conducted a study that examined the effect of B&W- and color-coding of information. They found that FI students, on the whole, scored significantly higher than FD students in both visually and verbally oriented tests. In addition, significant performance differences were found between FI and FD students in the B&W-coded treatment but not in the color-coded treatment. The results also showed that external support could be important for FD people in processing information. Liu and Reed (1994) found that participants of different cognitive styles employed different learning strategies in accomplishing the same task in a hypermedia environment. H2. Participants with FI cognitive style will score significantly higher on learning performance
and computer self-efficacy measures than participants with FD cognitive style. In addition to the direct effects of cognitive style on achievement performance, the relationship between cognitive style and training approach has been studied. Studies that have attempted to
H.-W. Chou / Computers & Education 37 (2001) 11–25
15
match situational factors to learning styles have only inconsistently found any benefits for learning (Knight, Halpin, & Halpin, 1992; Snow & Swanson, 1992). However, the search for such aptitude-treatment interaction goes on, and a few studies have found positive effects for programs that adapt instruction to an individual’s learning style (Chou & Wang, 1999; Lipsky, 1989). Lipsky (1989) found FD participants performed better with the outlining technique, and FI students performed better with the mapping technique. The above results confirmed that one instructional method suitable for FI learners might not necessarily be beneficial for FD learners. Learners with different cognitive styles pursue quite different ways of learning. Snow (1991) proposed that learners differ profoundly in what they do in learning and in their success in any particular learning situation. The present study employs the interactional psychology perspective that emphasizes the importance of considering both the person characteristics (cognitive style) and the situation factor (training method) as joint eterminants of behavior (Lee, Kim, & Lee, 1995). H3. FD participants will score significantly higher on learning performance and computer
self-efficacy with the behavior-modeling method, whereas FI participants will score significantly higher on learning performance and computer self-efficacy with the instruction-based approach. 1.4. Gender Harrison and Rainer (1992) found that male gender is associated with higher computer skill. In a meta-analysis, Whiteley (1997) found men and boys exhibited higher computer self-efficacy than did women and girls. The author claimed gender differences appear in attitude towards computers, knowledge about computers, and skills to work with computers. Gender was introduced in this study because it may contribute to the understanding of self-efficacy exerted in improving the training benefits of computer skills (Rattanapian & Gibbs, 1995, p. 60). Research in instructional psychology has demonstrated that individual characteristics could call for one or the other method of instruction. The moderating variable is a kind of independent variable that has a significant contributory or contingent effect on the independent-dependent relationship. In the present study, gender is proposed to be a moderating variable that moderates the effects of cognitive style and/or training method on learning performance and computer self-efficacy. In addition, the interaction effect of cognitive style and training method was proposed. H4. FI cognitive style will have stronger positive effect on learning performance and computer
self-efficacy for male participants than for female participants. H5. The behavior-modeling method has a stronger positive effect on learning performance and
computer self-efficacy for male students than for female students. H6. FI cognitive style will have stronger positive effect on learning performance and computer
self-efficacy for male participants in the behavior-modeling group than for male participants in the instruction-based group, or female participants in either training condition.
16
H.-W. Chou / Computers & Education 37 (2001) 11–25
1.5. Conceptual research model Fig. 1 presents the research model. Four constructs are included in the model. Cognitive style and training method each are posited to directly impact self-efficacy and learning performance. Gender is posited to moderate the causal effects of cognitive style and/or training method on selfefficacy and learning performance. Two training methods, instruction-based and behavior-modeling, are developed. Cognitive style is classified as field dependent and field independent. 2. Research methodology 2.1. Participants To obtain permission for conducting the field experiment, the researcher contacted local senior high schools in a northern city of Taiwan. All of the school principals expressed their willingness to participate. One school that had a computer classroom equipped with 55 Pentium-level networked PCs was chosen for the experiment. Two classes of 10th graders were randomly chosen and assigned to one of the two training conditions. The number of participants in each class was 53 and 55, respectively. Twenty-four participants either did not complete the entire training process or had missing data and were therefore dropped from the final statistical analysis. The two groups were comparable in terms of gender distribution (approximately 57 and 63% male, and 43% and 37% female for each class, respectively) and past performance (P=0.790 and P=0.821 for last semester’s math and Chinese grades, respectively). 2.2. Experimental design Due to practical limitations, a field experiment, instead of a laboratory experiment, was conducted to test the hypotheses. The experiment is a factorial design with training method and
Fig. 1. Conceptual research model.
H.-W. Chou / Computers & Education 37 (2001) 11–25
17
cognitive style as independent variables, and learning performance and computer self-efficacy as dependent variables. The present study proposes that the training method and cognitive style will have differential effects on self-efficacy and learning performance. That is, the research employs the interactional psychology perspective that emphasizes the importance of considering both the personal characteristics (cognitive style) and the situation factor (training method) as joint determinants of behavior (Lee et al., 1995). 2.3. Training material Three segments of training materials were developed based on a commercialized reference book for WWW homepage design (Horton, Taylor, Ignacio, & Hoft, 1996). The first segment included introduction of Netscape Composer, primary attributes, and document background. The second segment contained paragraph definition and image insertion. The final segment included hyperlink and table manipulation. Each segment was designed for a 1-h training program. 2.4. Procedure The three-session workshop was held on three consequent weeks, with 2 h in each session. At the beginning of the first session, all participants were given a background questionnaire, a computer self-efficacy scale (CSE; Murphy, Coover, & Owen, 1989), and a cognitive style inventory, the Group Embedded Figures Test (GEFT; Witkin, Oltman, Raskin, & Karp, 1971). A lecture on the basics of computing concepts, introduction of Netscape Composer, primary attributes, and document background was delivered thereafter. In the remaining two sessions, the manipulation was introduced. Each of the sessions began with 60 min of instruction in a specific training approach, followed by 30 min of practice, and concluding with a 30-min performance test. All training was conducted using a single instructor, providing continuity throughout all training sessions. The instructor followed a fixed outline, but questions were answered as they arose. Participants were told that their performance on TEST1 and TEST2 would contribute to their course final grades. Following the manipulation, a CSE test was given again after the third session. 2.5. Treatment intervention Uniform content of information was provided to both training groups in the same computer room. The training intervention was implemented through the following procedure. The two training approaches differ in the presentation sequence, instructional philosophy, and instructional media. The instruction-based training method employs a deductive approach in the lecture format. At each session, the instructor began by briefly outlining the key learning points, followed by descriptions on general knowledge, which included concepts, general rules, and command features. Next, procedural knowledge and the accompanying examples were presented step-by-step using computer-driven transparencies as the primary instructional media. In the behavior-modeling condition, the instructor started with specific examples and proceeded to general rules, which is termed an inductive approach. From their computer monitors,
18
H.-W. Chou / Computers & Education 37 (2001) 11–25
participants observed how the instructor (model) demonstrated examples and executed corresponding step-by-step procedures on the computer. Next, the instructor gave a lecture on general knowledge, followed by a summary on key learning points. The computer-driven demonstration was the principal instructional media. 2.6. Measures Cognitive style and computer self-efficacy were assessed by two inventories: the Group Embedded Figures Test (GEFT; Witkin et al., 1971) and the Computer Self-efficacy scale (CSE, Murphy et al., 1989). Two self-developed tests, TEST1 and TEST2, including hands-on and objective questions to test procedural and general knowledge, were self-developed to evaluate learning performance. 2.6.1. GEFT The GEFT, comprised of 18 complex figures, measures the cognitive style ‘‘field independence’’. Within each complex figure is embedded a previously seen simple figure. The participants’ task was to locate the embedded figures that were hidden with each of the complex figures. People who were successful at disembedding the hidden figures were considered to be more fieldindependent. A score was obtained by counting the number of correctly traced embedded figures. Scores range from 0 to 18, with higher scores indicating more field-independence. The GEFT grand mean score was employed as the cutting point for FD/FI group classification. 2.6.2. Computer self-efficacy The CSE self-reported scale was translated and slightly modified to include questions associated with working in a networked environment. This five-point Likert-type scale inventory, including 32 items, was administered twice to the participants, before and after the experiment. CSE.net, defined as the difference between the pre- and post-experiment CSE scores (CSE.pre and CSE.post), is the change in expectation for performance due to the treatment effects. The range of possible total scores for CSE.pre and CSE.post is between 32 and 160, with higher scores indicating more computer self-efficacy. Positive CSE.net scores indicate higher computer self-efficacy due to the experiment. 2.6.3. Learning performance Two learning performance tests, TEST1 and TEST2, were administered during the last 30 min of the second and third sessions, respectively. Both tests were self-developed and included objective questions for testing general knowledge and skill-based tasks for testing procedural knowledge. The range of possible total scores for each test is between 0 and 100, with higher scores indicating better learning performance. Examples of the objective questions and skill-based tasks for TEST1 and TEST2 include the following: 1.TEST1: Name the basic edit functions of FrontPage Express, explain the meaning of ‘‘attributes’’ (objective questions), and set the text format as directed (skill-based task). 2. TEST2: Explain the functions of hyperlink (objective question), and insert figures and tables as requested (skill-based task).
19
H.-W. Chou / Computers & Education 37 (2001) 11–25
2.7. Statistical techniques t Tests, correlation analysis, and the analysis of covariance (ANCOVA) procedure were used to analyze data. The ANCOVA procedure was used to remove potential sources of bias from the experiment so that unbiased estimates of treatment effects could be obtained. In the present study, subjects’ pre-experiment computer self-efficacy, chosen as the covariate, represents one of the potential sources that must be partitioned out to purify the effects of the experimental variables, the training method and learning style. The moderating variable, gender, is included because it may have a significant contributory or contingent effect on the independent-dependent relationship. In the present study, it is posited that training method and/or cognitive style will have different effects on learning performance and computer self-efficacy, depending on the subject’s gender.
3. Results Reliability measures for GEFT, CSE.pre, CSE.post, CSE.net, TEST1, and TEST2 were assessed. The Cronbach a coefficients are 0.75, 0.95, 0.93, 0.85, 0.81, and 0.82, respectively, with all >0.70. Table 1 provides a correlation matrix for all variables. The matrix displays the strength of association for the entire group as well as for the individual treatment group. 3.1. Correlation analysis The significant correlations between TEST1 and TEST2, and among CSE.pre, CSE.post, and CSE.net proved the validity of those measures. Compared with TEST1, TEST2 has stronger association with CSE.pre and CSE.post, which implies that people who performed better in Table 1 Correlation matrix among studied variablesa Variable
1.
2.
1. TEST1 2. TEST2
– 0.587**
–
3.
4.
5.
0.614**(0.603**)
3. CSE.pre 4. CSE.post 5. CSE.net
0.066
0.161
0.031(0.109)
0.118(0.102)
0.004
0.188
0.782**
008(0.011)
0.049(0.181)
708**(0.800**)
0.086 0.029( 0.141)
6. GEFT
0.080 0.020(0.173)
a
–
0.072 0.085(0.149)
0.137 0.044(0.3066
*)
–
0.163
488**
0.333**( 0.144)
0.430**(0.478**)
–
0.280**
0.190
0.091
0.299**(0.3346*)
0.198(0.276*)
0.118( 0.034)
The first row is for the entire group, whereas the second row is for instruction group and behavior modeling group (in parentheses). *P< 0.05. **P< 0.01.
20
H.-W. Chou / Computers & Education 37 (2001) 11–25
TEST2 would have higher perception on their computer efficacy. CSE.pre and CSE.post are strongly correlated with GEFT, which suggests that people who are more field-independent tend to develop higher computer self-efficacy. An opposite correlation pattern was found between the two training methods in the following variable sets. 3.1.1. CSE.net and TEST2 The directions of the correlation between the CSE.net and TEST2 in two treatment groups were opposite (r=0.149 and =0.085 for behavior-modeling and instruction-based, respectively), although neither was significant. This suggested that in the behavior-modeling group, students who performed better in TEST2 also scored higher on CSE.net, whereas the instructionbased group students who performed better in TEST2 showed smaller gains in CSE.net. 3.1.2. GEFT and TEST1, TEST2 GEFT has a higher association with TEST1 and TEST2 in the behavior-modeling group (r=0.173 and r=0.306) but is much weaker in the instruction group (r= 0.020 and r=0.044), which implies that in the behavior-modeling group, the more field-independent participants performed better in both tests. On the other hand, no significant difference was found between different learning style participants in the instruction-based group. 3.2. Main effects study The three-way ANCOVA technique was employed to examine whether different training methods and personal traits would generate significant differences in learning performance and computer self-efficacy change. The effects of gender on TEST1 [F (1,75)=4.29, MSE=890.1, P<0.05], and of training method on TEST2 [F (1,75)=5.51, MSEs=609.7, P<0.05) and CSE.net (F (1, 75)=3.29, MSE=647.8, P<0.05] were significant. These partially supported the propositions displayed in the previous section. Since some main effects were found to be significant, t-tests on the outcome measures by training condition, gender, and cognitive style were conducted. Table 2 displays gender by training method subgroup means in TEST1, TEST2, and CSE.net. Participants with GEFT scores greater than 12.32 (grand means) were assigned to the FI group, whereas the remaining participants were assigned to the FD group. Overall, the behavior-modeling method results in better learning outcomes and FI participants performed better than FD participants. Male participants obtained higher scores in TEST2, CSE.pre, and CSE.post, and female participants performed better in TEST1 and CSE.net. Table 2 Gender by treatment subgroup means Male
Behavior modeling Instruction-based
Female
CSE.net
TEST1
TEST2
CSE.net
TEST1
TEST2
12.03 1.82
58.55 67.50
55.97 56.36
9.71 11.88
80.29 67.50
68.71 37.13
H.-W. Chou / Computers & Education 37 (2001) 11–25
21
3.2.1. Training method effects Participants in the behavior-modeling group performed consistently better in all of the studied variables. The differences in TEST2 (t (1,82)= 2.24, P<0.05) and CSE.post were significant [t (1,82)= 3.34, P<0.00]. The significant differences in CSE.pre [t (1,82)= 2.35, P<0.05] indicated pre-existing class differences in computer self-efficacy. Nevertheless, the significant difference in CSE.net [t (1,82)= 2.19, P<0.05] implies that participants in the behavior-modeling group obtained a much higher computer self-efficacy score change than participants in the instruction-based group. The superiority of behavior modeling in computer learning performance and computer self-efficacy partially supported H1. 3.2.2. Gender effects Gender effects were found in all studied variables. Male participants scored significantly higher on CSE.pre [t (1,82)=4.66, P< 0.00] and CSE.post [t (1,82)=3.12, {P <0.00] whereas female participants scored higher on CSE.net and TEST1. This suggests that the female participants learned better due to the experiment, whereas male participants in general rated significantly higher in computer self-efficacy. 3.2.3. Cognitive style differences The difference was significant on CSE.pre [t (1, 82)= 2.53, P<0.01] and on CSE.post [t (1, 82)= 1.98, P<0.05]. This finding is consistent with the literature that suggested people with FI cognitive style in general tend to academically outperform FD people (Tinajero & Pa´ramo, 1997; Wieseman et al., 1992). Thus, H2 was partially supported. 3.3. Interaction effects study 3.3.1. Two-way interaction Cognitive style by training method interaction effect was found insignificant on any of the three dependent variables. Therefore, H3 was not supported. The hypothesized moderating effects of gender were significant on the relationships between training method and TEST1 [F (1,75)=3.35, P<0.05], TEST2 [F (1,75)=9.04, P<0.00], and CSE.net [F (1,75)=3.30, P<0.05]. Participants of different gender would favor different training methods, depending on the training objective. H5 then was supported. Concerning TEST1 and TEST2, male participants preferred instruction-based, whereas female participants favored behavior-modeling conditions. On the other hand, when self-efficacy is the training objective, instruction-based is more appropriate for female participants, and behavior modeling is the better approach for male learners. ANCOVA results showed that the other gender-moderating effects on the causal relation between cognitive style and dependent variables were not significant. Therefore, H4 was not supported. 3.3.2. Three-way interaction The significant three-way interaction effects of gender by training method by cognitive style on TEST1 [F (1,75)=3.966, P<0.05] partially supported H6. Table 3 provides subgroup means.
22
H.-W. Chou / Computers & Education 37 (2001) 11–25
Table 3 Gender by treatment by cognitive style subgroup meansa Male
Female
FD
FI
FD
FI
Instruction-based
n=8 0.88 72.5(61.38)
n=14 3.36 64.64(53.50)
n=8 6.75 55.0(33.50)
n=8 17.0 80.0(40.75)
Behavior-modeling
n=13 12.69 51.92(50.54)
n=16 11.50 63.94(60.37)
n=8 14.13 82.5(68.75)
n=9 5.78 78.33(68.67)
a The second row represents group means of CSE.net; the third row represents group means of TEST1 and TEST2 (in parentheses).
Regarding CSE.net, FI male participants preferred the behavior-modeling approach, whereas their female counterparts favored the instruction-based condition. On the other hand, FD male participants performed better on TEST1 and TEST2 in the instruction-based condition, whereas their female counterparts preferred the behavior-modeling method. In addition, the cognitive by training method interaction effects were most evident for female participants in CSE.net. Female FI participants preferred the instruction-based condition, whereas FD participants favored the behavior-modeling approach.
4. Conclusions The study results showed that while the behavior-modeling training method is superior with respect to learning performance and computer self-efficacy, the significant two- and three-way interactions indicate the critical roles that personal characteristics and situation factors play as joint determinants of behavior. This finding confirms the contingency effects of gender, cognitive style, training approach, and training objectives on behavior. Gender did significantly moderate the effects of training method on performance and selfefficacy, as hypothesized. Concerning TEST1 and TEST2, male participants benefited more from the instruction-based condition, and female participants learned better in the behaviormodeling condition. Nevertheless, the combined effects of gender and training method on computer self-efficacy were reversed, that is, female participants preferred the instruction-based condition, whereas male participants were more suited to the behavior-modeling condition. Cognitive style by training method interaction effect was not significant on any of the three dependent variables. Regarding the three-way interaction, gender did significantly moderate the effects of training method on TEST1. The three-way interaction indicates that, regarding CSE.net, FI male participants preferred the behavior-modeling approach, whereas their female counterparts favored the instruction-based condition. On the other hand, FD male participants performed better on TEST1 and TEST2 in the instruction-based condition, whereas their female counterparts
H.-W. Chou / Computers & Education 37 (2001) 11–25
23
preferred the behavior-modeling method. The cognitive by training method effects were most significant for female participants in CSE.net. Female FI participants preferred the instruction-based condition, whereas female FD participants favored the behavior-modeling approach. 4.1. Limitations Although the results of this research were clear-cut, the study is not without limitations. Specific weaknesses include the artificial nature of a laboratory setting, the use of students instead of employed individuals, and a relatively small sample. Although some evidence exists for the external validity of laboratory research and participant populations, caution is warranted when applying these results in different settings (Goldstein & Musicante, 1986). 4.2. Recommendations for future research Recommendations for future research come from the present findings. First, the moderating influence of software differences on learning outcomes is worth exploring. Another avenue of research is to understand some aspects of the model that were not supported. Finally, the effects of different types of cognitive style, as well as other individual differences, on training outcomes is worth examining. As with any scientific finding, replication is needed in different settings and with diverse populations. In particular, it would be helpful to test the hypotheses in an organizational setting to increase the external validity.
References Baldwin, T. T. (1992). Effects of alternative modeling strategies on outcomes of interpersonal-skills training. Journal of Applied Psychology, 77(2), 147–154. Bandura, A. (1986). Social foundations of thought and action: a social cognitive theory. Englewood Cliffs, NJ: PrenticeHall. Chou, H. W., & Wang, Y. F. (1999). The effects of learning style and training method on computer attitude and performance in WWW page design training. Journal of Educational Computing Research, 21(3), 323–342. Chou, H. W., & Wang, T. B. (2001). The influence of learning style and training method on self-efficacy and learning performance in WWW homepage design training. International Journal of Information Management, in press. Christoph, R. T., Schoenfeld Jr., G. A., & Tansky, J. W. (1998). Overcoming barriers to training utilizing technology: the influence of self-efficacy factors on multimedia-based training receptiveness. Human Resource Development Quarterly, 9(1), 25–38. Coffin, R. J., & MacIntyre, P. D. (1999). Motivational influences on computer-related affect states. Computers in Human Behavior, 15, 549–569. Compeau, D. R., & Higgins, C. A. (1995). Application of social cognitive theory to training for computer skills. Information Systems Research, 6(2), 118–143. Compeau, D. R., Higgins, C. A., & Huff, S. (1999). Social cognitive theory and individual reactions to computing technology: A longitudinal study. MIS Quarterly, 23(2), 145–158. Gist, M. E., Schwoerer, C., & Rosen, B. (1989). Effects of alternative training methods on self-efficacy and performance in computer software training. Journal of Applied Psychology, 74, 884–891. Gist, M. E., Stevens, C. K., & Bavetta, A. G. (1991). Effects of self-efficacy and post training intervention on the acquisition and maintenance of complex interpersonal skills. Personnel Psychology, 44, 837–861.
24
H.-W. Chou / Computers & Education 37 (2001) 11–25
Goldstein, I. L., & Musicante, G. R. (1986). The applicability of a training transfer model to issues concerning rater training. In E. A. Locke, Generalizing from laboratory to field setting (pp. 83–98). Lexington, MA: Lexington. Grover, V., & Teng, T. C. (1994). Facilitating the implementation of customer-based inter-organizational systems: an empirical analysis of innovation and support factors. Information Systems Journal, 4, 61–89. Harrison, A. W., & Rainer, R. K. Jr. (1992). The influence of individual differences on skill in end-user computing. Journal of Management Information Systems, 9(1), 93–111. Horton, W. L., Taylor, A. L., Ignacio, A. & Hoft, N. L. (1996). The web page design cookbook (pp. 31–50). New York: John Wiley & Sons. Igbaria, M., & Iivari, J. (1995). The effects of self-efficacy on computer usage. Omega, 23(6), 587–605. Knight, C. B., Halpin, G. & Halpin, G. (1997). The effects of learning environment accommodations on the achievement of second graders. Research in the Schools, 3(2), 9–14. Lee, S. M., Kim, Y. R., & Lee, J. (1995). An empirical study of the relationships among end-user information systems acceptance, training, and effectiveness. Journal of Management Information Systems, 12(2), 189–202. Lipsky, S. A. (1989). Effect of field independence/dependence on two textbook notetaking techniques. (ERIC Document Reproduction Service No. ED 311 983). Liu, M., & Reed, W. M. (1994). The relationship between the learning strategies and learning styles in a hypermedia environment. Computers in Human Behavior, 10(4), 419–434. Martocchio, J. J. (1994). Effects of conceptions of ability on anxiety, self-efficacy, and learning in training. Journal of Applied Psychology, 79(6), 819–825. Moore, D. M. & Dwyer, F. M. (1992). Effects of color coding on cognitive style. Paper presented at the annual meeting of the Eastern Education Research Association, Hilton Head, SC. Murphy, C. A., Coover, D., & Owen, S. V. (1989). Development and validation of the computer self-efficacy scale. Educational and Psychological Measurement, 49, 893–899. Olfman, L., & Mandviwalla, M. (1994). Conceptual versus procedural software training for graphical user interfaces: a longitudinal field experiment. Management Information Systems Quarterly, 18, 405–426. Pa´ramo, M. F., & Tinajero, C. (1990). Field dependence/independence and performance in school: an argument against neutrality of cognitive style. Perceptual and Motor Skills, 70, 1079–1087. Rattanapian, V., & Gibbs, W. (1995). Computerized drill and practice: design options and learner characteristics. International Journal of Instructional Media, 22(1), 59–77. Simon, S. J., Grover, V., Teng, J. T. C., & Whitcomb, K. (1996). The relationship of information system training methods and cognitive ability to end-user satisfaction, comprehension, and skill transfer: a longitudinal field study. Information Systems Research, 7(4), 466–490. Simon, S. J., & Werner, J. M. (1996). Computer training through behavior modeling, self-paced, and instructional approaches: a field experiment. Journal of Applied Psychology, 81(6), 648–659. Snow, R. E. (1991). Aptitude-treatment interaction as a framework for research on individual differences in psychotherapy. Journal of Consulting and Clinical Psychology, 59(2), 205–216. Snow, R. E., & Swanson, J. (1992). Instructional psychology: aptitude, adaptation, and assessment. Annual Review of Psychology, 43, 583–626. Tinajero, C., & Pa´ramo, M. F. (1997). Field dependence-independence and academic achievement: a re-examination of their relationship. British Journal of Educational Psychology, 67, 199–212. Torkzadeh, G., & Koufteros, X. (1994). Factorial validity of a computer self-efficacy scale and the impact of computer training. Educational & Psychological Measurement, 54(3), 813–821. Webster, J., & Martocchio, J. J. (1993). Turning work into play: implications for microcomputer software training. Journal of Management, 19(1), 127–146. Wieseman, R. A., Portis, S. C., & Simpson, F. M. (1992). An analysis of the relationship between cognitive styles and grades: new perspectives on success or failure of preservice education majors. College Student Journal, 26(4), 512–517. Witkin, H. A. (1964). Origins of cognitive style. In C. Scheerer, Cognition: theory, research, promise (pp. 172–205). New York: Harper and Row. Witkin, H. A., & Goodenough, D. R. (1981). Cognitive style: essence and origins. New York: International Universities Press.
H.-W. Chou / Computers & Education 37 (2001) 11–25
25
Witkin, H. A., Moore, C. A., Goodenough, D. R., & Cox, P. W. (1977). Field-dependent and field-independent cognitive styles and their educational implications. Review of Educational Research, 47, 1–64. Witkin, H. A., Oltman, P., Raskin, E., & Karp, S. (1971). A manual for the embedded figures test. Palo Alto, CA: Consulting Psychologist Press. Whiteley Jr., B. E. (1997). Gender differences in computer-related attitudes and behavior: a meta-analysis. Computers in Human Behavior, 13(1), 1–22.