Influence of training schedule on development of perceptual–motor control skills for construction equipment operators in a virtual training system

AUTCON-01602; No of Pages 9 Automation in Construction xxx (2013) xxx–xxx

Contents lists available at SciVerse ScienceDirect

Automation in Construction journal homepage: www.elsevier.com/locate/autcon

Influence of training schedule on development of perceptual–motor control skills for construction equipment operators in a virtual training system Xing Su a,⁎, Phillip S. Dunston a, Robert W. Proctor b, Xiangyu Wang c,d a

School of Civil Engineering, Purdue University, 550 Stadium Mall Dr., West Lafayette, IN 47907-2051, United States Department of Psychological Sciences, Purdue University, 703 Third St., West Lafayette, IN 47907-2081, United States School of Built Environment, Curtin University, GPO Box U1987 Perth, WA 6845, Australia d Department of Housing Interior Design, Kyung Hee University, Seoul, Republic of Korea b c

a r t i c l e

i n f o

Article history: Accepted 30 May 2013 Available online xxxx Keywords: Construction equipment Contextual interference Task complexity Intra-task interference Practice schedule Simulator Training Virtual environment Virtual training system

a b s t r a c t Virtual Training Systems have emerged as alternative tools for construction equipment operator training that may reduce costs, avoid risks, and provide flexible environments for various training purposes. However, principles for determining training-schedule design for efficient utilization of these systems are not well established. The present research compared performance of two groups, one adopting a mixed practice schedule (with high contextual interference) and the other employing a blocked practice schedule (presumably with lower contextual interference) for learning construction excavator control skills in a computer-based virtual environment. No significant difference was found with regard to achieved skill level and skill consistency. A possible reason is that the complexity of the training task created a degree of intra-task interference in the blocked practice schedule comparable to the interference created by the mixed practice schedule. This result implies a need to further assay construction equipment operation to critically isolate influences on operator skill development. © 2013 Elsevier B.V. All rights reserved.

1. Introduction 1.1. Background Training methods for construction equipment operation have historically included classroom instruction and hands-on training in a field setting. This hands-on practice plays an important role in construction equipment operator training [1], but it involves risk of personal injury and equipment damage, cost of fuel and site, and assignment of training personnel. It would be more economical to adopt an alternative approach to replace at least part of the hands-on practice time on real equipment [2]. The emergence of virtual reality (VR)-based construction equipment simulators, a type of Virtual Training System (VTS), addresses this issue [3]. Training on a simulator avoids expenses for fuel, equipment rental, site, and more importantly, the risks from real site operation hazards. These training simulators can model hazardous situations in a virtual environment, and trainees can practice such tasks repeatedly without exposing themselves, the equipment, or other personnel to any risk of harm. It follows that the practice be programmed in a manner that maximizes skill acquisition from the virtual environment [4].

⁎ Corresponding author. E-mail addresses: [email protected] (X. Su), [email protected] (P.S. Dunston), [email protected] (R.W. Proctor), [email protected] (X. Wang).

Several international construction equipment manufacturing companies (i.e., Caterpillar, John Deere, Komatsu, and Volvo) have collaborated with technology firms to develop simulators for various equipment types. The types of equipment simulated for operator training include hydraulic excavator, wheel loader, articulated hauler, and tracked feller, among others. Previous research in simulator training has focused predominantly on generating and applying new technologies to improve the simulation graphics and system functionality [5–8]. However, proof of the training principles for efficient utilization of a VTS, especially for operating heavy construction equipment, is still not found in the published literature [3]. An increasing number of studies have been conducted regarding perceptual–motor skills development through simulators and other VTSs since the turn of this century [9]. The results have shown that VTSs are a promising medium for training, but there is as of yet little empirical and conclusive research with respect to the effectiveness of such elements as training exercise design, training task sequence, mechanisms of feedback, and so on, within the medium. Verifying influences of the general principles that have been established in the literature on learning and training [9,10] may provide evidence for using VTSs more efficiently in construction equipment operator training [1,4]. The practical implications of the work presented in this paper are as follows: 1. The results contribute to a scientific basis for construction researchers to further investigate how construction equipment operation skills can be developed and transferred within a VTS.

0926-5805/$ – see front matter © 2013 Elsevier B.V. All rights reserved. http://dx.doi.org/10.1016/j.autcon.2013.05.029

Please cite this article as: X. Su, et al., Influence of training schedule on development of perceptual–motor control skills for construction equipment operators in a virtual training system, Automation in Construction (2013), http://dx.doi.org/10.1016/j.autcon.2013.05.029

2

X. Su et al. / Automation in Construction xxx (2013) xxx–xxx

2. The findings illuminate principles for training firms to customize (plan and devise) their program's training schedules for a specific set of skills, so as to achieve the maximum learning and performance from their trainees. 3. The results inform strategies and guidelines for VTS developers/ designers to craft built-in programs to enable different schedules to facilitate realization of skill objectives. Specifically, this work takes a step toward clarifying principles that can inform the design of the training schedule, i.e., the patterns of practice, to yield the most efficient skill acquisition for equipment operator training via a VTS. Toward that end, the concept of skill development must be clarified and specific principles of learning must be tested. 1.2. The psychology of skill development and memory Psychologists have classified skills into two main classes: cognitive and perceptual–motor [10,11]. Cognitive skills relate to problem solving in intellectual tasks, where a person's knowledge and reasoning capacity play more vital roles in success than do their physical capabilities. Perceptual–motor skills relate to executing physical actions in response to perceptual information, and they depend on factors such as hand–eye coordination and working memory abilities. For example, cognitive skills are involved in solving mathematical problems, whereas driving a car mostly relies extensively on a person's perceptual–motor skills. Operation of construction equipment is also considered to fall primarily under the category of perceptual–motor skill. Devising a program of practice in accordance with appropriate training principles is critical for perceptual–motor skill learning. Since the study of Snoddy in 1926 [12], it has been found that group mean performance time, when considered as the task criterion in perceptual–motor skills, tends to decrease with practice as a function of a power law. This finding has been replicated in many perceptual– motor experiments [13]. Different practice principles have also been investigated by researchers such as deliberate practice, errorless learning, variability of practice, and contextual interference (CI) in practice [14–16]. Among them, the phenomenon of CI is one of the most frequently studied topics in the field of perceptual–motor learning [9,17]. This training principle is highly related to training schedule design and has the potential to be adopted for employing VTSs. The key aspects of CI are further introduced in the next section. 1.3. Contextual interference and practice schedule CI is a principle of learning that refers to the relative amount of interference introduced during practice of two or more tasks [18]. When practice for a set of task components is designed with high levels of interference between the individual components, there is poor performance during training but better skill transfer and retention [9]. This phenomenon, called the CI effect, was first noted by Battig [19] and further demonstrated by Shea and Morgan [18]. To date, the CI effect has been shown to benefit learning of many physical skills such as execution of snowboarding turns, kayaking, and tying knots [20–22]. The CI effect has drawn considerable interest from not only researchers studying perceptual–motor skill learning, but also researchers studying cognitive skill acquisition. Studies of the CI effect have been conducted in learning many simple or high-level cognitive skills such as foreign language vocabulary [23] and how to troubleshoot complex systems [24]. The results have demonstrated that CI is a general phenomenon that affects learning of a variety of skills [but see 25]. A training approach for complex tasks might involve a schedule of practice of the whole task or a schedule of practice of the individual component tasks [26]. With respect to the latter, there are two

popular sequencing methods called mixed and blocked practice schedules. The CI effect is most commonly introduced by adopting a mixed practice schedule, alternating practice of the component tasks, often in a purely random sequence, which is presumed to create high CI. The blocked practice schedule, typified by complete practice of one component task before moving to the next, is considered to produce lower CI. Previous research indicates that learning a set of task (or skill) components with high levels of CI between the individual components leads to improved skill retention and transfer when tested, with the noted effect of poor performance during training (i.e., practice) [19,27]. Research has been conducted with the goal of finding an explanation for the CI effect. Lee and Magill [28] proposed that, in a blocked practice schedule, movement information related to a given task can be held in working memory for several trials during which the task remains unchanged. On the contrary, loss of information occurs when the task changes on successive trials in a random or mixed practice schedule; thus, participants are forced to retrieve information from long-term memory frequently because they must partially or completely reconstruct an action plan at the beginning of each trial. These complex and effortful cognitive processes, which do not happen in a blocked practice schedule, lead to better performance on subsequent retention tests [29]. This conclusion was supported by a study conducted by Immink and Wright [30] in which participants had to execute sequences of four keypresses with the fingers of the left hand: The mixed-schedule participants required longer study time but demonstrated better performance on a retention test. Cross et al. [31] measured brain activity using functional magnetic resonance brain imaging in a similar sequencing task and found that the mixed-schedule group showed more activity in regions of the brain associated with motor preparation and sequencing than did the blocked-schedule group. Cross et al. concluded that their results support the hypothesis that the benefits of skill retention displayed by participants in the mixed-schedule group can be attributed to the extra time spent on reconstructing the component movements or actions during training. 1.4. Problem statement and hypothesis In light of conclusions from prior research investigations of practice schedule effects, we assumed that skill acquisition for construction equipment operators follows the noted principles of learning for perceptual–motor skills and hypothesized that trainees should benefit more from a mixed practice schedule than from a blocked one. The class of construction equipment training simulators described earlier typically presents various modules for the trainee to practice simple tasks to develop skills that subsequently can be combined to perform more complex tasks. The question raised in the present research is whether the simple tasks should be practiced according to a mixed or blocked schedule. To examine the potency of the CI effect and different practice schedules, the following hypothesis was tested: Adopting a mixed practice schedule in a hydraulic excavator training simulator enhances skill learning compared to a blocked practice schedule, as demonstrated by participants who received the mixed practice schedule performing better in the final complex test (i.e., better transfer to an integrated skill) than those who received the blocked schedule, although a blocked practice schedule in the simulator yields better performance during training. The confidence levels of all participants regarding their operating skills at the end of training were also recorded to further evaluate the latter point. 2. Method 2.1. Participants Participants were recruited from the Purdue University student population. A total of 30 students participated, aged from 21 to 29.

Please cite this article as: X. Su, et al., Influence of training schedule on development of perceptual–motor control skills for construction equipment operators in a virtual training system, Automation in Construction (2013), http://dx.doi.org/10.1016/j.autcon.2013.05.029

X. Su et al. / Automation in Construction xxx (2013) xxx–xxx

None of them had any previous experience operating a hydraulic excavator. There were 10 males and 5 females in the mixed group, 12 males and 3 females in the blocked group, randomly assigned. The mean ages of the participants in the respective groups were 23.5 and 24.9 years, with corresponding standard deviations of 1.46 and 2.40. 2.2. Experiment apparatus and training modules Simlog's Hydraulic Excavator Personal Simulator (http://www. simlog.com/ index.html) was used to conduct the experiment. This system, as shown in Fig. 1, was set up on a desktop computer with a 19-in. LCD monitor and original equipment manufacturer (OEM) joystick controls. The Simlog system was chosen for its high quality graphics, OEM joystick controls, and affordability. Replica pedal controls and a chair base assembly are also available from Simlog but were not part of the setup used in this experiment. There are also other simulator models on the market. Some are fully equipped with cabin, pedals, a large display, and more virtual training task modules, but at a much higher price. The focus of the study is to compare performance of two groups with different training schedules on learning of construction excavator control skills through a virtual environment using a computer-based simulator. The virtual simulator is an independent device, and therefore, it does not matter whether the simulator is particularly sophisticated in its functions, as long as it provides a fully functional and usable platform for the devised experimental procedure and valid data collection can be implemented. An expert operator of hydraulic excavators was interviewed after the expert first examined the Simlog hydraulic excavator simulator used in this study [1]. The interviewee rated several features of the simulator. The similarity of the controls to real ones was rated as excellent (5 on 1–5 scale), and the comfort, graphics, and field of view were rated as good (4 on a 1–5 scale). Considering all these factors, the Simlog system was deemed to be a sufficiently functional platform for the study. Simlog's hydraulic excavator simulator has 10 built-in virtual training modules. Among those, the bucket-position and orientation task “Bucket Placement” and the bucket-movement tasks “Raking the Green” and “Over the Moon” were selected to serve as the

3

training tasks in this experiment; “Single Pass Digging” was selected to be the final test. In the “Bucket Placement” (Fig. 2a), trainees need to move the bucket of the hydraulic excavator to a target position, which is indicated by a red “wire-frame” in the 3D space, with correct bucket position and angle of rotation. “Raking the Green” (Fig. 2b) requires trainees to move the bucket following a designated straight line toward the excavator cabin. “Over the Moon” (Fig. 2c) is similar to “Raking the Green” but with a difference that the bucket needs to be positioned following the trajectory of an arc instead of a straight line. This arc is oriented at an oblique angle relative to the face of the cabin. The objective of “Single Pass Digging” (Fig. 2d), the final test, is to learn to fill the bucket of the hydraulic excavator by digging just once within a designated area. According to the developers of the simulator, the module “Bucket Placement” is inspired by their experience with mobile crane simulation software (putting a payload in a target space), whereas the modules “Raking the Green” and “Over the Moon” both have their corresponding training tasks in the real world. These training modules together are aimed at developing the operating skills of dexterity and precision in moving the bucket. The “Bucket Placement” module instills how to be precise in positioning the bucket in a specific location and orientation. “Raking the Green” requires coordination of “stick” and “boom” functions to move along a straight line. “Over the Moon” further requires coordination of all “swing”, “stick”, “boom”, and “bucket” functions to make a rather long and smooth bucket motion along an arc. Simultaneous operation of the joystick controls with both hands is necessary to finish these tasks quickly and accurately. In the final test of “Single Pass Digging,” the participant first needs to place the bucket above the ground with a proper attack angle and then cut into the ground surface and make a smooth motion by activating the “stick in”, “boom raise”, and “bucket close” functions in a coordinated fashion. The task ends when the bucket is being raised; and “Bucket Close” should be executed to eliminate soil spillage. This module does not require the operator to dump the soil from the bucket. The three practice modules require distinctly different tool manipulation capabilities. In the final test, the skills should be combined smoothly to form a new motion—digging. Those skills learned from the training tasks for manipulating the bucket are fundamental and

Fig. 1. Simlog's hydraulic excavator personal simulator.

Please cite this article as: X. Su, et al., Influence of training schedule on development of perceptual–motor control skills for construction equipment operators in a virtual training system, Automation in Construction (2013), http://dx.doi.org/10.1016/j.autcon.2013.05.029

4

X. Su et al. / Automation in Construction xxx (2013) xxx–xxx

Fig. 2. Screenshots of trials for each of the three training tasks, (a) Bucket Placement, (b) Raking the Green, and c) Over the Moon, and the final test, (d) Single Pass Digging. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

transferrable. They are designed for developing what might be called “coordination skills” which can generally help trainees improve their efficient and smooth performance of more complex operating skills. 2.3. Procedure The experiment contained four steps: formal instruction, hands-on exploration, virtual tasks practice, and final test. At first, participants were randomly assigned to a mixed-practice group or a blockedpractice group. Both groups received the same formal orientation and were given the same time (3 min) to perform hands-on exploration on the Simlog Hydraulic Excavator Personal Simulator. Then both groups were required to practice the three coordination training tasks: “Bucket Placement”, “Raking the Green”, and “Over the Moon” without taking any breaks. Participants in the mixed group were trained by employing a mixed sequence of these coordination skill practices such that no task (or skill) was performed twice in a row. Each participant performed nine cycles for the whole training process, with each cycle containing one trial each of the “Bucket Placement”, “Raking the Green”, and “Over the Moon” tasks, in that fixed sequence. The blocked group was required to perform each task nine times consecutively before moving to the next task. This sequence of the three tasks corresponds to their order of presentation among the simulator training modules, presumably based upon some rationalization of increasing difficulty of complexity. This progression becomes evident as one considers the coordinated motions required to accomplish each task (refer to the module explanations in Section 2.2). The task sequence for the mixed group was not designed to be purely random based on the desire to avoid repeating a task two times in a row, in order to maximize CI within the limited number of trials. After the training and a brief explanation of the final test, each participant was required to fill out a questionnaire. They provided a rating of their level of agreement with two statements: “The training

practice was effective in helping me to master the controls” and “I should have been given more time for practice.” They also provided a rating response to the question “What is your confidence level for performing the test in an effective manner?” All three ratings were on a 5-point scale. After the questionnaire, all the participants were directed to the final test that drew upon the coordination skills that had been practiced. The time interval between the training and the final test was about 5 min, including the time for filling out the questionnaire. The results of the final test were recorded. The number of trials for each task was determined based on the results of pilot tests and previous experiences with the simulator. The whole training process was long enough for participants' execution times on the practice tasks to reach asymptotic levels. Our aim was to test for differences at this point of skill acquisition and then test for performance of the final task that consolidates those skills. We designed the test to involve the same number of trials in order to capture not only initial differences in skill transfer but also trends that might appear. The experiment, including the training and the final test, lasted approximately 1.5 h for each participant. This amount of time is not enough for a novice to reach an expert level but is sufficient to test which schedule method—blocked or mixed— works better in facilitating skill development. 2.4. Formal instruction Before being seated at the simulator, both groups were given an audio–visual instructional presentation about the hydraulic excavator's basic control functions. The presentation first explained the excavator's basic control components—cabin, boom, stick, and bucket. It also explained the movements of those components, which are “bucket dump”, “bucket close”, “stick in”, “stick out”, “boom raise”, “boom lower”, “swing left”, and “swing right”—eight movements in all. Then it introduced how the control joysticks are used to execute

Please cite this article as: X. Su, et al., Influence of training schedule on development of perceptual–motor control skills for construction equipment operators in a virtual training system, Automation in Construction (2013), http://dx.doi.org/10.1016/j.autcon.2013.05.029

X. Su et al. / Automation in Construction xxx (2013) xxx–xxx

the eight movements or functions explained in the first part of the presentation. Each control joystick can move in four directions: up (forward), down (backward), left, and right. Therefore, the two

5

control joysticks have eight basic movements which are mapped to the eight hydraulic excavator functions. Fig. 3 depicts the control mapping that each participant was required to learn.

a(1)

a(2)

a(3)

a(4)

b(1)

b(2)

b(3)

b(4)

Fig. 3. Screenshots from the formal instruction: a(1) through a(4) show components and control functions; b(1) through b(4) show manual control actions and their mappings to functions.

Please cite this article as: X. Su, et al., Influence of training schedule on development of perceptual–motor control skills for construction equipment operators in a virtual training system, Automation in Construction (2013), http://dx.doi.org/10.1016/j.autcon.2013.05.029

6

X. Su et al. / Automation in Construction xxx (2013) xxx–xxx

2.5. Data collection

Table 2 Analysis results for effects revealed in the training data.

3. Results 3.1. Training performance The first step of analysis focused on effects that could be observed in the training (practice) phase of the experiment. Table 2 shows the results of the ANOVA of execution time for each of the three training

Table 1 Requirements of the training tasks. Task

Requirement

Bucket Placement

Positioning error b0.3 m Alignment error b10° Attack angle error b10° Bucket middle tooth must reach the target indicator Bucket middle tooth must reach the target indicator

Raking the Green Over the Moon

F value

Group Trial Group ∗ Trial Group Trial Group ∗ Trial Group Trial Group ∗ Trial

F(1,28) F(8,28) F(8,28) F(1,28) F(8,28) F(8,28) F(1,28) F(8,28) F(8,28)

Raking the Green (task 2)

Over the Moon (task 3)

p-Value = = = = = = = = =

1.46 9.41 2.68 2.27 5.70 3.27 0.80 6.84 2.28

0.2368 b.0001 0.0254 0.1431 0.0002 0.0093 0.3783 b.0001 0.0508

tasks. The Group (blocked or mixed) main effects were not significant (p > 0.05) in the case of all three training tasks. In contrast, the Trial (1 through 9) main effects were all significant, and the interactions of the two variables were significant for all but the “Over the Moon” task, for which the p value barely exceeded the .05 level. The trial effect reflects an incremental improvement with practice. The interaction effect is shown in Fig. 4. There were slight trends for the mixed group to show more improvement than the blocked group from the first to last trial in the “Bucket Placement” and “Over the Moon” tasks, but less improvement in “Raking the Green”. However, these differences were mainly due to the first trial, for which execution time was long and variable. By the last trial performance was similar for the two groups. The blocked group did not show any clear and persistent performance advantage over the mixed group. 3.2. Participants' evaluations and confidence level after practice Table 3 shows the ratings of the participants' agreement with the statements “The training practice was effective in helping me to master the controls” and “I should have been given more time for

a) Bucket Placement Execution time (s)

The training execution time for each task was submitted to a 2 (Group) × 9 (Trial) ANOVA with repeated measures on the Trial factor. The confidence levels of the two groups were compared using a one-way ANOVA. For the final test, the productivity was submitted to a 2 (Group) × 9 (Trial) ANOVA with repeated measures on the Trial factor. Observations of trends in the data prompted additional comparisons of some critical trials, namely the 1st through 4th trials and the 7th, 8th, and 9th trials. Finally, the standard deviations of the initial attack angle were compared for the two groups using a one-way ANOVA.

Treatment

Bucket Placement (task 1)

240.00 210.00 180.00 150.00 120.00 90.00 60.00 30.00 0.00

Mixed Group Blocked Group

1

2

3

4

5

6

7

8

9

No. of trials

b) Raking the Green Execution time (s)

2.6. Data analysis

Task

120.00 90.00 60.00 Mixed Group

30.00 Blocked Group

0.00

1

2

3

4

5

6

7

8

9

No. of trials

c) Over the Moon Execution time (s)

The execution time of each training task was collected and considered to be the most appropriate factor representing a participant's performance during training. There are some requirements, as shown in Table 1, that the participants had to satisfy to complete the tasks and that link the execution time with successful performance, thus justifying use of the execution time for comparison. When a participant felt that the task was successfully completed, she/he could activate the horn button (on the top of the left joystick) to end the trial. If all requirements were satisfied, the clock would stop and results data would be presented on the display. If any requirements were not satisfied, pressing the button only caused the horn to sound, and then the trainee would need to make adjustments. During the bucket movement training tasks (task 2 and task 3), a cursor is presented with its color changing to signify whether the bucket movement is tracking within acceptable proximity to the indicated path (line or arc). If the bucket moved too far away from the trajectory guide line, the cursor would turn red to prompt the participant to improve the performance accuracy. Two measures were collected and analyzed to represent the performances in the final test: execution time and soil volume in the bucket. Each trainee was informed that the execution time and soil volume were equally important. The goal of this test was to maximize productivity (i.e., volume of soil per unit of time). Therefore, “Productivity”, bucket volume divided by execution time, was the chief indicator of the participant's digging skill. Initial attack angle is another measure worth consideration because attacking the soil at the correct angle is a mark of skillful efficiency. The fact that the participants were not instructed regarding this parameter, however, argues against emphasizing its use as a primary measure of performance for this experiment, but an initial attack angle data series with a small standard deviation would indicate that the participant has the ability to control the bucket in a consistent manner. All these data were automatically recorded and provided by the simulator software.

180.00 150.00 120.00 90.00 60.00 30.00 0.00

Mixed Group Blocked Group

1

2

3

4

5

6

7

8

9

No. of trials Fig. 4. Execution time on the three practice tasks as a function of training group and number of trials performing the task.

Please cite this article as: X. Su, et al., Influence of training schedule on development of perceptual–motor control skills for construction equipment operators in a virtual training system, Automation in Construction (2013), http://dx.doi.org/10.1016/j.autcon.2013.05.029

X. Su et al. / Automation in Construction xxx (2013) xxx–xxx Table 3 Participants' evaluations and confidence. Mixed group

Blocked group

Training effectiveness Mean SD Median

4.47 0.64 5

4.47 0.64 5

Inadequate time Mean SD Median

2.47 1.19 2

2.67 1.17 2

Confidence level Mean SD Median

3.93 0.59 4

3.73 0.80 4

practice” and their responses to the question “What is your confidence level for performing the test in an effective manner?” Both groups gave exactly the same mean rating to the first question about training effectiveness. The ratings are high, and more than half of the participants considered the training practice to be very effective (median = 5/5). The ratings to the second question about inadequate time are more diverse, but still more than half asserted that the time was not inadequate (median = 2/5). The rating results regarding confidence level basically match the feedback from question 2. Both groups gave high average confidence levels. More than half of the participants rated their confidence levels greater than or equal to 4/5, and none rated their confidence less than 3/5. A one-way ANOVA on the confidence level revealed no significant difference (p = 0.44).

7

common to both groups between the 7th and 8th trials and the divergence between the 8th and 9th trials provoked deeper questions regarding conditions of the test. The decreasing trend on the 8th trial can be attributed, at least in part, to the software setting. Fig. 6 depicts the scene setup for the 7th (a), 8th (b), and 9th (c) test trials. The target area marked by the red border is narrower on the 8th trial as compared to the target areas for the 7th and 9th trials, only slightly wider than the excavator bucket. Further review of the “Single Pass Digging” module reveals that the target areas in other trials are all similar in width to those for the 7th and the 9th trials. It is reasonable to assume this change for the 8th trial, i.e., a more restrictive digging area, could affect the performance of our novice participants, who reasonably would slow down as they intensify their focus on the smaller target. The average results for the 9th trials, however, are more difficult to explain. The blocked group improved their performance over that of their 8th trial and even over their 7th, whereas the mixed group, on average, continued their worsening trend. Inspection of the data,

3.3. Final test performance The ANOVA results for productivity on all nine trials of the final test showed no group main effect, F(1,28) b 1.0, a significant trial effect, F(8224) = 3.74, p b 0.001, and no interaction, F(8224) = 1.25, p = 0.2690. As shown in Fig. 5, the main trend was for performance to increase across trials to a similar extent for the two groups. Although practice schedule was not a significant factor influencing performance in the overall analysis, there was a tendency toward better performance for the mixed group than for the blocked group on the first four trials of the final test (M = .023 m3/s versus .020 m3/s). However, an ANOVA performed only on those trials showed neither a group main effect nor interaction with trials, Fs b 1.0. The analysis of the standard deviation of the initial attack angles further confirms that there was no significance difference between the two groups (p = 0.5910). The interaction plot of the productivity analysis, however, shows a peculiar trend on the last two trials, as seen in Fig. 5. The trend

Productivity (m3/S)

0.05

0.04

0.03

Mixed Group Blocked Group

0.02

0.01

1

2

3

4

5

6

7

8

9

No. of Trials Fig. 5. Productivity on the final test of Single Pass Digging as a function of training group and number of trials performing the task.

Fig. 6. Digging target scenarios in the final test of Single Pass Digging for trials 7 (a), 8 (b), and 9 (c).

Please cite this article as: X. Su, et al., Influence of training schedule on development of perceptual–motor control skills for construction equipment operators in a virtual training system, Automation in Construction (2013), http://dx.doi.org/10.1016/j.autcon.2013.05.029

8

X. Su et al. / Automation in Construction xxx (2013) xxx–xxx

revealed no outlier that could be responsible for the worse performance in the 9th trial for the mixed group. For reasons not made clear from the available data, the blocked group appears to have recovered immediately on the 9th trial while the mixed group did not. It is plausible that the mixed group became more tentative on their 9th trial, maybe expecting another complicating parameter. However, it should be noted that pair-wise comparisons between groups for each of the last three trials did not show any significant difference. Even the 9th trial comparison, under the Bonferroni adjustment, yielded p = 0.437. So, the trends cannot be regarded with any certainty at this time. 4. Discussion Contrary to our hypothesis, no advantage of the mixed over the blocked practice schedule on the final test was found for the training tasks employed in this experiment, and the possible reasons are discussed here. The presence of intra-task interference is one of the more plausible explanations for the experiment results. Current theoretical explanations regarding the effect of practice schedules during training consider that more cognitive effort is needed in the mixed or random practice compared to the blocked practice, which enhances later transfer and retention of the skill [28–33]. Participants have to form a partially or completely new plan for the next trial that is different from the most recent one in a random or mixed practice schedule; in contrast, participants following the blocked practice schedule are able to repeatedly use the same information in their working memory as the task does not change substantially. For the three built-in training modules used for practice in the present study, the background scenery changes across trials, as does the configuration of the display and the specific control actions required. For example, for the Bucket Placement module the location of the red target varies, and for the Over the Moon module the trajectory is to the left on some trials and the right on others. This variability across trials in the blocked training condition is akin to different specific movement distances or timings within a movement class that typically are regarded as mixed practice in learning of basic motor skills compared to blocked practice of the same exact movement [18,33]. This within-task variability may therefore have been sufficient to induce contextual interference. Albaret and Thon [32] similarly proposed that task complexity generates intra-task interference, which demands cognitive effort as well and may rival the interference created by a random practice schedule. Task complexity, as they defined it, depends on the number of movements or movement combinations required to complete the task. The situation of a more complex task may place both random (mixed) and blocked practice groups in similar conditions with little difference regarding interference. A blocked practice condition is more often considered to engage participants in shallow cognitive processes of movement information as mentioned above [28]. If a task is sufficiently complex, however, participants may have difficulty maintaining all related movement information in the working memory to the next trial, which forces them to construct a new action plan, even though the task does not change in the next trial. In this study, the coordinated manipulations of the joysticks for each of the training tasks is arguably more complex than typical sports training tasks such as golf club swinging and basketball shooting. This could be a possible reason why no CI effect was found in this experiment. In the training tasks, every motion practiced requires multiple steps in combination of the basic movements. None of the training tasks can be completed without combining and coordinating at least four joystick movement functions along a correct performing sequence. It might be difficult for participants to hold all the information about the controls of a task in the working memory from one trial to the next trial, even for those in the blocked group. It is possible that the complexity of the combination of four or more control joystick movements generated high enough intra-task interference to make

the CI introduced by the mixed practice schedule to be of no effect. In Wulf and Shea's [25] words, “The intratask interference inherent in complex tasks may be sufficient for effective learning under blocked conditions” (p. 189). It is not possible to say whether the varying conditions within each task or the task complexity was responsible for high CI within the blocked schedule, but any additional CI introduced by the mixed practice schedule was not sufficient for its effect to be evident over and above that due to the intra-task interference effect. This finding highlights the need for assessment of exactly what constitutes repetition and task complexity when investigating influences on the learning of construction equipment operating skills. Ollis et al. [22] suggested that the interrelationships between expertise, complexity, task needs, and extent of CI and other variables should be understood before a particular amount of CI could be prescribed as most beneficial. However, the problem remains of how to measure the task complexity and its impact. Wulf and Shea [25] stated that “complexity” is a multidimensional construct, and hence it is difficult to quantify its impact to the CI effect. More work is needed to formalize this assessment of the required perceptual– motor skills and thus facilitate more careful experimentation to determine more conclusively the CI effect as well as other perceptual– motor skill training principles and possibly exploit them. Based on the experiences and lessons from this research, two research directions are recommended to be explored in the future: 1. Investigate the existence of intra-task complexity and its effect in construction equipment operator skill training using VR-based simulators. Hierarchical Task Analysis (HTA) [34,35] might be the key to understanding this problem. HTA can be applied to decompose complex tasks into a hierarchy of goals and sub-goals. More fundamental understanding is expected to be thus obtained with regard to the structure of the executed tasks, coordination skill elements, and the cognitive efforts involved in operator skill training. 2. Investigate the distinction between task complexity and “task difficulty”. As task complexity is defined by Albaret and Thon [32] to depend on the number of movements or movement combinations required to complete the task, task difficulty should refer to the constraints of a movement in the task. The constraints of a movement can be characterized by the number of degrees of freedom of the joints, with one degree of freedom representing one dimension in which a movement can occur [36]. The more degrees of freedom the task has, the more difficult it is. For example, shooting a basketball near the three-point line may be considered as a difficult but not a complex task. There is only one movement but this movement has many constraints. Failure in following those constraints leads to an unsuccessful shot. Intuitively the task difficulty is one of the factors that determine the amount of practice required to achieve the asymptotic performance level for a trained skill. Research along this line is expected to help better explain and assess the components of each construction equipment operation training task. 3. The performance divergence after the 8th trials of the “Single Pass Digging” test, though not statistically significant, remains a puzzle at this time. Assuming the abovementioned rationale regarding the impact of the smaller digging target on the 8th trial, two open questions follow from the comparison of the 9th trial performances: 1) Did the practice experience somehow influence the capacity of the participants for recovering from their downward turn in performance on the 8th trial? 2) Would the mixed group have exhibited a recovery if more subsequent trials had been conducted? Unfortunately, this experiment did not collect data that could provide answers for these two questions. We are left to wonder whether the blocked practice facilitates a quicker recovery due to a persistent plan that has been reinforced by the numerous similar preceding trials.

Please cite this article as: X. Su, et al., Influence of training schedule on development of perceptual–motor control skills for construction equipment operators in a virtual training system, Automation in Construction (2013), http://dx.doi.org/10.1016/j.autcon.2013.05.029

X. Su et al. / Automation in Construction xxx (2013) xxx–xxx

5. Conclusions The overarching objective of this paper has been to establish scientific principles for efficient skill development and transfer with scientifically based utilization of a Virtual Training System (VTS). This paper has focused on development of complex perceptual motor-skills for the operation of a hydraulic excavator. Two training practice schedules were experimentally compared with respect to training effectiveness and the trainee's confidence level. The training effectiveness comparison further included comparisons of achieved skill levels and performance consistency. The results of this study indicate that a mixed practice schedule and a blocked practice schedule of coordination skills for training made no difference in respect to training effectiveness. A reflection on the present findings in light of a re-examination of literature suggests that intra-task variability may have been sufficient to induce high CI and that there is a need to better understand the task complexity and task difficulty for construction equipment operation prior to investigating and prescribing task and practice schedule designs. In particular, investigations of the CI effect may need to consider the intra-task interference resulting from the complexity of tasks under examination. Most of the operator training tasks are comparatively complex, which emphasizes the importance of study in task complexity versus task difficulty and the presence of intra-task interference. Acknowledgment The authors acknowledge the support for this research from the National Science Foundation (NSF), under Grant No. CMMI-0700492. Opinions, findings, conclusions, or recommendations are those of the authors and do not necessarily reflect the views of the National Science Foundation. References [1] P.S. Dunston, R.W. Proctor, X. Wang, Challenges in evaluating skill transfer from construction equipment simulators, Theoretical Issues in Ergonomics Science (Oct. 25 2011), http://dx.doi.org/10.1080/1463922X.2011.624647, (published online). [2] X. Wang, P.S. Dunston, Design, strategies, and issues towards an augmented reality-based construction training platform, Journal of Information Technology in Construction (ITcon) 12 (2007) 363–380, (International Council for Research and Innovation in Building and Construction (CIB), Rotterdam, Netherlands). [3] P.S. Dunston, R.W. Proctor, X. Su, M. Yamaguchi, X. Wang, R.(.I.). Chen, Principles for utilization of construction equipment operator training simulators, Proc. of the Construction Research Congress, 2, 2010, pp. 1039–1046. [4] J.C.Y. So, R.W. Proctor, P.S. Dunston, X. Wang, Better retention of skill operating a simulated hydraulic excavator after part-task than whole-task training, Human Factors 55 (2) (2013) 449–460. [5] J.J. Kozak, P.A. Hancock, E.J. Arthur, S.T. Chrysler, Transfer of training from virtual reality, Ergonomics 36 (7) (1993) 777–784. [6] R. Kenyon, M. Afenya, Training in virtual and real environments, Annals of Biomedical Engineering 23 (4) (1995) 445–455. [7] In: D.A. Vincenzi, J.A. Wise, M. Mouloua, P.A. Hancock (Eds.), Human Factors in Simulation and Training, CRC Press/Taylor & Francis, Boca Raton,FL, 2008. [8] X. Wang, P.S. Dunston, R. Proctor, L. Hou, Reflections on using a game engine to develop a virtual training system for construction excavator operators, Proc. of the 28th International Symposium on Automation and Robotics in Construction (ISARC 2011), June 29th–July 2nd, Seoul, Korea, 2011. [9] E.L. Abrahamse, M.L. Noordzij, Designing training programs for perceptual–motor skills: practical implications from the serial reaction time task, Revue Européenne de Psychologie Appliquée 61 (2011) 65–76.

9

[10] In: A.F. Healy, L.E. Bourne Jr. (Eds.), Training Cognition: Optimizing Efficiency, Durability, and Generalizability, Psychology Press, New York, 2012. [11] D.A. Rosenbaum, R.A. Carlson, R.O. Gilmore, Acquisition of intellectual and perceptual motor skills, Annual Review of Psychology 52 (2001) 453–470. [12] G.S. Snoddy, Learning and stability, Journal of Applied Psychology 10 (1926) 1–36. [13] K.M. Newell, Motor skill acquisition, Annual Review of Psychology 42 (1991) 217–237. [14] A. Baddeley, B.A. Wilson, When implicit learning fails: amnesia and the problem of error elimination, Neuropsychologia 32 (1994) 53–68. [15] B.A. Boyce, C.A. Coker, L.K. Bunker, Implications for variability of practice from pedagogy and motor learning perspectives: finding a common ground, Quest 58 (2006) 330–343. [16] N. Keith, K.A. Ericsson, A deliberate practice account of typing proficiency in everyday typists, Journal of Experimental Psychology. Applied 13 (3) (2007) 135–145. [17] R.A. Magill, Practice schedule considerations for enhancing human performance in sport, in: R.W. Christina, H.M. Eckert (Eds.), Enhancing Human Performance in Sport: New Concepts and Developments, Human Kinetics, Champaign, IL, 1992. [18] J.B. Shea, R.L. Morgan, Contextual interference effects on the acquisition, retention and transfer of a motor skill, Journal of Experimental Psychology: Human Learning and Memory 5 (2) (1979) 179–187. [19] W.F. Battig, Intratask interference as a source of facilitation in transfer and retention, in: R.F. Thompson, J.F. Voss (Eds.), Topics in Learning and Performance, Academic Press, New York, 1972, pp. 131–159. [20] P.J. Smith, Applying contextual interference to snowboarding skills, Perceptual and Motor Skills 95 (3) (2002) 999–1005. [21] P.J. Smith, M. Davies, Applying contextual interference to the Pawlata roll, Journal of Sports Sciences 13 (1995) 455–462. [22] S. Ollis, C. Button, M. Fairweather, The influence of professional expertise and task complexity upon the potency of the contextual interference effect, Acta Psychologica 118 (3) (2005) 229–244. [23] V.I. Schneider, A.F. Healy, K.A. Ericsson, L.E. Bourne Jr., What is learned under difficult conditions is hard to forget: contextual interference effects in foreign vocabulary acquisition, retention and transfer, Journal of Memory and Language 46 (2) (2002) 419–440. [24] M.B.M. de Crook, J.J.H. van Merriënboer, F.G.W.C. Paas, High versus low contextual interference in simulation-based training of troubleshooting skills: effects of transfer performance and invested mental effort, Computers in Human Behavior 14 (2) (1998) 249–267. [25] G. Wulf, C.H. Shea, Principles derived from the study of simple skills do not generalize complex skill learning, Psychonomic Bulletin and Review 9 (2) (2002) 195–211. [26] R. Brydges, H. Carnahan, D. Backstein, A. Dubrowski, Application of motor learning principles to complex surgical tasks: searching for the optimal practice schedule, Journal of Motor Behavior 39 (1) (2007) 40–48. [27] R.A. Magill, K.G. Hall, A review of the contextual interference effect in motor skill acquisition, Human Movement Science 9 (3–5) (1990) 241–289. [28] T.D. Lee, R.A. Magill, Can forgetting facilitate skill acquisition? in: D. Goodman, R.B. Wilberg, I.M. Franks (Eds.), Differing Perspectives in Motor Learning, Memory, and Control, North Holland, Amsterdam, 1985, pp. 3–22. [29] T.D. Lee, S.P. Swinnen, D.J. Serrien, Cognitive effort and motor learning, Quest 46 (1994) 328–344. [30] M.A. Immink, D.L. Wright, Contextual interference: a response planning account, Quarterly Journal of Experimental Psychology 51A (4) (1998) 735–754. [31] E. Cross, P. Schmitt, S. Grafton, Neural substrates of contextual interference during motor learning support a model of active preparation, Journal of Cognitive Neuroscience 19 (11) (2007) 1854–1871. [32] J.M. Albaret, B. Thon, Differential effects of task complexity on contextual interference in a drawing task, Acta Psychologica 100 (1–2) (1998) 9–24. [33] F.W. Wu, D.E. Young, S.L. Schandler, G. Meir, R.L.M. Judy, J. Perez, M.J. Cohen, Contextual interference and augmented feedback: is there an additive effect for motor learning? Human Movement Science 30 (2011) 1092–1101. [34] J. Annett, Hierarchical task analysis, in: D. Diaper, N.A. Stanton (Eds.), The handbook of task analysis in human-computer interaction, Lawrence Erlbaum Associates, London, 2004, pp. 67–82. [35] N.A. Stanton, Hierarchical task analysis: developments, applications, and extensions, Applied Ergonomics 37 (2006) 55–79. [36] R.W. Proctor, A. Dutta, Skill Acquisition and Human Performance, Sage, Thousand Oaks, CA, 1995..

Please cite this article as: X. Su, et al., Influence of training schedule on development of perceptual–motor control skills for construction equipment operators in a virtual training system, Automation in Construction (2013), http://dx.doi.org/10.1016/j.autcon.2013.05.029