- Email: [email protected]
Innovative system for real-time ergonomic feedback in industrial manufacturing
Applied Ergonomics 44 (2013) 566e574
Contents lists available at SciVerse ScienceDirect
Applied Ergonomics journal homepage: www.elsevier.com/locate/apergo
Innovative system for real-time ergonomic feedback in industrial manufacturing Nicolas Vignais a, *, Markus Miezal b, Gabriele Bleser b, Katharina Mura c, Dominic Gorecky c, Frédéric Marin a a
UMR CNRS 7338 Biomechanics and Bioengineering, University of Technology of Compiègne, Research Center, Dct Schweitzer Street, 60200 Compiègne, France DFKI GmbH, German Research Center for Artificial Intelligence, Trippstadter Strasse 122, D-67663 Kaiserslautern, Germany c SmartFactoryKL, Trippstadter Strasse 122, 67663 Kaiserslautern, Germany b
a r t i c l e i n f o
a b s t r a c t
Article history: Received 24 April 2012 Accepted 26 November 2012
This work presents a system that permits a real-time ergonomic assessment of manual tasks in an industrial environment. First, a biomechanical model of the upper body has been developed by using inertial sensors placed at different locations on the upper body. Based on this model, a computerized RULA ergonomic assessment was implemented to permit a global risk assessment of musculoskeletal disorders in real-time. Furthermore, local scores were calculated per segment, e.g. the neck region, and gave information on the local risks for musculoskeletal disorders. Visual information was fed back to the user by using a see-through head mounted display. Additional visual highlighting and auditory warnings were provided when some predefined thresholds were exceeded. In a user study (N ¼ 12 participants) a group with the RULA feedback was compared to a control group. Results demonstrate that the real-time ergonomic feedback significantly decreased the outcome of both globally as well as locally hazardous RULA values that are associated with increased risk for musculoskeletal disorders. Task execution time did not differ between groups. The real-time ergonomic tool introduced in this study has the potential to considerably reduce the risk of musculoskeletal disorders in industrial settings. Implications for ergonomics in manufacturing and user feedback modalities are further discussed. Ó 2012 Elsevier Ltd and The Ergonomics Society. All rights reserved.
Keywords: Ergonomics Real-time Augmented reality
1. Introduction The human worker plays an important role as operator and troubleshooter in modern and complex manufacturing systems. Many aspects of industrial work are of a physical nature, especially manual tasks, e.g. when the human worker has to pick up a component or arrange it in the assembly position. More generally, any kind of physical activity, such as lifting, pushing, pulling, carrying, moving, manipulating, holding or restraining objects, is considered to be a manual task (COSH, 2010). However, musculoskeletal disorders (MSDs) caused by manual tasks represent a large part of all work-related MSDs (Burgess-Limerick, 2007; Euzenat, 2010) and are a central issue for public health. Ergonomics specialists may identify conditions under which the risk of workrelated MSDs is high and develop adequate interventions (Waters, 2012). Moreover, these specialists play an increasing role in industry since they can also influence motion efficiency, operational safety and productivity (Battini et al., 2011; Thun et al., 2011).
* Corresponding author. Tel.: þ33 3 44 23 43 89; fax: þ3 33 44 23 79 42. E-mail address: [email protected] (N. Vignais).
Thus, developing supportive tools for the identification and evaluation of potentially hazardous motor tasks and postures is crucial for ergonomics researchers. Different methods and tools exist for the ergonomic assessment of manual tasks. A non-exhaustive list of them includes QEC, manTRA, RULA, REBA, HAL-TLV, OWAS, LUBA, OCRA, Strain Index, SNOOK tables and the NIOSH lifting equation (Andreoni et al., 2009). They can be classified into self-reports, observational methods and direct measurements (Li and Buckle, 1999; David, 2005). Firstly, self-reports involve worker diaries, interviews and questionnaires. This method has been used for the ergonomic assessment of handling work by Balogh et al. (2001) which constructed valid indices for mechanical exposure of the shouldere neck region. Nevertheless, self-reports entail some drawbacks like the unreliability of exposure perception and comprehension or interpretation according to the worker’s literacy. Secondly, observational methods aim to assess workplace exposure by evaluating the worker’s behavior on pro-forma sheets either while observing in the field or replaying videos. The Rapid Upper Limb Assessment (RULA) index is one of the most cited. This tool is based on the observation of the postures during a certain task and outputs biomechanical and postural load values on the whole body with
0003-6870/$ e see front matter Ó 2012 Elsevier Ltd and The Ergonomics Society. All rights reserved. http://dx.doi.org/10.1016/j.apergo.2012.11.008
N. Vignais et al. / Applied Ergonomics 44 (2013) 566e574
particular attention to the neck, trunk and upper limbs (McAtamney and Corlett, 1993). Observational methods based on videotaped work sequences analyze different kinds of manual tasks with specific software (Yen and Radwin, 1995; Chang et al., 2010; Radwin, 2011). Observational methods are affordable and practical for use in a wide range of professional situations (David, 2005). However, the scoring system may be questionable according to limited original epidemiological data. Video-based observational methods are also time consuming. Direct methods, in contrast, measure the risk of exposure in real-time by using sensors directly attached to the worker’s body. Some of these methods have been employed to study upper limbs (Radwin and Lin, 1993; Freivalds et al., 2000; Bernmark and Wiktorin, 2002). Nevertheless, all these methods require a complex and cost-intensive hardware setup and a lot of effort to analyze and interpret recorded data in real-time (David, 2005). As a consequence, the ergonomic evaluation of the directly assessed behavior has to be performed offline. Yet, postural evaluation that can be carried out in real-time provides benefits in practice (Mullineaux et al., 2012). If the system provides the worker with information concerning his current ergonomic behavior, then postures could be modified immediately. Furthermore, in the long run, associations between certain postures and their hazardousness could be learned. In order to receive the immediate feedback, Augmented Reality (AR) technology can be used during the actual work execution (Udani et al., 2012). Recent developments in sensor technology offer potential for regular industrial use in contrast to other tracking devices, such as range cameras or magnetic sensors, that are more effective in virtual environments (Ray and Teizer, 2012; Jayaram et al., 2006). For instance, an inertial measurement unit (IMUs) is a small-sized, inexpensive and low-power device suitable for monitoring the kinematics of a segment in real-time (Breen et al., 2009). If several inertial measurement units are connected, biomechanical models can be developed to capture a wide range of movements (Roetenberg et al., 2009). The aim of this study is to introduce an innovative and practical system for ergonomic assessment of a worker’s activity in realtime. The worker’s movements are recorded by using a lightweight sensor network composed of synchronized inertial measurements units (IMUs) and goniometers linked to an upper body biomechanical model. In order to perform the ergonomic evaluation in real-time, the RULA index is continuously computed
567
from this biomechanical model. The RULA assessment is simultaneously fed back to the user via a see-through head-mounted display (STHMD). A user study was conducted in order to reveal the influence of the proposed ergonomic feedback on workers’ motions during manual task execution. 2. Materials and methods In the first part of this paper, the data collection process based on the on-body sensor network and the biomechanical model of the upper body are introduced. Then, the real-time ergonomic computation based on the RULA sheet is described. Finally, a user study aiming at the evaluation of the real-time ergonomic feedback is presented. 2.1. Data collection 2.1.1. On-body sensor network The RULA score computation requires information about joint angles and segment orientations. These two parameters were derived using an on-body sensor network and a mobile processing unit (in this case, a standard laptop). This system is composed of seven wireless Colibri IMUs (Trivisio GmbH, Trier, Germany). Each lightweight sensor (48 g, 56 42 19 mm) contains a tri-axial accelerometer, a tri-axial gyroscope and a tri-axial magnetoinductive magnetic sensor. All sensors are sampled at 100 Hz. The IMUs are placed on the worker’s body as follows: one IMU for each upper arm, one IMU for each forearm, one IMU for the head, placed on the STHMD, one IMU for the trunk, located on the chest, and one IMU for the pelvis, placed on the sacrum (see Fig. 1a). This last IMU is necessary to define the movement of the trunk with respect to the pelvis segment. In order to record wrist angles (flexion/extension, radial/ulnar deviation), bi-axial SG65 goniometers (Biometrics Ltd., Newport, UK) have been added to the on-body sensor network and synchronized with the IMUs. 2.1.2. Biomechanical model of the upper body The worker’s upper body is then represented by a biomechanical model composed of ten rigid segments (trunk, clavicles, upper arms, forearms, hands and head) connected by anatomically motivated restricted articulations (pelvis, neck joint, sternoclavicular joints, shoulders, elbows and wrists) (see Fig. 1b). This
Fig. 1. On-body sensor network composed of IMUs coupled with goniometers (a) and underlying biomechanical model of the upper body (b) with rotation axes for local body frames, and DoF of each joint.
568
N. Vignais et al. / Applied Ergonomics 44 (2013) 566e574
biomechanical model provides 20 degrees of freedom (DoF): three for the pelvis, three for the neck joint, three for each shoulder (composed of acromioclavicular and glenohumeral joints), two for each elbow and two for each wrist. The sternoclavicular articulation depends on the upper arm elevation (Klopcar and Lenarcic, 2005) and therefore has only one dependent degree of mobility (see Fig. 1b). All rotation axes of local body frames are considered orthogonal in the model. In the neutral pose, which corresponds to the standard anatomical position, all local body frames are aligned with the global body frame, with the z-axes aligned with the longitudinal axes of the segments and gravity, the x-axes pointing laterally, and the y-axes posteriorly. 2.1.3. Calibration process Segmental lengths are derived from an anthropometric database by using the worker’s height as input (Winter, 2009). The relative positions of the IMUs with respect to adjoining segments are directly measured on the subject. Two postures performed by the subject are necessary to calibrate the orientations of the IMUs with respect to the body. In a first step, orientation axes of pelvis, chest and head IMUs are obtained as follows: - In the neutral pose, the z-axis is defined by gravity, which is given by the accelerometer measurements when the subject is stationary; - Then, by bending forward, the orthogonal x-axis is computed using a cross product; - Finally, by computing the cross product between z- and x-axes, the corresponding y-axis is calculated. In a second step, upper arm and forearm IMUs are aligned using accelerations and magnetic measurements recorded during the neutral posture. More precisely, z-axes (longitudinal axes of the segments) are aligned with gravity and x- and y-axes of upper limbs joints are aligned using magnetometer measurements. 2.1.4. Joint angle estimation Once the on-body sensor network has been fully calibrated, joint angles are computed in real-time. These are derived from the measured IMU data using model-based sensor fusion, in particular using a set of loosely coupled extended Kalman filters (EKFs) (Jazwinski, 1970). Firstly, the orientations of the trunk, the pelvis, and the head are estimated from the respective IMU data, each in a separate EKF, using a state-of-the-art orientation estimation algorithm (Harada et al., 2007). Neck and pelvis angles are deduced from the differential rotations. By using the chest orientation and measurements from the arm IMUs, joint angles of left and right elbows and shoulders relative to the trunk reference frame are computed. Each arm is handled in a separate EKF. The inertial measurement models are based on forward kinematic equations derived from the biomechanical model of the upper body (see Fig. 1b). By providing a global heading direction based on the earth or the local magnetic field, magnetometer information is used to compensate for drift caused by noise and fluctuating offsets in the inertial data. Further details about the data collection method and the particularities of the chosen state-space models for the joint angle estimation can be found in a previous publication (Bleser et al., 2011). 2.2. RULA assessment in real-time 2.2.1. Computation The RULA ergonomic tool estimates the exposure to upper limb MSDs by computing a global risk score. For a current posture, this global score ranges from one to seven, one being most comfortable.
This score is based on posture, muscle use, weight of loads, task duration, and repetitiveness (see Fig. 2). Originally based on a discrete observation of postures, the RULA tool has been implemented into the biomechanical model in order to compute the score of risk exposure to MSDs in real time. To this aim, the described biomechanical model provides joint angles necessary to locate the head (called ‘neck’ in the RULA sheet), trunk, upper arm, forearm and hand (called ‘wrist’ in the RULA sheet) positions. These angles allow computation of a score for each articulation, also called local score. The muscle-use scores are deduced from the task, i.e. a posture is considered as being ‘mainly static’ if variation of joint positions is under a specific threshold for 10 min. Additionally, shocks to the hand segments are detected during the movement based on the accelerations measured by the forearm IMUs. Sudden changes of acceleration magnitude indicate a shock. In the current system, these are determined based on an exponential sliding window over the differences in measured acceleration magnitudes between subsequent time steps with an experimentally determined threshold of 4 m/s2. However, in this study, it has to be noted that the manual tasks will mainly focus on the evaluation of posture and will not contain actions that imply a high risk of shocks. 2.2.2. RULA feedback For the current study, two feedback modalities were selected: an auditory signal linked to the global score and a visual cue related to local scores provided through the STHMD (see the video on http:// www.ict-cognito.org/demo.html for further details). The auditory signal was given as a warning when the global score was 7 for a period of at least 0.5 s. It reflects the recommendation of the RULA table to change the posture immediately. For a score of 5 or 6, the RULA table suggests modifying the posture soon. To address this, the auditory warning was given after spending at least 5 s in this range. In the same way, each time a local score exceeded a predefined threshold, the concerned joint and segment were highlighted in red in a schematic representation of the operator’s upper body on the STHMD (see Fig. 3). Consequently, the subject was able to understand which posture was inappropriate during the manual task and which segments had to be moved to decrease the RULA score. Based on a preliminary study, the visualization on the STHMD has been designed to be easily recognizable and not distractive. The red visual highlighting was presented if the local score was higher than the following thresholds: -
Shoulder and upper arm: 5 Elbow and lower arm: 3 Wrist and hand: 5 Neck and head: 4 Pelvis and trunk: 4
2.3. User study 2.3.1. Subjects All participants were male students aged between 20 and 27 years (M ¼ 22.5 years, SD ¼ 2.5). Participants’ mean height and mass were 180.42 cm (SD ¼ 6.53) and 76.58 kg (SD ¼ 12.71), respectively. They were randomly divided into two groups of six workers. Subsequent t-tests revealed no significant differences in subject characteristics (see Table 1). The first group had to perform an industrial manual task with the real-time auditory warnings and the RULA display shown in the STHMD. This group was subsequently called WR group (with RULA feedback). Participants of the second group had no access to the visual feedback and auditory warnings. The second group was subsequently called WOR group (without RULA feedback).
N. Vignais et al. / Applied Ergonomics 44 (2013) 566e574
569
Fig. 2. RULA sheet (Source: McAtamney and Corlett, 1993; with permission from Neese Consulting Inc., Leawood, KS, USA).
2.3.2. Study scenario Data were collected in the SmartFactoryKL living lab (Kaiserslautern, Germany) which is a manufacturer-independent demonstration and research platform (Zuehlke, 2008). This platform allowed testing of the on-body sensor network system in a realistic industrial production environment. The experimental task scenario was chosen to contain different postures and motions that are performed in industrial manufacturing. The manual task was composed of four subtasks (see Fig. 4): - turning two hand levers by 90 (A), - removing four fuses at knee level (B) and putting them into a box (C), - taking a screwdriver (D), unscrewing four screws of a transducer’s covering (E) and putting it down near the box (C), - unscrewing four screws of an upper transducer’s covering (F) and putting it down near the box (C). For this last subtask, a ladder may be used to reach a more comfortable working position (G). Forces and loads exerted during the task were lower than 4.4 lbs. The standard execution time was below 4 min.
2.3.3. Procedure Each participant signed an informed consent and answered anthropometric questions before the experimentation. Afterward, the participant had to put on the on-body sensor network and run through the calibration process. Then he received an overview and instructions of the task, the tools (screwdrivers) and equipment (ladder). He was asked to perform the task at normal pace and as accurately as possible. All participants were informed that the system will measure their kinematic behavior during task execution. The WR group was additionally informed that the system gives visual and auditory signals concerning the ergonomics. After participants finished the scenario they filled out a final questionnaire which included additional questions for the WR group concerning the feedback. 2.3.4. Metrics and statistical analysis For WOR and WR groups, the execution times and the percentage of time spent at each range defined by the RULA table (1e2, 3e4, 5e6 and 7) were computed. Moreover, an articulationbased analysis was carried out. For each articulation generating a local score higher than the predefined values, the total time of appearance and the frequency of appearance were computed.
570
N. Vignais et al. / Applied Ergonomics 44 (2013) 566e574
Fig. 3. Visual feedback representation on the STHMD. When a movement is performed in a hazardous way, segments exposed to a musculoskeletal risk are highlighted in red. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)
Subjective comments on the usefulness of visual and auditory feedback were also assessed through a questionnaire. On a 5-point Likert scale (“agree” to “disagree”) participants rated their impression of the appropriateness of the feedback concerning visibility, attention, attraction, and ergonomic support (e.g. “The information on the display supported me in executing the task in an ergonomic way.”). Additionally, they reported their experience concerning the combination of both feedback modalities (visual and auditory) and suggested potential improvements. Statistical analyses were performed to compare the results of both groups using t-tests or ManneWhitney U-tests when normality failed. 3. Results 3.1. Execution time The WR group needed significantly (p < 0.005) more total time to execute the whole task (M ¼ 227.99, SD ¼ 33.65 s) compared to the WOR group (M ¼ 156.98, SD ¼ 28.87 s). Nevertheless, when observing the execution time (accumulated time to complete each subtask) results showed that there was no significant difference (p ¼ 0.07) between the WOR group (129.38 26.11 s) and the WR group (175.35 38.65 s) (see Fig. 5). 3.2. Mean RULA score On average, participants in the WR group performed the task with a significantly lower global RULA score (3.95 0.83) than the WOR group (4.35 0.54) (p < 0.05). Concerning the right and the left RULA scores separately, these were higher for the WOR group (right: 4.4 0.65, left: 4.31 0.46) than for the WR group (right: 3.99 0.86, left: 3.9 0.88), but not significantly different. 3.3. Percentage of time spent in each RULA range The RULA table defines four score ranges (see Fig. 2): 1e2 corresponds to an acceptable posture; 3e4 means that the current posture needs further investigation and that a change may be needed; 5e6 means that the current posture needs further
investigation and that it has to be changed soon; 7 means that the movement can lead to MSDs, that it has to be investigated and changed immediately. Results demonstrated that there were some significant differences between WR and WOR groups for ranges 3e 4 (WOR ¼ 56.91 13.64%; WR ¼ 76.42 17.77%; p < 0.05) and 5e6 (WOR ¼ 30.48 6.92%; WR ¼ 16.76 13.22%; p < 0.05) (see Fig. 6). For score 7, on average it is higher for the WOR group (10.37 12.2) than for the WR group (3.4 5.47) without significance (p ¼ 0.07). It has to be noted that score 7 was rarely attained during the proposed task. 3.4. Local scores The RULA ergonomic tool allowed the calculation of a local score for each articulation or segment (see Fig. 2). By defining a value beyond which a local score was considered as potentially hazardous (see Section 2.2.2), it was possible to compute the percentage of time spent at a risky level per articulation or segment during the task (see Fig. 7). Results showed that subjects’ articulations and segments of the WOR group were significantly more exposed to a risk of MSDs during the task, especially for the left upper arm (WR ¼ 17.10 21.77%; WOR ¼ 38.14 6.14; p < 0.05), the left lower arm (WR ¼ 31.73 23.85%; WOR ¼ 75.13 16.74; p < 0.01), the right upper arm (WR ¼ 20.81 24.04%; WOR ¼ 49.09 12.64; p < 0.05), the right lower arm (WR ¼ 37.4 17.53%; WOR ¼ 74.7 18.66; p < 0.01) and the neck (WR ¼ 12.24 15.89%; WOR ¼ 34.03 10.8; p < 0.05). 3.5. Subjective reports All participants strongly agreed or agreed that the visual feedback was sufficiently visible (see Fig. 8). Most of the participants (N ¼ 5) approved that the visual feedback was suitable to attract attention. Half of the subjects stated that the visual information supported more ergonomic behavior, whereas the rest of subjects were undecided. Concerning the auditory feedback, almost all participants (N ¼ 5) agreed that it attracted their attention (one participant disagreed). As can be seen from Fig. 8, more than half of the participants (N ¼ 4) agreed that the auditory feedback helped
N. Vignais et al. / Applied Ergonomics 44 (2013) 566e574
571
Table 1 Characteristics of WR and WOR group participants: mean (standard deviation).
Age (years) Height (cm) Weight (kg) Visual impairment Shortsighted Farsighted No impairment Handedness Right Left Both Previous experience with AR Yes No
WR group
WOR group
21.5 (1.6) 180.7 (6.15) 82.83 (12.64)
22.8 (2.7) 180.2 (7.47) 70.33 (10.09)
n¼2 n¼0 n¼4
n¼2 n¼0 n¼4
n¼5 n¼1 n¼0
n¼4 n¼1 n¼1
n¼2 n¼4
n¼2 n¼4 Fig. 5. Total times and execution times for WR and WOR groups (***p < 0.001).
them to behave in a more ergonomic way; two participants were either undecided or disagreed respectively. Five out of six subjects reported that a combination of visual and auditory feedback was preferable to using only one modality. Two subjects argued that both feedback types complemented each other; while the auditory feedback warned reliably that there was a risky posture, the visual feedback implied more detailed information on which segments were involved.
In this study, an innovative system for real-time ergonomic evaluation and feedback in an industrial environment was introduced. The ergonomic assessment was based on biomechanical parameter estimation of the upper body using an on-body sensor network of IMUs and goniometers. By using the RULA tool, this evaluation permitted computation of a global score reflecting the risk of exposure to MSDs in real-time during the movement. Moreover, some local scores were calculated and linked to the specific risk of MSDs per articulation or segment. Visual and auditory feedback was implemented into a STHMD and tested in a realistic, industrial task. For the testing, two groups have been defined: a group with the RULA feedback (WR group), and a group without the RULA feedback (WOR group). Results demonstrated that the total time to perform the task was significantly higher for the WR group than for the WOR group. However, when analyzing the execution time during the task, there was no more significant difference, even if this time was higher for the WR group. Hence,
this latter group seemed to adapt to the ergonomic feedback during the task, i.e. paying attention to a changing display, reacting to warning signals, improving their working position by using a ladder, without the feedback significantly influencing the execution time of the task. Concerning the global risk of MSDs during the movement, results showed that the mean RULA score was significantly greater for the WOR group than for the WR group. It can be suggested that the real-time RULA feedback has influenced WR subjects by decreasing the risk of MSDs during the task. Indeed, WR group subjects had the possibility to modify their movement when they obtained an auditory cue during the movement. Moreover, the visual feedback indicated which parts of the body are subject to a MSD risk. By considering different ranges provided by the RULA table, it can be demonstrated that this decrease of risk comes from the fact that the WR group spent significantly more time at range 3e4, whereas the WOR group is significantly more exposed to the range 5e6. From the RULA table, this last range can be considered as a risky range as the posture of the subject needed “further investigation and has to be changed soon”. Furthermore, the WR group spent less time at a RULA score of 7, even if this trend was not significant. By allowing the operator to spend less time at a risky level, the real-time ergonomic feedback allowed prevention of MSDs. The main contribution of this new tool comes from the fact that this kind of prevention can be performed in real time in an industrial environment, which has, to our knowledge, never been
Fig. 4. Localization of the four subtasks on the production line.
Fig. 6. Percentage of time spent at each range during the task for WR and WOR groups (*p < 0.05).
4. Discussion
572
N. Vignais et al. / Applied Ergonomics 44 (2013) 566e574
Fig. 7. Percentage of time spent at a risky level per articulation or segment (*p < 0.05; **p < 0.01).
suggested in the past. Lately, proactive ergonomics has been used to prevent the risk of MSDs in industry (Nomura and Sawada, 2001; Chaffin, 2005). Virtual environments allow workplaces and tasks to be simulated even before the facilities are physically in place. Some essential parameters in ergonomics such as reach, visibility and spatial workplace organization have been proactively tested in virtual environments (Jayaram et al., 2006). Concerning ergonomic feedback in a virtual environment, Hu et al. (2012) have evaluated the impact of multimodal feedback (visual, auditory, tactile) on ergonomic measurements for a simulated drilling task. The authors demonstrated that task completion time, maximum force capacity reduction, body part discomfort, rated perceived exertion and rated task difficulty were significantly higher in the virtual environment than in a real environment. These findings indicate that the
subjects felt more discomfort, experienced faster fatigue and required longer time in the virtual environment than in the real environment when executing the same task. Thus, even if the proactive manner presents great potential in evaluating humane machine interfaces, this tool requires still further improvements to completely integrate into simulation and ergonomic evaluation methods (Hansson et al., 2001; Chaffin, 2005). In contrast, it has been demonstrated that, in an in situ experiment, using a visual feedback in real-time can significantly decreases the percentage of time spent in bad postures (Breen et al., 2009). Consequently, the proposed approach can be considered as a complementary tool for ergonomic evaluation in industry. After designing an ergonomic workplace in a virtual environment according to the proactive manner, the current real-time ergonomic feedback could be
Fig. 8. Graphical representation of subjective reports in the WR group (A: auditory feedback; V: visual feedback; 1: strongly disagree; 2: disagree; 3: neutral; 4: agree; 5: strongly agree). Values represent averages overall participants.
N. Vignais et al. / Applied Ergonomics 44 (2013) 566e574
employed to validate the virtual design, refine the industrial environment and improve required movements to decrease the risk of MSDs. By using local scores to compute the global score, the RULA table permitted an evaluation of the risk exposure of each articulation or segment in real-time during the task. Indeed, results demonstrated that the percentage of time spent at a risky level was significantly higher for the upper and lower arms and the neck of the WOR group subjects’ during the specific task chosen for the user test. The current study permitted identification of anatomical parts that were protected by the use of the real-time ergonomic feedback. This is a critical point for the efficiency of the real-time ergonomic feedback, especially for the neck. Neck pain is a common health problem associated with significant disability in the general population (Côté et al., 2004; Breen et al., 2009) and it has been proved that there is a trend for a positive relation between neck movement and neck pain at work (Ariëns et al., 2001). Moreover, the current real-time ergonomic tool resulted in a decrease in the risk of MSDs at upper and lower arms, which represent a significant benefit given the prevalent proportion of these pathologies in all claims at work (Burgess-Limerick, 2007; Euzenat, 2010). Thus, by decreasing the risk of exposure to MSDs at critical anatomical positions during an industrial task, the real-time ergonomic feedback may contribute to resolving this public health issue. Concerning the feedback modalities, user reports suggested that the combination of visual and auditory feedback was a good way to provide information concerning the occurrence of hazardous behavior and the affected segments. Although visual-auditory feedback has been proven to improve user performance in augmented reality studies (Burke et al., 2006), further developments concerning multimodal feedback presented to the worker are necessary. More precisely, in an industrial environment, information fed back to the worker must be both salient and not distracting. 4.1. Limitations Although the RULA table has been widely employed in the ergonomic literature (David, 2005), inferred computations are still questionable. The risk factors described by the RULA table are: posture, muscle use, weight of load, shock, task duration and repetitiveness. The RULA approach attempts to quantify the combination of these factors with the aim of obtaining an overall exposure score. Nevertheless, epidemiological data supporting the suggested patterns is missing (Li and Buckle, 1999). Moreover, the RULA table lacks precision, since some angle thresholds, such as upper arm abduction or neck twist, have to be subjectively chosen during the implementation of the tool. The RULA table uses basic calculations in order to provide an ergonomic assessment of the whole upper body. These calculations can be considered as a weakness for some specific anatomical areas like the lumbar zone. Indeed, low back pain is the most common MSD around the world (Brooks, 2006) and some particular risk factors, like lateral trunk velocity, timing of the maximum dynamic asymmetric load moment exposure, the magnitude of the dynamic sagittal bending moment or spinal compression forces, would have to be taken into account for a more comprehensive ergonomic evaluation (Nelson and Hughes, 2009; Marras et al., 2010). A system using the Kinect technology has recently been developed to prevent back injuries at work in real-time (Martin et al., 2012). However, this system is not able to individually evaluate a lift and it has not yet been tested in an industrial environment. Another questionable point when using the RULA tool concerns the fact that it is not possible to know the influence of the cumulative time spent at each range on the risk of MSDs exposure. Indeed, the RULA tool only took into account static postures to
573
compute local and global scores. Although time-sensitive aspects have been shown as risk factors for MSDs (Wells et al., 2007; Rhen et al., 2011), most ergonomic evaluation methods do not consider the dynamics of the work (Svensson et al., 2010). Some quantification concepts currently developed in the literature could be used to implement the time-factor into the real-time ergonomic process. Using IMUs to capture the movement of the worker provides advantages, such as freedom of movement and in-field application. However, as mentioned in Section 2.1.4, inertial sensors and magnetometers can suffer from drawbacks such as magnetic disturbances. These limitations could be overcome by using additional video signals. For example, Bleser et al. (2011) have developed an inertial system supported by an egocentric camera for compensating magnetic disturbances. This system is more robust and at the same time applicable in the field. 4.2. Future research In order to be more robust, this real-time ergonomic system must be task-adaptive. To this aim, some improvements have to be implemented into the RULA calculations. Concerning repetitiveness, an action detection algorithm has been developed to detect which action is currently performed by the subject (Behera et al., 2012) and is currently under implementation. In the same way, by knowing a priori the weight of devices that the worker will use, force/load scores can be inferred from an object recognition process integrated into the system (Damen et al., 2011). Finally, some dynamic indications, i.e. suggested postures for the next subtask, could be implemented into the system and provided to the operator in real-time in order to guide his behavior even more efficiently. In the user test, a RULA score of 7 has been rarely attained during the proposed task. In order to completely assess the efficiency of the real-time ergonomic feedback, some more difficult tasks, including also risks for shocks, have to be investigated with this new tool. Moreover, the proposed system could be tested on industrial training processes. By using the system during training of new tasks, an operator would be able to learn how to perform the task in an ergonomic and healthy way. The influence of the proposed real-time ergonomic tool might in the long run provide important contributions to the decrease of MSDs. However, some epidemiological studies will be necessary to assess this influence on a larger population. 5. Conclusion This study presented an innovative system permitting real-time ergonomic assessment and feedback in an industrial environment. Based on a biomechanical model of the upper body and parameter estimation from body-worn IMU and goniometer data, RULA score computations have been implemented in order to assess the risk of exposure to MSDs in real-time during the movement. This new tool permitted global ergonomic evaluation, as well as calculating the risk of MSDs per articulation or segment. These global and local scores were fed back to the user in the form of auditory and visual warnings, respectively, in the case of hazardous postures. This realtime tool has been tested through a user study in which ergonomic variables have been compared between a group receiving real-time feedback (WR) and a control (WOR) group. Results demonstrated that the real-time ergonomic feedback significantly decreased the risk of MSDs at global and segmental levels. The real-time ergonomic tool presented in this study could be used to directly reduce the risk of MSDs in industry and to support and optimize the longterm performance of workers.
574
N. Vignais et al. / Applied Ergonomics 44 (2013) 566e574
Acknowledgments The research leading to these results has received funding from the European Union FP7 2007e2013 grant agreement n 248290 e Project COGNITO. References Andreoni, G., Mazzola, M., Ciani, O., Zambetti, M., Romero, M., Costa, F., Preatoni, E., 2009. Method for movement and gesture assessment (MMGA) in ergonomics. In: International Conference on Digital Human Modeling, San Diego, CA, USA, pp. 591e598. Ariëns, G.A.M., Bongers, P.M., Douwes, M., Miedema, M.C., Hoogendoorn, W.E., van der Wal, G., Bouter, L.M., van Mechelen, W., 2001. Are neck flexion, neck rotation, and sitting at work risk factors for neck pain? Results of a prospective cohort study. Occup. Environ. Med. 58, 200e207. Balogh, I., Orbaek, P., Winkel, J., Nordander, C., Ohlsson, K., Ektor-Andersen, J., Grp, M.S.-N.S., 2001. Questionnaire-based mechanical exposure indices for large population studies e reliability, internal consistency and predictive validity. Scand. J. Work Environ. Health 27, 41e48. Battini, D., Faccio, M., Persona, A., Sgarbossa, F., 2011. New methodological framework to improve productivity and ergonomics in assembly system design. Int. J. Ind. Ergon. 41, 30e42. Behera, A., Cohn, A., Hogg, D., 2012. Workflow activity monitoring using dynamics of pair-wise qualitative spatial relations. In: International Conference on Advances in Multimedia Modeling, Klagenfurt, Austria, pp. 196e209. Bernmark, E., Wiktorin, C.C.P.S., 2002. A triaxial accelerometer for measuring arm movements. Appl. Ergon. 33, 541e547. Bleser, G., Hendeby, G., Miezal, M., 2011. Using egocentric vision to achieve robust inertial body tracking under magnetic disturbances. In: IEEE International Symposium on Mixed and Augmented Reality, Basel, Switzerland, pp. 103e109. Breen, P.P., Nisar, A., Olaighin, G., 2009. Evaluation of a single accelerometer based biofeedback system for real-time correction of neck posture in computer users. In: IEEE Engineering in Medicine and Biology Society Annual International Conference, Minneapolis, Minnesota, USA, pp. 7269e7272. Brooks, P.M., 2006. The burden of musculoskeletal disease e a global perspective. Clin. Rheumatol. 25, 778e781. Burgess-Limerick, R.J., 2007. Ergonomics for manual tasks. In: Mayhew, C. (Ed.), Australian Master of OHS and Environment Guide. CCH Australia, North Ryde, pp. 261e278. Burke, J.L., Prewett, M.S., Gray, A.A., Yang, L., Stilson, F.R.B., Coovert, M.D., Elliot, L.R., Redden, E., 2006. Comparing the effects of visual-auditory and visual-tactile feedback on user performance: a meta-analysis. In: International Conference on Multimodal Interfaces, Banff, AB, Canada, pp. 108e117. Chaffin, D.B., 2005. Improving digital human modelling for proactive ergonomics in design. Ergonomics 48, 478e491. Chang, C.C., McGorry, R.W., Lin, J.H., Xu, X., Hsiang, S.C.P., 2010. Prediction accuracy in estimating joint angle trajectories using a video posture coding method for sagittal lifting tasks. Ergonomics 53, 1039e1047. Council for Occupational Safety and Health (COSH), 2010. Code of Practice: Manual Tasks, Government of Western Australia. Department of Commerce, Perth. Côté, P., Cassidy, J.D., Carroll, L.J., Kristman, V., 2004. The annual incidence and course of neck pain in the general population: a population-based cohort study. Pain 112, 267e273. Damen, D., Gee, A., Calway, A., Mayol-Cuevas, W., 2011. Detecting and localising multiple 3D objects: a fast and scalable approach. In: IROS Workshop on Active Semantic Perception and Object Search in the Real World, San Francisco, California, USA. David, G.C., 2005. Ergonomic methods for assessing exposure to risk factors for work-related musculoskeletal disorders. Occup. Med-Oxford 55, 190e199. Euzenat, D., 2010. L’exposition des salaries aux maladies professionnelles en 2007. In: Direction de l’Animation de la Recherche, des Etudes et des Statistiques (Ed.), DARES Analyses 56. Freivalds, A., Kong, Y., You, H., Park, S., 2000. A comprehensive risk assessment model for work-related musculoskeletal disorders of the upper extremities. Hum. Fac. Erg. Soc. P 44, 5728e5731.
Hansson, G.A., Asterland, P., Holmer, N.G., Skerfving, S., 2001. Validity and reliability of triaxial accelerometers for inclinometry in posture analysis. Med. Biol. Eng. Comput. 39, 405e413. Harada, T., Mori, T., Sato, T., 2007. Development of a tiny orientation estimation device to operate under motion and magnetic disturbance. Int. J. Robot Res. 26, 547e559. Hu, B., Zhang, W., Salvendy, G., 2012. Impact of multimodal feedback on simulated ergonomic measurements in a virtual environment: a case study with manufacturing workers. Hum. Factors Ergon. Manuf. Serv. Ind. 22, 145e155. Jayaram, U., Jayaram, S., Shaikh, I., Kim, Y.J., Palmer, C., 2006. Introducing quantitative analysis methods into virtual environments for real-time and continuous ergonomic evaluations. Comput. Ind. 57, 283e296. Jazwinski, A.H., 1970. Stochastic processes and filtering theory. Math. Sci. Eng. 64 Klopcar, N., Lenarcic, J., 2005. Kinematic model for determination of human arm reachable workspace. Meccanica 40, 203e219. Li, G.Y., Buckle, P., 1999. Current techniques for assessing physical exposure to workrelated musculoskeletal risks, with emphasis on posture-based methods. Ergonomics 42, 674e695. Marras, W.S., Lavender, S.A., Ferguson, S.A., Splittstoesser, R.E., Yang, G., 2010. Quantitative dynamic measures of physical exposure predict low back functional impairment. Spine 35, 914. Martin, C.C., Burkert, D.C., Choi, K.R., Wieczorek, N.B., McGregor, P.M., Herrmann, R.A., Beling, P.A., 2012. A real-time ergonomic monitoring system using the microsoft Kinect. In: IEEE Systems and Information Engineering Design Symposium, Charlottesville, VA, USA, pp. 50e55. McAtamney, L., Corlett, E.N., 1993. RULA e a survey method for the investigation of work-related upper limb disorders. Appl. Ergon. 24, 91e99. Mullineaux, D.R., Underwood, S.M., Shapiro, R., Hall, J.W., 2012. Real-time biomechanical biofeedback effects on top-level rifle shooters. Appl. Ergon. 43, 109e114. Nelson, N.A., Hughes, R.E., 2009. Quantifying relationships between selected work-related risk factors and back pain: a systematic review of objective biomechanical measures and cost-related health outcomes. Int. J. Ind. Ergon. 39, 202e210. Nomura, J., Sawada, K., 2001. Virtual reality technology and its application on aircraft assembling. Annu. Rev. Contr. 25, 99e109. Radwin, R.G., 2011. Automated video exposure assessment of repetitive motion. Hum. Fac. Erg. Soc. P 55, 995e996. Radwin, R.G., Lin, M.L., 1993. An analytical method for characterizing repetitive motion and postural stress using spectral-analysis. Ergonomics 36, 379e389. Ray, S.J., Teizer, J., 2012. Real-time construction worker posture analysis for ergonomics training. Adv. Eng. Inform. 26, 439e455. Rhen, I.M., Gyllensvärd, D., Hanson, L., Högberg, D., 2011. Time dependent exposure analysis and risk assessment of a manikin’s wrist movements. In: International Conference on Digital Human Modelling, Lyon, France. Roetenberg, D., Luinge, H., Slycke, P., 2009. Xsens MVN: full 6DOF human motion tracking using miniature inertial sensors. Xsens Motion Technologies BV, Technical Report. Svensson, E., Bertilsson, E., Högberg, D., Hanson, L., 2010. Review of the incorporation, utilization and future demands of ergonomic evaluation methods in digital human modeling. In: Annual Nordic Ergonomic Society Conference, Stavanger, Norway. Thun, J.H., Lehr, C.B., Bierwirth, M., 2011. Feel free to feel comfortable e an empirical analysis of ergonomics in the German automotive industry. Int. J. Prod. Econ. 133, 551e561. Udani, A.D., Harrison, T.K., Howard, S.K., Kim, T.E., Brock-Utne, J.G., Gaba, D.M., Mariano, E.R., 2012. Preliminary study of ergonomic behavior during simulated ultrasound-guided regional anesthesia using a head-mounted display. J. Ultras. Med. 31, 1277e1280. Waters, T.R., 2012. Ergonomics in design: interventions for youth working in the agricultural industry. Theoret. Iss. Ergon. Sci. 13, 270e285. Wells, R., Mathiassen, S.E., Medbo, L., Winkel, J., 2007. Time - a key issue for musculoskeletal health and manufacturing. Appl. Ergon. 38, 733e744. Winter, D.A., 2009. Biomechanics and Motor Control of Human Movement, fourth ed. Wiley & Sons, USA. Yen, T.Y., Radwin, R.G., 1995. A video-based system for acquiring biomechanical data synchronized with arbitrary events and activities. IEEE T. Bio-Med. Eng. 42, 944e948. Zuehlke, D., 2008. Smartfactory e from vision to reality in factory technologies. In: World Congress of International Federation of Automatic Control, Seoul, Korea.
Contents lists available at SciVerse ScienceDirect
Applied Ergonomics journal homepage: www.elsevier.com/locate/apergo
Innovative system for real-time ergonomic feedback in industrial manufacturing Nicolas Vignais a, *, Markus Miezal b, Gabriele Bleser b, Katharina Mura c, Dominic Gorecky c, Frédéric Marin a a
UMR CNRS 7338 Biomechanics and Bioengineering, University of Technology of Compiègne, Research Center, Dct Schweitzer Street, 60200 Compiègne, France DFKI GmbH, German Research Center for Artificial Intelligence, Trippstadter Strasse 122, D-67663 Kaiserslautern, Germany c SmartFactoryKL, Trippstadter Strasse 122, 67663 Kaiserslautern, Germany b
a r t i c l e i n f o
a b s t r a c t
Article history: Received 24 April 2012 Accepted 26 November 2012
This work presents a system that permits a real-time ergonomic assessment of manual tasks in an industrial environment. First, a biomechanical model of the upper body has been developed by using inertial sensors placed at different locations on the upper body. Based on this model, a computerized RULA ergonomic assessment was implemented to permit a global risk assessment of musculoskeletal disorders in real-time. Furthermore, local scores were calculated per segment, e.g. the neck region, and gave information on the local risks for musculoskeletal disorders. Visual information was fed back to the user by using a see-through head mounted display. Additional visual highlighting and auditory warnings were provided when some predefined thresholds were exceeded. In a user study (N ¼ 12 participants) a group with the RULA feedback was compared to a control group. Results demonstrate that the real-time ergonomic feedback significantly decreased the outcome of both globally as well as locally hazardous RULA values that are associated with increased risk for musculoskeletal disorders. Task execution time did not differ between groups. The real-time ergonomic tool introduced in this study has the potential to considerably reduce the risk of musculoskeletal disorders in industrial settings. Implications for ergonomics in manufacturing and user feedback modalities are further discussed. Ó 2012 Elsevier Ltd and The Ergonomics Society. All rights reserved.
Keywords: Ergonomics Real-time Augmented reality
1. Introduction The human worker plays an important role as operator and troubleshooter in modern and complex manufacturing systems. Many aspects of industrial work are of a physical nature, especially manual tasks, e.g. when the human worker has to pick up a component or arrange it in the assembly position. More generally, any kind of physical activity, such as lifting, pushing, pulling, carrying, moving, manipulating, holding or restraining objects, is considered to be a manual task (COSH, 2010). However, musculoskeletal disorders (MSDs) caused by manual tasks represent a large part of all work-related MSDs (Burgess-Limerick, 2007; Euzenat, 2010) and are a central issue for public health. Ergonomics specialists may identify conditions under which the risk of workrelated MSDs is high and develop adequate interventions (Waters, 2012). Moreover, these specialists play an increasing role in industry since they can also influence motion efficiency, operational safety and productivity (Battini et al., 2011; Thun et al., 2011).
* Corresponding author. Tel.: þ33 3 44 23 43 89; fax: þ3 33 44 23 79 42. E-mail address: [email protected] (N. Vignais).
Thus, developing supportive tools for the identification and evaluation of potentially hazardous motor tasks and postures is crucial for ergonomics researchers. Different methods and tools exist for the ergonomic assessment of manual tasks. A non-exhaustive list of them includes QEC, manTRA, RULA, REBA, HAL-TLV, OWAS, LUBA, OCRA, Strain Index, SNOOK tables and the NIOSH lifting equation (Andreoni et al., 2009). They can be classified into self-reports, observational methods and direct measurements (Li and Buckle, 1999; David, 2005). Firstly, self-reports involve worker diaries, interviews and questionnaires. This method has been used for the ergonomic assessment of handling work by Balogh et al. (2001) which constructed valid indices for mechanical exposure of the shouldere neck region. Nevertheless, self-reports entail some drawbacks like the unreliability of exposure perception and comprehension or interpretation according to the worker’s literacy. Secondly, observational methods aim to assess workplace exposure by evaluating the worker’s behavior on pro-forma sheets either while observing in the field or replaying videos. The Rapid Upper Limb Assessment (RULA) index is one of the most cited. This tool is based on the observation of the postures during a certain task and outputs biomechanical and postural load values on the whole body with
0003-6870/$ e see front matter Ó 2012 Elsevier Ltd and The Ergonomics Society. All rights reserved. http://dx.doi.org/10.1016/j.apergo.2012.11.008
N. Vignais et al. / Applied Ergonomics 44 (2013) 566e574
particular attention to the neck, trunk and upper limbs (McAtamney and Corlett, 1993). Observational methods based on videotaped work sequences analyze different kinds of manual tasks with specific software (Yen and Radwin, 1995; Chang et al., 2010; Radwin, 2011). Observational methods are affordable and practical for use in a wide range of professional situations (David, 2005). However, the scoring system may be questionable according to limited original epidemiological data. Video-based observational methods are also time consuming. Direct methods, in contrast, measure the risk of exposure in real-time by using sensors directly attached to the worker’s body. Some of these methods have been employed to study upper limbs (Radwin and Lin, 1993; Freivalds et al., 2000; Bernmark and Wiktorin, 2002). Nevertheless, all these methods require a complex and cost-intensive hardware setup and a lot of effort to analyze and interpret recorded data in real-time (David, 2005). As a consequence, the ergonomic evaluation of the directly assessed behavior has to be performed offline. Yet, postural evaluation that can be carried out in real-time provides benefits in practice (Mullineaux et al., 2012). If the system provides the worker with information concerning his current ergonomic behavior, then postures could be modified immediately. Furthermore, in the long run, associations between certain postures and their hazardousness could be learned. In order to receive the immediate feedback, Augmented Reality (AR) technology can be used during the actual work execution (Udani et al., 2012). Recent developments in sensor technology offer potential for regular industrial use in contrast to other tracking devices, such as range cameras or magnetic sensors, that are more effective in virtual environments (Ray and Teizer, 2012; Jayaram et al., 2006). For instance, an inertial measurement unit (IMUs) is a small-sized, inexpensive and low-power device suitable for monitoring the kinematics of a segment in real-time (Breen et al., 2009). If several inertial measurement units are connected, biomechanical models can be developed to capture a wide range of movements (Roetenberg et al., 2009). The aim of this study is to introduce an innovative and practical system for ergonomic assessment of a worker’s activity in realtime. The worker’s movements are recorded by using a lightweight sensor network composed of synchronized inertial measurements units (IMUs) and goniometers linked to an upper body biomechanical model. In order to perform the ergonomic evaluation in real-time, the RULA index is continuously computed
567
from this biomechanical model. The RULA assessment is simultaneously fed back to the user via a see-through head-mounted display (STHMD). A user study was conducted in order to reveal the influence of the proposed ergonomic feedback on workers’ motions during manual task execution. 2. Materials and methods In the first part of this paper, the data collection process based on the on-body sensor network and the biomechanical model of the upper body are introduced. Then, the real-time ergonomic computation based on the RULA sheet is described. Finally, a user study aiming at the evaluation of the real-time ergonomic feedback is presented. 2.1. Data collection 2.1.1. On-body sensor network The RULA score computation requires information about joint angles and segment orientations. These two parameters were derived using an on-body sensor network and a mobile processing unit (in this case, a standard laptop). This system is composed of seven wireless Colibri IMUs (Trivisio GmbH, Trier, Germany). Each lightweight sensor (48 g, 56 42 19 mm) contains a tri-axial accelerometer, a tri-axial gyroscope and a tri-axial magnetoinductive magnetic sensor. All sensors are sampled at 100 Hz. The IMUs are placed on the worker’s body as follows: one IMU for each upper arm, one IMU for each forearm, one IMU for the head, placed on the STHMD, one IMU for the trunk, located on the chest, and one IMU for the pelvis, placed on the sacrum (see Fig. 1a). This last IMU is necessary to define the movement of the trunk with respect to the pelvis segment. In order to record wrist angles (flexion/extension, radial/ulnar deviation), bi-axial SG65 goniometers (Biometrics Ltd., Newport, UK) have been added to the on-body sensor network and synchronized with the IMUs. 2.1.2. Biomechanical model of the upper body The worker’s upper body is then represented by a biomechanical model composed of ten rigid segments (trunk, clavicles, upper arms, forearms, hands and head) connected by anatomically motivated restricted articulations (pelvis, neck joint, sternoclavicular joints, shoulders, elbows and wrists) (see Fig. 1b). This
Fig. 1. On-body sensor network composed of IMUs coupled with goniometers (a) and underlying biomechanical model of the upper body (b) with rotation axes for local body frames, and DoF of each joint.
568
N. Vignais et al. / Applied Ergonomics 44 (2013) 566e574
biomechanical model provides 20 degrees of freedom (DoF): three for the pelvis, three for the neck joint, three for each shoulder (composed of acromioclavicular and glenohumeral joints), two for each elbow and two for each wrist. The sternoclavicular articulation depends on the upper arm elevation (Klopcar and Lenarcic, 2005) and therefore has only one dependent degree of mobility (see Fig. 1b). All rotation axes of local body frames are considered orthogonal in the model. In the neutral pose, which corresponds to the standard anatomical position, all local body frames are aligned with the global body frame, with the z-axes aligned with the longitudinal axes of the segments and gravity, the x-axes pointing laterally, and the y-axes posteriorly. 2.1.3. Calibration process Segmental lengths are derived from an anthropometric database by using the worker’s height as input (Winter, 2009). The relative positions of the IMUs with respect to adjoining segments are directly measured on the subject. Two postures performed by the subject are necessary to calibrate the orientations of the IMUs with respect to the body. In a first step, orientation axes of pelvis, chest and head IMUs are obtained as follows: - In the neutral pose, the z-axis is defined by gravity, which is given by the accelerometer measurements when the subject is stationary; - Then, by bending forward, the orthogonal x-axis is computed using a cross product; - Finally, by computing the cross product between z- and x-axes, the corresponding y-axis is calculated. In a second step, upper arm and forearm IMUs are aligned using accelerations and magnetic measurements recorded during the neutral posture. More precisely, z-axes (longitudinal axes of the segments) are aligned with gravity and x- and y-axes of upper limbs joints are aligned using magnetometer measurements. 2.1.4. Joint angle estimation Once the on-body sensor network has been fully calibrated, joint angles are computed in real-time. These are derived from the measured IMU data using model-based sensor fusion, in particular using a set of loosely coupled extended Kalman filters (EKFs) (Jazwinski, 1970). Firstly, the orientations of the trunk, the pelvis, and the head are estimated from the respective IMU data, each in a separate EKF, using a state-of-the-art orientation estimation algorithm (Harada et al., 2007). Neck and pelvis angles are deduced from the differential rotations. By using the chest orientation and measurements from the arm IMUs, joint angles of left and right elbows and shoulders relative to the trunk reference frame are computed. Each arm is handled in a separate EKF. The inertial measurement models are based on forward kinematic equations derived from the biomechanical model of the upper body (see Fig. 1b). By providing a global heading direction based on the earth or the local magnetic field, magnetometer information is used to compensate for drift caused by noise and fluctuating offsets in the inertial data. Further details about the data collection method and the particularities of the chosen state-space models for the joint angle estimation can be found in a previous publication (Bleser et al., 2011). 2.2. RULA assessment in real-time 2.2.1. Computation The RULA ergonomic tool estimates the exposure to upper limb MSDs by computing a global risk score. For a current posture, this global score ranges from one to seven, one being most comfortable.
This score is based on posture, muscle use, weight of loads, task duration, and repetitiveness (see Fig. 2). Originally based on a discrete observation of postures, the RULA tool has been implemented into the biomechanical model in order to compute the score of risk exposure to MSDs in real time. To this aim, the described biomechanical model provides joint angles necessary to locate the head (called ‘neck’ in the RULA sheet), trunk, upper arm, forearm and hand (called ‘wrist’ in the RULA sheet) positions. These angles allow computation of a score for each articulation, also called local score. The muscle-use scores are deduced from the task, i.e. a posture is considered as being ‘mainly static’ if variation of joint positions is under a specific threshold for 10 min. Additionally, shocks to the hand segments are detected during the movement based on the accelerations measured by the forearm IMUs. Sudden changes of acceleration magnitude indicate a shock. In the current system, these are determined based on an exponential sliding window over the differences in measured acceleration magnitudes between subsequent time steps with an experimentally determined threshold of 4 m/s2. However, in this study, it has to be noted that the manual tasks will mainly focus on the evaluation of posture and will not contain actions that imply a high risk of shocks. 2.2.2. RULA feedback For the current study, two feedback modalities were selected: an auditory signal linked to the global score and a visual cue related to local scores provided through the STHMD (see the video on http:// www.ict-cognito.org/demo.html for further details). The auditory signal was given as a warning when the global score was 7 for a period of at least 0.5 s. It reflects the recommendation of the RULA table to change the posture immediately. For a score of 5 or 6, the RULA table suggests modifying the posture soon. To address this, the auditory warning was given after spending at least 5 s in this range. In the same way, each time a local score exceeded a predefined threshold, the concerned joint and segment were highlighted in red in a schematic representation of the operator’s upper body on the STHMD (see Fig. 3). Consequently, the subject was able to understand which posture was inappropriate during the manual task and which segments had to be moved to decrease the RULA score. Based on a preliminary study, the visualization on the STHMD has been designed to be easily recognizable and not distractive. The red visual highlighting was presented if the local score was higher than the following thresholds: -
Shoulder and upper arm: 5 Elbow and lower arm: 3 Wrist and hand: 5 Neck and head: 4 Pelvis and trunk: 4
2.3. User study 2.3.1. Subjects All participants were male students aged between 20 and 27 years (M ¼ 22.5 years, SD ¼ 2.5). Participants’ mean height and mass were 180.42 cm (SD ¼ 6.53) and 76.58 kg (SD ¼ 12.71), respectively. They were randomly divided into two groups of six workers. Subsequent t-tests revealed no significant differences in subject characteristics (see Table 1). The first group had to perform an industrial manual task with the real-time auditory warnings and the RULA display shown in the STHMD. This group was subsequently called WR group (with RULA feedback). Participants of the second group had no access to the visual feedback and auditory warnings. The second group was subsequently called WOR group (without RULA feedback).
N. Vignais et al. / Applied Ergonomics 44 (2013) 566e574
569
Fig. 2. RULA sheet (Source: McAtamney and Corlett, 1993; with permission from Neese Consulting Inc., Leawood, KS, USA).
2.3.2. Study scenario Data were collected in the SmartFactoryKL living lab (Kaiserslautern, Germany) which is a manufacturer-independent demonstration and research platform (Zuehlke, 2008). This platform allowed testing of the on-body sensor network system in a realistic industrial production environment. The experimental task scenario was chosen to contain different postures and motions that are performed in industrial manufacturing. The manual task was composed of four subtasks (see Fig. 4): - turning two hand levers by 90 (A), - removing four fuses at knee level (B) and putting them into a box (C), - taking a screwdriver (D), unscrewing four screws of a transducer’s covering (E) and putting it down near the box (C), - unscrewing four screws of an upper transducer’s covering (F) and putting it down near the box (C). For this last subtask, a ladder may be used to reach a more comfortable working position (G). Forces and loads exerted during the task were lower than 4.4 lbs. The standard execution time was below 4 min.
2.3.3. Procedure Each participant signed an informed consent and answered anthropometric questions before the experimentation. Afterward, the participant had to put on the on-body sensor network and run through the calibration process. Then he received an overview and instructions of the task, the tools (screwdrivers) and equipment (ladder). He was asked to perform the task at normal pace and as accurately as possible. All participants were informed that the system will measure their kinematic behavior during task execution. The WR group was additionally informed that the system gives visual and auditory signals concerning the ergonomics. After participants finished the scenario they filled out a final questionnaire which included additional questions for the WR group concerning the feedback. 2.3.4. Metrics and statistical analysis For WOR and WR groups, the execution times and the percentage of time spent at each range defined by the RULA table (1e2, 3e4, 5e6 and 7) were computed. Moreover, an articulationbased analysis was carried out. For each articulation generating a local score higher than the predefined values, the total time of appearance and the frequency of appearance were computed.
570
N. Vignais et al. / Applied Ergonomics 44 (2013) 566e574
Fig. 3. Visual feedback representation on the STHMD. When a movement is performed in a hazardous way, segments exposed to a musculoskeletal risk are highlighted in red. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)
Subjective comments on the usefulness of visual and auditory feedback were also assessed through a questionnaire. On a 5-point Likert scale (“agree” to “disagree”) participants rated their impression of the appropriateness of the feedback concerning visibility, attention, attraction, and ergonomic support (e.g. “The information on the display supported me in executing the task in an ergonomic way.”). Additionally, they reported their experience concerning the combination of both feedback modalities (visual and auditory) and suggested potential improvements. Statistical analyses were performed to compare the results of both groups using t-tests or ManneWhitney U-tests when normality failed. 3. Results 3.1. Execution time The WR group needed significantly (p < 0.005) more total time to execute the whole task (M ¼ 227.99, SD ¼ 33.65 s) compared to the WOR group (M ¼ 156.98, SD ¼ 28.87 s). Nevertheless, when observing the execution time (accumulated time to complete each subtask) results showed that there was no significant difference (p ¼ 0.07) between the WOR group (129.38 26.11 s) and the WR group (175.35 38.65 s) (see Fig. 5). 3.2. Mean RULA score On average, participants in the WR group performed the task with a significantly lower global RULA score (3.95 0.83) than the WOR group (4.35 0.54) (p < 0.05). Concerning the right and the left RULA scores separately, these were higher for the WOR group (right: 4.4 0.65, left: 4.31 0.46) than for the WR group (right: 3.99 0.86, left: 3.9 0.88), but not significantly different. 3.3. Percentage of time spent in each RULA range The RULA table defines four score ranges (see Fig. 2): 1e2 corresponds to an acceptable posture; 3e4 means that the current posture needs further investigation and that a change may be needed; 5e6 means that the current posture needs further
investigation and that it has to be changed soon; 7 means that the movement can lead to MSDs, that it has to be investigated and changed immediately. Results demonstrated that there were some significant differences between WR and WOR groups for ranges 3e 4 (WOR ¼ 56.91 13.64%; WR ¼ 76.42 17.77%; p < 0.05) and 5e6 (WOR ¼ 30.48 6.92%; WR ¼ 16.76 13.22%; p < 0.05) (see Fig. 6). For score 7, on average it is higher for the WOR group (10.37 12.2) than for the WR group (3.4 5.47) without significance (p ¼ 0.07). It has to be noted that score 7 was rarely attained during the proposed task. 3.4. Local scores The RULA ergonomic tool allowed the calculation of a local score for each articulation or segment (see Fig. 2). By defining a value beyond which a local score was considered as potentially hazardous (see Section 2.2.2), it was possible to compute the percentage of time spent at a risky level per articulation or segment during the task (see Fig. 7). Results showed that subjects’ articulations and segments of the WOR group were significantly more exposed to a risk of MSDs during the task, especially for the left upper arm (WR ¼ 17.10 21.77%; WOR ¼ 38.14 6.14; p < 0.05), the left lower arm (WR ¼ 31.73 23.85%; WOR ¼ 75.13 16.74; p < 0.01), the right upper arm (WR ¼ 20.81 24.04%; WOR ¼ 49.09 12.64; p < 0.05), the right lower arm (WR ¼ 37.4 17.53%; WOR ¼ 74.7 18.66; p < 0.01) and the neck (WR ¼ 12.24 15.89%; WOR ¼ 34.03 10.8; p < 0.05). 3.5. Subjective reports All participants strongly agreed or agreed that the visual feedback was sufficiently visible (see Fig. 8). Most of the participants (N ¼ 5) approved that the visual feedback was suitable to attract attention. Half of the subjects stated that the visual information supported more ergonomic behavior, whereas the rest of subjects were undecided. Concerning the auditory feedback, almost all participants (N ¼ 5) agreed that it attracted their attention (one participant disagreed). As can be seen from Fig. 8, more than half of the participants (N ¼ 4) agreed that the auditory feedback helped
N. Vignais et al. / Applied Ergonomics 44 (2013) 566e574
571
Table 1 Characteristics of WR and WOR group participants: mean (standard deviation).
Age (years) Height (cm) Weight (kg) Visual impairment Shortsighted Farsighted No impairment Handedness Right Left Both Previous experience with AR Yes No
WR group
WOR group
21.5 (1.6) 180.7 (6.15) 82.83 (12.64)
22.8 (2.7) 180.2 (7.47) 70.33 (10.09)
n¼2 n¼0 n¼4
n¼2 n¼0 n¼4
n¼5 n¼1 n¼0
n¼4 n¼1 n¼1
n¼2 n¼4
n¼2 n¼4 Fig. 5. Total times and execution times for WR and WOR groups (***p < 0.001).
them to behave in a more ergonomic way; two participants were either undecided or disagreed respectively. Five out of six subjects reported that a combination of visual and auditory feedback was preferable to using only one modality. Two subjects argued that both feedback types complemented each other; while the auditory feedback warned reliably that there was a risky posture, the visual feedback implied more detailed information on which segments were involved.
In this study, an innovative system for real-time ergonomic evaluation and feedback in an industrial environment was introduced. The ergonomic assessment was based on biomechanical parameter estimation of the upper body using an on-body sensor network of IMUs and goniometers. By using the RULA tool, this evaluation permitted computation of a global score reflecting the risk of exposure to MSDs in real-time during the movement. Moreover, some local scores were calculated and linked to the specific risk of MSDs per articulation or segment. Visual and auditory feedback was implemented into a STHMD and tested in a realistic, industrial task. For the testing, two groups have been defined: a group with the RULA feedback (WR group), and a group without the RULA feedback (WOR group). Results demonstrated that the total time to perform the task was significantly higher for the WR group than for the WOR group. However, when analyzing the execution time during the task, there was no more significant difference, even if this time was higher for the WR group. Hence,
this latter group seemed to adapt to the ergonomic feedback during the task, i.e. paying attention to a changing display, reacting to warning signals, improving their working position by using a ladder, without the feedback significantly influencing the execution time of the task. Concerning the global risk of MSDs during the movement, results showed that the mean RULA score was significantly greater for the WOR group than for the WR group. It can be suggested that the real-time RULA feedback has influenced WR subjects by decreasing the risk of MSDs during the task. Indeed, WR group subjects had the possibility to modify their movement when they obtained an auditory cue during the movement. Moreover, the visual feedback indicated which parts of the body are subject to a MSD risk. By considering different ranges provided by the RULA table, it can be demonstrated that this decrease of risk comes from the fact that the WR group spent significantly more time at range 3e4, whereas the WOR group is significantly more exposed to the range 5e6. From the RULA table, this last range can be considered as a risky range as the posture of the subject needed “further investigation and has to be changed soon”. Furthermore, the WR group spent less time at a RULA score of 7, even if this trend was not significant. By allowing the operator to spend less time at a risky level, the real-time ergonomic feedback allowed prevention of MSDs. The main contribution of this new tool comes from the fact that this kind of prevention can be performed in real time in an industrial environment, which has, to our knowledge, never been
Fig. 4. Localization of the four subtasks on the production line.
Fig. 6. Percentage of time spent at each range during the task for WR and WOR groups (*p < 0.05).
4. Discussion
572
N. Vignais et al. / Applied Ergonomics 44 (2013) 566e574
Fig. 7. Percentage of time spent at a risky level per articulation or segment (*p < 0.05; **p < 0.01).
suggested in the past. Lately, proactive ergonomics has been used to prevent the risk of MSDs in industry (Nomura and Sawada, 2001; Chaffin, 2005). Virtual environments allow workplaces and tasks to be simulated even before the facilities are physically in place. Some essential parameters in ergonomics such as reach, visibility and spatial workplace organization have been proactively tested in virtual environments (Jayaram et al., 2006). Concerning ergonomic feedback in a virtual environment, Hu et al. (2012) have evaluated the impact of multimodal feedback (visual, auditory, tactile) on ergonomic measurements for a simulated drilling task. The authors demonstrated that task completion time, maximum force capacity reduction, body part discomfort, rated perceived exertion and rated task difficulty were significantly higher in the virtual environment than in a real environment. These findings indicate that the
subjects felt more discomfort, experienced faster fatigue and required longer time in the virtual environment than in the real environment when executing the same task. Thus, even if the proactive manner presents great potential in evaluating humane machine interfaces, this tool requires still further improvements to completely integrate into simulation and ergonomic evaluation methods (Hansson et al., 2001; Chaffin, 2005). In contrast, it has been demonstrated that, in an in situ experiment, using a visual feedback in real-time can significantly decreases the percentage of time spent in bad postures (Breen et al., 2009). Consequently, the proposed approach can be considered as a complementary tool for ergonomic evaluation in industry. After designing an ergonomic workplace in a virtual environment according to the proactive manner, the current real-time ergonomic feedback could be
Fig. 8. Graphical representation of subjective reports in the WR group (A: auditory feedback; V: visual feedback; 1: strongly disagree; 2: disagree; 3: neutral; 4: agree; 5: strongly agree). Values represent averages overall participants.
N. Vignais et al. / Applied Ergonomics 44 (2013) 566e574
employed to validate the virtual design, refine the industrial environment and improve required movements to decrease the risk of MSDs. By using local scores to compute the global score, the RULA table permitted an evaluation of the risk exposure of each articulation or segment in real-time during the task. Indeed, results demonstrated that the percentage of time spent at a risky level was significantly higher for the upper and lower arms and the neck of the WOR group subjects’ during the specific task chosen for the user test. The current study permitted identification of anatomical parts that were protected by the use of the real-time ergonomic feedback. This is a critical point for the efficiency of the real-time ergonomic feedback, especially for the neck. Neck pain is a common health problem associated with significant disability in the general population (Côté et al., 2004; Breen et al., 2009) and it has been proved that there is a trend for a positive relation between neck movement and neck pain at work (Ariëns et al., 2001). Moreover, the current real-time ergonomic tool resulted in a decrease in the risk of MSDs at upper and lower arms, which represent a significant benefit given the prevalent proportion of these pathologies in all claims at work (Burgess-Limerick, 2007; Euzenat, 2010). Thus, by decreasing the risk of exposure to MSDs at critical anatomical positions during an industrial task, the real-time ergonomic feedback may contribute to resolving this public health issue. Concerning the feedback modalities, user reports suggested that the combination of visual and auditory feedback was a good way to provide information concerning the occurrence of hazardous behavior and the affected segments. Although visual-auditory feedback has been proven to improve user performance in augmented reality studies (Burke et al., 2006), further developments concerning multimodal feedback presented to the worker are necessary. More precisely, in an industrial environment, information fed back to the worker must be both salient and not distracting. 4.1. Limitations Although the RULA table has been widely employed in the ergonomic literature (David, 2005), inferred computations are still questionable. The risk factors described by the RULA table are: posture, muscle use, weight of load, shock, task duration and repetitiveness. The RULA approach attempts to quantify the combination of these factors with the aim of obtaining an overall exposure score. Nevertheless, epidemiological data supporting the suggested patterns is missing (Li and Buckle, 1999). Moreover, the RULA table lacks precision, since some angle thresholds, such as upper arm abduction or neck twist, have to be subjectively chosen during the implementation of the tool. The RULA table uses basic calculations in order to provide an ergonomic assessment of the whole upper body. These calculations can be considered as a weakness for some specific anatomical areas like the lumbar zone. Indeed, low back pain is the most common MSD around the world (Brooks, 2006) and some particular risk factors, like lateral trunk velocity, timing of the maximum dynamic asymmetric load moment exposure, the magnitude of the dynamic sagittal bending moment or spinal compression forces, would have to be taken into account for a more comprehensive ergonomic evaluation (Nelson and Hughes, 2009; Marras et al., 2010). A system using the Kinect technology has recently been developed to prevent back injuries at work in real-time (Martin et al., 2012). However, this system is not able to individually evaluate a lift and it has not yet been tested in an industrial environment. Another questionable point when using the RULA tool concerns the fact that it is not possible to know the influence of the cumulative time spent at each range on the risk of MSDs exposure. Indeed, the RULA tool only took into account static postures to
573
compute local and global scores. Although time-sensitive aspects have been shown as risk factors for MSDs (Wells et al., 2007; Rhen et al., 2011), most ergonomic evaluation methods do not consider the dynamics of the work (Svensson et al., 2010). Some quantification concepts currently developed in the literature could be used to implement the time-factor into the real-time ergonomic process. Using IMUs to capture the movement of the worker provides advantages, such as freedom of movement and in-field application. However, as mentioned in Section 2.1.4, inertial sensors and magnetometers can suffer from drawbacks such as magnetic disturbances. These limitations could be overcome by using additional video signals. For example, Bleser et al. (2011) have developed an inertial system supported by an egocentric camera for compensating magnetic disturbances. This system is more robust and at the same time applicable in the field. 4.2. Future research In order to be more robust, this real-time ergonomic system must be task-adaptive. To this aim, some improvements have to be implemented into the RULA calculations. Concerning repetitiveness, an action detection algorithm has been developed to detect which action is currently performed by the subject (Behera et al., 2012) and is currently under implementation. In the same way, by knowing a priori the weight of devices that the worker will use, force/load scores can be inferred from an object recognition process integrated into the system (Damen et al., 2011). Finally, some dynamic indications, i.e. suggested postures for the next subtask, could be implemented into the system and provided to the operator in real-time in order to guide his behavior even more efficiently. In the user test, a RULA score of 7 has been rarely attained during the proposed task. In order to completely assess the efficiency of the real-time ergonomic feedback, some more difficult tasks, including also risks for shocks, have to be investigated with this new tool. Moreover, the proposed system could be tested on industrial training processes. By using the system during training of new tasks, an operator would be able to learn how to perform the task in an ergonomic and healthy way. The influence of the proposed real-time ergonomic tool might in the long run provide important contributions to the decrease of MSDs. However, some epidemiological studies will be necessary to assess this influence on a larger population. 5. Conclusion This study presented an innovative system permitting real-time ergonomic assessment and feedback in an industrial environment. Based on a biomechanical model of the upper body and parameter estimation from body-worn IMU and goniometer data, RULA score computations have been implemented in order to assess the risk of exposure to MSDs in real-time during the movement. This new tool permitted global ergonomic evaluation, as well as calculating the risk of MSDs per articulation or segment. These global and local scores were fed back to the user in the form of auditory and visual warnings, respectively, in the case of hazardous postures. This realtime tool has been tested through a user study in which ergonomic variables have been compared between a group receiving real-time feedback (WR) and a control (WOR) group. Results demonstrated that the real-time ergonomic feedback significantly decreased the risk of MSDs at global and segmental levels. The real-time ergonomic tool presented in this study could be used to directly reduce the risk of MSDs in industry and to support and optimize the longterm performance of workers.
574
N. Vignais et al. / Applied Ergonomics 44 (2013) 566e574
Acknowledgments The research leading to these results has received funding from the European Union FP7 2007e2013 grant agreement n 248290 e Project COGNITO. References Andreoni, G., Mazzola, M., Ciani, O., Zambetti, M., Romero, M., Costa, F., Preatoni, E., 2009. Method for movement and gesture assessment (MMGA) in ergonomics. In: International Conference on Digital Human Modeling, San Diego, CA, USA, pp. 591e598. Ariëns, G.A.M., Bongers, P.M., Douwes, M., Miedema, M.C., Hoogendoorn, W.E., van der Wal, G., Bouter, L.M., van Mechelen, W., 2001. Are neck flexion, neck rotation, and sitting at work risk factors for neck pain? Results of a prospective cohort study. Occup. Environ. Med. 58, 200e207. Balogh, I., Orbaek, P., Winkel, J., Nordander, C., Ohlsson, K., Ektor-Andersen, J., Grp, M.S.-N.S., 2001. Questionnaire-based mechanical exposure indices for large population studies e reliability, internal consistency and predictive validity. Scand. J. Work Environ. Health 27, 41e48. Battini, D., Faccio, M., Persona, A., Sgarbossa, F., 2011. New methodological framework to improve productivity and ergonomics in assembly system design. Int. J. Ind. Ergon. 41, 30e42. Behera, A., Cohn, A., Hogg, D., 2012. Workflow activity monitoring using dynamics of pair-wise qualitative spatial relations. In: International Conference on Advances in Multimedia Modeling, Klagenfurt, Austria, pp. 196e209. Bernmark, E., Wiktorin, C.C.P.S., 2002. A triaxial accelerometer for measuring arm movements. Appl. Ergon. 33, 541e547. Bleser, G., Hendeby, G., Miezal, M., 2011. Using egocentric vision to achieve robust inertial body tracking under magnetic disturbances. In: IEEE International Symposium on Mixed and Augmented Reality, Basel, Switzerland, pp. 103e109. Breen, P.P., Nisar, A., Olaighin, G., 2009. Evaluation of a single accelerometer based biofeedback system for real-time correction of neck posture in computer users. In: IEEE Engineering in Medicine and Biology Society Annual International Conference, Minneapolis, Minnesota, USA, pp. 7269e7272. Brooks, P.M., 2006. The burden of musculoskeletal disease e a global perspective. Clin. Rheumatol. 25, 778e781. Burgess-Limerick, R.J., 2007. Ergonomics for manual tasks. In: Mayhew, C. (Ed.), Australian Master of OHS and Environment Guide. CCH Australia, North Ryde, pp. 261e278. Burke, J.L., Prewett, M.S., Gray, A.A., Yang, L., Stilson, F.R.B., Coovert, M.D., Elliot, L.R., Redden, E., 2006. Comparing the effects of visual-auditory and visual-tactile feedback on user performance: a meta-analysis. In: International Conference on Multimodal Interfaces, Banff, AB, Canada, pp. 108e117. Chaffin, D.B., 2005. Improving digital human modelling for proactive ergonomics in design. Ergonomics 48, 478e491. Chang, C.C., McGorry, R.W., Lin, J.H., Xu, X., Hsiang, S.C.P., 2010. Prediction accuracy in estimating joint angle trajectories using a video posture coding method for sagittal lifting tasks. Ergonomics 53, 1039e1047. Council for Occupational Safety and Health (COSH), 2010. Code of Practice: Manual Tasks, Government of Western Australia. Department of Commerce, Perth. Côté, P., Cassidy, J.D., Carroll, L.J., Kristman, V., 2004. The annual incidence and course of neck pain in the general population: a population-based cohort study. Pain 112, 267e273. Damen, D., Gee, A., Calway, A., Mayol-Cuevas, W., 2011. Detecting and localising multiple 3D objects: a fast and scalable approach. In: IROS Workshop on Active Semantic Perception and Object Search in the Real World, San Francisco, California, USA. David, G.C., 2005. Ergonomic methods for assessing exposure to risk factors for work-related musculoskeletal disorders. Occup. Med-Oxford 55, 190e199. Euzenat, D., 2010. L’exposition des salaries aux maladies professionnelles en 2007. In: Direction de l’Animation de la Recherche, des Etudes et des Statistiques (Ed.), DARES Analyses 56. Freivalds, A., Kong, Y., You, H., Park, S., 2000. A comprehensive risk assessment model for work-related musculoskeletal disorders of the upper extremities. Hum. Fac. Erg. Soc. P 44, 5728e5731.
Hansson, G.A., Asterland, P., Holmer, N.G., Skerfving, S., 2001. Validity and reliability of triaxial accelerometers for inclinometry in posture analysis. Med. Biol. Eng. Comput. 39, 405e413. Harada, T., Mori, T., Sato, T., 2007. Development of a tiny orientation estimation device to operate under motion and magnetic disturbance. Int. J. Robot Res. 26, 547e559. Hu, B., Zhang, W., Salvendy, G., 2012. Impact of multimodal feedback on simulated ergonomic measurements in a virtual environment: a case study with manufacturing workers. Hum. Factors Ergon. Manuf. Serv. Ind. 22, 145e155. Jayaram, U., Jayaram, S., Shaikh, I., Kim, Y.J., Palmer, C., 2006. Introducing quantitative analysis methods into virtual environments for real-time and continuous ergonomic evaluations. Comput. Ind. 57, 283e296. Jazwinski, A.H., 1970. Stochastic processes and filtering theory. Math. Sci. Eng. 64 Klopcar, N., Lenarcic, J., 2005. Kinematic model for determination of human arm reachable workspace. Meccanica 40, 203e219. Li, G.Y., Buckle, P., 1999. Current techniques for assessing physical exposure to workrelated musculoskeletal risks, with emphasis on posture-based methods. Ergonomics 42, 674e695. Marras, W.S., Lavender, S.A., Ferguson, S.A., Splittstoesser, R.E., Yang, G., 2010. Quantitative dynamic measures of physical exposure predict low back functional impairment. Spine 35, 914. Martin, C.C., Burkert, D.C., Choi, K.R., Wieczorek, N.B., McGregor, P.M., Herrmann, R.A., Beling, P.A., 2012. A real-time ergonomic monitoring system using the microsoft Kinect. In: IEEE Systems and Information Engineering Design Symposium, Charlottesville, VA, USA, pp. 50e55. McAtamney, L., Corlett, E.N., 1993. RULA e a survey method for the investigation of work-related upper limb disorders. Appl. Ergon. 24, 91e99. Mullineaux, D.R., Underwood, S.M., Shapiro, R., Hall, J.W., 2012. Real-time biomechanical biofeedback effects on top-level rifle shooters. Appl. Ergon. 43, 109e114. Nelson, N.A., Hughes, R.E., 2009. Quantifying relationships between selected work-related risk factors and back pain: a systematic review of objective biomechanical measures and cost-related health outcomes. Int. J. Ind. Ergon. 39, 202e210. Nomura, J., Sawada, K., 2001. Virtual reality technology and its application on aircraft assembling. Annu. Rev. Contr. 25, 99e109. Radwin, R.G., 2011. Automated video exposure assessment of repetitive motion. Hum. Fac. Erg. Soc. P 55, 995e996. Radwin, R.G., Lin, M.L., 1993. An analytical method for characterizing repetitive motion and postural stress using spectral-analysis. Ergonomics 36, 379e389. Ray, S.J., Teizer, J., 2012. Real-time construction worker posture analysis for ergonomics training. Adv. Eng. Inform. 26, 439e455. Rhen, I.M., Gyllensvärd, D., Hanson, L., Högberg, D., 2011. Time dependent exposure analysis and risk assessment of a manikin’s wrist movements. In: International Conference on Digital Human Modelling, Lyon, France. Roetenberg, D., Luinge, H., Slycke, P., 2009. Xsens MVN: full 6DOF human motion tracking using miniature inertial sensors. Xsens Motion Technologies BV, Technical Report. Svensson, E., Bertilsson, E., Högberg, D., Hanson, L., 2010. Review of the incorporation, utilization and future demands of ergonomic evaluation methods in digital human modeling. In: Annual Nordic Ergonomic Society Conference, Stavanger, Norway. Thun, J.H., Lehr, C.B., Bierwirth, M., 2011. Feel free to feel comfortable e an empirical analysis of ergonomics in the German automotive industry. Int. J. Prod. Econ. 133, 551e561. Udani, A.D., Harrison, T.K., Howard, S.K., Kim, T.E., Brock-Utne, J.G., Gaba, D.M., Mariano, E.R., 2012. Preliminary study of ergonomic behavior during simulated ultrasound-guided regional anesthesia using a head-mounted display. J. Ultras. Med. 31, 1277e1280. Waters, T.R., 2012. Ergonomics in design: interventions for youth working in the agricultural industry. Theoret. Iss. Ergon. Sci. 13, 270e285. Wells, R., Mathiassen, S.E., Medbo, L., Winkel, J., 2007. Time - a key issue for musculoskeletal health and manufacturing. Appl. Ergon. 38, 733e744. Winter, D.A., 2009. Biomechanics and Motor Control of Human Movement, fourth ed. Wiley & Sons, USA. Yen, T.Y., Radwin, R.G., 1995. A video-based system for acquiring biomechanical data synchronized with arbitrary events and activities. IEEE T. Bio-Med. Eng. 42, 944e948. Zuehlke, D., 2008. Smartfactory e from vision to reality in factory technologies. In: World Congress of International Federation of Automatic Control, Seoul, Korea.