- Email: [email protected]
Sharif-Human movement instrumentation system (SHARIF-HMIS): Development and validation
ARTICLE IN PRESS
JID: JJBE
[m5G;August 31, 2018;18:29]
Medical Engineering and Physics 0 0 0 (2018) 1–8
Contents lists available at ScienceDirect
Medical Engineering and Physics journal homepage: www.elsevier.com/locate/medengphy
Technical note
Sharif-Human movement instrumentation system (SHARIF-HMIS): Development and validation Mohammad Iman Mokhlespour Esfahani a,b,∗, Ali Akbari c, Omid Zobeiri d, Ehsan Rashedi e, Mohamad Parnianpour a a
Laboratory of Wearable Technologies and Neuromusculoskeletal Research, School of Mechanical Engineering, Sharif University of Technology, Tehran P.O. Box: 11155-9567, Iran Grado Department of Industrial and Systems Engineering, Virginia Tech, Blacksburg, VA 24061, USA c Department of Biomedical Engineering, Texas A&M University, College Station, TX 77843, USA d Department of Biomedical Engineering, McGill University, Montreal, QC H3A 2B4, Canada e Department of Industrial and Systems Engineering, Rochester Institute of Technology, Rochester, NY 14623, USA b
a r t i c l e
i n f o
Article history: Received 2 May 2017 Revised 21 July 2018 Accepted 24 July 2018 Available online xxx Keywords: Wearable system Inertial Measurement Unit Human Movement Analysis Kalman filter
a b s t r a c t The interest in wearable systems among the biomedical engineering and clinical community continues to escalate as technical refinements enhance their potential use for both indoor and outdoor applications. For example, an important wearable technology known as a microelectromechanical system (MEMS) is demonstrating promising applications in the area of biomedical engineering. Accordingly, this study was designed to investigate the Sharif-Human Movement Instrumentation System (SHARIF-HMIS), consisting of inertial measurement units (IMUs), stretchable clothing, and a data logger—all of which can be used outside the controlled environment of a laboratory, thus enhancing its overall utility. This system is lightweight, portable, able to be deliver data for almost 10 h, and features a new data-fusion algorithm using the Kalman filter with an adaptive approach. In specific terms, the data from the system’s gyroscope, accelerometer, and magnetometer sensors can be combined to estimate total-body orientation; additionally, the noise level of these sensors can be changed to accommodate faster motions as well as magnetic disturbances. These variations can be incorporated within the extended Kalman filter by changing the parameters of the filter adaptively. In specific terms, the system’s interface was developed to acquire data from eighteen IMUs located on the body to collect kinematic data associated with human motion. Meanwhile, a validation test involving one subject performing different shoulder motions was designed to compare data captured by SHARIF-HMIS and the VICON motion-capture system. This validation test demonstrated correlation values of >0.9. Results also confirmed that the output accuracy of the new system’s sensor was <0.55, 1.5 and 3.5° for roll, pitch, and yaw directions, respectively. In summary, SHARIF-HMIS successfully collected kinematic data for specific human movements, which has promising implications for a range of sporting, biomedical, and healthcare-related applications. © 2018 IPEM. Published by Elsevier Ltd. All rights reserved.
1. Introduction Measuring human movement and activity levels has become increasingly important for several disciplines—notably biomedical engineering and healthcare (e.g., in designing a rehabilitation protocol for patients). In general, the current systems for obtaining such data fall under two broad umbrellas: non-wearable and wearable systems. Non-wearable systems are almost exclusively used
∗ Corresponding author at: Grado Department of Industrial and Systems Engineering, Virginia Tech, 250 Durham Hall, Blacksburg 24061, VA, USA. E-mail addresses: [email protected] (M.I.M. Esfahani), [email protected] (A. Akbari), [email protected] (O. Zobeiri), [email protected] (E. Rashedi), [email protected] (M. Parnianpour).
for indoor applications as they often must capture data through wired connections and may require ongoing adjustments. In contrast, wearable systems are portable and capture data remotely, making them far more effective for military, biomedical and rehabilitation applications [1]. Wearable sensors have also been used in the field of ergonomics and task design [2], and have demonstrated utility in evaluating the impact of assistive devices on the reduction of physical demands and the potential risk for injury [3,4]. Moreover, quantifying human kinematics may facilitate the prevention of adverse gait perturbation events such as slips and falls [5,6]. A diverse range of sensors and systems have been utilized in wearable devices, including accelerometers, gyroscopes, flexible angular sensors, electromagnetic tracking systems, and sensing
https://doi.org/10.1016/j.medengphy.2018.07.008 1350-4533/© 2018 IPEM. Published by Elsevier Ltd. All rights reserved.
Please cite this article as: M.I.M. Esfahani et al., Sharif-Human movement instrumentation system (SHARIF-HMIS): Development and validation, Medical Engineering and Physics (2018), https://doi.org/10.1016/j.medengphy.2018.07.008
ARTICLE IN PRESS
JID: JJBE 2
[m5G;August 31, 2018;18:29]
M.I.M. Esfahani et al. / Medical Engineering and Physics 000 (2018) 1–8 Table 1 The specification of sensors which are used in the system. Dynamic range Acceleration Rate of turn Magnetic field
Pitch ± 90°-Roll/Heading ± 180° 3 axes ± 16 gNon-linearity ± 0.5% 3 axes ± 20 0 0°/sZero-rate output variation ± 40°/s 3 axes ± 8 Gauss
fabrics [1,7–10]. Meanwhile, researchers have confirmed that sensor wearers (e.g., patients and workers) overwhelmingly prefer small, portable, lightweight devices that can be easily operated and maintained, while also being compatible with the performance of daily activities [11–13]. Based on recent studies devoted to the development of such devices for monitoring activities of daily living, two wearable systems have come to the forefront: smart textile sensors [8,14,15] and micro electromechanical system (MEMS) [16]. Research is ongoing into the design of textile-based sensors, which range from the intrinsic level featuring “smart” fibers that are able to measure fabric strain (among other factors), to the extrinsic level with discrete components that are attached to the fabric. However, such flexible electronics are currently limited by a number of environmental factors—for example, humidity caused by the external environment or human sweat, both of which could negatively impact durability [17]. MEMS represents a prominent technology in developing measurement systems to be utilized in biomedical applications [16,18]. Now almost two decades old, the accelerometer represents one of the first MEMS sensors designed for a wearable application, sparking the design of additional commercial devices intended to measure human activities during ambulatory applications [19,20]. More recently, researchers have combined accelerometer and gyroscope signals to generate more accurate kinematic data (Inertial sensors) [21]. More than ten years ago researchers combined a magnetometer (for measuring the earth’s magnetic field) with an accelerometer and gyroscope to increase the accuracy of orientation estimations, which is now known as an Inertial Measurement Unit (IMU) [22]. Researchers have also utilized the Kalman filter for estimating orientation data based on accelerometer, gyroscope, and magnetometer signals [22,23]. Regrettably, however, some did not account for the impact of high acceleration movements and magnetic disturbance, which can result in errors in orientation estimates using the Kalman filter [24]. Recently, our group demonstrated the reliability of two SHARIFHMIS IMU sensors using a simple data fusion algorithm known as the Direction Cosine Matrix (DCM) [25,26]. We have since extended this work by incorporating a new adaptive Kalman filter algorithm, which is based on an existing approach by Li and Wang [24], into a study whereby we attached eighteen sensors and a data logger to a stretchable smart garment. This approach can be used to compensate for the effects of magnetic field disturbance and high acceleration movements—further improving the accuracy of angle orientations in IMUs. As described herein, a sensorcalibration process was performed to remove any drift and signal inaccuracy; after which system performance was validated using a VICON motion capture system. 2. Materials and methods 2.1. Material 2.1.1. Sensor A sensor board, which was created to measure 3D orientations, was comprised of three different triple-axes sensors: gyroscope (InvenSense Inc., San Jose, CA, USA), accelerometer (Analog Devices Inc., Norwood, MA, USA), and magnetometer (Honeywell Inc., Morris Plains, NJ, USA) (Table 1). Sensor output was collected at the rate of 100 samples per second and transmitted to a PC-based data
logger using an AVR microcontroller (Atmel Corp., San Jose, California, USA), which employed the RS485 serial protocol. Since these sensors were connected as parallel and shared the same data bus, each sensor board had two equivalent and distinct connections (including power supply and data ports). Hence, each board could be connected to two different boards, which resulted in the proposed sensor configuration, as described in the following section. 2.1.2. System configuration The location for the various sensors was selected based on an earlier study by Roetenberg et al. [27]—primarily to ensure that the user would not be restricted in his or her body movements and that the required comprehensive data could be captured with high accuracy. As shown in Fig. 1, the design featured five sensor branches, four of which were mounted on the limbs (12 sensor boards), while other sensor boards—fixed to the head, shoulders, and back—comprised the fifth sensor branch. Upper-limb branches were linked together in the chest sensor and then joined to other three branches in a hub, which was placed on the front of the body. This hub connected the various sensor boards to the data logger. 2.2. Methods 2.2.1. Data fusion method: Kalman filter with adaptive approach Roetenberg et al. [22] designed a complimentary filter based on Kalman in order to more accurately estimate gyroscope bias error, orientation error, and magnetic disturbance error. They then tested their error-based Kalman filter algorithm under dynamic and ferromagnetic disturbance conditions. Under static conditions, their system delivered 0.6° accuracy. The researchers then tested the system close to ferromagnetic disturbances under dynamic conditions, with error reporting increasing up to 40°. However, they compensated for this magnetic disturbance in their algorithm by modeling the magnetic field, which resulted in reducing the system error under static and dynamic testing to 1.4 and 2.6°, respectively. Similarly, Li and Wang [24] proposed an effective adaptive Kalman filter for IMUs by adjusting the measurement covariance matrix (R) based on the magnitude of acceleration across three levels. The assay results showed a 0.1° error under stationary testing, 0.5° error under low dynamic testing, and 0.7° error under high dynamic testing. While their adaptive scheme clearly overcame the problem of high acceleration and high dynamic movements, they did not take into account magnetic disturbances and the resulting changes in the measurement covariance matrix (R). In this study we implemented a Kalman filter based on Marcard’s study [28], coupled with an extension of the adaptive Kalman filter (AKF) approach proposed by Li and Wang [24]. In Li and Wang’s AKF (and in contrast to previous studies), observation noise variance was not assumed to be a constant. As such, Li and Wang computed the observation noise variance as a variable based on the absolute amount of acceleration. However, the acceleration data only contributed to improving the accuracy of estimating the roll and pitch angles, but had no impact on the yaw angle. The reason for this outcome is that by rotating the sensor around the axis of the earth’s gravity (i.e., the yaw angle), the gravity-related components in the sensor body frame do not change. Thus, this rotation cannot be measured by an accelerometer. Instead, a magnetometer is required to improve the accuracy of estimating the yaw angle. Similar to an accelerometer, data obtained from a magnetometer can suffer from disturbances caused by environmental magnetic errors. However, Li and Wang did not take into account these magnetic field disturbances, which tend to be ubiquitous when inertial sensors are utilized. To counter this drawback, we extended Li and Wang’s AKF method by considering the absolute amount of magnetic field, in addition to acceler-
Please cite this article as: M.I.M. Esfahani et al., Sharif-Human movement instrumentation system (SHARIF-HMIS): Development and validation, Medical Engineering and Physics (2018), https://doi.org/10.1016/j.medengphy.2018.07.008
ARTICLE IN PRESS
JID: JJBE
[m5G;August 31, 2018;18:29]
M.I.M. Esfahani et al. / Medical Engineering and Physics 000 (2018) 1–8
3
Fig. 1. System configuration. In current five branches, four were mounted on the limbs (12 sensor boards) and other sensor boards were fixed on the head, shoulders, and the back (the fifth branch).
ation data in adjusting the observation noise variance. Using this method is expected to improve measurement accuracy in presence of magnetic disturbances—for instance, when in the vicinity of cell phones. Therefore, our proposed algorithm utilizes the advantage of adaptive scheme in acceleration, while at the same time taking into account the influence of any magnetic disturbance in adjusting the R matrix. We believe that this approach will enhance the robustness of our system against ferromagnetic disturbance. In short, our system does not feature the complexity of the error-based Extended Kalman filter (as seen in the Roetenberg et al. study [29]); nor does it require any extra source for fusion as reported by Plamondon et al. [30]. To compute measurement noise, an extension of adaptive approach proposed by Li and Wang [24] was used. Therefore, the measurement noise covariance matrix can be defined by Eqs. (1) and (2).
2 2 2 , σg,y , σg,z , β , β , β , β if ||ab | − g| < Th σg,x
R = diag
(1)
2 2 2 , σg,y , σg,z , kβ , kβ , kβ , kβ , k > 1 if ||ab | − g|> Th σg,x
R = diag
(2) 2 Where σg,x,y,z represents the variances in the gyroscope sensor in each axis; β is the variance of quaternion representation of orientation, which is dependent on the noise of the accelerometer and
magnetometer (adjusted experimentally); ab refers to accelerometer data, g is gravity, Hb refers to magnetometer data, Hground is the magnetic field of the earth, and Th is the threshold determined experimentally. Li and Wang [24] developed Eq. (1) for situations lacking any significant acceleration movement; in contrast, they introduced Eq. (2) for the situations with high acceleration movement. We therefore added two additional conditions to Eqs. (1) and (2) in order to take into account the effects of magnetic disturbance. They are presented here as Eqs. (3) and (4):
2 2 2 , σg,y , σg,z , β, β, β, β σg,x if ||ab | − g| < Th & |Hb | − Hground < Th
(3)
2 2 2 , σg,y , σg,z , kβ , kβ , kβ , kβ , σg,x k> 1 if ||ab | − g| > Th or |Hb | − Hground > Th
(4)
R = diag
R = diag
Therefore, we developed Eq. (3) as an extension of Eq. (1) for situations lacking any significant acceleration movement by adding another condition to consider the effect of magnetic disturbance— namely, by incorporating the term ||Hb | − Hground | < Th. This term indicates that the magnetometer disturbance is not considerable, since the difference between the magnetometer’s signal and the magnetic field of the earth is less than a constant threshold (determined experimentally). Furthermore, we introduced Eq. (4) as an extension of Eq. (2) for situations with high acceleration
Please cite this article as: M.I.M. Esfahani et al., Sharif-Human movement instrumentation system (SHARIF-HMIS): Development and validation, Medical Engineering and Physics (2018), https://doi.org/10.1016/j.medengphy.2018.07.008
JID: JJBE 4
ARTICLE IN PRESS
[m5G;August 31, 2018;18:29]
M.I.M. Esfahani et al. / Medical Engineering and Physics 000 (2018) 1–8
Fig. 2. View of tilt table and the coordinate system.
movement by incorporating the term||Hb | − Hground | > Th , to compensate for the effect of magnetic disturbance by increasing the noise variance through a constant coefficient (k > 1). In such cases, the magnetometer disturbance is considerable because the difference between the magnetometer’s signal and the magnetic field of the earth is more than a constant threshold (determined experimentally). 2.2.2. Calibration We utilized two calibration processes in this investigation: sensor calibration and system calibration. Sensor calibration was conducted by calibrating the accelerometer and gyroscope, using turn and tilt tables, to validate the single sensor in our developed system. For system calibration, we designed a validation process using a single human subject, as detailed in Section 2.2.3. 2.2.2.1. Accelerometer calibration procedure. For the accelerometer calibration process, an IMU sensor was attached to the tilt table, which could be turned with two degrees of freedom (Fig. 2). Since the accelerometer can only measure roll and pitch angles—and not the yaw angle—this type of table was deemed to be appropriate for calibrating this sensor. The rotation about the x and y directions, which refer to roll and pitch, are indicated by the symbols ϕ and θ , respectively. 2.2.2.2. Gyroscope calibration procedure. The gyroscope was calibrated using the turn table, which could rotate with different speeds. The sensor was tested from 10 °/s to 150 °/s, and data were collected for each velocity. 2.2.3. Validation testing A correlation coefficient statistic was utilized to evaluate how well the IMU results agreed with findings obtained from the VICON motion-capture system. For this verification assay, we singled out shoulder movements by placing two IMUs on the arm and sternum. One subject performed the movements that are illustrated in Fig. 3(a). (Note that the age, weight, and height of this participant was 24 years, 105 kg, and 188 cm, respectively.) This study was approved by the Sharif University of Technology Review Board and the subject gave their written informed consent to participate in the study. Specifically, the test consisted of simple movements that were conducted in just one anatomical plane; with
supplementary data obtained from performing complex movements in two and three planes. The simple tasks consisted of flexion and abduction movements that were performed in six different ranges, extension, adduction, rotation, horizontal flexion, horizontal extension, and horizontal rotation. The scapular task was conducted in two planes—namely through a combination of flexion and abduction. The final task (i.e., a mixed task) was performed along three anatomical planes. For this particular assay, the subject was required to point to 9 dots on the wall that were placed in a 3 × 3 matrix. We later validated this system using the VICON system (VICON Motion Systems Ltd. UK) including 8 cameras. The cameras were calibrated statically to measure the accurate 3D position of the reflective cameras in a lab environment. Specifically, five markers were attached to the subject’s sternum, clavicle, shoulder, arm and epicondyle (Fig. 3(b)). To calculate the shoulder angle, the elbow and the trunk are assumed as two rigid links and their 3D coordinates were computed based on the markers’ coordinates. Finally, the angle between the two links was calculated and utilized as the reference angle. For calculating the 3D coordination of the elbow link, the Z axis was created using two markers (m3 and m2 as shown in Fig. 3), after which another temporary vector (a) was built into the similar plane by using the m1 and m2 markers. The Y axis served as the outer product of the Z axis and the temporary vector. The X axis was obtained by determining the cross product of the Y and Z axes. Then, we used the matrix (R1 ) as the rotation matrix to convert the local coordinate system to the global coordinate system (i.e., the VICON coordinate system) (Eq. (6)). Similarly, the rotation matrix of the trunk (R2 ) was calculated using m1 , m4 , m5 . The angle of the shoulder, which represents the relative angle between the elbow and trunk, was calculated using Eq. (7). We then compared the angle extracted from the camera to our computed angles.
Z = m3 − m2 a = m1 − m2 Y = Z×a X = Y ×Z
R=
Xx Yx Zx
(5) Xy Yy Zy
Xz Yz Zz
Rrel = R1 × RT2
(6)
(7)
Please cite this article as: M.I.M. Esfahani et al., Sharif-Human movement instrumentation system (SHARIF-HMIS): Development and validation, Medical Engineering and Physics (2018), https://doi.org/10.1016/j.medengphy.2018.07.008
JID: JJBE
ARTICLE IN PRESS
[m5G;August 31, 2018;18:29]
M.I.M. Esfahani et al. / Medical Engineering and Physics 000 (2018) 1–8
5
Fig. 3. (a) The experimental tasks. For the mixed task, 9 dots were placed on the wall in a 3 × 3 matrix and the subject pointed to each dot. (b) The placement of markers on the body and assumed links for computing reference angles. 3D coordinates of these markers were collected using the VICON system.
Please cite this article as: M.I.M. Esfahani et al., Sharif-Human movement instrumentation system (SHARIF-HMIS): Development and validation, Medical Engineering and Physics (2018), https://doi.org/10.1016/j.medengphy.2018.07.008
JID: JJBE 6
ARTICLE IN PRESS
[m5G;August 31, 2018;18:29]
M.I.M. Esfahani et al. / Medical Engineering and Physics 000 (2018) 1–8
Fig. 4. The results of the calibration tests for the accelerometer in different positions of tilt table.
Table 2 The RMSE ± standard deviation for roll, pitch and yaw angles. Velocity (°/s)
Roll RMSE (°)
Pitch RMSE (°)
20 30 40 50 60 70 80 90 100 150
0.519 ± 0.158 0.523 ± 0.169 0.528 ± 0.162 0.560 ± 0.155 0.611 ± 0.161 0.555 ± 0.166 0.609 ± 0.168 0.544 ± 0.169 0.590 ± 0.167 0.613 ± 0.133
1.457 1.464 1.455 1.477 1.488 1.502 1.524 1.585 1.588 1.646
± 0.183 ± 0.183 ± 0.194 ± 0.178 ± 0.183 ± 0.180 ± 0.190 ± 0.196 ± 0.193 ± 0.195
Yaw RMSE (°) 4.811 ± 2.857 4.619 ± 2.688 4.002 ± 2.448 3.438 ± 2.194 2.342 ± 1.409 2.398 ± 1.377 2.564 ± 1.609 2.210 ± 1.284 3.443 ± 2.585 2.750 ± 1.619
dicates angular velocity; note that a fitted line was added to compare the results with the outputs of the turn table as a reference. Results obtained from our angle calibration efforts are presented in Table 2, which included the root mean square errors (RMSE) between the computed angles of sensors and referenced values in different angular velocities. Fig. 5. The results of gyroscope calibration process in different angular velocities for yaw direction.
3. Results 3.1. Results of calibration The results of the accelerometer calibration are presented in Fig. 4, including 3D accelerations, 3D reference acceleration by camera system, and errors among them in each position of tilt. A sample of results associated with the gyroscope calibration process for the Yaw direction is illustrated in Fig. 5. Each point in-
3.2. Results of validation test The angle outcome extracted from the IMU system were compared to the angles obtained by the VICON system (as described earlier). The average (SD) of RMSE for all tests was 2.9° (Table 3). The ‘R’ column provides data on the level of correlation between the IMU system and the camera outputs. 4. Discussion Wearable devices are evolving in sophistication and, thus, in their ability to accurately monitor kinematic movement. Such data is being used across a range of applications, including the military,
Please cite this article as: M.I.M. Esfahani et al., Sharif-Human movement instrumentation system (SHARIF-HMIS): Development and validation, Medical Engineering and Physics (2018), https://doi.org/10.1016/j.medengphy.2018.07.008
ARTICLE IN PRESS
JID: JJBE
[m5G;August 31, 2018;18:29]
M.I.M. Esfahani et al. / Medical Engineering and Physics 000 (2018) 1–8
7
Table 3 Results of validation test. Task
Test
RMSE
R2
Task
Test
RMSE
R2
Flexion A
Roll Pitch Yaw Roll Pitch Yaw Roll Pitch Yaw Roll Pitch Yaw Roll Pitch Yaw Roll Pitch Yaw Roll Pitch Yaw Roll Pitch Yaw
3.21 2.84 3.53 3.1 2.5 3.38 2.69 2.09 2.97 2.78 2.43 3.1 3.1 2.15 2.92 2.11 1.75 3.73 2.79 3.04 3 3.5 2.34 3.74
0.96 0.97 0.94 0.99 0.99 0.97 0.98 0.96 0.95 0.97 0.98 0.95 0.99 0.97 0.97 0.96 0.95 0.95 0.97 0.95 0.97 0.99 0.97 0.95
Abduction D
Roll Pitch Yaw Roll Pitch Yaw Roll Pitch Yaw Roll Pitch Yaw Roll Pitch Yaw Roll Pitch Yaw Roll Pitch Yaw Roll Pitch Yaw
3.5 2.79 3.61 3.1 2.93 3.13 2.94 3.15 3.86 2.71 3.6 3.86 1.41 3.36 3.16 3.4 2.89 3.56 1.94 2.86 4.11 3.45 2.54 3.69
0.97 0.95 0.98 0.97 0.98 0.96 0.96 0.93 0.94 0.99 0.95 0.97 0.97 0.96 0.95 0.97 0.98 0.94 0.94 0.92 0.93 0.96 0.97 0.94
Flexion B
Flexion C
Flexion D
Extension
Abduction A
Abduction B
Abduction C
the sports/fitness arena, and increasingly in the healthcare domain for continuous monitoring of a patient’s movement. For this investigation, we developed and tested a body-worn system known as SHARIF-HMIS featuring inertial measurement units (IMUs), a data logger, and stretchable clothing. We also introduced an extension of the adaptive Kalman filter (AKF) approach suggested by Li and Wang [24] in order to take into account any localized magnetic disturbances in the filter. This filter could be potentially used to compensate for the effects of magnetic field disturbance, in addition to high acceleration movements, on the accuracy of angle orientations in IMUs. For this study, we considered the observation noise variance of AKF to be variable, which differs from prior studies that have assumed a constant variance [22]. Therefore, our algorithm computed observation noise variance as a variable according to the absolute amount of acceleration and the absolute amount of magnetic field. In contrast, the Li and Wang method (AKF) only considered the absolute amount of acceleration. As such, our modified approach may improve the system against ferromagnetic disturbances. The gyroscope and accelerometer were calibrated using known laboratory approaches. This initial step was essential for validating the general performance of this newly developed system. Results obtained from accelerometer calibration indicated that the highest error in measurement of acceleration was 0.05 g in the orthogonal direction. The linear data shown in Fig. 5 also confirm the accuracy of the gyroscope in the Yaw direction. The highest error for roll, pitch, and yaw directions were 0.6, 1.6, and 4.8°, respectively. Note that, overall, the error average was less than 3° (Table 2). Testing was also conducted to validate shoulder motion, which was selected because it generally known to be one of the most complicated joint in the human body. Our findings indicate that the mean error for measuring the angle of shoulder motions was 2.9°. The results shown in Table 3 confirm that the angles measured by this system are consistent with those obtained by the VICON motioncapture system, as evidenced by the fact that all correlation coefficients between the IMU system camera outputs were more than 0.9. The results of these calibration tests verified that the SHARIFHMIS is a valid system when compared to available commercial devices such Xsens [31]. Thus, we may conclude that our modified algorithm resulted in an accurate measurement of specific human body movements. However, a future study should be designed to
Adduction
Rotation
Horizontal flexion
Horizontal extension
Horizontal rotation
Scapular
Mix
test the SHARIF-HMIS in the presence of magnetic fields to further evaluate the performance of the algorithm. A number of limitations must be acknowledged in this study. First, the magnetometer should be calibrated in the presence of environmental magnetic noise to more fully assess and confirm the performance of the algorithm. Based on practical limitations of time and financial resources, the validation test was completed by just one healthy, young participant. Thus, the findings described herein cannot be extrapolated to older adults with more restrictive movements. Furthermore, future work is needed to evaluate the filtering algorithm under more complex working conditions, such as those involving high acceleration movements and in the presence of magnetic disturbance. To reiterate, the main purpose of this investigation was to validate the hardware and modified algorithm under “normal” conditions in order to determine whether it was reasonably accurate compared to existing commercial devices. Logically, a future study should be designed to assess its reproducibility with more participants in diverse applications and involving high acceleration movements. In summary, our SHARIF-HMIS wearable system demonstrated acceptable measurement accuracy against a validated commercial motion-capture system (i.e., VICON). Our device may help to better quantify patient activity, which can be useful for several applications such as sports/fitness assessments, biomechanical research, and medical rehabilitation and progress evaluation in physical therapy settings.
Conflicts of interest The authors have no conflicts of interest.
Acknowledgments Authors would like to acknowledge the technical and financial support of the Iran National Science Foundation (INSF) under Grant 890 0 0867. We deeply appreciate the contributions of Mrs. Narimani and the staff of the Laboratory of Wearable Technologies and Neuromusculoskeletal Research, as well as the assistance of Mr. Sharifi at Precise Elements Lab.
Please cite this article as: M.I.M. Esfahani et al., Sharif-Human movement instrumentation system (SHARIF-HMIS): Development and validation, Medical Engineering and Physics (2018), https://doi.org/10.1016/j.medengphy.2018.07.008
JID: JJBE 8
ARTICLE IN PRESS
[m5G;August 31, 2018;18:29]
M.I.M. Esfahani et al. / Medical Engineering and Physics 000 (2018) 1–8
References [1] Patel S, Park H, Bonato P, Chan L, Rodgers M. A review of wearable sensors and systems with application in rehabilitation. J Neuroeng Rehabil 2012;9:21. [2] Alabdulkarim S, Nussbaum MA, Rashedi E, Kim S, Agnew M, Gardner R. Impact of task design on task performance and injury risk: case study of a simulated drilling task. Ergonomics 2017;60:851–66. [3] Kim S, Nussbaum MA, Mokhlespour Esfahani MI, Alemi MM, Alabdulkarim S, Rashedi E. Assessing the influence of a passive, upper extremity exoskeletal vest for tasks requiring arm elevation: part I – “Expected” effects on discomfort, shoulder muscle activity, and work task performance. Appl Ergon 2018. [4] Kim S, Nussbaum MA, Mokhlespour Esfahani MI, Alemi MM, Jia B, Rashedi E. Assessing the influence of a passive, upper extremity exoskeletal vest for tasks requiring arm elevation: part II – “Unexpected” effects on shoulder motion, balance, and spine loading. Appl Ergon 2018. [5] Shany T, Redmond SJ, Narayanan MR, Lovell NH. Sensors-based wearable systems for monitoring of human movement and falls. IEEE Sens J 2012;12:658–70. [6] Ehsani H, Mohler J, Marlinski V, Rashedi E, Toosizadeh N. The influence of mechanical vibration on local and central balance control. J Biomech 2018. [7] Godfrey A, Conway R, Meagher D, G OL. Direct measurement of human movement by accelerometry. Med. Eng Phys 2008;30:1364–86. [8] Mokhlespour Esfahani MI, Zobeiri O, Moshiri B, Narimani R, Mehravar M, Rashedi E, et al. Trunk motion system (TMS) using printed body worn sensor (BWS) via data fusion approach. Sensors 2017;17. [9] Mokhlespour Esfahani MI, Narimani R, Ramezanzadehkoldeh M. A wearable respiratory plethysmography using flexible sensor. Int J Biomed Eng Technol 2013;11:364–80. [10] Mokhlespour Esfahani MI, Taghinedjad S, Mottaghitalab V, Narimani R, Parnianpour M. Novel printed body worn sensor for measuring the human movement orientation. Sens Rev 2016;36:321–31. [11] Bergmann JH, McGregor AH. Body-worn sensor design: what do patients and clinicians want? Ann Biomed Eng 2011;39:2299–312. [12] Bergmann JH, Chandaria V, McGregor A. Wearable and implantable sensors: the patient’s perspective. Sensors 2012;12:16695–709. [13] Mokhlespour Esfahani MI, Nussbaum MA. Preferred placement and usability of a smart textile system vs. inertial measurement units for activity monitoring. Sensors 2018;18. [14] Mokhlespour Esfahani MI, Zobeiri O, Narimani R, Hoviattalab M, Moshiri B, Parnianpour M. Design and prototyping of wearable measuring system for trunk movement using textile sensors. In: Proceedings of the Twentieth Iranian Conference on Electrical Engineering (ICEE). IEEE; 2012. p. 1571–5. [15] Mokhlespour Esfahani MI, Nussbaum MA. A “Smart” undershirt for tracking upper body motions: task classification and angle estimation. IEEE Sens J 2018. [16] De Rossi D, Veltink P. Wearable technology for biomechanics: e-textile or micromechanical sensors? IEEE Eng Med Biol Mag: Q Mag Eng Med Biol Soc 2010;29:37–43.
[17] Castano LM, Flatau AB. Smart Textile Transducers: Design, Techniques, and Applications. Industrial Applications for Intelligent Polymers and Coatings. Cham: Springer International Publishing; 2016. p. 121–46. [18] Hosseinzadegan H, Pierron ON, Hosseinian E. Accurate modeling of air shear damping of a silicon lateral rotary micro-resonator for MEMS environmental monitoring applications. Sens Actuators A – Phys 2014;216:342–8. [19] Foerster F, Smeja M, Fahrenberg J. Detection of posture and motion by accelerometry: a validation study in ambulatory monitoring. Comput Hum Behav 1999;15:571–83. [20] Wong WY, Wong MS, Lo KH. Clinical applications of sensors for human posture and movement analysis: a review. Prosthet Orthot Int 2007;31:62–75. [21] Coley B, Jolles BM, Farron A, Bourgeois A, Nussbaumer F, Pichonnaz C, et al. Outcome evaluation in shoulder surgery using 3D kinematics sensors. Gait Posture 2007;25:523–32. [22] Roetenberg D, Luinge HJ, Baten CTM, Veltink PH. Compensation of magnetic disturbances improves inertial and magnetic sensing of human body segment orientation. IEEE Trans Neural Syst Rehabil Eng 2005;13:395–405. [23] Meng X, Zhang ZQ, Wu JK, Wong WC, Yu H. Self-contained pedestrian tracking during normal walking using an inertial/magnetic sensor module. IEEE Trans Bio-med Eng 2014;61:892–9. [24] Li W, Wang JL. Effective adaptive Kalman filter for MEMS-IMU/magnetometers integrated attitude and heading reference systems. J Navig 2013;66:99–113. [25] Abedi M, Manshadi FD, Zavieh MK, Ashouri S, Azimi H, Parnanpour M. A reliability study of the new sensors for movement analysis (SHARIF–HMIS). J Bodyw Mov Ther 2016;20:341–5. [26] Mokhlespour Esfahani MI, Zobeiri O, Akbari A, Moshiri B, Parnianpour M. Wearable human movement instrumentation system. Iran J Biomed Eng 2014;7:361–9. [27] Roetenberg D, Luinge H, Slycke P. Xsens MVN: full 6DOF human motion tracking using miniature inertial sensors. Xsens Technol: Xsens Co 2013:1–9. [28] Marcard TV. Design and implementation of an attitude estimation system to control orthopedic components. Goteborg, Sweden: Chalmers University of Technology; 2010. [29] Roetenberg D, Luinge HJ, Baten CT, Veltink PH. Compensation of magnetic disturbances improves inertial and magnetic sensing of human body segment orientation. IEEE Trans Neural Syst Rehabil Eng: Publ IEEE Eng Med Biol Soc 2005;13:395–405. [30] Plamondon A, Delisle A, Larue C, Brouillette D, McFadden D, Desjardins P, et al. Evaluation of a hybrid system for three-dimensional measurement of trunk posture in motion. Appl Ergon 2007;38:697–712. [31] Robert-Lachaine X, Mecheri H, Larue C, Plamondon A. Validation of inertial measurement units with an optoelectronic system for whole-body motion analysis. Med Biol Eng Comput 2017;55:609–19.
Please cite this article as: M.I.M. Esfahani et al., Sharif-Human movement instrumentation system (SHARIF-HMIS): Development and validation, Medical Engineering and Physics (2018), https://doi.org/10.1016/j.medengphy.2018.07.008
JID: JJBE
[m5G;August 31, 2018;18:29]
Medical Engineering and Physics 0 0 0 (2018) 1–8
Contents lists available at ScienceDirect
Medical Engineering and Physics journal homepage: www.elsevier.com/locate/medengphy
Technical note
Sharif-Human movement instrumentation system (SHARIF-HMIS): Development and validation Mohammad Iman Mokhlespour Esfahani a,b,∗, Ali Akbari c, Omid Zobeiri d, Ehsan Rashedi e, Mohamad Parnianpour a a
Laboratory of Wearable Technologies and Neuromusculoskeletal Research, School of Mechanical Engineering, Sharif University of Technology, Tehran P.O. Box: 11155-9567, Iran Grado Department of Industrial and Systems Engineering, Virginia Tech, Blacksburg, VA 24061, USA c Department of Biomedical Engineering, Texas A&M University, College Station, TX 77843, USA d Department of Biomedical Engineering, McGill University, Montreal, QC H3A 2B4, Canada e Department of Industrial and Systems Engineering, Rochester Institute of Technology, Rochester, NY 14623, USA b
a r t i c l e
i n f o
Article history: Received 2 May 2017 Revised 21 July 2018 Accepted 24 July 2018 Available online xxx Keywords: Wearable system Inertial Measurement Unit Human Movement Analysis Kalman filter
a b s t r a c t The interest in wearable systems among the biomedical engineering and clinical community continues to escalate as technical refinements enhance their potential use for both indoor and outdoor applications. For example, an important wearable technology known as a microelectromechanical system (MEMS) is demonstrating promising applications in the area of biomedical engineering. Accordingly, this study was designed to investigate the Sharif-Human Movement Instrumentation System (SHARIF-HMIS), consisting of inertial measurement units (IMUs), stretchable clothing, and a data logger—all of which can be used outside the controlled environment of a laboratory, thus enhancing its overall utility. This system is lightweight, portable, able to be deliver data for almost 10 h, and features a new data-fusion algorithm using the Kalman filter with an adaptive approach. In specific terms, the data from the system’s gyroscope, accelerometer, and magnetometer sensors can be combined to estimate total-body orientation; additionally, the noise level of these sensors can be changed to accommodate faster motions as well as magnetic disturbances. These variations can be incorporated within the extended Kalman filter by changing the parameters of the filter adaptively. In specific terms, the system’s interface was developed to acquire data from eighteen IMUs located on the body to collect kinematic data associated with human motion. Meanwhile, a validation test involving one subject performing different shoulder motions was designed to compare data captured by SHARIF-HMIS and the VICON motion-capture system. This validation test demonstrated correlation values of >0.9. Results also confirmed that the output accuracy of the new system’s sensor was <0.55, 1.5 and 3.5° for roll, pitch, and yaw directions, respectively. In summary, SHARIF-HMIS successfully collected kinematic data for specific human movements, which has promising implications for a range of sporting, biomedical, and healthcare-related applications. © 2018 IPEM. Published by Elsevier Ltd. All rights reserved.
1. Introduction Measuring human movement and activity levels has become increasingly important for several disciplines—notably biomedical engineering and healthcare (e.g., in designing a rehabilitation protocol for patients). In general, the current systems for obtaining such data fall under two broad umbrellas: non-wearable and wearable systems. Non-wearable systems are almost exclusively used
∗ Corresponding author at: Grado Department of Industrial and Systems Engineering, Virginia Tech, 250 Durham Hall, Blacksburg 24061, VA, USA. E-mail addresses: [email protected] (M.I.M. Esfahani), [email protected] (A. Akbari), [email protected] (O. Zobeiri), [email protected] (E. Rashedi), [email protected] (M. Parnianpour).
for indoor applications as they often must capture data through wired connections and may require ongoing adjustments. In contrast, wearable systems are portable and capture data remotely, making them far more effective for military, biomedical and rehabilitation applications [1]. Wearable sensors have also been used in the field of ergonomics and task design [2], and have demonstrated utility in evaluating the impact of assistive devices on the reduction of physical demands and the potential risk for injury [3,4]. Moreover, quantifying human kinematics may facilitate the prevention of adverse gait perturbation events such as slips and falls [5,6]. A diverse range of sensors and systems have been utilized in wearable devices, including accelerometers, gyroscopes, flexible angular sensors, electromagnetic tracking systems, and sensing
https://doi.org/10.1016/j.medengphy.2018.07.008 1350-4533/© 2018 IPEM. Published by Elsevier Ltd. All rights reserved.
Please cite this article as: M.I.M. Esfahani et al., Sharif-Human movement instrumentation system (SHARIF-HMIS): Development and validation, Medical Engineering and Physics (2018), https://doi.org/10.1016/j.medengphy.2018.07.008
ARTICLE IN PRESS
JID: JJBE 2
[m5G;August 31, 2018;18:29]
M.I.M. Esfahani et al. / Medical Engineering and Physics 000 (2018) 1–8 Table 1 The specification of sensors which are used in the system. Dynamic range Acceleration Rate of turn Magnetic field
Pitch ± 90°-Roll/Heading ± 180° 3 axes ± 16 gNon-linearity ± 0.5% 3 axes ± 20 0 0°/sZero-rate output variation ± 40°/s 3 axes ± 8 Gauss
fabrics [1,7–10]. Meanwhile, researchers have confirmed that sensor wearers (e.g., patients and workers) overwhelmingly prefer small, portable, lightweight devices that can be easily operated and maintained, while also being compatible with the performance of daily activities [11–13]. Based on recent studies devoted to the development of such devices for monitoring activities of daily living, two wearable systems have come to the forefront: smart textile sensors [8,14,15] and micro electromechanical system (MEMS) [16]. Research is ongoing into the design of textile-based sensors, which range from the intrinsic level featuring “smart” fibers that are able to measure fabric strain (among other factors), to the extrinsic level with discrete components that are attached to the fabric. However, such flexible electronics are currently limited by a number of environmental factors—for example, humidity caused by the external environment or human sweat, both of which could negatively impact durability [17]. MEMS represents a prominent technology in developing measurement systems to be utilized in biomedical applications [16,18]. Now almost two decades old, the accelerometer represents one of the first MEMS sensors designed for a wearable application, sparking the design of additional commercial devices intended to measure human activities during ambulatory applications [19,20]. More recently, researchers have combined accelerometer and gyroscope signals to generate more accurate kinematic data (Inertial sensors) [21]. More than ten years ago researchers combined a magnetometer (for measuring the earth’s magnetic field) with an accelerometer and gyroscope to increase the accuracy of orientation estimations, which is now known as an Inertial Measurement Unit (IMU) [22]. Researchers have also utilized the Kalman filter for estimating orientation data based on accelerometer, gyroscope, and magnetometer signals [22,23]. Regrettably, however, some did not account for the impact of high acceleration movements and magnetic disturbance, which can result in errors in orientation estimates using the Kalman filter [24]. Recently, our group demonstrated the reliability of two SHARIFHMIS IMU sensors using a simple data fusion algorithm known as the Direction Cosine Matrix (DCM) [25,26]. We have since extended this work by incorporating a new adaptive Kalman filter algorithm, which is based on an existing approach by Li and Wang [24], into a study whereby we attached eighteen sensors and a data logger to a stretchable smart garment. This approach can be used to compensate for the effects of magnetic field disturbance and high acceleration movements—further improving the accuracy of angle orientations in IMUs. As described herein, a sensorcalibration process was performed to remove any drift and signal inaccuracy; after which system performance was validated using a VICON motion capture system. 2. Materials and methods 2.1. Material 2.1.1. Sensor A sensor board, which was created to measure 3D orientations, was comprised of three different triple-axes sensors: gyroscope (InvenSense Inc., San Jose, CA, USA), accelerometer (Analog Devices Inc., Norwood, MA, USA), and magnetometer (Honeywell Inc., Morris Plains, NJ, USA) (Table 1). Sensor output was collected at the rate of 100 samples per second and transmitted to a PC-based data
logger using an AVR microcontroller (Atmel Corp., San Jose, California, USA), which employed the RS485 serial protocol. Since these sensors were connected as parallel and shared the same data bus, each sensor board had two equivalent and distinct connections (including power supply and data ports). Hence, each board could be connected to two different boards, which resulted in the proposed sensor configuration, as described in the following section. 2.1.2. System configuration The location for the various sensors was selected based on an earlier study by Roetenberg et al. [27]—primarily to ensure that the user would not be restricted in his or her body movements and that the required comprehensive data could be captured with high accuracy. As shown in Fig. 1, the design featured five sensor branches, four of which were mounted on the limbs (12 sensor boards), while other sensor boards—fixed to the head, shoulders, and back—comprised the fifth sensor branch. Upper-limb branches were linked together in the chest sensor and then joined to other three branches in a hub, which was placed on the front of the body. This hub connected the various sensor boards to the data logger. 2.2. Methods 2.2.1. Data fusion method: Kalman filter with adaptive approach Roetenberg et al. [22] designed a complimentary filter based on Kalman in order to more accurately estimate gyroscope bias error, orientation error, and magnetic disturbance error. They then tested their error-based Kalman filter algorithm under dynamic and ferromagnetic disturbance conditions. Under static conditions, their system delivered 0.6° accuracy. The researchers then tested the system close to ferromagnetic disturbances under dynamic conditions, with error reporting increasing up to 40°. However, they compensated for this magnetic disturbance in their algorithm by modeling the magnetic field, which resulted in reducing the system error under static and dynamic testing to 1.4 and 2.6°, respectively. Similarly, Li and Wang [24] proposed an effective adaptive Kalman filter for IMUs by adjusting the measurement covariance matrix (R) based on the magnitude of acceleration across three levels. The assay results showed a 0.1° error under stationary testing, 0.5° error under low dynamic testing, and 0.7° error under high dynamic testing. While their adaptive scheme clearly overcame the problem of high acceleration and high dynamic movements, they did not take into account magnetic disturbances and the resulting changes in the measurement covariance matrix (R). In this study we implemented a Kalman filter based on Marcard’s study [28], coupled with an extension of the adaptive Kalman filter (AKF) approach proposed by Li and Wang [24]. In Li and Wang’s AKF (and in contrast to previous studies), observation noise variance was not assumed to be a constant. As such, Li and Wang computed the observation noise variance as a variable based on the absolute amount of acceleration. However, the acceleration data only contributed to improving the accuracy of estimating the roll and pitch angles, but had no impact on the yaw angle. The reason for this outcome is that by rotating the sensor around the axis of the earth’s gravity (i.e., the yaw angle), the gravity-related components in the sensor body frame do not change. Thus, this rotation cannot be measured by an accelerometer. Instead, a magnetometer is required to improve the accuracy of estimating the yaw angle. Similar to an accelerometer, data obtained from a magnetometer can suffer from disturbances caused by environmental magnetic errors. However, Li and Wang did not take into account these magnetic field disturbances, which tend to be ubiquitous when inertial sensors are utilized. To counter this drawback, we extended Li and Wang’s AKF method by considering the absolute amount of magnetic field, in addition to acceler-
Please cite this article as: M.I.M. Esfahani et al., Sharif-Human movement instrumentation system (SHARIF-HMIS): Development and validation, Medical Engineering and Physics (2018), https://doi.org/10.1016/j.medengphy.2018.07.008
ARTICLE IN PRESS
JID: JJBE
[m5G;August 31, 2018;18:29]
M.I.M. Esfahani et al. / Medical Engineering and Physics 000 (2018) 1–8
3
Fig. 1. System configuration. In current five branches, four were mounted on the limbs (12 sensor boards) and other sensor boards were fixed on the head, shoulders, and the back (the fifth branch).
ation data in adjusting the observation noise variance. Using this method is expected to improve measurement accuracy in presence of magnetic disturbances—for instance, when in the vicinity of cell phones. Therefore, our proposed algorithm utilizes the advantage of adaptive scheme in acceleration, while at the same time taking into account the influence of any magnetic disturbance in adjusting the R matrix. We believe that this approach will enhance the robustness of our system against ferromagnetic disturbance. In short, our system does not feature the complexity of the error-based Extended Kalman filter (as seen in the Roetenberg et al. study [29]); nor does it require any extra source for fusion as reported by Plamondon et al. [30]. To compute measurement noise, an extension of adaptive approach proposed by Li and Wang [24] was used. Therefore, the measurement noise covariance matrix can be defined by Eqs. (1) and (2).
2 2 2 , σg,y , σg,z , β , β , β , β if ||ab | − g| < Th σg,x
R = diag
(1)
2 2 2 , σg,y , σg,z , kβ , kβ , kβ , kβ , k > 1 if ||ab | − g|> Th σg,x
R = diag
(2) 2 Where σg,x,y,z represents the variances in the gyroscope sensor in each axis; β is the variance of quaternion representation of orientation, which is dependent on the noise of the accelerometer and
magnetometer (adjusted experimentally); ab refers to accelerometer data, g is gravity, Hb refers to magnetometer data, Hground is the magnetic field of the earth, and Th is the threshold determined experimentally. Li and Wang [24] developed Eq. (1) for situations lacking any significant acceleration movement; in contrast, they introduced Eq. (2) for the situations with high acceleration movement. We therefore added two additional conditions to Eqs. (1) and (2) in order to take into account the effects of magnetic disturbance. They are presented here as Eqs. (3) and (4):
2 2 2 , σg,y , σg,z , β, β, β, β σg,x if ||ab | − g| < Th & |Hb | − Hground < Th
(3)
2 2 2 , σg,y , σg,z , kβ , kβ , kβ , kβ , σg,x k> 1 if ||ab | − g| > Th or |Hb | − Hground > Th
(4)
R = diag
R = diag
Therefore, we developed Eq. (3) as an extension of Eq. (1) for situations lacking any significant acceleration movement by adding another condition to consider the effect of magnetic disturbance— namely, by incorporating the term ||Hb | − Hground | < Th. This term indicates that the magnetometer disturbance is not considerable, since the difference between the magnetometer’s signal and the magnetic field of the earth is less than a constant threshold (determined experimentally). Furthermore, we introduced Eq. (4) as an extension of Eq. (2) for situations with high acceleration
Please cite this article as: M.I.M. Esfahani et al., Sharif-Human movement instrumentation system (SHARIF-HMIS): Development and validation, Medical Engineering and Physics (2018), https://doi.org/10.1016/j.medengphy.2018.07.008
JID: JJBE 4
ARTICLE IN PRESS
[m5G;August 31, 2018;18:29]
M.I.M. Esfahani et al. / Medical Engineering and Physics 000 (2018) 1–8
Fig. 2. View of tilt table and the coordinate system.
movement by incorporating the term||Hb | − Hground | > Th , to compensate for the effect of magnetic disturbance by increasing the noise variance through a constant coefficient (k > 1). In such cases, the magnetometer disturbance is considerable because the difference between the magnetometer’s signal and the magnetic field of the earth is more than a constant threshold (determined experimentally). 2.2.2. Calibration We utilized two calibration processes in this investigation: sensor calibration and system calibration. Sensor calibration was conducted by calibrating the accelerometer and gyroscope, using turn and tilt tables, to validate the single sensor in our developed system. For system calibration, we designed a validation process using a single human subject, as detailed in Section 2.2.3. 2.2.2.1. Accelerometer calibration procedure. For the accelerometer calibration process, an IMU sensor was attached to the tilt table, which could be turned with two degrees of freedom (Fig. 2). Since the accelerometer can only measure roll and pitch angles—and not the yaw angle—this type of table was deemed to be appropriate for calibrating this sensor. The rotation about the x and y directions, which refer to roll and pitch, are indicated by the symbols ϕ and θ , respectively. 2.2.2.2. Gyroscope calibration procedure. The gyroscope was calibrated using the turn table, which could rotate with different speeds. The sensor was tested from 10 °/s to 150 °/s, and data were collected for each velocity. 2.2.3. Validation testing A correlation coefficient statistic was utilized to evaluate how well the IMU results agreed with findings obtained from the VICON motion-capture system. For this verification assay, we singled out shoulder movements by placing two IMUs on the arm and sternum. One subject performed the movements that are illustrated in Fig. 3(a). (Note that the age, weight, and height of this participant was 24 years, 105 kg, and 188 cm, respectively.) This study was approved by the Sharif University of Technology Review Board and the subject gave their written informed consent to participate in the study. Specifically, the test consisted of simple movements that were conducted in just one anatomical plane; with
supplementary data obtained from performing complex movements in two and three planes. The simple tasks consisted of flexion and abduction movements that were performed in six different ranges, extension, adduction, rotation, horizontal flexion, horizontal extension, and horizontal rotation. The scapular task was conducted in two planes—namely through a combination of flexion and abduction. The final task (i.e., a mixed task) was performed along three anatomical planes. For this particular assay, the subject was required to point to 9 dots on the wall that were placed in a 3 × 3 matrix. We later validated this system using the VICON system (VICON Motion Systems Ltd. UK) including 8 cameras. The cameras were calibrated statically to measure the accurate 3D position of the reflective cameras in a lab environment. Specifically, five markers were attached to the subject’s sternum, clavicle, shoulder, arm and epicondyle (Fig. 3(b)). To calculate the shoulder angle, the elbow and the trunk are assumed as two rigid links and their 3D coordinates were computed based on the markers’ coordinates. Finally, the angle between the two links was calculated and utilized as the reference angle. For calculating the 3D coordination of the elbow link, the Z axis was created using two markers (m3 and m2 as shown in Fig. 3), after which another temporary vector (a) was built into the similar plane by using the m1 and m2 markers. The Y axis served as the outer product of the Z axis and the temporary vector. The X axis was obtained by determining the cross product of the Y and Z axes. Then, we used the matrix (R1 ) as the rotation matrix to convert the local coordinate system to the global coordinate system (i.e., the VICON coordinate system) (Eq. (6)). Similarly, the rotation matrix of the trunk (R2 ) was calculated using m1 , m4 , m5 . The angle of the shoulder, which represents the relative angle between the elbow and trunk, was calculated using Eq. (7). We then compared the angle extracted from the camera to our computed angles.
Z = m3 − m2 a = m1 − m2 Y = Z×a X = Y ×Z
R=
Xx Yx Zx
(5) Xy Yy Zy
Xz Yz Zz
Rrel = R1 × RT2
(6)
(7)
Please cite this article as: M.I.M. Esfahani et al., Sharif-Human movement instrumentation system (SHARIF-HMIS): Development and validation, Medical Engineering and Physics (2018), https://doi.org/10.1016/j.medengphy.2018.07.008
JID: JJBE
ARTICLE IN PRESS
[m5G;August 31, 2018;18:29]
M.I.M. Esfahani et al. / Medical Engineering and Physics 000 (2018) 1–8
5
Fig. 3. (a) The experimental tasks. For the mixed task, 9 dots were placed on the wall in a 3 × 3 matrix and the subject pointed to each dot. (b) The placement of markers on the body and assumed links for computing reference angles. 3D coordinates of these markers were collected using the VICON system.
Please cite this article as: M.I.M. Esfahani et al., Sharif-Human movement instrumentation system (SHARIF-HMIS): Development and validation, Medical Engineering and Physics (2018), https://doi.org/10.1016/j.medengphy.2018.07.008
JID: JJBE 6
ARTICLE IN PRESS
[m5G;August 31, 2018;18:29]
M.I.M. Esfahani et al. / Medical Engineering and Physics 000 (2018) 1–8
Fig. 4. The results of the calibration tests for the accelerometer in different positions of tilt table.
Table 2 The RMSE ± standard deviation for roll, pitch and yaw angles. Velocity (°/s)
Roll RMSE (°)
Pitch RMSE (°)
20 30 40 50 60 70 80 90 100 150
0.519 ± 0.158 0.523 ± 0.169 0.528 ± 0.162 0.560 ± 0.155 0.611 ± 0.161 0.555 ± 0.166 0.609 ± 0.168 0.544 ± 0.169 0.590 ± 0.167 0.613 ± 0.133
1.457 1.464 1.455 1.477 1.488 1.502 1.524 1.585 1.588 1.646
± 0.183 ± 0.183 ± 0.194 ± 0.178 ± 0.183 ± 0.180 ± 0.190 ± 0.196 ± 0.193 ± 0.195
Yaw RMSE (°) 4.811 ± 2.857 4.619 ± 2.688 4.002 ± 2.448 3.438 ± 2.194 2.342 ± 1.409 2.398 ± 1.377 2.564 ± 1.609 2.210 ± 1.284 3.443 ± 2.585 2.750 ± 1.619
dicates angular velocity; note that a fitted line was added to compare the results with the outputs of the turn table as a reference. Results obtained from our angle calibration efforts are presented in Table 2, which included the root mean square errors (RMSE) between the computed angles of sensors and referenced values in different angular velocities. Fig. 5. The results of gyroscope calibration process in different angular velocities for yaw direction.
3. Results 3.1. Results of calibration The results of the accelerometer calibration are presented in Fig. 4, including 3D accelerations, 3D reference acceleration by camera system, and errors among them in each position of tilt. A sample of results associated with the gyroscope calibration process for the Yaw direction is illustrated in Fig. 5. Each point in-
3.2. Results of validation test The angle outcome extracted from the IMU system were compared to the angles obtained by the VICON system (as described earlier). The average (SD) of RMSE for all tests was 2.9° (Table 3). The ‘R’ column provides data on the level of correlation between the IMU system and the camera outputs. 4. Discussion Wearable devices are evolving in sophistication and, thus, in their ability to accurately monitor kinematic movement. Such data is being used across a range of applications, including the military,
Please cite this article as: M.I.M. Esfahani et al., Sharif-Human movement instrumentation system (SHARIF-HMIS): Development and validation, Medical Engineering and Physics (2018), https://doi.org/10.1016/j.medengphy.2018.07.008
ARTICLE IN PRESS
JID: JJBE
[m5G;August 31, 2018;18:29]
M.I.M. Esfahani et al. / Medical Engineering and Physics 000 (2018) 1–8
7
Table 3 Results of validation test. Task
Test
RMSE
R2
Task
Test
RMSE
R2
Flexion A
Roll Pitch Yaw Roll Pitch Yaw Roll Pitch Yaw Roll Pitch Yaw Roll Pitch Yaw Roll Pitch Yaw Roll Pitch Yaw Roll Pitch Yaw
3.21 2.84 3.53 3.1 2.5 3.38 2.69 2.09 2.97 2.78 2.43 3.1 3.1 2.15 2.92 2.11 1.75 3.73 2.79 3.04 3 3.5 2.34 3.74
0.96 0.97 0.94 0.99 0.99 0.97 0.98 0.96 0.95 0.97 0.98 0.95 0.99 0.97 0.97 0.96 0.95 0.95 0.97 0.95 0.97 0.99 0.97 0.95
Abduction D
Roll Pitch Yaw Roll Pitch Yaw Roll Pitch Yaw Roll Pitch Yaw Roll Pitch Yaw Roll Pitch Yaw Roll Pitch Yaw Roll Pitch Yaw
3.5 2.79 3.61 3.1 2.93 3.13 2.94 3.15 3.86 2.71 3.6 3.86 1.41 3.36 3.16 3.4 2.89 3.56 1.94 2.86 4.11 3.45 2.54 3.69
0.97 0.95 0.98 0.97 0.98 0.96 0.96 0.93 0.94 0.99 0.95 0.97 0.97 0.96 0.95 0.97 0.98 0.94 0.94 0.92 0.93 0.96 0.97 0.94
Flexion B
Flexion C
Flexion D
Extension
Abduction A
Abduction B
Abduction C
the sports/fitness arena, and increasingly in the healthcare domain for continuous monitoring of a patient’s movement. For this investigation, we developed and tested a body-worn system known as SHARIF-HMIS featuring inertial measurement units (IMUs), a data logger, and stretchable clothing. We also introduced an extension of the adaptive Kalman filter (AKF) approach suggested by Li and Wang [24] in order to take into account any localized magnetic disturbances in the filter. This filter could be potentially used to compensate for the effects of magnetic field disturbance, in addition to high acceleration movements, on the accuracy of angle orientations in IMUs. For this study, we considered the observation noise variance of AKF to be variable, which differs from prior studies that have assumed a constant variance [22]. Therefore, our algorithm computed observation noise variance as a variable according to the absolute amount of acceleration and the absolute amount of magnetic field. In contrast, the Li and Wang method (AKF) only considered the absolute amount of acceleration. As such, our modified approach may improve the system against ferromagnetic disturbances. The gyroscope and accelerometer were calibrated using known laboratory approaches. This initial step was essential for validating the general performance of this newly developed system. Results obtained from accelerometer calibration indicated that the highest error in measurement of acceleration was 0.05 g in the orthogonal direction. The linear data shown in Fig. 5 also confirm the accuracy of the gyroscope in the Yaw direction. The highest error for roll, pitch, and yaw directions were 0.6, 1.6, and 4.8°, respectively. Note that, overall, the error average was less than 3° (Table 2). Testing was also conducted to validate shoulder motion, which was selected because it generally known to be one of the most complicated joint in the human body. Our findings indicate that the mean error for measuring the angle of shoulder motions was 2.9°. The results shown in Table 3 confirm that the angles measured by this system are consistent with those obtained by the VICON motioncapture system, as evidenced by the fact that all correlation coefficients between the IMU system camera outputs were more than 0.9. The results of these calibration tests verified that the SHARIFHMIS is a valid system when compared to available commercial devices such Xsens [31]. Thus, we may conclude that our modified algorithm resulted in an accurate measurement of specific human body movements. However, a future study should be designed to
Adduction
Rotation
Horizontal flexion
Horizontal extension
Horizontal rotation
Scapular
Mix
test the SHARIF-HMIS in the presence of magnetic fields to further evaluate the performance of the algorithm. A number of limitations must be acknowledged in this study. First, the magnetometer should be calibrated in the presence of environmental magnetic noise to more fully assess and confirm the performance of the algorithm. Based on practical limitations of time and financial resources, the validation test was completed by just one healthy, young participant. Thus, the findings described herein cannot be extrapolated to older adults with more restrictive movements. Furthermore, future work is needed to evaluate the filtering algorithm under more complex working conditions, such as those involving high acceleration movements and in the presence of magnetic disturbance. To reiterate, the main purpose of this investigation was to validate the hardware and modified algorithm under “normal” conditions in order to determine whether it was reasonably accurate compared to existing commercial devices. Logically, a future study should be designed to assess its reproducibility with more participants in diverse applications and involving high acceleration movements. In summary, our SHARIF-HMIS wearable system demonstrated acceptable measurement accuracy against a validated commercial motion-capture system (i.e., VICON). Our device may help to better quantify patient activity, which can be useful for several applications such as sports/fitness assessments, biomechanical research, and medical rehabilitation and progress evaluation in physical therapy settings.
Conflicts of interest The authors have no conflicts of interest.
Acknowledgments Authors would like to acknowledge the technical and financial support of the Iran National Science Foundation (INSF) under Grant 890 0 0867. We deeply appreciate the contributions of Mrs. Narimani and the staff of the Laboratory of Wearable Technologies and Neuromusculoskeletal Research, as well as the assistance of Mr. Sharifi at Precise Elements Lab.
Please cite this article as: M.I.M. Esfahani et al., Sharif-Human movement instrumentation system (SHARIF-HMIS): Development and validation, Medical Engineering and Physics (2018), https://doi.org/10.1016/j.medengphy.2018.07.008
JID: JJBE 8
ARTICLE IN PRESS
[m5G;August 31, 2018;18:29]
M.I.M. Esfahani et al. / Medical Engineering and Physics 000 (2018) 1–8
References [1] Patel S, Park H, Bonato P, Chan L, Rodgers M. A review of wearable sensors and systems with application in rehabilitation. J Neuroeng Rehabil 2012;9:21. [2] Alabdulkarim S, Nussbaum MA, Rashedi E, Kim S, Agnew M, Gardner R. Impact of task design on task performance and injury risk: case study of a simulated drilling task. Ergonomics 2017;60:851–66. [3] Kim S, Nussbaum MA, Mokhlespour Esfahani MI, Alemi MM, Alabdulkarim S, Rashedi E. Assessing the influence of a passive, upper extremity exoskeletal vest for tasks requiring arm elevation: part I – “Expected” effects on discomfort, shoulder muscle activity, and work task performance. Appl Ergon 2018. [4] Kim S, Nussbaum MA, Mokhlespour Esfahani MI, Alemi MM, Jia B, Rashedi E. Assessing the influence of a passive, upper extremity exoskeletal vest for tasks requiring arm elevation: part II – “Unexpected” effects on shoulder motion, balance, and spine loading. Appl Ergon 2018. [5] Shany T, Redmond SJ, Narayanan MR, Lovell NH. Sensors-based wearable systems for monitoring of human movement and falls. IEEE Sens J 2012;12:658–70. [6] Ehsani H, Mohler J, Marlinski V, Rashedi E, Toosizadeh N. The influence of mechanical vibration on local and central balance control. J Biomech 2018. [7] Godfrey A, Conway R, Meagher D, G OL. Direct measurement of human movement by accelerometry. Med. Eng Phys 2008;30:1364–86. [8] Mokhlespour Esfahani MI, Zobeiri O, Moshiri B, Narimani R, Mehravar M, Rashedi E, et al. Trunk motion system (TMS) using printed body worn sensor (BWS) via data fusion approach. Sensors 2017;17. [9] Mokhlespour Esfahani MI, Narimani R, Ramezanzadehkoldeh M. A wearable respiratory plethysmography using flexible sensor. Int J Biomed Eng Technol 2013;11:364–80. [10] Mokhlespour Esfahani MI, Taghinedjad S, Mottaghitalab V, Narimani R, Parnianpour M. Novel printed body worn sensor for measuring the human movement orientation. Sens Rev 2016;36:321–31. [11] Bergmann JH, McGregor AH. Body-worn sensor design: what do patients and clinicians want? Ann Biomed Eng 2011;39:2299–312. [12] Bergmann JH, Chandaria V, McGregor A. Wearable and implantable sensors: the patient’s perspective. Sensors 2012;12:16695–709. [13] Mokhlespour Esfahani MI, Nussbaum MA. Preferred placement and usability of a smart textile system vs. inertial measurement units for activity monitoring. Sensors 2018;18. [14] Mokhlespour Esfahani MI, Zobeiri O, Narimani R, Hoviattalab M, Moshiri B, Parnianpour M. Design and prototyping of wearable measuring system for trunk movement using textile sensors. In: Proceedings of the Twentieth Iranian Conference on Electrical Engineering (ICEE). IEEE; 2012. p. 1571–5. [15] Mokhlespour Esfahani MI, Nussbaum MA. A “Smart” undershirt for tracking upper body motions: task classification and angle estimation. IEEE Sens J 2018. [16] De Rossi D, Veltink P. Wearable technology for biomechanics: e-textile or micromechanical sensors? IEEE Eng Med Biol Mag: Q Mag Eng Med Biol Soc 2010;29:37–43.
[17] Castano LM, Flatau AB. Smart Textile Transducers: Design, Techniques, and Applications. Industrial Applications for Intelligent Polymers and Coatings. Cham: Springer International Publishing; 2016. p. 121–46. [18] Hosseinzadegan H, Pierron ON, Hosseinian E. Accurate modeling of air shear damping of a silicon lateral rotary micro-resonator for MEMS environmental monitoring applications. Sens Actuators A – Phys 2014;216:342–8. [19] Foerster F, Smeja M, Fahrenberg J. Detection of posture and motion by accelerometry: a validation study in ambulatory monitoring. Comput Hum Behav 1999;15:571–83. [20] Wong WY, Wong MS, Lo KH. Clinical applications of sensors for human posture and movement analysis: a review. Prosthet Orthot Int 2007;31:62–75. [21] Coley B, Jolles BM, Farron A, Bourgeois A, Nussbaumer F, Pichonnaz C, et al. Outcome evaluation in shoulder surgery using 3D kinematics sensors. Gait Posture 2007;25:523–32. [22] Roetenberg D, Luinge HJ, Baten CTM, Veltink PH. Compensation of magnetic disturbances improves inertial and magnetic sensing of human body segment orientation. IEEE Trans Neural Syst Rehabil Eng 2005;13:395–405. [23] Meng X, Zhang ZQ, Wu JK, Wong WC, Yu H. Self-contained pedestrian tracking during normal walking using an inertial/magnetic sensor module. IEEE Trans Bio-med Eng 2014;61:892–9. [24] Li W, Wang JL. Effective adaptive Kalman filter for MEMS-IMU/magnetometers integrated attitude and heading reference systems. J Navig 2013;66:99–113. [25] Abedi M, Manshadi FD, Zavieh MK, Ashouri S, Azimi H, Parnanpour M. A reliability study of the new sensors for movement analysis (SHARIF–HMIS). J Bodyw Mov Ther 2016;20:341–5. [26] Mokhlespour Esfahani MI, Zobeiri O, Akbari A, Moshiri B, Parnianpour M. Wearable human movement instrumentation system. Iran J Biomed Eng 2014;7:361–9. [27] Roetenberg D, Luinge H, Slycke P. Xsens MVN: full 6DOF human motion tracking using miniature inertial sensors. Xsens Technol: Xsens Co 2013:1–9. [28] Marcard TV. Design and implementation of an attitude estimation system to control orthopedic components. Goteborg, Sweden: Chalmers University of Technology; 2010. [29] Roetenberg D, Luinge HJ, Baten CT, Veltink PH. Compensation of magnetic disturbances improves inertial and magnetic sensing of human body segment orientation. IEEE Trans Neural Syst Rehabil Eng: Publ IEEE Eng Med Biol Soc 2005;13:395–405. [30] Plamondon A, Delisle A, Larue C, Brouillette D, McFadden D, Desjardins P, et al. Evaluation of a hybrid system for three-dimensional measurement of trunk posture in motion. Appl Ergon 2007;38:697–712. [31] Robert-Lachaine X, Mecheri H, Larue C, Plamondon A. Validation of inertial measurement units with an optoelectronic system for whole-body motion analysis. Med Biol Eng Comput 2017;55:609–19.
Please cite this article as: M.I.M. Esfahani et al., Sharif-Human movement instrumentation system (SHARIF-HMIS): Development and validation, Medical Engineering and Physics (2018), https://doi.org/10.1016/j.medengphy.2018.07.008