- Email: [email protected]
Ankle Joint Moment Estimation Based on Smart Shoes*
Proceedings of the 20th World The International Federation of Congress Automatic Control Proceedings of the 20th World The International Federation of Congress Automatic Control Control The International of Automatic Toulouse, France,Federation July 9-14, 2017 The International Federation of Automatic Control Proceedings of the 20th9-14, World Congress Toulouse, Toulouse, France, France, July July 9-14, 2017 2017 Available online at www.sciencedirect.com Toulouse, France,Federation July 9-14, 2017 The International of Automatic Control Toulouse, France, July 9-14, 2017
ScienceDirect
PapersOnLine 50-1 (2017) 1366–1371 Ankle IFAC Joint Moment Estimation Based on Ankle Estimation Based Ankle Joint Joint Moment Moment Estimation Based on on Smart Shoes Ankle Joint Moment Estimation Based on Smart Shoes Smart Shoes Smart Shoes Hyunjin Choi ∗∗∗ Kanghyun Kim ∗∗∗ Pyeong-Gook Jung ∗∗∗
Hyunjin Choi Kim Pyeong-Gook Jung ∗∗∗ Hyunjin Kanghyun Kim Pyeong-Gook Jung ∗ ∗∗ ∗ Byeonghun Na∗∗∗∗Kanghyun Dong-wook Rha Kyungmo Jung Hyunjin Choi Choi Kanghyun Kim Pyeong-Gook Jung∗∗∗ ∗∗ ∗∗ ∗∗∗ Byeonghun Na Dong-wook Rha Kyungmo Jung ∗ Byeonghun Na Rha ∗ ∗ Dong-wook ∗ ∗∗ Kyungmo Jung ∗∗∗ Kyoungchul Kong Hyunjin Choi Kanghyun Kim Pyeong-Gook Jung Byeonghun Na Dong-wook Kyungmo Jung ∗ ∗ ∗ KyoungchulRha Kong Kyoungchul Kong ∗∗ ∗ Byeonghun Na ∗ Dong-wook Kyungmo Jung ∗∗∗ KyoungchulRha Kong ∗ ∗ Engineering, Sogang University, Seoul, Kyoungchul Kong ∗ Department of Mechanical ∗ of Mechanical Engineering, Sogang University, Seoul, Department of Mechanical Engineering, Sogang University, ∗ Department Korea (e-mail: [email protected], [email protected] Department of Mechanical Engineering, Sogang University, Seoul, Seoul, Korea (e-mail: [email protected], [email protected] [email protected], [email protected] ∗ Korea (e-mail: {pgjung, nbh87,Engineering, kckong}@sogang.ac.kr). Department of Mechanical Sogang University, Seoul, Korea (e-mail: [email protected], [email protected] {pgjung, nbh87, kckong}@sogang.ac.kr). kckong}@sogang.ac.kr). ∗∗ {pgjung, nbh87, Department and Research Institute [email protected] Rehabilitation Medicine, Korea (e-mail: [email protected], {pgjung, nbh87, kckong}@sogang.ac.kr). ∗∗ ∗∗ Department and Research Research Institute of Rehabilitation Rehabilitation Medicine, Department and Institute of Medicine, ∗∗ Severance Rehabilitation Hospital, Yonsei University College of {pgjung, nbh87, kckong}@sogang.ac.kr). Department and Research Institute of Rehabilitation Medicine, Severance Rehabilitation Hospital, Yonsei University College of Severance Rehabilitation Hospital, Yonsei University College of ∗∗ Medicine, Seoul, Korea (e-mail: [email protected]) Department and Research Institute of Rehabilitation Medicine, Severance Rehabilitation Hospital, Yonsei University College of Seoul, Korea (e-mail: [email protected]) ∗∗∗ Medicine, Medicine, Seoul, Korea (e-mail: [email protected]) Human Factors & Device Research Team, Hyundai Motor Severance Rehabilitation Hospital, Yonsei University College of Medicine, Seoul, Korea (e-mail: [email protected]) ∗∗∗ ∗∗∗ Human Factors & Device Research Team, Hyundai Motor Factors & Device Research Team, Hyundai Motor ∗∗∗ Human Company, Korea (e-mail: [email protected]) Medicine, Seoul, Korea (e-mail: [email protected]) Human Factors & Device Research Team, Hyundai Motor Company, Korea Korea (e-mail: (e-mail: [email protected]) [email protected]) Company, ∗∗∗ Human Factors & Device Research Team, Hyundai Motor Company, Korea (e-mail: [email protected]) Company, Korea (e-mail: [email protected]) Abstract: For analyses and diagnoses of a human gait, the precise measurement of ground Abstract: For analyses analyses and and diagnoses of a human human gait, the precise precise measurement of ground ground Abstract: For diagnoses of the measurement of reaction forces essential, because GRFgait, is the significant external in Abstract: For (GRFs) analysesis diagnoses of aa the human themost precise measurement of force ground reaction forces (GRFs) isand essential, because the GRFgait, is the most significant external force in reaction forces (GRFs) is essential, because the GRF is the most significant external force in the human body dynamics. The GRF is not only useful information by itself, but also utilized to Abstract: For analyses and diagnoses of a human gait, the precise measurement of ground reaction forces (GRFs) is essential, because the GRF is the most significant external force in the human body dynamics. The GRF is not only useful information by itself, but also utilized to the human body dynamics. The is not only useful information by but also obtain further useful information, such center pressure (CoP) and the joint moments. reaction forces (GRFs) is essential, because the GRFof is the most significant external force to in the human body dynamics. The GRF GRF is as notthe only useful information by itself, itself, but also utilized utilized to obtain further useful information, such as the center ofcapture pressure (CoP) and the the joint moments. obtain useful information, such as the center of pressure (CoP) and joint moments. For human thisfurther purpose, a force plate and anisoptical motion system have been considered as the body dynamics. The GRF not only useful information by itself, but also utilized to obtain further useful information, such as the center of pressure (CoP) and the joint moments. For this purpose, purpose,However, a force plate plate and an an optical motion capture system have limits. been considered considered as For this a and optical motion have been as a gold the force plate inconvenient due tosystem its spatial To moments. overcome obtain further useful information, as is the center pressure (CoP) andlimits. the For thisstandard. purpose, a force force plate andsuch an optical motionofcapture capture have beenjoint considered as a gold standard. However, theSmart force plate is inconvenient dueGRFs tosystem its by spatial To overcome overcome a gold standard. However, the force plate is inconvenient due to its spatial limits. To the limitation of force plates, Shoes that measure the four air-pressure sensors For thisstandard. purpose, a force plate and an optical motion capture have been considered as a gold theSmart force plate isthat inconvenient dueGRFs tosystem its by spatial To overcome the limitation of However, force plates, Shoes measure the four limits. air-pressure sensors the limitation of force plates, Smart Shoes that measure the GRFs by four air-pressure sensors in silicone insoles are used to measure the GRFs intothis paper. Then, To the overcome dynamic aembedded gold standard. However, the force plate is inconvenient due its spatial limits. the limitation of force plates, Smart Shoes that measure the GRFs by four air-pressure sensors embedded in silicone insoles are used used to measure the GRFs in this paper. paper. Then, the dynamic dynamic embedded in are to the GRFs in Then, the characteristics the insoles ankle areShoes modeled used to estimate the joint the limitation ofof force plates,joint Smart thatand measure the GRFs fourankle air-pressure sensors embedded in silicone silicone insoles are used to measure measure the GRFs in this thisby paper. Then, the moments dynamic characteristics of the ankle joint are modeled and used to estimate the ankle joint moments characteristics of the ankle joint are modeled and used to estimate the ankle joint moments from the Smart Shoes measurement. The performance oftothe proposed method isthe verified by embedded in silicone insoles are used to measure the GRFs in this paper. Then, dynamic characteristics of the ankle joint are modeled and used estimate the ankle joint moments from the Smart Smart Shoes measurement. The performance performance of of the the proposed proposed method method is is verified verified by from the Shoes measurement. The by experimental results with a human subject. characteristics of the ankle joint are modeled and used to estimate the ankle joint moments from the Smart Shoes measurement. The performance of the proposed method is verified by experimental results results with with aa human human subject. subject. experimental from the Smart Shoes measurement. The performance of the proposed method is verified by experimental results with a human subject. © 2017, IFAC (International Federation of Automatic Control) Hosting by Elsevier Ltd. All rights reserved. experimental results with a human subject. Keywords: Gait monitoring, mobile sensing system, ground reaction force, sensor, signal Keywords: Gait monitoring, mobile sensing system, ground reaction force, sensor, signal Keywords: Gait monitoring, monitoring, mobile mobile sensing sensing system, system, ground ground reaction reaction force, force, sensor, sensor, signal signal processing. Gait Keywords: processing. processing. Keywords: Gait monitoring, mobile sensing system, ground reaction force, sensor, signal processing. processing. 1. INTRODUCTION To develop a sensor-integrated shoe, Kim et al. (2014) used 1. INTRODUCTION To develop aa sensor-integrated et al.of(2014) used 1. To develop shoe, Kim et used triaxial coin-type load cells atshoe, the Kim outsole the shoe. 1. INTRODUCTION INTRODUCTION To develop a sensor-integrated sensor-integrated shoe, Kim et al. al.of(2014) (2014) used triaxial coin-type load cells at the outsole the shoe. triaxial coin-type load cells at the outsole of the shoe. Although the load cells guarantee the high precision in 1. INTRODUCTION To develop a sensor-integrated shoe, Kim et al. (2014) used coin-type load cells at the outsole of the shoe. The measurement of ground reaction forces (GRFs) triaxial Although the but loadthe cells guaranteerigidity the high high precision in The measurement of ground ground reaction forcesof (GRFs) (GRFs) Although the load cells guarantee the precision in The measurement of reaction forces measurement, mechanical of the load cells triaxial coin-type load cells at the outsole of the shoe. Although the load cells guarantee the high precision in plays a significant role in many applications biomeThe measurement of ground reaction forcesof (GRFs) measurement, but the mechanical rigidity of the load cells plays a significant role in many applications biomemeasurement, but the mechanical rigidity of the load cells plays a significant role in many applications of biomeoften cause discomfort and disturb the human motion. To Although the load cells guarantee the high precision in measurement, but the mechanical rigidity of the load cells chanics, healthcare mechatronics, robotics, rehabilitation The measurement of ground forces plays a significant role in manyreaction applications of (GRFs) biome- often cause and disturb the human motion. To chanics, healthcare mechatronics, robotics, rehabilitation cause discomfort and disturb the human motion. To chanics, healthcare mechatronics, address the discomfort rigidity issue, many rigidity researchers like Howell measurement, but the mechanical of the load cells often cause discomfort and disturb the human motion. To medicines, and so on. The foot isrobotics, the mostrehabilitation distal body often plays a significant role in many applications of biomechanics, healthcare mechatronics, robotics, rehabilitation address the rigidity issue, many researchers like Howell medicines, and so on. The The foot is the most most distal body address the rigidity issue, many researchers like Howell medicines, and so foot the distal al. cause (2013) and Zheng al. (2014) developed insoleoften andet disturb the human motion. To address the discomfort rigidity issue, many researchers like Howell segment that frequently interacts with ground in body daily et chanics, healthcare mechatronics, robotics, rehabilitation medicines, and so on. on. The foot is is thethe most distal body et al. (2013) and Zheng et al. (2014) developed insolesegment that frequently interacts with the ground in daily et al. (2013) and Zheng et al. (2014) developed insolesegment that frequently interacts with the ground in daily type GRF sensor systems with force resistive sensors address the rigidity issue, many researchers like Howell et al. (2013) and Zheng et al. (2014) developed insoleactivities, and thus the external forces applied to the medicines, and so on. The foot is the most distal body segment that frequently interacts with the ground in daily type GRF GRF sensor systems with only force for resistive sensors activities, andGRFs) thus the external forces forces applied to and the type systems with force resistive sensors activities, and thus external applied the butsensor most of them the detection et al. GRF (2013) and Zheng et were al. (2014) developed insoletype sensor systems with force for resistive sensors foot (i.e.,that the areinteracts important forthe the analyses segment ground into activities, andfrequently thus the the externalwith forces applied todaily the (FSRs), (FSRs), but most of them were only the detection foot (i.e., the GRFs) are important for the analyses and (FSRs), but most of them were only for the detection foot (i.e., the GRFs) are important for the analyses and of certain postures due to the poor accuracy of the FSR type GRF sensor systems with force resistive sensors (FSRs), but most of them were only for the detection diagnoses of dynamic characteristics of human motions. activities, and thus the external forces applied to the foot (i.e., the GRFs) are important for the analyses and of certain postures due to the poor accuracy of the FSR diagnoses of of dynamic dynamic characteristics characteristics of of human human motions. motions. of certain postures due to the poor accuracy of the diagnoses measurement. (FSRs), but most of them were only for the detection certain postures due to the poor accuracy of the FSR FSR foot (i.e.,plate thedynamic GRFs) are regarded importantasof forthe themost analyses and of diagnoses of characteristics human motions. measurement. A force has been accurate measurement. of certain postures due to the poor accuracy of thewhich FSR measurement. A force plate has been regarded as the most accurate diagnoses of dynamic characteristics of human motions. A force plate has been regarded as the most accurate Kong and Tomizuka (2009) developed Smart Shoes, means measure the GRFs; however, their price and Kong A forceto plate has been regarded as the their mostprice accurate and Tomizuka (2009) developed Smart Shoes, which measurement. means to measure the GRFs; however, and Kong and Tomizuka (2009) developed Smart Shoes, which means measure the GRFs; however, price and anTomizuka insole-type GRFdeveloped sensor system airKong and (2009) Smartbased Shoes,on which spatial limitations drawbacks. Moreover, A forceto plate has are been regarded as the their mostMoreover, accurate means to measure theunavoidable GRFs; however, their price and include include an insole-type GRF sensor sensor system based on airspatial limitations are unavoidable drawbacks. include an insole-type GRF system based on airspatial limitations are unavoidable drawbacks. Moreover, bladder sensors. The air-bladder sensor consists of an airKong and Tomizuka (2009) developed Smart Shoes, which include an insole-type GRF sensor system based on the force plate measures the interaction forces between means to measure theunavoidable GRFs; however, their price and bladder spatial limitations are drawbacks. Moreover, sensors. The air-bladder sensor consists of an airthe force plate measures the interaction forces between bladder sensors. The air-bladder sensor consists of an airthe force plate measures the interaction forces between sensor and silicone tube. Each of the Smart include an insole-type GRF sensor system based bladder sensors. The aaair-bladder sensor consists of on an airthe shoes and the ground, while the desired information spatial limitations are unavoidable drawbacks. Moreover, force plate measures the interaction forces between pressure pressure sensor silicone tube. Each of the Smart the shoes andinteraction the ground, while the desired information sensor and silicone tube. Each of the Smart the and the ground, while the desired information Shoes pair has and four aaair-bladder air-bladder sensors placed at the bladder sensors. The sensor consists of an pressure sensor and silicone tube. Each of the Smart is inshoes fact the forces between theforces shoes and the pressure force plate measures the interaction between the shoes and the ground, while the desired information Shoes pair has four four air-bladder sensors placed at airthe is in fact the interaction forces between the shoes and the Shoes pair has air-bladder sensors placed at is in fact the interaction forces between the shoes and the locations of major bones or joints that contact the ground. pressure sensor and a silicone tube. Each of the Smart pair has four air-bladder sensors placed at the the feet. The the force plate measurement is available only when the andinteraction the ground, while the desired information is inshoes fact forces between the shoes and the Shoes locations of major bones or joints that contact the ground. feet. The force plate measurement is available only when locations of major bones or joints that contact the ground. feet. The force plate measurement is available only when The Smart Shoes areair-bladder useful practice due Shoes pair has four sensors placed attheir the locations of major bones or jointsin that contact the to ground. a human subject steps on it, which disturbs the natural is in fact the interaction forces between the shoes and the feet. The force plate measurement is available only when The Smart Shoes are useful in practice due to their a human subject steps on it, it,aspects, which researches disturbs the the natural The Smart Shoes are useful in practice due to their afeet. human subject steps on which disturbs natural superior durability and wide sensing range. However, Kong locations of major bones or joints that contact the ground. The Smart Shoes are useful in practice due to their human gait motion. In these on sensorThe force plate measurement is available only when ahuman human subject steps on it,aspects, which researches disturbs the natural superior durability and wide range. However, Kong gait motion. In these these on sensordurability and wide sensing range. However, human gait motion. In on sensorand Tomizuka (2009) notsensing validate the performance of The Smart Shoes aredid useful in practice due to Kong their superior durability and wide sensing range. However, Kong shoes have by Jacobsthe andnatural Ferris superior aintegrated human subject steps on conducted it,aspects, which researches disturbs human gait motion. Inbeen these aspects, researches on sensorand Tomizuka (2009) did not validate the performance of integrated shoes have been conducted by Jacobs and Ferris and Tomizuka (2009) did not validate the performance of integrated shoes have been conducted by Jacobs and Ferris the Smart Shoes comparing with a gold standard method superior durability and wide sensing range. However, Kong and Tomizuka (2009) did not validate the performance of (2015), gait Jung et al. (2014), and so on. The sensor-integrated human motion. Inbeen these aspects, researches on integrated shoes have conducted by Jacobs andsensorFerris the the Smart Shoes comparing with a gold standard method (2015), Jung et al. (2014), and so on. The sensor-integrated Smart Shoes comparing with a gold standard method (2015), Jung et al. (2014), and so on. The sensor-integrated (i.e., a force plate and an optical motion capture system). and Tomizuka (2009) did not validate the performance of the Smart Shoes comparing with a gold standard method shoes are noteworthy, because they can be used without integrated shoes have been conducted by Jacobs and Ferris (i.e., aa force (2015), Jung et al. (2014), and sothey on. The sensor-integrated plate and an optical motion capture system). shoes are noteworthy, because can bereaction used without force plate and an optical motion capture system). shoes noteworthy, because they can be used without the Smart Shoes comparing with a gold standard method (i.e., a force plate and an optical motion capture system). limits are inJung time and space, and so measure the forces (i.e., (2015), et al. (2014), on. The sensor-integrated shoes are noteworthy, because they can be used without sensors available for the insole-type GRF sensor limits in time and and space, and measure the the reaction forces forces The limits in time space, and The sensors available for the insole-type insole-type GRF sensor (i.e., asensors force and anfor optical motionsensors, capture system). acting are actually on the feet. available the GRF sensor shoes noteworthy, because they canthebereaction used without limits in time and space, and measure measure reaction forces The systems (e.g.,plate the FSRs, the air-bladder etc.) show The sensors available for the insole-type GRF sensor acting actually on the feet. acting actually on the the feet. (e.g., the FSRs, the air-bladder sensors, etc.) show systems (e.g., the FSRs, the air-bladder sensors, etc.) show limits in time and space, and measure the reaction forces systems acting actually on feet. larger nonlinearity compared to the load cells, because The sensors available for the insole-type GRF sensor systems (e.g., the FSRs, the air-bladder sensors, etc.) show larger nonlinearity compared to the load cells, because larger nonlinearity compared to the load cells, because acting actually on the feet. of the sensing principles and the structural uncertainties. systems (e.g., the FSRs, the air-bladder sensors, etc.) show larger nonlinearity compared to the load cells, because This work was supported by Hyundai Motors Company. of the the sensing sensing principles principles and and the the structural structural uncertainties. uncertainties. This work was supported by Hyundai Motors Company. of work was supported by Hyundai Motors Company. larger nonlinearity compared to structural the load cells, because of the sensing principles and the uncertainties. This This work was supported by Hyundai Motors Company. of the sensing principles and the structural uncertainties. This work was supported by Hyundai Motors Company. Copyright © 2017 IFAC 1402 Copyright © 2017, 2017 IFAC IFAC 1402Hosting by Elsevier Ltd. All rights reserved. Copyright © 2017 1402 2405-8963 © IFAC (International Federation of Automatic Control) Copyright © 2017 IFAC 1402 Peer review under responsibility of International Federation of Automatic Control. Copyright © 2017 IFAC 1402 10.1016/j.ifacol.2017.08.234
Proceedings of the 20th IFAC World Congress Toulouse, France, July 9-14, 2017 Hyunjin Choi et al. / IFAC PapersOnLine 50-1 (2017) 1366–1371
1367
Fig. 1. A Smart Shoe system for motion capturing; (a) an insole-type GRF sensor system. The proposed force sensing unit (b) is constituted by (c) soft elastomers with an air chamber, (d) soft covers for sealing, and (e) a small-sized barometer to measure the pressure change. Various methods have been studied to compensate for the nonlinearity. For example, Jacobs and Ferris (2015) used a regression model that linearizes the measurement of their insole-type GRF sensor system. They validated the regression model with the GRF signals obtained during different motions, such as walking and calf raising. This method was useful, but it was limited to a subject-specific case and required an initial trial on a force plate. In fact, however, it is an unanswered question if the measurement of an insole-type GRF sensor system should match that of a force plate, because of the dynamic effects of the cushion material (i.e., the sole of a shoe). It is reasonable and convincing that the measurement of the insole-type GRF sensor system is more reliable than the force plate, as long as its performance is validated, because the force actually acting on the foot is the interaction force between the foot and the insole. In this paper, an improved version of the Smart Shoes is introduced, and it is utilized for analyzing the ankle joint moment, as shown in Figs. 1 and 2. The new Smart Shoes include four modularized air-bladder sensors, as shown in Fig. 1(a), which is similar to the Smart Shoes developed by Kong and Tomizuka (2009), but its manufacturing process and sensing performance have been improved. The gait phase detection algorithm proposed by Kong and Tomizuka (2009) is identically applied in this paper as well. Unlike the previous work, however, the GRF measurements are compared and calibrated with a gold standard method (i.e., a force plate), and thus the GRF sensing performance of the Smart Shoes is validated in this paper. Based on the GRF measurement, the position of the center of pressure (CoP) is calculated in the laboratory coordinate system. For accurate comparison of the GRF sensing performance with a force plate, the two sensing methods are used to measure the same physical quantity at the same time. In addition, in order to check if the Smart Shoes can replace the force plate in an optical motion capture system, the gait analysis performance with the Smart Shoes (in particular, the ankle joint moment analysis) is verified in this paper also. The performance of the proposed system is verified by experimental results with a human subject in this paper. The subject is asked to walk for several steps wearing the Smart Shoes and sixteen infrared markers in a laboratory setting. For accurate verification, the subject is asked to step on a force plate during walking. The graphs and
Fig. 2. Configuration of an experimental setting; (a) optical motion capture system and force plate in a laboratory, and (b) a human subject stepping on force plates with sixteen markers and the Smart Shoes. data shown in this paper are all the experimental results obtained with the human subject. 2. HARDWARE CONFIGURATION The proposed Smart Shoes system consists of four force sensing units and one inertial measurement unit (IMU) at each side. The force sensing units are made of casted silicone and have air chambers and small-sized barometer, which is the absolute pressure sensor manufactured by Measurement Specialities Inc., as shown in Fig 1. If a force is applied, the whole sensing unit is compressed and thus the air chamber is pressurized. This pressure change is measured by the absolute pressure sensor embedded in each sensing unit. As the sensor measures the absolute pressure, the sensing unit can be completely encapsulated, which greatly improves the practicality. The sensing performance, such as dynamic range, hysteresis, linearity, etc., is dependent on the shape of the air chamber and the material property. Through multiple trials and errors, an appropriate material was sought; the RTV silicone rubber KE-12 manufactured by ShinEtsu Chemical, which is a hyper-elastic material with the hardness of Shore A40 and the tensile strength of 2.5M P a, was utilized for fabricating the proposed force sensing unit. Each force sensing unit is calibrated with a load cell while measuring the same force in series, as shown in Fig. 3(a); a regression model with a second-order polynomial function is applied for the calibration. The parameters of the regression model were found by Matlab polyfit.m function; the obtained regression model is s = a2 m2 + a1 m + a0 , (1) where m is the raw measurement from the absolute pressure sensor, s is the calibrated signal, and the parameters are a2 = 2.8018 × 10−13 , a1 = 9.6250 × 10−6 , and a0 = 1.1960 × 10−1 , respectively. Figure 3(b) shows the linearity of force measurement by the proposed sensor unit. The linearity was calculated as 3.8 %FSO (full scale output). Both the left and right sides of Smart Shoes utilize the same force sensing units. In each side, four force sensing units are placed on the locations of the major bones and joints, such as, the hallux, the first-second and the
1403
Proceedings of the 20th IFAC World Congress Toulouse, France, July 9-14, 2017 1368 Hyunjin Choi et al. / IFAC PapersOnLine 50-1 (2017) 1366–1371
60
Proposed sensor(kgf)
Table 1. Parameters for GRF measurement Sensor(i) 1 2 3 4
40
20
0
0
20
40
60
Loadcell(kgf)
(a)
(b)
Fig. 3. Force calibration and linearity test of sensing units; (a) a test-bed for calibrating the proposed sensor unit with a load cell, and (b) the calibration result. fourth-fifth metatarsophalangeal joints, and the heel [see Fig. 1(a)]. 3. MEASUREMENT OF GROUND REACTION FORCES The proposed force sensing unit enables the Smart Shoes to measure the ground reaction pressure distributed over the foot area. Since the ground reaction pressure is mainly concentrated at the locations of major bones and joints, the ground reaction forces (GRFs) can be approximately measured by the four force sensing units placed at the locations of the major bones and joints, as shown in Fig. 1. In a strict sense, however, the four sensing units cannot measure the whole ground reaction pressure distributed over the foot area due to the limited size of the sensing units. For example, when the whole foot contacts the ground during a mid-stance phase, the ground reaction pressure is almost uniformly distributed over the foot area, which cannot be accurately measured by the sensors placed at the discrete locations. Therefore, an algorithm that estimates the ground reaction pressure from the GRFs measured by the proposed sensing units is introduced in this section. Since the GRFs are transferred to the body through the foot bones and joints contacting the ground, the peaks of the GRFs may occur at the locations of the bones and joints. In practice, however, the GRFs are distributed over a large area, because the sole of a foot is covered by soft skin tissues, and the sole of a shoe is made of soft cushion materials. Nevertheless, the centers of the concentrated pressures are not changed (i.e., the locations of the bones and joints, at which the force sensing units are placed). If the area of the ground reaction pressure distribution is smaller than the area of a force sensing unit, the sensor measurement is reliable. However, if the area of the pressure distribution is larger than the sensing area, which is most likely when the GRF is significantly large, the sensor reading is no longer reliable. In order to address this issue, a kernel density function, K(x, y), is applied, i.e. (x−xi )2 +(y−yi )2 − 1 2 2σ i e Ki (x, y) = , (2) 2πσi2
Position(xi , yi )[m] (-0.016, 0.244) (-0.021, 0.199) (0.027, 0.182) (0, 0.043)
λi 1.24 1.24 1.24 1.44
σi 15.84 15.84 15.84 19.94
where xi and yi represent the center location of ith force sensing unit, and σi is the standard deviation of pressure distribution, which is related to the size of the pressure distribution area. The values of xi ’s and yi ’s of the Smart Shoes used in the experiments in this paper are shown in Table 1. The index i = {1, ..., 4} represents the sensing units located at the hallux, the first-second and the fourthfifth metatarsophalangeal joints, and the heel, respectively. Notice that the kernel function in (2) is a Gaussian distribution function, the spatial integral of which overall all area is unitary. The use of such kernel functions were introduced in our previous work by Kim et al. (2016). Based on the kernel function, the ground reaction force at the ith sensing area, Fi , is estimated as si Ki (x, y)dxdy, (3) Fi = λi where λi is a scaling factor, and si is the measurement value calibrated by (1). Since si is scalar, (3) can be rewritten as Fi = λi si
Ki (x, y)da + ∆i ,
(4)
Di
where Di is the area of each sensing unit defined by Di = {(x, y)|(x − xi )2 + (y − yi )2 ≤ r}, (5) where r is the radius of a force sensing unit. The first part on the right-hand-side of (4) represents the force applied on the sensing area, while the second part represents the force applied outside of the sensing area. It should be noted again that the forces applied on the sensing area cannot represent the whole distribution of the ground reaction pressure, as shown in Fig. 4(a). The force applied on the sensing area is the measurement of the sensing unit, i.e. si = λi si Ki (x, y)da. (6) Di
Therefore, the scaling factor, λi , can be calculated as 1 λi = . (7) Ki da Di
The standard deviation, σi , in (2) is determined from the size of bones and joint contacting the ground. Applying si and λi to (4), the distribution of the ground reaction pressure at each sensing area can be partially recovered, as shown in Fig. 4(b). By the estimated ground reaction pressures, the total GRF can be calculated more accurately, i.e. 4 Ftotal = Fi , (8) i
where Fi is the foot pressure distribution calculated by (4).
1404
Proceedings of the 20th IFAC World Congress Toulouse, France, July 9-14, 2017 Hyunjin Choi et al. / IFAC PapersOnLine 50-1 (2017) 1366–1371
1369
(a)
(b)
Fig. 4. Kernel density estimation of foot pressure distribution; (a) pressure distribution on sensor areas, and (b) estimated pressure distribution of foot.
where Fi is the sensor measurement calibrated as in (4), and xi and yi are the locations of the sensing units shown in Table 1. Figure 6 shows the CoP calculated as in (9). The red circles represent the locations of sensing units, and the black dots represent the CoP during a gait cycle.
150
GRF(%BW)
125 100 75 50
Notice that the CoP calculated by (9) is limited in the range of sensor locations, i.e., xcop ∈ [mini (xi ), maxi (xi )] and ycop ∈ [mini (yi ), maxi (yi )], while the actual CoP can be placed anywhere on the foot area. This problem is also shown in the experimental result in Fig. 6.
Smart Shoes Force Plate
25 0 0
Ankle angle(rad)
Fig. 6. Center of pressure measured by Smart Shoes during stance phase. The red dots represent the position of center of each sensor units.
25
50
75
100
1 0.5 0 -0.5 0
25
50
75
100
Stance(%)
Fig. 5. Ground reaction force measurement and foot angle measurement during stance phase; black line represents force plate measurements, and gray line represents Smart Shoes measurements. To validate the GRF measurement of the proposed system, a subject with the body weight of 60kg performed a walking experiment wearing the Smart Shoes in a laboratory environment where a force plate manufactured by Advance Mechanical Technology, Inc. is installed, as shown in Fig. 2. Figure 5 shows the GRFs measured by the proposed method and the force plate. It should be noted that the two different systems measured very similar values throughout the gait cycle. The measurements shown in Fig. 5 cannot be directly compared, because the proposed Smart Shoes measured the interaction force between the foot and the insole, while the force plate measured the interaction force between the sole of a shoe and the floor. Nevertheless, the measurement error was as small as 5.23 %BW (body weight) in terms of root-mean-squares (RMS). 4. CENTER OF PRESSURE ESTIMATION The center of pressure (CoP) is important for analyzing the human body dynamics, because it is the location where all the ground reaction pressure is assumed to be concentrated. The CoP can be calculated from the GRFs as xcop =
ΣFi xi ΣFi
and
ycop =
ΣFi yi , ΣFi
(9)
The limitation in the range of CoP is compensated by utilizing the gait phase information. If the gait of a subject is normal and thus it shows certain phases, called gait phases, repetitively, the CoP trajectory of the current stride may be identical to that of the previous stride. Assuming that 1) the gait period, Tperiod , is constant (at least the same as the previous stride) and that 2) the CoP trajectory should start from the rear end of the shoe and end at the front end of the shoe, a CoP compensator, k(t), can be utilized as yˆcop (t) = ycop (t) + k(t)
(10)
where k(t) is defined as a function of the gait phases, such as the initial contact (IC) phase, the mid-stance (MS) phase, the terminal stance (TS) phase, and the swing (SW) phase, i.e. ycop (t) (t − TIC ), TIC k(t) = 0, L − ycop (t) (t − TT S ), Tperiod − TT S
for 0 ≤ t < TIC for TIC ≤ t < TT S (11) for TT S ≤ t < Tperiod ,
where t, TIC , TT S , and Tperiod are the current gait cycle time, the time when the IC phase ends, the time when the TS phase starts, and the gait cycle period, respectively. L is the foot length of the human subject. TIC and TT S can be detected by the gait phase detection method developed by Kong and Tomizuka (2009). Figure 7 shows the CoP of the anterior-posterior direction, i.e., ycop ; the CoP trajectories with and without a correction by the proposed method are shown in the figure. According to the gait phases detected as in Fig. 7(b), the CoP compensator, k(t), was calculated as shown in Fig. 7(c). The x-axis is scaled by the percent stance cycle for the sake of simple representation. It should be noted that the CoP trajectories calculated by the proposed method and the force plate match well.
1405
Proceedings of the 20th IFAC World Congress 1370 Hyunjin Choi et al. / IFAC PapersOnLine 50-1 (2017) 1366–1371 Toulouse, France, July 9-14, 2017
0.3
(a)
CoP y(m)
Smart Shoes Estimated Force Plate
0.1
(a)
(b)
Fig. 8. Foot reference coordinate system and the positions of the rear heel(rcalcn ) and center of mass(rcom ).
1
(b)
IC MS TS
0.5
Table 2. The parameters of a reference human body model
0
0.05
Gait phases
0.2
0
Reference Model rref calcn rref com Iref com mref f oot
(c)
0 -0.05
k(t)
0
25
50
75
100
Stance(%)
Fig. 7. (a) Estimated center of pressure, (b) gait phases, and (c) CoP compensator during stance phase. While the CoP of the anterior-posterior direction, i.e., ycop , should be corrected as in (10), it is not necessary to correct xcop , because it is within the range of sensor locations in general. 5. ANKLE MOMENT ESTIMATION 5.1 Moment Calculation Assuming that the foot is a rigid body and that it rotates around the ankle axis, the ankle joint moment, T ∈ R3 , can be calculated as T(t) = Iα(t) − rcop (t) × Fgrf (t) − rcom (t) × mf oot g (12) where I is the moment inertia of the foot around the ankle axis, α(t) is the angular acceleration, rcop (t) is the position vector of the CoP, rcom (t) is the position vector of the center of mass of the foot, mf oot is the mass of the foot and g is the gravitational acceleration, as shown in Fig. 8. The origin of the foot reference frame in Fig. 6 is the rear end of the calcaneus (i.e., heel bone), and the origin vector with respect to the ankle joint position is represented as rcalcn ∈ R3 . The CoP vector, rcop (t), can be obtained from rcalcn and the location of the CoP calculated in (9) and (10), i.e. oot rcop (t) = rcalcn + rfcop (t) (13) xcop (t) f oot where rcop (t) = yˆcop (t) . 5.2 Scaled Body Parameters The anthropometric parameters of the human subject, such as I, rcom , mf oot , and rcalcn should be determined to calculate the ankle moment. Since it is difficult to accurately measure the required anthropometric parameters for each human subject, an approximated foot model, which is scaled from a reference foot model with the reference
Value [−0.0488, −0.0419, 0.00792]m [0.0638, −0.0148, 0.00514]m [0.0025, 0.0051, 0.0052]kgm2 1.56kg
parameters shown in Table 1, is utilized in this paper. The reference foot model used in this paper is identical to the human body model of OpenSim software developed by Delp et al. (2007) of which the body weight is mref = 75kg and the foot length is Lref = 0.275m. Among various scaling factors, the foot length ratio, Rl , and the body weight ratio, Rw , defined as L m and Rw = (14) Lref mref are utilized to obtain a scaled foot model. Based on these scaling factors, the required anthropometric parameters can be approximately obtained as Rl =
rcalcn = Rl rref calcn rcom = Rl rref com mf oot = Rw mref f oot (15) Icom = Rw Rl2 Iref com . The human subject in the experiments of this paper had the body weight of 60kg and the foot length of L = 0.26m. Since Icom is the moment of inertia of the foot around the axis of the center of mass, I around the ankle joint axis should be calculated by the parallel axis theorem. Table 2 and (15) show the parameters of a reference human body model and the associated scaling factors. By applying the estimated parameters to (12), the ankle joint moment can be calculated approximately. 6. EXPERIMENTAL RESULTS A subject with the body weight of 60kg performed four walking trials wearing the proposed Smart Shoes system. In order to validate the performance of the proposed system accurately, an infrared camera-based motion capture system, VICON, was utilized. The GRF applied on the foot and the associated ankle joint moment during a stance phase were measured by the proposed system and by VICON with the force plates, and the results are represented in Figs. 5 and 9. Table 3 shows the root-mean-squared
1406
Proceedings of the 20th IFAC World Congress Hyunjin Choi et al. / IFAC PapersOnLine 50-1 (2017) 1366–1371 Toulouse, France, July 9-14, 2017
Moment(Nm/kg)
3
Smart Shoes Vicon
2
Table 3. Root-mean-squared errors of trials Trial 1 2 3 4
1 0 -1 0
25
50
75
Fig. 9. Comparison of the ankle joint moments calculated by the proposed method and the VICON gait analysis result during a stance phase.
GRF(%BW)
100 Smart Shoes Force Plate
50
0 0.5
1
1.5
2
2.5
3
3.5
Time(s)
Moment(Nm/kg)
(a) 2 1 0 0.5
1
1.5
2
2.5
3
method were introduced. The GRFs measured by the proposed system were further processed to analyze gait patterns, to calculate the center of pressure on the foot, and to estimate the ankle joint moments. The processed data were validated from experimental results. Unlike a conventional motion capture system based on an optical motion capture system and force plates, the proposed system can be applied without limits in time and space.
REFERENCES
Smart Shoes Force Plate
0
Moment (Nm/kg) 0.29 0.39 0.29 0.41
The GRF measurement system and the gait analysis method can be applied to many other mobile robot systems or wearable devices, in particular ones that interact with humans. Such information provide necessary information for monitoring the human motion and intention. With additional sensors for measuring the joint angles, such as the knee and hip joint angles, the joint moments of the whole lower extremity can also be estimated.
25 0
GRF (%BW) 12.3 10.2 5.23 7.36
100
Stance(%)
75
1371
3.5
Time(s) (b)
Fig. 10. Experimental results of multiple strides; (a)GRF measurements and (b) ankle joint moment measurements. errors of the four trials. The average RMS errors for the four trials were 8.77 %BW and 0.35N m/kg for the GRF measurement and the ankle joint moment measurement, respectively. Figure 10(a) shows the GRFs measured by the proposed system and the force plates. Notice that an error between the two measurements is large between the two peaks of the GRFs, which represents the mid-stance phase where the whole foot contacts the ground at the same time. This may be because the ground reaction pressure of the midfoot cannot be measured by the four discrete sensing units in the proposed system. However, it should be noted that the proposed system measured the GRFs and ankle joint moments for all the strides continuously, while the force plates could measure the GRFs only when stepped by the subject. Therefore, Fig. 10 represents the benefits of the proposed system in gait analysis. 7. CONCLUSION In this paper, an insole-type GRF measurement system, called Smart Shoes, and its validation with a gold standard
Delp, S.L., Anderson, F.C., Arnold, A.S., Loan, P., Habib, A., John, C.T., Guendelman, E., and Thelen, D.G. (2007). Opensim: open-source software to create and analyze dynamic simulations of movement. Biomedical Engineering, IEEE Transactions on, 54(11), 1940–1950. Howell, A.M., Kobayashi, T., Hayes, H.A., Foreman, K.B., and Bamberg, S.J. (2013). Kinetic gait analysis using a low-cost insole. Biomedical Engineering, IEEE Transactions on, 60(12), 3284–3290. Jacobs, D.A. and Ferris, D.P. (2015). Estimation of ground reaction forces and ankle moment with multiple, lowcost sensors. Journal of neuroengineering and rehabilitation, 12(1), 1. Jung, Y., Jung, M., Lee, K., and Koo, S. (2014). Ground reaction force estimation using an insole-type pressure mat and joint kinematics during walking. Journal of biomechanics, 47(11), 2693–2699. Kim, J.C., Kim, K.S., and Kim, S. (2014). Wearable sensor system including optical 3-axis grf sensor for joint torque estimation in real-time gait analysis. In Advanced Intelligent Mechatronics (AIM), 2014 IEEE/ASME International Conference on, 112–117. IEEE. Kim, Y., Choi, H., Jung, K., and Kong, K. (2016). Estimation of foot pressure distribution based on an ergonomic kernel density function. Transactions of the Korean Society of Mechanical Engineers. Will be presented. Kong, K. and Tomizuka, M. (2009). A gait monitoring system based on air pressure sensors embedded in a shoe. Mechatronics, IEEE/ASME Transactions on, 14(3), 358–370. Zheng, E., Chen, B., Wang, X., Huang, Y., and Wang, Q. (2014). On the design of a wearable multi-sensor system for recognizing motion modes and sit-to-stand transition. International Journal of Advanced Robotic Systems, 11.
1407
ScienceDirect
PapersOnLine 50-1 (2017) 1366–1371 Ankle IFAC Joint Moment Estimation Based on Ankle Estimation Based Ankle Joint Joint Moment Moment Estimation Based on on Smart Shoes Ankle Joint Moment Estimation Based on Smart Shoes Smart Shoes Smart Shoes Hyunjin Choi ∗∗∗ Kanghyun Kim ∗∗∗ Pyeong-Gook Jung ∗∗∗
Hyunjin Choi Kim Pyeong-Gook Jung ∗∗∗ Hyunjin Kanghyun Kim Pyeong-Gook Jung ∗ ∗∗ ∗ Byeonghun Na∗∗∗∗Kanghyun Dong-wook Rha Kyungmo Jung Hyunjin Choi Choi Kanghyun Kim Pyeong-Gook Jung∗∗∗ ∗∗ ∗∗ ∗∗∗ Byeonghun Na Dong-wook Rha Kyungmo Jung ∗ Byeonghun Na Rha ∗ ∗ Dong-wook ∗ ∗∗ Kyungmo Jung ∗∗∗ Kyoungchul Kong Hyunjin Choi Kanghyun Kim Pyeong-Gook Jung Byeonghun Na Dong-wook Kyungmo Jung ∗ ∗ ∗ KyoungchulRha Kong Kyoungchul Kong ∗∗ ∗ Byeonghun Na ∗ Dong-wook Kyungmo Jung ∗∗∗ KyoungchulRha Kong ∗ ∗ Engineering, Sogang University, Seoul, Kyoungchul Kong ∗ Department of Mechanical ∗ of Mechanical Engineering, Sogang University, Seoul, Department of Mechanical Engineering, Sogang University, ∗ Department Korea (e-mail: [email protected], [email protected] Department of Mechanical Engineering, Sogang University, Seoul, Seoul, Korea (e-mail: [email protected], [email protected] [email protected], [email protected] ∗ Korea (e-mail: {pgjung, nbh87,Engineering, kckong}@sogang.ac.kr). Department of Mechanical Sogang University, Seoul, Korea (e-mail: [email protected], [email protected] {pgjung, nbh87, kckong}@sogang.ac.kr). kckong}@sogang.ac.kr). ∗∗ {pgjung, nbh87, Department and Research Institute [email protected] Rehabilitation Medicine, Korea (e-mail: [email protected], {pgjung, nbh87, kckong}@sogang.ac.kr). ∗∗ ∗∗ Department and Research Research Institute of Rehabilitation Rehabilitation Medicine, Department and Institute of Medicine, ∗∗ Severance Rehabilitation Hospital, Yonsei University College of {pgjung, nbh87, kckong}@sogang.ac.kr). Department and Research Institute of Rehabilitation Medicine, Severance Rehabilitation Hospital, Yonsei University College of Severance Rehabilitation Hospital, Yonsei University College of ∗∗ Medicine, Seoul, Korea (e-mail: [email protected]) Department and Research Institute of Rehabilitation Medicine, Severance Rehabilitation Hospital, Yonsei University College of Seoul, Korea (e-mail: [email protected]) ∗∗∗ Medicine, Medicine, Seoul, Korea (e-mail: [email protected]) Human Factors & Device Research Team, Hyundai Motor Severance Rehabilitation Hospital, Yonsei University College of Medicine, Seoul, Korea (e-mail: [email protected]) ∗∗∗ ∗∗∗ Human Factors & Device Research Team, Hyundai Motor Factors & Device Research Team, Hyundai Motor ∗∗∗ Human Company, Korea (e-mail: [email protected]) Medicine, Seoul, Korea (e-mail: [email protected]) Human Factors & Device Research Team, Hyundai Motor Company, Korea Korea (e-mail: (e-mail: [email protected]) [email protected]) Company, ∗∗∗ Human Factors & Device Research Team, Hyundai Motor Company, Korea (e-mail: [email protected]) Company, Korea (e-mail: [email protected]) Abstract: For analyses and diagnoses of a human gait, the precise measurement of ground Abstract: For analyses analyses and and diagnoses of a human human gait, the precise precise measurement of ground ground Abstract: For diagnoses of the measurement of reaction forces essential, because GRFgait, is the significant external in Abstract: For (GRFs) analysesis diagnoses of aa the human themost precise measurement of force ground reaction forces (GRFs) isand essential, because the GRFgait, is the most significant external force in reaction forces (GRFs) is essential, because the GRF is the most significant external force in the human body dynamics. The GRF is not only useful information by itself, but also utilized to Abstract: For analyses and diagnoses of a human gait, the precise measurement of ground reaction forces (GRFs) is essential, because the GRF is the most significant external force in the human body dynamics. The GRF is not only useful information by itself, but also utilized to the human body dynamics. The is not only useful information by but also obtain further useful information, such center pressure (CoP) and the joint moments. reaction forces (GRFs) is essential, because the GRFof is the most significant external force to in the human body dynamics. The GRF GRF is as notthe only useful information by itself, itself, but also utilized utilized to obtain further useful information, such as the center ofcapture pressure (CoP) and the the joint moments. obtain useful information, such as the center of pressure (CoP) and joint moments. For human thisfurther purpose, a force plate and anisoptical motion system have been considered as the body dynamics. The GRF not only useful information by itself, but also utilized to obtain further useful information, such as the center of pressure (CoP) and the joint moments. For this purpose, purpose,However, a force plate plate and an an optical motion capture system have limits. been considered considered as For this a and optical motion have been as a gold the force plate inconvenient due tosystem its spatial To moments. overcome obtain further useful information, as is the center pressure (CoP) andlimits. the For thisstandard. purpose, a force force plate andsuch an optical motionofcapture capture have beenjoint considered as a gold standard. However, theSmart force plate is inconvenient dueGRFs tosystem its by spatial To overcome overcome a gold standard. However, the force plate is inconvenient due to its spatial limits. To the limitation of force plates, Shoes that measure the four air-pressure sensors For thisstandard. purpose, a force plate and an optical motion capture have been considered as a gold theSmart force plate isthat inconvenient dueGRFs tosystem its by spatial To overcome the limitation of However, force plates, Shoes measure the four limits. air-pressure sensors the limitation of force plates, Smart Shoes that measure the GRFs by four air-pressure sensors in silicone insoles are used to measure the GRFs intothis paper. Then, To the overcome dynamic aembedded gold standard. However, the force plate is inconvenient due its spatial limits. the limitation of force plates, Smart Shoes that measure the GRFs by four air-pressure sensors embedded in silicone insoles are used used to measure the GRFs in this paper. paper. Then, the dynamic dynamic embedded in are to the GRFs in Then, the characteristics the insoles ankle areShoes modeled used to estimate the joint the limitation ofof force plates,joint Smart thatand measure the GRFs fourankle air-pressure sensors embedded in silicone silicone insoles are used to measure measure the GRFs in this thisby paper. Then, the moments dynamic characteristics of the ankle joint are modeled and used to estimate the ankle joint moments characteristics of the ankle joint are modeled and used to estimate the ankle joint moments from the Smart Shoes measurement. The performance oftothe proposed method isthe verified by embedded in silicone insoles are used to measure the GRFs in this paper. Then, dynamic characteristics of the ankle joint are modeled and used estimate the ankle joint moments from the Smart Smart Shoes measurement. The performance performance of of the the proposed proposed method method is is verified verified by from the Shoes measurement. The by experimental results with a human subject. characteristics of the ankle joint are modeled and used to estimate the ankle joint moments from the Smart Shoes measurement. The performance of the proposed method is verified by experimental results results with with aa human human subject. subject. experimental from the Smart Shoes measurement. The performance of the proposed method is verified by experimental results with a human subject. © 2017, IFAC (International Federation of Automatic Control) Hosting by Elsevier Ltd. All rights reserved. experimental results with a human subject. Keywords: Gait monitoring, mobile sensing system, ground reaction force, sensor, signal Keywords: Gait monitoring, mobile sensing system, ground reaction force, sensor, signal Keywords: Gait monitoring, monitoring, mobile mobile sensing sensing system, system, ground ground reaction reaction force, force, sensor, sensor, signal signal processing. Gait Keywords: processing. processing. Keywords: Gait monitoring, mobile sensing system, ground reaction force, sensor, signal processing. processing. 1. INTRODUCTION To develop a sensor-integrated shoe, Kim et al. (2014) used 1. INTRODUCTION To develop aa sensor-integrated et al.of(2014) used 1. To develop shoe, Kim et used triaxial coin-type load cells atshoe, the Kim outsole the shoe. 1. INTRODUCTION INTRODUCTION To develop a sensor-integrated sensor-integrated shoe, Kim et al. al.of(2014) (2014) used triaxial coin-type load cells at the outsole the shoe. triaxial coin-type load cells at the outsole of the shoe. Although the load cells guarantee the high precision in 1. INTRODUCTION To develop a sensor-integrated shoe, Kim et al. (2014) used coin-type load cells at the outsole of the shoe. The measurement of ground reaction forces (GRFs) triaxial Although the but loadthe cells guaranteerigidity the high high precision in The measurement of ground ground reaction forcesof (GRFs) (GRFs) Although the load cells guarantee the precision in The measurement of reaction forces measurement, mechanical of the load cells triaxial coin-type load cells at the outsole of the shoe. Although the load cells guarantee the high precision in plays a significant role in many applications biomeThe measurement of ground reaction forcesof (GRFs) measurement, but the mechanical rigidity of the load cells plays a significant role in many applications biomemeasurement, but the mechanical rigidity of the load cells plays a significant role in many applications of biomeoften cause discomfort and disturb the human motion. To Although the load cells guarantee the high precision in measurement, but the mechanical rigidity of the load cells chanics, healthcare mechatronics, robotics, rehabilitation The measurement of ground forces plays a significant role in manyreaction applications of (GRFs) biome- often cause and disturb the human motion. To chanics, healthcare mechatronics, robotics, rehabilitation cause discomfort and disturb the human motion. To chanics, healthcare mechatronics, address the discomfort rigidity issue, many rigidity researchers like Howell measurement, but the mechanical of the load cells often cause discomfort and disturb the human motion. To medicines, and so on. The foot isrobotics, the mostrehabilitation distal body often plays a significant role in many applications of biomechanics, healthcare mechatronics, robotics, rehabilitation address the rigidity issue, many researchers like Howell medicines, and so on. The The foot is the most most distal body address the rigidity issue, many researchers like Howell medicines, and so foot the distal al. cause (2013) and Zheng al. (2014) developed insoleoften andet disturb the human motion. To address the discomfort rigidity issue, many researchers like Howell segment that frequently interacts with ground in body daily et chanics, healthcare mechatronics, robotics, rehabilitation medicines, and so on. on. The foot is is thethe most distal body et al. (2013) and Zheng et al. (2014) developed insolesegment that frequently interacts with the ground in daily et al. (2013) and Zheng et al. (2014) developed insolesegment that frequently interacts with the ground in daily type GRF sensor systems with force resistive sensors address the rigidity issue, many researchers like Howell et al. (2013) and Zheng et al. (2014) developed insoleactivities, and thus the external forces applied to the medicines, and so on. The foot is the most distal body segment that frequently interacts with the ground in daily type GRF GRF sensor systems with only force for resistive sensors activities, andGRFs) thus the external forces forces applied to and the type systems with force resistive sensors activities, and thus external applied the butsensor most of them the detection et al. GRF (2013) and Zheng et were al. (2014) developed insoletype sensor systems with force for resistive sensors foot (i.e.,that the areinteracts important forthe the analyses segment ground into activities, andfrequently thus the the externalwith forces applied todaily the (FSRs), (FSRs), but most of them were only the detection foot (i.e., the GRFs) are important for the analyses and (FSRs), but most of them were only for the detection foot (i.e., the GRFs) are important for the analyses and of certain postures due to the poor accuracy of the FSR type GRF sensor systems with force resistive sensors (FSRs), but most of them were only for the detection diagnoses of dynamic characteristics of human motions. activities, and thus the external forces applied to the foot (i.e., the GRFs) are important for the analyses and of certain postures due to the poor accuracy of the FSR diagnoses of of dynamic dynamic characteristics characteristics of of human human motions. motions. of certain postures due to the poor accuracy of the diagnoses measurement. (FSRs), but most of them were only for the detection certain postures due to the poor accuracy of the FSR FSR foot (i.e.,plate thedynamic GRFs) are regarded importantasof forthe themost analyses and of diagnoses of characteristics human motions. measurement. A force has been accurate measurement. of certain postures due to the poor accuracy of thewhich FSR measurement. A force plate has been regarded as the most accurate diagnoses of dynamic characteristics of human motions. A force plate has been regarded as the most accurate Kong and Tomizuka (2009) developed Smart Shoes, means measure the GRFs; however, their price and Kong A forceto plate has been regarded as the their mostprice accurate and Tomizuka (2009) developed Smart Shoes, which measurement. means to measure the GRFs; however, and Kong and Tomizuka (2009) developed Smart Shoes, which means measure the GRFs; however, price and anTomizuka insole-type GRFdeveloped sensor system airKong and (2009) Smartbased Shoes,on which spatial limitations drawbacks. Moreover, A forceto plate has are been regarded as the their mostMoreover, accurate means to measure theunavoidable GRFs; however, their price and include include an insole-type GRF sensor sensor system based on airspatial limitations are unavoidable drawbacks. include an insole-type GRF system based on airspatial limitations are unavoidable drawbacks. Moreover, bladder sensors. The air-bladder sensor consists of an airKong and Tomizuka (2009) developed Smart Shoes, which include an insole-type GRF sensor system based on the force plate measures the interaction forces between means to measure theunavoidable GRFs; however, their price and bladder spatial limitations are drawbacks. Moreover, sensors. The air-bladder sensor consists of an airthe force plate measures the interaction forces between bladder sensors. The air-bladder sensor consists of an airthe force plate measures the interaction forces between sensor and silicone tube. Each of the Smart include an insole-type GRF sensor system based bladder sensors. The aaair-bladder sensor consists of on an airthe shoes and the ground, while the desired information spatial limitations are unavoidable drawbacks. Moreover, force plate measures the interaction forces between pressure pressure sensor silicone tube. Each of the Smart the shoes andinteraction the ground, while the desired information sensor and silicone tube. Each of the Smart the and the ground, while the desired information Shoes pair has and four aaair-bladder air-bladder sensors placed at the bladder sensors. The sensor consists of an pressure sensor and silicone tube. Each of the Smart is inshoes fact the forces between theforces shoes and the pressure force plate measures the interaction between the shoes and the ground, while the desired information Shoes pair has four four air-bladder sensors placed at airthe is in fact the interaction forces between the shoes and the Shoes pair has air-bladder sensors placed at is in fact the interaction forces between the shoes and the locations of major bones or joints that contact the ground. pressure sensor and a silicone tube. Each of the Smart pair has four air-bladder sensors placed at the the feet. The the force plate measurement is available only when the andinteraction the ground, while the desired information is inshoes fact forces between the shoes and the Shoes locations of major bones or joints that contact the ground. feet. The force plate measurement is available only when locations of major bones or joints that contact the ground. feet. The force plate measurement is available only when The Smart Shoes areair-bladder useful practice due Shoes pair has four sensors placed attheir the locations of major bones or jointsin that contact the to ground. a human subject steps on it, which disturbs the natural is in fact the interaction forces between the shoes and the feet. The force plate measurement is available only when The Smart Shoes are useful in practice due to their a human subject steps on it, it,aspects, which researches disturbs the the natural The Smart Shoes are useful in practice due to their afeet. human subject steps on which disturbs natural superior durability and wide sensing range. However, Kong locations of major bones or joints that contact the ground. The Smart Shoes are useful in practice due to their human gait motion. In these on sensorThe force plate measurement is available only when ahuman human subject steps on it,aspects, which researches disturbs the natural superior durability and wide range. However, Kong gait motion. In these these on sensordurability and wide sensing range. However, human gait motion. In on sensorand Tomizuka (2009) notsensing validate the performance of The Smart Shoes aredid useful in practice due to Kong their superior durability and wide sensing range. However, Kong shoes have by Jacobsthe andnatural Ferris superior aintegrated human subject steps on conducted it,aspects, which researches disturbs human gait motion. Inbeen these aspects, researches on sensorand Tomizuka (2009) did not validate the performance of integrated shoes have been conducted by Jacobs and Ferris and Tomizuka (2009) did not validate the performance of integrated shoes have been conducted by Jacobs and Ferris the Smart Shoes comparing with a gold standard method superior durability and wide sensing range. However, Kong and Tomizuka (2009) did not validate the performance of (2015), gait Jung et al. (2014), and so on. The sensor-integrated human motion. Inbeen these aspects, researches on integrated shoes have conducted by Jacobs andsensorFerris the the Smart Shoes comparing with a gold standard method (2015), Jung et al. (2014), and so on. The sensor-integrated Smart Shoes comparing with a gold standard method (2015), Jung et al. (2014), and so on. The sensor-integrated (i.e., a force plate and an optical motion capture system). and Tomizuka (2009) did not validate the performance of the Smart Shoes comparing with a gold standard method shoes are noteworthy, because they can be used without integrated shoes have been conducted by Jacobs and Ferris (i.e., aa force (2015), Jung et al. (2014), and sothey on. The sensor-integrated plate and an optical motion capture system). shoes are noteworthy, because can bereaction used without force plate and an optical motion capture system). shoes noteworthy, because they can be used without the Smart Shoes comparing with a gold standard method (i.e., a force plate and an optical motion capture system). limits are inJung time and space, and so measure the forces (i.e., (2015), et al. (2014), on. The sensor-integrated shoes are noteworthy, because they can be used without sensors available for the insole-type GRF sensor limits in time and and space, and measure the the reaction forces forces The limits in time space, and The sensors available for the insole-type insole-type GRF sensor (i.e., asensors force and anfor optical motionsensors, capture system). acting are actually on the feet. available the GRF sensor shoes noteworthy, because they canthebereaction used without limits in time and space, and measure measure reaction forces The systems (e.g.,plate the FSRs, the air-bladder etc.) show The sensors available for the insole-type GRF sensor acting actually on the feet. acting actually on the the feet. (e.g., the FSRs, the air-bladder sensors, etc.) show systems (e.g., the FSRs, the air-bladder sensors, etc.) show limits in time and space, and measure the reaction forces systems acting actually on feet. larger nonlinearity compared to the load cells, because The sensors available for the insole-type GRF sensor systems (e.g., the FSRs, the air-bladder sensors, etc.) show larger nonlinearity compared to the load cells, because larger nonlinearity compared to the load cells, because acting actually on the feet. of the sensing principles and the structural uncertainties. systems (e.g., the FSRs, the air-bladder sensors, etc.) show larger nonlinearity compared to the load cells, because This work was supported by Hyundai Motors Company. of the the sensing sensing principles principles and and the the structural structural uncertainties. uncertainties. This work was supported by Hyundai Motors Company. of work was supported by Hyundai Motors Company. larger nonlinearity compared to structural the load cells, because of the sensing principles and the uncertainties. This This work was supported by Hyundai Motors Company. of the sensing principles and the structural uncertainties. This work was supported by Hyundai Motors Company. Copyright © 2017 IFAC 1402 Copyright © 2017, 2017 IFAC IFAC 1402Hosting by Elsevier Ltd. All rights reserved. Copyright © 2017 1402 2405-8963 © IFAC (International Federation of Automatic Control) Copyright © 2017 IFAC 1402 Peer review under responsibility of International Federation of Automatic Control. Copyright © 2017 IFAC 1402 10.1016/j.ifacol.2017.08.234
Proceedings of the 20th IFAC World Congress Toulouse, France, July 9-14, 2017 Hyunjin Choi et al. / IFAC PapersOnLine 50-1 (2017) 1366–1371
1367
Fig. 1. A Smart Shoe system for motion capturing; (a) an insole-type GRF sensor system. The proposed force sensing unit (b) is constituted by (c) soft elastomers with an air chamber, (d) soft covers for sealing, and (e) a small-sized barometer to measure the pressure change. Various methods have been studied to compensate for the nonlinearity. For example, Jacobs and Ferris (2015) used a regression model that linearizes the measurement of their insole-type GRF sensor system. They validated the regression model with the GRF signals obtained during different motions, such as walking and calf raising. This method was useful, but it was limited to a subject-specific case and required an initial trial on a force plate. In fact, however, it is an unanswered question if the measurement of an insole-type GRF sensor system should match that of a force plate, because of the dynamic effects of the cushion material (i.e., the sole of a shoe). It is reasonable and convincing that the measurement of the insole-type GRF sensor system is more reliable than the force plate, as long as its performance is validated, because the force actually acting on the foot is the interaction force between the foot and the insole. In this paper, an improved version of the Smart Shoes is introduced, and it is utilized for analyzing the ankle joint moment, as shown in Figs. 1 and 2. The new Smart Shoes include four modularized air-bladder sensors, as shown in Fig. 1(a), which is similar to the Smart Shoes developed by Kong and Tomizuka (2009), but its manufacturing process and sensing performance have been improved. The gait phase detection algorithm proposed by Kong and Tomizuka (2009) is identically applied in this paper as well. Unlike the previous work, however, the GRF measurements are compared and calibrated with a gold standard method (i.e., a force plate), and thus the GRF sensing performance of the Smart Shoes is validated in this paper. Based on the GRF measurement, the position of the center of pressure (CoP) is calculated in the laboratory coordinate system. For accurate comparison of the GRF sensing performance with a force plate, the two sensing methods are used to measure the same physical quantity at the same time. In addition, in order to check if the Smart Shoes can replace the force plate in an optical motion capture system, the gait analysis performance with the Smart Shoes (in particular, the ankle joint moment analysis) is verified in this paper also. The performance of the proposed system is verified by experimental results with a human subject in this paper. The subject is asked to walk for several steps wearing the Smart Shoes and sixteen infrared markers in a laboratory setting. For accurate verification, the subject is asked to step on a force plate during walking. The graphs and
Fig. 2. Configuration of an experimental setting; (a) optical motion capture system and force plate in a laboratory, and (b) a human subject stepping on force plates with sixteen markers and the Smart Shoes. data shown in this paper are all the experimental results obtained with the human subject. 2. HARDWARE CONFIGURATION The proposed Smart Shoes system consists of four force sensing units and one inertial measurement unit (IMU) at each side. The force sensing units are made of casted silicone and have air chambers and small-sized barometer, which is the absolute pressure sensor manufactured by Measurement Specialities Inc., as shown in Fig 1. If a force is applied, the whole sensing unit is compressed and thus the air chamber is pressurized. This pressure change is measured by the absolute pressure sensor embedded in each sensing unit. As the sensor measures the absolute pressure, the sensing unit can be completely encapsulated, which greatly improves the practicality. The sensing performance, such as dynamic range, hysteresis, linearity, etc., is dependent on the shape of the air chamber and the material property. Through multiple trials and errors, an appropriate material was sought; the RTV silicone rubber KE-12 manufactured by ShinEtsu Chemical, which is a hyper-elastic material with the hardness of Shore A40 and the tensile strength of 2.5M P a, was utilized for fabricating the proposed force sensing unit. Each force sensing unit is calibrated with a load cell while measuring the same force in series, as shown in Fig. 3(a); a regression model with a second-order polynomial function is applied for the calibration. The parameters of the regression model were found by Matlab polyfit.m function; the obtained regression model is s = a2 m2 + a1 m + a0 , (1) where m is the raw measurement from the absolute pressure sensor, s is the calibrated signal, and the parameters are a2 = 2.8018 × 10−13 , a1 = 9.6250 × 10−6 , and a0 = 1.1960 × 10−1 , respectively. Figure 3(b) shows the linearity of force measurement by the proposed sensor unit. The linearity was calculated as 3.8 %FSO (full scale output). Both the left and right sides of Smart Shoes utilize the same force sensing units. In each side, four force sensing units are placed on the locations of the major bones and joints, such as, the hallux, the first-second and the
1403
Proceedings of the 20th IFAC World Congress Toulouse, France, July 9-14, 2017 1368 Hyunjin Choi et al. / IFAC PapersOnLine 50-1 (2017) 1366–1371
60
Proposed sensor(kgf)
Table 1. Parameters for GRF measurement Sensor(i) 1 2 3 4
40
20
0
0
20
40
60
Loadcell(kgf)
(a)
(b)
Fig. 3. Force calibration and linearity test of sensing units; (a) a test-bed for calibrating the proposed sensor unit with a load cell, and (b) the calibration result. fourth-fifth metatarsophalangeal joints, and the heel [see Fig. 1(a)]. 3. MEASUREMENT OF GROUND REACTION FORCES The proposed force sensing unit enables the Smart Shoes to measure the ground reaction pressure distributed over the foot area. Since the ground reaction pressure is mainly concentrated at the locations of major bones and joints, the ground reaction forces (GRFs) can be approximately measured by the four force sensing units placed at the locations of the major bones and joints, as shown in Fig. 1. In a strict sense, however, the four sensing units cannot measure the whole ground reaction pressure distributed over the foot area due to the limited size of the sensing units. For example, when the whole foot contacts the ground during a mid-stance phase, the ground reaction pressure is almost uniformly distributed over the foot area, which cannot be accurately measured by the sensors placed at the discrete locations. Therefore, an algorithm that estimates the ground reaction pressure from the GRFs measured by the proposed sensing units is introduced in this section. Since the GRFs are transferred to the body through the foot bones and joints contacting the ground, the peaks of the GRFs may occur at the locations of the bones and joints. In practice, however, the GRFs are distributed over a large area, because the sole of a foot is covered by soft skin tissues, and the sole of a shoe is made of soft cushion materials. Nevertheless, the centers of the concentrated pressures are not changed (i.e., the locations of the bones and joints, at which the force sensing units are placed). If the area of the ground reaction pressure distribution is smaller than the area of a force sensing unit, the sensor measurement is reliable. However, if the area of the pressure distribution is larger than the sensing area, which is most likely when the GRF is significantly large, the sensor reading is no longer reliable. In order to address this issue, a kernel density function, K(x, y), is applied, i.e. (x−xi )2 +(y−yi )2 − 1 2 2σ i e Ki (x, y) = , (2) 2πσi2
Position(xi , yi )[m] (-0.016, 0.244) (-0.021, 0.199) (0.027, 0.182) (0, 0.043)
λi 1.24 1.24 1.24 1.44
σi 15.84 15.84 15.84 19.94
where xi and yi represent the center location of ith force sensing unit, and σi is the standard deviation of pressure distribution, which is related to the size of the pressure distribution area. The values of xi ’s and yi ’s of the Smart Shoes used in the experiments in this paper are shown in Table 1. The index i = {1, ..., 4} represents the sensing units located at the hallux, the first-second and the fourthfifth metatarsophalangeal joints, and the heel, respectively. Notice that the kernel function in (2) is a Gaussian distribution function, the spatial integral of which overall all area is unitary. The use of such kernel functions were introduced in our previous work by Kim et al. (2016). Based on the kernel function, the ground reaction force at the ith sensing area, Fi , is estimated as si Ki (x, y)dxdy, (3) Fi = λi where λi is a scaling factor, and si is the measurement value calibrated by (1). Since si is scalar, (3) can be rewritten as Fi = λi si
Ki (x, y)da + ∆i ,
(4)
Di
where Di is the area of each sensing unit defined by Di = {(x, y)|(x − xi )2 + (y − yi )2 ≤ r}, (5) where r is the radius of a force sensing unit. The first part on the right-hand-side of (4) represents the force applied on the sensing area, while the second part represents the force applied outside of the sensing area. It should be noted again that the forces applied on the sensing area cannot represent the whole distribution of the ground reaction pressure, as shown in Fig. 4(a). The force applied on the sensing area is the measurement of the sensing unit, i.e. si = λi si Ki (x, y)da. (6) Di
Therefore, the scaling factor, λi , can be calculated as 1 λi = . (7) Ki da Di
The standard deviation, σi , in (2) is determined from the size of bones and joint contacting the ground. Applying si and λi to (4), the distribution of the ground reaction pressure at each sensing area can be partially recovered, as shown in Fig. 4(b). By the estimated ground reaction pressures, the total GRF can be calculated more accurately, i.e. 4 Ftotal = Fi , (8) i
where Fi is the foot pressure distribution calculated by (4).
1404
Proceedings of the 20th IFAC World Congress Toulouse, France, July 9-14, 2017 Hyunjin Choi et al. / IFAC PapersOnLine 50-1 (2017) 1366–1371
1369
(a)
(b)
Fig. 4. Kernel density estimation of foot pressure distribution; (a) pressure distribution on sensor areas, and (b) estimated pressure distribution of foot.
where Fi is the sensor measurement calibrated as in (4), and xi and yi are the locations of the sensing units shown in Table 1. Figure 6 shows the CoP calculated as in (9). The red circles represent the locations of sensing units, and the black dots represent the CoP during a gait cycle.
150
GRF(%BW)
125 100 75 50
Notice that the CoP calculated by (9) is limited in the range of sensor locations, i.e., xcop ∈ [mini (xi ), maxi (xi )] and ycop ∈ [mini (yi ), maxi (yi )], while the actual CoP can be placed anywhere on the foot area. This problem is also shown in the experimental result in Fig. 6.
Smart Shoes Force Plate
25 0 0
Ankle angle(rad)
Fig. 6. Center of pressure measured by Smart Shoes during stance phase. The red dots represent the position of center of each sensor units.
25
50
75
100
1 0.5 0 -0.5 0
25
50
75
100
Stance(%)
Fig. 5. Ground reaction force measurement and foot angle measurement during stance phase; black line represents force plate measurements, and gray line represents Smart Shoes measurements. To validate the GRF measurement of the proposed system, a subject with the body weight of 60kg performed a walking experiment wearing the Smart Shoes in a laboratory environment where a force plate manufactured by Advance Mechanical Technology, Inc. is installed, as shown in Fig. 2. Figure 5 shows the GRFs measured by the proposed method and the force plate. It should be noted that the two different systems measured very similar values throughout the gait cycle. The measurements shown in Fig. 5 cannot be directly compared, because the proposed Smart Shoes measured the interaction force between the foot and the insole, while the force plate measured the interaction force between the sole of a shoe and the floor. Nevertheless, the measurement error was as small as 5.23 %BW (body weight) in terms of root-mean-squares (RMS). 4. CENTER OF PRESSURE ESTIMATION The center of pressure (CoP) is important for analyzing the human body dynamics, because it is the location where all the ground reaction pressure is assumed to be concentrated. The CoP can be calculated from the GRFs as xcop =
ΣFi xi ΣFi
and
ycop =
ΣFi yi , ΣFi
(9)
The limitation in the range of CoP is compensated by utilizing the gait phase information. If the gait of a subject is normal and thus it shows certain phases, called gait phases, repetitively, the CoP trajectory of the current stride may be identical to that of the previous stride. Assuming that 1) the gait period, Tperiod , is constant (at least the same as the previous stride) and that 2) the CoP trajectory should start from the rear end of the shoe and end at the front end of the shoe, a CoP compensator, k(t), can be utilized as yˆcop (t) = ycop (t) + k(t)
(10)
where k(t) is defined as a function of the gait phases, such as the initial contact (IC) phase, the mid-stance (MS) phase, the terminal stance (TS) phase, and the swing (SW) phase, i.e. ycop (t) (t − TIC ), TIC k(t) = 0, L − ycop (t) (t − TT S ), Tperiod − TT S
for 0 ≤ t < TIC for TIC ≤ t < TT S (11) for TT S ≤ t < Tperiod ,
where t, TIC , TT S , and Tperiod are the current gait cycle time, the time when the IC phase ends, the time when the TS phase starts, and the gait cycle period, respectively. L is the foot length of the human subject. TIC and TT S can be detected by the gait phase detection method developed by Kong and Tomizuka (2009). Figure 7 shows the CoP of the anterior-posterior direction, i.e., ycop ; the CoP trajectories with and without a correction by the proposed method are shown in the figure. According to the gait phases detected as in Fig. 7(b), the CoP compensator, k(t), was calculated as shown in Fig. 7(c). The x-axis is scaled by the percent stance cycle for the sake of simple representation. It should be noted that the CoP trajectories calculated by the proposed method and the force plate match well.
1405
Proceedings of the 20th IFAC World Congress 1370 Hyunjin Choi et al. / IFAC PapersOnLine 50-1 (2017) 1366–1371 Toulouse, France, July 9-14, 2017
0.3
(a)
CoP y(m)
Smart Shoes Estimated Force Plate
0.1
(a)
(b)
Fig. 8. Foot reference coordinate system and the positions of the rear heel(rcalcn ) and center of mass(rcom ).
1
(b)
IC MS TS
0.5
Table 2. The parameters of a reference human body model
0
0.05
Gait phases
0.2
0
Reference Model rref calcn rref com Iref com mref f oot
(c)
0 -0.05
k(t)
0
25
50
75
100
Stance(%)
Fig. 7. (a) Estimated center of pressure, (b) gait phases, and (c) CoP compensator during stance phase. While the CoP of the anterior-posterior direction, i.e., ycop , should be corrected as in (10), it is not necessary to correct xcop , because it is within the range of sensor locations in general. 5. ANKLE MOMENT ESTIMATION 5.1 Moment Calculation Assuming that the foot is a rigid body and that it rotates around the ankle axis, the ankle joint moment, T ∈ R3 , can be calculated as T(t) = Iα(t) − rcop (t) × Fgrf (t) − rcom (t) × mf oot g (12) where I is the moment inertia of the foot around the ankle axis, α(t) is the angular acceleration, rcop (t) is the position vector of the CoP, rcom (t) is the position vector of the center of mass of the foot, mf oot is the mass of the foot and g is the gravitational acceleration, as shown in Fig. 8. The origin of the foot reference frame in Fig. 6 is the rear end of the calcaneus (i.e., heel bone), and the origin vector with respect to the ankle joint position is represented as rcalcn ∈ R3 . The CoP vector, rcop (t), can be obtained from rcalcn and the location of the CoP calculated in (9) and (10), i.e. oot rcop (t) = rcalcn + rfcop (t) (13) xcop (t) f oot where rcop (t) = yˆcop (t) . 5.2 Scaled Body Parameters The anthropometric parameters of the human subject, such as I, rcom , mf oot , and rcalcn should be determined to calculate the ankle moment. Since it is difficult to accurately measure the required anthropometric parameters for each human subject, an approximated foot model, which is scaled from a reference foot model with the reference
Value [−0.0488, −0.0419, 0.00792]m [0.0638, −0.0148, 0.00514]m [0.0025, 0.0051, 0.0052]kgm2 1.56kg
parameters shown in Table 1, is utilized in this paper. The reference foot model used in this paper is identical to the human body model of OpenSim software developed by Delp et al. (2007) of which the body weight is mref = 75kg and the foot length is Lref = 0.275m. Among various scaling factors, the foot length ratio, Rl , and the body weight ratio, Rw , defined as L m and Rw = (14) Lref mref are utilized to obtain a scaled foot model. Based on these scaling factors, the required anthropometric parameters can be approximately obtained as Rl =
rcalcn = Rl rref calcn rcom = Rl rref com mf oot = Rw mref f oot (15) Icom = Rw Rl2 Iref com . The human subject in the experiments of this paper had the body weight of 60kg and the foot length of L = 0.26m. Since Icom is the moment of inertia of the foot around the axis of the center of mass, I around the ankle joint axis should be calculated by the parallel axis theorem. Table 2 and (15) show the parameters of a reference human body model and the associated scaling factors. By applying the estimated parameters to (12), the ankle joint moment can be calculated approximately. 6. EXPERIMENTAL RESULTS A subject with the body weight of 60kg performed four walking trials wearing the proposed Smart Shoes system. In order to validate the performance of the proposed system accurately, an infrared camera-based motion capture system, VICON, was utilized. The GRF applied on the foot and the associated ankle joint moment during a stance phase were measured by the proposed system and by VICON with the force plates, and the results are represented in Figs. 5 and 9. Table 3 shows the root-mean-squared
1406
Proceedings of the 20th IFAC World Congress Hyunjin Choi et al. / IFAC PapersOnLine 50-1 (2017) 1366–1371 Toulouse, France, July 9-14, 2017
Moment(Nm/kg)
3
Smart Shoes Vicon
2
Table 3. Root-mean-squared errors of trials Trial 1 2 3 4
1 0 -1 0
25
50
75
Fig. 9. Comparison of the ankle joint moments calculated by the proposed method and the VICON gait analysis result during a stance phase.
GRF(%BW)
100 Smart Shoes Force Plate
50
0 0.5
1
1.5
2
2.5
3
3.5
Time(s)
Moment(Nm/kg)
(a) 2 1 0 0.5
1
1.5
2
2.5
3
method were introduced. The GRFs measured by the proposed system were further processed to analyze gait patterns, to calculate the center of pressure on the foot, and to estimate the ankle joint moments. The processed data were validated from experimental results. Unlike a conventional motion capture system based on an optical motion capture system and force plates, the proposed system can be applied without limits in time and space.
REFERENCES
Smart Shoes Force Plate
0
Moment (Nm/kg) 0.29 0.39 0.29 0.41
The GRF measurement system and the gait analysis method can be applied to many other mobile robot systems or wearable devices, in particular ones that interact with humans. Such information provide necessary information for monitoring the human motion and intention. With additional sensors for measuring the joint angles, such as the knee and hip joint angles, the joint moments of the whole lower extremity can also be estimated.
25 0
GRF (%BW) 12.3 10.2 5.23 7.36
100
Stance(%)
75
1371
3.5
Time(s) (b)
Fig. 10. Experimental results of multiple strides; (a)GRF measurements and (b) ankle joint moment measurements. errors of the four trials. The average RMS errors for the four trials were 8.77 %BW and 0.35N m/kg for the GRF measurement and the ankle joint moment measurement, respectively. Figure 10(a) shows the GRFs measured by the proposed system and the force plates. Notice that an error between the two measurements is large between the two peaks of the GRFs, which represents the mid-stance phase where the whole foot contacts the ground at the same time. This may be because the ground reaction pressure of the midfoot cannot be measured by the four discrete sensing units in the proposed system. However, it should be noted that the proposed system measured the GRFs and ankle joint moments for all the strides continuously, while the force plates could measure the GRFs only when stepped by the subject. Therefore, Fig. 10 represents the benefits of the proposed system in gait analysis. 7. CONCLUSION In this paper, an insole-type GRF measurement system, called Smart Shoes, and its validation with a gold standard
Delp, S.L., Anderson, F.C., Arnold, A.S., Loan, P., Habib, A., John, C.T., Guendelman, E., and Thelen, D.G. (2007). Opensim: open-source software to create and analyze dynamic simulations of movement. Biomedical Engineering, IEEE Transactions on, 54(11), 1940–1950. Howell, A.M., Kobayashi, T., Hayes, H.A., Foreman, K.B., and Bamberg, S.J. (2013). Kinetic gait analysis using a low-cost insole. Biomedical Engineering, IEEE Transactions on, 60(12), 3284–3290. Jacobs, D.A. and Ferris, D.P. (2015). Estimation of ground reaction forces and ankle moment with multiple, lowcost sensors. Journal of neuroengineering and rehabilitation, 12(1), 1. Jung, Y., Jung, M., Lee, K., and Koo, S. (2014). Ground reaction force estimation using an insole-type pressure mat and joint kinematics during walking. Journal of biomechanics, 47(11), 2693–2699. Kim, J.C., Kim, K.S., and Kim, S. (2014). Wearable sensor system including optical 3-axis grf sensor for joint torque estimation in real-time gait analysis. In Advanced Intelligent Mechatronics (AIM), 2014 IEEE/ASME International Conference on, 112–117. IEEE. Kim, Y., Choi, H., Jung, K., and Kong, K. (2016). Estimation of foot pressure distribution based on an ergonomic kernel density function. Transactions of the Korean Society of Mechanical Engineers. Will be presented. Kong, K. and Tomizuka, M. (2009). A gait monitoring system based on air pressure sensors embedded in a shoe. Mechatronics, IEEE/ASME Transactions on, 14(3), 358–370. Zheng, E., Chen, B., Wang, X., Huang, Y., and Wang, Q. (2014). On the design of a wearable multi-sensor system for recognizing motion modes and sit-to-stand transition. International Journal of Advanced Robotic Systems, 11.
1407