Comparison of the performance of 3D camera systems II

Gait & Posture 5 (1997)

251-255

Technical note

Comparison of the performance of 3D camera s~s~s II Yoshihiro Ehara*a, Hiroshi Fujimotob, Shinji Miyazaki”, Masaaki Mo&~arub, Shigeru Tanakad, Sumiko Yamamotoe ‘Rehabiliiation h’nginerring Department, Kanagawa RehubihWion Institute, 514, Nanaaewa. Ate@, 243.01. Japan bNatiomd InMute of B&science ami Hlcman-~cchADJqFy.Tokyo, Jkpn %.Wute~or Medical am3 Lkntd B&neering, Tokyo h&died ad f&n& Vm’kersity9 T&p, Jqmtz dIntemational University of Health ad We&we, Tokp, Jm ‘Tokyo Metropolitan Prosthetic ami Orthotic Research Instiente, Tokyo, Japan Accepted

I5 April

1996

The acauraq and processing time of 11 commercially available 3D camera systems w test& to in clinic&gait &t&on. The systems tested were Ariel APAS, Dynas 3D&, Elite Plus, ExpertVision MAG, YICON 140, VKON 370, color Video Locus and reflective Video Locus, The 3D lc&oas a rigid bar weredetermined. The distance betweenthese markers wascalculated from Thea da&aand which was measured with a slide caliper prior to the measurement. For the esti4~ of noise, 30 txxwd@W of the markers weremeasured while the same rigid bar wasplaoed on the floor, and the standard deviations were calculated. The processing time for dating 30 coordinates from data obtained during normal gait was measured. Keywork

3D wra

systems; Comparison; Standardized condition; Gait measurements

1. In 1993, the accuracy and processing time of eight 3D camera systems were measured and the data were repmted in this journal [ 421. Since recent progress on these systems has bacn sipifmt in terms of both hardware and tiftwarc, these data must be updated. The purpose of this technical note is to update old data in order to cmpare and evaluate the basic performances of each system. The data will be useful to users who want to purchase a 3D camera system for clinical rehabilitation use. Beca~ the 3D camera system is utilized in the analysis and evaluation of a patient’s gait in a clinical situation, it is important to evaluate the performance of eachsystem from such a viewpoint. However, it is too dif%ult to deal with this kind of performance in a technical note. Here, we defined basic performances in terms

* Cocres@ag 492290.

at&or,

0966~6362/97/%17.00 PII SO966-6362(96)01093-4

Tel.:

+81

0 1997 Elsevier

462

492594;

Science

Fax:

+81

B.V. Ai1 rights

462

reserved

of the accuracy of data and user ftiendiiness in obtaining data. Two accuracy tests were pcrfonntd. First, the relative distance between two markers on a rigid bar was calculated and compared with t&e true valse while moving the bar in the field of view. F&T&, the noise of marker data was measured. The same bax was pl floor and the standard deviatMn of the cmdinates of each marker was calcuWd. ‘I%$ test was ef%ctive because noise in data could be caztc&iod out in the ca4culation of the distance between two:I8lukets wh*m the noise of the markers was syachroniml. Accuracy and noise the final 3D data, not estimations were performed lasers twe CQncerned intermediate data, because only with the final data. All data were obtaimred in an environment which simulated real gait laboratory conditions. User friendliness entails my aspects. However, since it is impossible to consi&r ai.l the& aslpects, we considered the processing time raqtired to obtain final results. The&-ore, the time req&ed fbp c&&&g 3D coordinates and for displaying 3D s&k Fssgureson the monitor after gait measurement was determined.

Y. Ehara

252

et al. I Gait & Posture 5 (1997) 251-255

2. Methods 2.1. Systems tested

The sysmnstestedwere Ariel APAS (Ariel Dynamic, Inc.), Dynas 3D/h (Shin Osaka Syokai), Elite Plus (BTS s.r.l.), ExpertVision (Motion Analysis Corp.), PEAK5 (Peak Performance Technologies, Inc.), PRIMAS (DELFT Motion Analysis), Quick MAG (Ohyou Keisoku Kenkyusyo),VICON 140,VICON 370(Oxford Metrics Ltd.), color Video Locus, and reflective Video Locus (Anima Co. Ltd.). 2.2. Procedures for testing measurement accuracy The number of cameras ranged from two to six depending on the 3D camera system.The spaceavailable for each systemwas 10 x 6 m including the gait walkway. Adhesive tape was stuck onto the floor, as shown in Fig. 1, to indicate the direction of movement (long tape) and the positions (short tapes) where a subject was instructed to step. For each system,a l-m-lung black aluminum bar with a matte finish, on which a marker was attached at each end, was prepared - the approximate distance between markers being 900 mm. The exactdistance between two markers was measured again with a slide caliper prior to the measurement. First, the measurementaccuracyin the vertical dire0 tion was determined. A subject held the aluminum bar so that his forearm was parallei to his torso, as shown in Fig. 2a. The upper end of the bar was approximately level with his head. The subject stood straight at ‘position 0’ asshown in Fig. 3. He then started walking, step-

30

‘la

Fig. 1. Walkway and tape markers on the floor.

ping sequentiallyon tape no’s. 1,2,3 and 4. Finat&, ,he stopped at tape no. 5, maintaining an uprig& rrtance. The samesubject performed this task for all systems. Data were obtained during this period. Measurement lasted for 5 s and the sampling frequency was from 50-100 Hz depending on the system. Next, the measurementaccuracy in the direction of motion was determined. The subject walked while holding the bar parallel to the sag&al and horizontal planes (Fig. 2b), but moving it in the vertical direction betweenthe middle of his thigh and - 500mm above the thigh. The subject was instructed to step sequentially only on tape no’s. 1,2 and 3 so that no marker was outsidethe measurementarea. Measurementtime and sampling frequency were the sameas those in the previous test. Finally, the measurement accuracy in the mediolateral direction was determined. The subject walked while holding the bar in front of him, keeping it parallel to the coronal and horizontal planes(Fig. 2c), but moving it in the vertical direction between 150 mm above and below the middle of his thigh. For some systems, tools were used to prevent the markers from being hidden by the subject’s body. The subject walked in the same manner as described above. Measurement time and sampling fquency were the same as those in the previous tests. 2.3. Evaluation of noise

The samebar was placed on the floor, except in the caseof ExpertVision for which the bar was placed - 200 mm above the floor. The bar was placed in the middle of the measurementareaso that its longitudinal axiswas in the direction of motion. Three hundred setsof data, which correspondedto a duration of 6 swith 50Hz sampling frequency, were obtained. W&n necessary,the floor was covered with black cloth to preveni rellcctions contributing to noise. With the systemwhich used the manual tracking method, positions of static markers in every frame could be determined without re&acking the position after the operator tracked the positions in the first frame. However, we cannot evaluate noise with those data. Hence, the markers were tracked in every frame. 2.4. Data processing procedures After the measurements,the data obtained were converted to DIFF [3] (Data Interface File Format proposed by the Clinical Gait Analysis Forum of Japan). No data train was filtered or averaged. We used our own computer program to calculate the foIlowing values for all systems. Data for accuracy. Mean value of absolute error = maximum value of (mean value of &so&e errors in distancebetween the markers) fro& ail tfials (1) Positive maximum error = maximum value of (distance

Y. Ehara

(a)

et al. I Gait

& Posrure

5 (1997)

251

251-255

(b)

Cc)

Fig. 2. Subject holding the bar with markers for accuracy measurement.

betweentwo markers - true value) from all samplesof all trials for each system(2) Negdive maximum error = minimum value of (distance true value) from all samplesof all trials for e& qF.em (3) ima Nihse = maximum S.D. of the data No&e es for two RI&WX for al three coordinates (4)

their placementwere the sameas those in the meaaurement of accuracy.Markers were atta&ed to the right side of a healthy subject at the shoukler, the major

Spetd.HeMfrO

shown in Fig. 1 and 2.5. Measurementof data processing time The experimental site, the number of cameras and

VTRs, the time reqded for ~M&&gg the tape was excluded. 3. R The results are shown in Table 1. 4.

in this wok. In whereasf&rward and considered in the n

Fig. 3. Sequence of steps.

be worse than the previous ones,but FBlB to be closer to the actuaJmeasumd values. Noise evaluation was ah added to the new protocol.

254

Y. Ehara

er al. / Gait & Posrure 5 (1997) 251-255

Table 1 Performance of 3D camera systems System

Ariel APAS Dynas 3Dih ELITE PLUS ExpertVision

Marker

Reflective absorption Reflective absorption Retlective Reflective

Marker ID process

Mean abs. error (mm)

Max error (mm)

Fi)

Processing time

11.61

13.41/-24.01

5.36

28

min

60

Auto manual Manual

18.42

27.20/-X48

0.24

16

min

CCD CCD

50 60

Auto Auto

0.53 1.14

1.19/-0.96

0.31

2.8Ob9.42

0.53

41 s 26 s

Auto manual Auto Real time

3.85

8.10/-10.39

2.04

1.79

2.25

4.09/-6.14 8.9Ok5.68

0.14 0.52

4.91/-5.96

1.82

No. of cameras

Input device

Sampling frequency (Hz)

2

VTR

60

2

VTR

4 4

7min

2

VTR

60

PRIMAS Quick MAG

Reflective absorption Reflective Color

2 2

100 60

VICON 140

Reflective

4

CCD Color CCD CCD

60

Auto

1.60

VICON 370 Video Locus

Reflective Color

6 2

60 60

Auto Auto

7.63

4.311-8.57 24.39/- 17.42

0.39 2.81

15 s 21 s

Video Locus

Rethtive

2

CCD Color CCD CCD

60

Auto

7.73

28.231-7.19

1.45

31 s

PEAK5

A systemwith low n&se generally seemsto exhibit better performance. In 3D camera systems,however, the non-uniformity of the data from the cameraswhich have poor resolution can be small beaHlsethey casmotdetect small perturbations. Therefore, noise should be consideredonly as addi6ional infwon to accuraq data. It is debatable whether or not we should have used a digital falter before the caklatkm of the error. However, we did not use any fxlter beeawe we consider the situation without f&em to be mwe &m&mental, and it is the option of each userwhether or not to &ilize a filter in real measurementsituati. When v&x&y an&k acceleration of markers is catcoastsd,however, fitters are indispensablein current 3D camera systems. For processingtime evaluation, we measured the gait of a healthy subject,becausethe measurementis essential in the rehabilitation field. Moreover, measuring the gait of healthy subjectsis one of the easiestmeasurementsfor camera systems.It was possibleto compare all systemsusing the sameprotocol. As a result, readerscan get a general idea of the relative performance of each system.However, if objects of mcasumment other than gait, such as movements in sports afztivitie dancing eating and outdoor activities were selected,the result might be different. In the erra of evalurrtiag such mwemen& it should be oh&& &at the system@anprovide accuratedata, and the test paclrlooalshoti be bytha~~thenreelv~Ourrawlsswili~~intbe dection of an appwptite camera system.

This researchwas cunductexlas a project of the Clini-

0.94

44 s IO s

1 min

calGaitAnal~~ForumofJapan.Allda%ainthisnote were obtained at the I Meeting ‘95 held in author thank all the vetwlws and trading wmpanies for their devotion and -ration in organizing the meeting.

Ariel Dynamics, Inc., Ariel Center, 6 Alicate Trabuco Canyon, CA, 92679, U.S.A. Tel.: +01 714 858 4216; Fax: +01 714 858 5022. Shin Oosaka Shoukai, l-6-24 Tamatsukuri, Chuou-ku, Oosaka540, Japan.Tel.: +816 7627453; Fax: +81 6 762 5822.

BTS s.r.l., Via Cristoforo Colombo lA, 20094 Corsico(MI)-Italy. Tel.: +39 2 458751. E-mail:bts@ bts.it Motion Analysis Corp., 3tN7 Westwind Boulevard, Santa Rosa, CA, 95403, USA. Tel.: +01 707 579 6HLo; Fax.: +01 707 526 0629. Peak Performance T , Inc., 7388 S. Revere , colorado 80112, Parkway, Suite 601, U.S.A. Tel.: +01 303 799 86f&; Fax: +Ol 303 799 8690. Delft Motion Analysis,Oude De@ 175 2611 HI3 Delft, The Nethe&&, Tel.: +31 15 78=, Fax: +31 13 786081.

Y. Ehura

et al. I Gait

Oh-you Keisoku Kenkyusho, 3-26- 12 Kitasenzoku, Ohta-ku, Tdtyo 145, Japan. Tel.: +81 3 3726 6776; Fax: +81 3 3727 6299.

& Posture

R@A [l]

Clinical Gait Analysis Forum of 3qw1. Current Srarns of 30 Camera

Oxford Metrics Ltd., Unit 14, 7 West Way Oxford OX2 OJB, United Kingdom. Tel.: +44 (0)865 244656; Fax: + 44 (0)865 240527. Anima, Co. Ltd., Naruko Bldg. 8-4-l Nishi-shinjuku, Shinjuku-ku, Tokyo 160, Japan. Tel.: +81 3 5330 5581; Fax: +81 3 5330 5582.

255

S /1997125/-2.iS

Systems

for

Clinical

Gait

Measurement

I993

version.

1994. [2]

Ehara Y, Fujimoto H, Miya&i S, Tawka S, Yamamoto S. Comparison of the performance of 3D camera systems.Gait and

[3]

Clinical Gait Analysis Forum of Japan.

Posfure maf

[4]

1995;

(DIFFj

3: 166-169. User%

Mamd

Data

hrerfuce

File

For-

1992.

Clinical Gait Analysis FoNm of Japan. Current Stafus of 3f) Camera 1995.

Systems

/or

Clinic&

Gait

Measwement

1395

version.