An automated motion measurement system for clinical gait analysis

.l BwnFchaurs VoL IS. No. Printed tn Great Bnlam.

7, pp. 505-516.

0021-9290’82fO70505-II 6 1982 Perpmon

1982.

AN AUTOMATED MOTION MEASUREMENT FOR CLINICAL GAIT ANALYSIS*

$0300!0 Press Ltd.

SYSTEM

KENNETH D. TAYLOR, FRANCOIS M. MOTTLER+DAN W. SIMMONSWILLIAMCOH~U, R.AYMOND PAVLAR, JR., DONALDP. CORNELLand G. BLAIR HANKINS

Instrumentation

Laboratory and Computer Center, United Technologies Research Center, East Hartford. CT 06108, U.S.A.

Abstract-An automated motion measurement system using a television camera interfaced to a computer was constructed and evaluated to determine its applicability to clinical gait analysis. Experimental data indicate that the motion measurement system has a resolution of 1 part in 2CKKl and a static accuracy of I part in I000 with a worst case dynamic error of I part in 300. The study has also shown that this system can reliably track multiple passive markers on a human illuminated with infrared light emitting diodes. This approach minimizes patient distraction, since the iliumination is not visible, and reduces patient discomfort since a marker telemetry back pack system, to control active markers, is not needed. Further, the motion measurement system has been successfully demonstrated under conditions similar to those expected in a clinical environment

6, X, Y, X Y’ X” r

NOMENCLATURE Horizontal length of a marker line. Last horizontal address of a marker line. Vertical address of a marker line. Horizontal marker centroid coordinate. Vertical marker centroid coordinate. Estimate of horizontal marker centroid coordinate. Estimate of vertical marker centroid coordinate.

INTRODUCTION In general, clinical gait analysis is performed by combining motion data with kinetic data, usually measured by force plates. and muscle activity data, obtained via the electromyogram (Simon et al., 1978 and Sutherland et al.. 1980). Acquiring kinetic and muscle activity data, in a clinical setting, can be accomplished without much difficulty, using commercially available equipment. In performing clinical gait analysis, the most difficult task is to acquire and analyse motion data. Motion data, or data describing the position of various body locations as a function of time, has been acquired using a variety of techniques. An excellent history of the methods used for motion data acquisition, dating from 384 B.C. to 1976, is presented by Jarrett (1976). The discussion presented here will be limited to automatic motion measurement systems used for gait analysis. In general, motion measurement systems can be categorized as either manual or automatic. Manual

* Received 1981.

1’7 March

1981;

in revised form

9 November

t Specifically the clinical gait laboratories located at the Children’s Hospital Medical Center, San Diego, CA 92123 and at the Children’s Hospital Medical Center, Boston, MA 02115, U.S.A. 505

systems require that motion data acquisition or analysis is manually done whereas automatic systems acquire and analyse motion data with minimal human intervention. For clinical assessment of gait disorders, either type of system was deemed to require an accuracy and resolution of 1 part in 500 at local velocities up to approximately 5 m/s to be suitable for gait assessment. The resolution specification of 1 part in 500 is necessary to achieve limb rotation errors of less than 0.5” assuming that the markers used to measure limb rotation are on approximately 1Ocm long wands, as is the current practice in some clinical gait laboratoriest. This specification is not applicable to systems which will be used for analytical studies involving rigid body kinematics and numerical differentiation. A number of manual motion measurement systems, using optical and mechanical measurement techniques have been developed. Since the motion measurement system described here is of the automatic type the reader is referred to the literature for discussions concerning manual motion measurement systems (Kasvand et al., 1972; Kettelkamp er al., 1970; Lamoreux, 1971; Simon et al., 1978; Sutherland and Hagy, 1972). Most of the automatic motion measurement systems employ an electro-optical sensor interfaced to a computer to record motion data and therefore eliminate manual data acquisition and analysis. The electro-optical sensors usually employed in these systems are either television cameras or lateral effect photodiodes. One of the first automatic motion measurement systems was developed by Winter et al. (1972). This system uses a television (TV) camera to observe markers on a subject and a video tape recorder (VTR) to record motion data. The video tape is then played back through a TV-computer interface. The area of a

306

KENNETHD. TAYLOR er al.

frame to be sampled is determined by positioning a 96 x 96 sampling matrix window over the appropriate portion of the image. In order to effectively use this coarse matrix, it is necessary that the camera field-ofview (FOV) only cover a small area. Consequently the operator must push the trolley mounted camera along the gait path so that the camera ‘tracks’ or follows the subject during the gait cycle to compensate for the narrow FOV. Although the system has a reported 1 mm resolution, the requirement of physical camera motion complicates the measurement process. Additionally, the use of a VTR, as an intermediate motion data storage element, extends the period between data acquisition and data analysis. Television systems that are directly interfaced to a computer have also been reported. The system described by Cheng (1974) uses an on-line minicomputer to obtain marker location data from a TV camera with horizontal and vertical resolution of approximately 1 part in 240. In this system, when a marker is detected in the TV camera video signal, the system stores the horizontal and vertical coordinates in a memory and inhibits further coordinate data acquisition, on that TV line. Marker location data are then transferred to the minicomputer memory under program control. This marker detector method restricts the system to detecting only one marker per line which limits the utility of the system in cases where different markers appear at equal height, such as in observing lateral view toe and ankle markers. Also the resolution is somewhat lower than the desired 1 part in 500. Another television system described by Jarrett (1976) is also directly interfaced to a computer but the horizontal and vertical resolution is increased to approximately 1 part in 300 and the restriction of one marker per TV line removed. Although this is a multiple view system yielding three-dimensional motion data whereas the previous two systems are twodimensional motion data systems, the resolution is still lower than desired. A modified version of this system is currently being marketed as the VICON system.? Systems using lateral effect photodiodes as described by Woltring (1974) and the modified SELSPOTS system, described by Andriacchi er al. (1979) and Woltring and Marsolais (1980), have the advantages of not requiring intermediate storage or image sensor manipulation while having the desired resolution capability and being able to measure threedimensional movement. In these systems, the motion sensor is a large area position sensing photodiode (PSD). The PSD produces four electrical signals

t Oxford Medical Systems, Nuffield Way, Abingdon, Oxon. OX14 lBZ, U.K. $ Selective Electronic Company, S-43325 Partille, Sweden. EjCODA-3 is manufactured by Movement Techniques Ltd. 17 South Street, Barrow-upon-Soar, Leiastershire LE12 LILY.U.K. 1)This laboratory will be located at Newington Children’s Hospital, Newington, CT 06111, U.S.A.

corresponding to the two-dimensional average position of a light spot imaged on the photodiode surface. Small light emitting diodes (LEDs) are used as markers and in order to monitor the position of more than 1 marker, time division multiplexing is used resulting in a sampling rate of 312.5 Hz for 30 markers. Although these systems have quoted resolutions of 1 part in 500 to 1 part in 1000, they have limitations which strongly impact their use in a clinical situation. Specifically, these systems use LEDs as markers which need either a telemetry power pack or umbilical cord to control their activation and provide electrical power and therefore are an encumbrance to the subject. Further, the accuracy of these systems can be reduced by ghost images (i.e. reflections) from surfaces adjacent to the gait path due to the weighted average response of the PSD. An improved SELSPOT system known as SELSPOT-2 has recently been announced with a claimed resolution of 1 part in 4000 and a claimed error of less than 1 part in 200. This system apparently minimizes the effects of ghost images though it still requires active (i.e.. LED) markers and therefore presents an encumbrance to the subject. Another opto-electronic motion measurement system, called CODA (Cartesian Optoelectronic Dynamic Anthropometer), has been developed by Mitchelson (1975). This system uses an array of photodiodes masked by an optical encoder plate. The photodiode array senses LED markers and produces signals proportional to the three-dimensional marker coordinates. This system only has a resolution of the order of 1 part in 200. It shares the same disadvantages of the lateral effect photodiode systems. An improved version of this system, CODA-3, has been recently announced; it uses passive retroreflective markers and has a claimed resolution of 1 part in 4000 (longitudinal horizontal axis) to 1 part in 8000 (vertical and transverse horizontal axes).# Further, the system has a high sampling rate (600 Hz per marker) though apparently only eight markers may be used. Other information on the system operation has not been published to the authors’ knowledge, though the system appears to be promising. The system described herein uses advanced direct video measurement technology which has evolved over the past several years as a consequence of advances in the field of industrial automation. This motion measurement system is being developed for clinical use as part of a gait analysis laboratory. // In use, a subject is fitted with passive retro-reflecting markers and marker motion is observed by three television cameras while the subject traverses the gait path shown in Fig. 1. Light emitted by infrared LEDs placed near the TV cameras is reflected by the markers and is detected by the cameras. The positions of detected markers are transferred from the cameras to the computer through three specially designed interfaces. The computer then tracks the position of each marker during the gait cycle and provides graphical data on the displacement of

An automated motion measurement system for clinical gait analysis

507

CAMERA

NO 2 l-l

_~=

-\-\

_

_

_

-I_- -I===----0 CAMERA NO. 3

I

/

:

,I

CAMERA NO 1

Fig. 1. Three channel motion measurement system.

TER MONITOR

1 85m

LABORATORY LED RING

t------------I

I

SWITCH TERMINAL

L---_--__-__--A

Fig. 2. Single channel experimental motion measurement system block diagram. The MST (multi-wire serial terminal) link is used to transmit data to the computer.

important anatomical locations and the rotation of various limb segments. Note that this system, in its final configuration, is designed to be a three-dimensional (3-D), threecamera motion measurement system. Motion information will be obtained using data from the three cameras shown in Fig. 1 and analyzed using standard photogrammetric techniques to produce 3-D data. The automated motion measurement system involved the use of new technology in achieving the 1 part in 500 resolution and the 0.2% error desired. To evaluate the motion measurement concept, a single channel twodimensional motion measurement system, shown in Fig. 2, was constructed and extensively tested as described in the following sections.

c Hamamatsu Systems Inc., Waltham, MA 02154, U.S.A.

SYSTEM DESCRIPTION AND OPERATION

The single channel motion measurement system uses a TV camera to observe a set of markers on the subject. The output of the camera contains marker location data which are collected by the camera interface and stored for transmission to the computer. These data are transmitted to a remote Digital Equipment Corporation PDP 11/70 computer via a MST (multi-wire serial terminal) link where the software package fpr marker tracking and display graphics analyses and displays the motion data. The analyzed data is returned to the Gait Laboratory and displayed on a graphics terminal. The system contains three major components; (1) the image sensor or motion camera; (2) the camera memory and interface; and (3) special purpose software resident in the computer. The Hamamatsu C-1000 TV camera’, equipped with a silicon target vidicon was selected as the motion camera because of its high near infrared (i.r.) sensitivity

-508

KENNETH

D.

and low image lag. The camera is packaged in two assemblies: (1) the camera head which contains the vidicon, deflection circuit and video preamplifier ; and (2) the camera control unit which contains the power supply, scan generators and digital X and Y address generators. Due to this construction, the camera is well suited to interface with a computer and therefore is particularly appropriate for photogrammetric applications. A scan format of a 60 Hz field/frame rate with 256 noninterlaced lines per field was chosen for its compatability with standard high quality television monitors. This allows flicker free viewing of the image while setting up or performing measurements. The Hamamatsu camera was equipped with a 24 mm focal lengthf/2.8 35 mm format still camera lens to produce a 1.85 m square FOV, with negligible loss of image luminance in the entire FOV. Marker illumination was provided by an annular array of 24 infrared LEDs, mounted around the camera lens, which were activated by a pulse driver. The LEDs were mounted on a printed circuit board assembly which was mounted in a lens shade attached to the camera lens. General Electric F5D2 LEDs were used which radiate 9 mW of optical power output at 880 nm for 100 mA d.c. drive. These diodes are capable of much higher peak optical power when driven by a high current pulse. The LEDs were series connected in pairs, with each pair driven by a high current driver using a 2A peak current, 0.2 to 2.2 ms duration pulse. This pulse occurred at a 30 Hz or 60 Hz rate and was triggered by the vertical sync (vertical retrace) signal from the Hamamatsu camera so that the illumination pulses were synchronized to the camera. To enhance marker to background contrast, near i.r. illumination was used in conjunction with an optical

I--

CAMERA CONTROL UNIT X ADDRESS Y ADDRESS

I r------l . .._..r_

I

MAnnCn

I ;

; DETECTOR ! L-me--.

.-..,.,e

-

-

-

TAYLOR

et al.

filter. Consequently the camera spectral sensitivity is adjusted by suppressing ultraviolet and visible wavelengths. This is accomplished by using the system in an environment free of energy sources emitting near i.r. wavelengths (0.8-l.Opm) or more specifically in an area with ceiling mounted fluorescent lighting. Fluorescent room lighting emits broad band radiation extending from the ultraviolet, around 320 nm, up to the infrared, around 800 nm, with spectral peaks in the blue to orange wavelengths. The infrared LEDs used for illumination of the passive markers emit light at 880 nm with a spectral width of +_80 nm. A short wavelength cut-off filter, such as the Schott RG 830, can separate these two spectral bands effectively. Consequently, all motion measurement system experiments were conducted in a room illuminated using fluorescent light ceiling fixtures and with a Schott RG830 filter installed between the camera lens and the vidicon. All markers were constructed of high gain Scotchlight 7610 retro-reflective tape. Flat markers 17.5 mm square as well as 28 mm diameter balls covered with the retro-reflective tape were used depending upon the experiment conducted. The outputs of the motion camera are interfaced to the computer through the camera interface. A simplifiid block diagram of the camera interface is shown in Fig. 3. The high camera data rate prevents the use of direct memory access (DMA) operations to store motion data. Further, since the computer is remotely located, DMA operations would be prohibited due to data transmission considerations. The technique, utilized for detecting and acquiring marker data from the camera control unit requires three sequential stages of storage to accommodate the 25 MHz camera data

CAMERA INTERFACE AND MEMORY - -

-

-

-1

1 & 1

I i

I

I

1

CAMERA INTERFACE

--

I

MULTIPLE DATA LINES SINGLE DATA LINE

I

I

?

’

I

L~+KJdiiq FRONT PANEL CONTROLS AND INDICATORS

I--_---__--

---

--------

_

Fig. 3. Camera interface block diagram. FIFOs are first in-first out memories.

-

--

Fig. 4.

landed EXP

vie w of marker

image. (a) Before marker

509

detection.

(b) Afte :r

:ker

511

An automated motion measurement system for clinical gait analysis rate. Since video information

from the markers appears in bursts, a small fast first in-first out (fast FIFO) memory is used to store the data temporarily. Information is then transferred to a slower intermediate memory (slow FIFO) prior to being transferred to the main memory. Data are then transferred to the main memory for storage until the measurement cycle is complete. The acquired motion data are transferred to the MST via the MST interface for transmission to the computer. In operation, signals from the interface front panel trigger the interface and initiate the data acquisition process. This can be accomplished by a front panel switch or by a remotely located photoelectric switch such as one placed at the beginning of the gait path. When the data acquisition process is initiated, the memory interface waits for a frame synchronization signal from the camera control unit. Upon receipt of this signal a beginning of frame (BOF) word, identifying frame number one, is produced by the camera interface. The BOF word is sent through the high speed buffer (fast and slow FIFOs) and into the main memory. Once the frame synchronization signal has occurred, the marker detector can initiate marker data acquisition. The marker detector circuit consists of a discriminator that transforms the camera video signal into a binary video signal. This is performed by comparing the video signal with either a constant threshold level or with a variable threshold level that is derived from the preceding frame peak video level and is updated every frame. The marker detector can be switched between the constant threshold mode and adaptive mode by changing the position of a switch located on the front panel of the camera control unit. The amount of information contained in the binary video signal is too large to be stored effectively in current solid state memories of reasonable size. Therefore marker data are compressed prior to storage. Figure 4 shows expanded views of a marker image, as reproduced by the television camera before and after the marker detector. As is apparent in Fig. 4(b) the

RAW CAMERA DATA

CAMERA FILE II

marker appears as a succession of longer or shorter sections of ‘1’ level signals on adjacent television lines. The output of the marker detector, in the camera control unit, is examined by the video detector in the camera interface. When the presence of a marker is sensed, a marker size counter starts to count pixels. At the end of the marker, the camera interface develops an X word which denotes the marker width in terms of X address increments and the ending X address of the marker. The resultant X words for all markers on a line are then transmitted through the high speed buffer to the main memory. At the end of a line that contained a marker signal, the Y address is stored as a Y word. The camera memory can store 8 s of gait data at a 60Hz sampling rate assuming 10 markers in the motion camera FOV with each marker signature 16 pixels by 4 lines. A front panel switch allows data to be sampled at a 30 Hz rate in which case 16 s of data can be acquired for the same marker size and number of markers. The acquisition process continues in this manner, recording a BOF word to identify the current frame and recording X and Y words for marker data until a stop signal is received from one of the front panel controls. Upon acquiring all of the data, the camera interface writes and stores an end of transmission word (EOT) and switches from the acquisition mode to the transmit mode. The stored data is then transferred to the computer via the MST link using a ‘handshake’ data transmission procedure, under software control. After initializing a data file for the camera data, one data word at a time is read from the MST link into a buffer. When the buffer is full, it is written into a disk file. The data acquisition process continues in this fashion until an end of transmission (EOT) word is accessed. Thus, the disk file contains a permanent data base of the video information recorded during the gait cycle and the first step of the motion analysis data processing procedure is completed (Fig. 5). After acquiring the video data and building a data base, the next step in the software processing chain is to generate a disk file containing the centroid of each marker signature. The centroids are calculated and stored in the data file on a frame-by-frame basis for each view. It should be noted that the centroid calculation is not a true centroid calculation since it is not amplitude weighted. Figure 6 shows a magnified section of a frame of video data and a marker signature. In a single pass by the centroid algorithm of the video data, a marker signature is defined by associating the proper horizontal and vertical addresses. During the same pass, information is accumulated to calculate the centroid of the marker using the relations

I

X’ =

c (X, - 6,/2) 6, k

f3 CENTROID FILE

B MOTION FILE

Fig. 5. Motion analysis data processing procedure.

1 ;“I

(1)

KENNETHD. TAYLOR et al.

512 1600

t

1775 --

F 2 5

1750--

6

8K

8 <

8K+l

. XK

1725 --

YK ‘K+l

‘K+l

6K+2

‘Kt2

‘K+2

2 5

6K+3

‘Kt3

‘K+3

>

1700 --

1675

III, 1450

.#I, 1470

III1 1460 HORIZONTAL

I,,,

II,,

1490 COORDINATE,

1500

1 IO

X

Fig. 6. Magnified marker signature display.

where 6, is the length of each marker line, X, is the ending horizontal address of each marker line and Yi is the vertical address of each marker line, with X’ and Y’ being the horizontal and vertical marker centroid coordinates, respectively. A floating point calculation of the centroid is made, using equations (1) and (2), and a coordinate stretching transformation is applied before storing the integer coordinates in the centroid data base. The coordinate stretching transformation formats the centroid data so that one horizontal or vertical increment corresponds to approximately 1 mm in the camera FOV (assuming a 4.3 m camera to gait path separation). Consequently the centroid data base contains the calculated centroids on a frame-byframe basis. The centroid software also contains an algorithm to delete camera data caused by vidicon image lag or persistence. The final step in the generation of a two-dimensional motion file is the monitoring of marker movement via the use of the tracking algorithm. The objective of the tracking algorithm is to anatomically identify each centroid and to track its position from frame to frame. The identified centroids are then written in a motion file for each view from which three-dimensional coordinates will subsequently be calculated in the threecamera system. The input to the tracking algorithm is the centroid file, which contains a list of centroid point coordinates for each frame. The tracking algorithm must be ‘intelligent’, since the number of centroid points will vary from frame to frame due to points being obscured from the FOV or points appearing in the FOV but being assigned to another view. In addition, since the recording of gait information can be initiated and stopped by foot switches in the gait path, the subject may be partially in or out of the gait path during the data acquisition process, providing data from a partial set of markers and further complicating the tracking process. The first task in the tracking procedure is to search

the selected view in the centroid file for two consecutive frames that contain the same number of centroids, and have minimum centroid movement. The objective of this task is to locate two frames where the position of corresponding centroids is close, so that a two point extrapolation to estimate the position of a point in the next frame will be reasonably accurate. Another criterion used in this task is that the number of points in the frame must be greater than or equal to the number of markers expected to be tracked. This test effectively eliminates those frames in which the subject is not fully in view. On the other hand, satisfaction of these tests does not necessarily mean that all expected markers are visible. It is possible that an expected marker will be obscured and a marker assigned to another view will be visible. However, in general, the search procedure will select a frame that is usually the best starting point for tracking. When the frame which initiates tracking has been found, the centroids are displayed. At this point in the process, the operator, interactively, may either identify the centroids in the display or select a new frame for display. In order for the operator to make the anatomical identification, the next frame must contain the same number of centroids. As the operator identifies the centroids the location identification is echoed on the screen. After the centroids have been identified, a stick figure is drawn and the centroids are numbered. The operator may either verify the identification or change the identification code of a marker. If the operator verifies the identification, the next frame is displayed for verification. Following verification of two adjacent frames, automatic tracking is initiated. Tracking of the markers proceeds first forward through the centroid file, beginning with the two verified frames, until the end of the file. The remaining frames are tracked backwards through the file beginning at the same point as the forward pass. For each

An automated motion measurement system for clinical gait analysis identified marker in the initial frame, a two point, twodimensional linear extrapolation is calculated using the equations x:,,

= x:, + 2(x;+,

- x;)

(3)

r;,,

= Y:, + 2(Yb+, - Y:)

(4)

where X:, and Y; are the horizontal and vertical marker umtroid coordinates of frame n, X:,, and Y’,+, are the horizontal and vertical marker centroid coordinates of frame n + 1, and X:,, and Y:,, are estimates of the identified horizontal and vertical marker centroid coordinates (Wiley, 1951). Subsequent estimates of the new position are calculated using the tracked points from the three previous frames. A three-point, two-dimensional linear least squares approximation generates this extrapolation. In this case the estimates of the new centroid coordinates are calculated using the equations,

Xb+3 = a, + 4b,,

(5)

Yk+3 = a, + 4b,,

(6)

where Xb+ 3 and Y:+ 3 are estimates of the horizontal and vertical coordinates in the new frame with, a, = (-XA

+ Xf+2)/2,

(7)

a? = (- YA i- Yi+z)/2,

(8)

b, = (Xi + X:, 1 + Xi+,

- 6a,)/3,

(9)

and b, = (Y:, + Yn+ I + Y,“+* - 6aJ3.

(10)

Given the estimated coordinates of the tracked centroids for the next frame, an association window is automatically placed around each estimate. The centroids of the new frame are obtained from the centroid file and a nearest neighbor algorithm is employed to associate the centroids with the estimates. Therefore if two or more centroids lie within an association window, the nearest centroid is associated with the estimate. Alternatively, if no centroid lies within an association window, then it is assumed the tracked point has been obscured for this frame. If a centroid is close to two estimates, it is associated only with the nearest estimate. When the centroids have been associated with the estimates, a series of tests are made to determine if the centroids violate any physical constraints on movement based on their anatomical identification. The physical constraints are determined by the type of view, either frontal or lateral. For example, one physical constraint assures the proper orientation of the ankle. heel and toe markers. The tests to determine whether a physical constraint has been violated are performed by comparing the appropriate centroid coordinates. If the tests fail, then the nearest neighbor algorithm is employed again to associate a new centroid, if possible, with the estimate and the tests are repeated.

513

When a marker is obscured for several frames. it is necessary for the algorithm to track through the obscurity and locate the appropriate centroid when it reappears. The technique for determining the estimated centroid position, when the centroid in the previous frame was not located. is dependent on the view. From the front view, the horizontal and vertical coordinates from the last tracked centroid are used for the next estimated position. Holding the coordinates constant for the front view recognizes that the expected changes in the locations of each marker should be minimal. From the lateral view. the vertical coordinate from the last tracked centroid is used for the next estimated position. However, to estimate the horizontal coordinate of the centroid. a two point linear extrapolation is computed. The velocity obtained from the last two tracked points is used in this extrapolation calculation to estimate the horizontal position for subsequent frames. As the centroids are tracked from frame to frame, the identified coordinates are written into the twodimensional motion file. A key in the motion file contains the identified marker codes and relates the identifications to the proper centroid in each frame. The contents of the motion file can be displayed as a sequence of stick figures or as trajectory plots of the anatomical locations marked. In the final implementation of the system, the contents of each camera’s motion file would be used to produce 3-D data (via photogrammetric techniques). From the 3-D data base, other information such as hip, knee and pelvic rotation can be obtained.

EXPERIMENTAL EVALUATION

The single channel motion measurement system was evaluated using the experimental system shown in Fig. 2. The camera head was mounted on an aluminum pedestal. and secured to the floor with the camera lens centerline 93 cm above the floor and the camera head parallel to the floor. The motion camera was connected to the camera interface in the manner shown in Fig. 3. Additionally the camera binary video signal was displayed using a monitor. Camera data were transmitted from the camera interface, located in the laboratory, to the PDP 11/70in the Computer Center, using the MST link. Analysed data were displayed on a Tektronix 4014-l graphic display terminal with permanent graphic data output provided by a Tektronix 4631 hard copy unit. The camera control unit, the illuminator electronics, and the camera interface were all placed in a rack located adjacent to the camera head. A large number of tests were performed using this system with the most important of these tests being the accuracy and resolution tests and the demonstration of multiple marker tracking of human motion. The first test performed measured static accuracy or geometric stability. System geometric stability is of paramount

KENNETHD. TAYLOR er al.

\

TV CAMERA OPTICAL AXIS

Fig. 7. Rotating meter stick used in dynamic accuracy test.

since the system must perform spatial measurements to an accuracy of at least one part in 500. Significant geometric drift would require frequent calibration which is time consuming and would disrupt normal clinical operation. Therefore the repeatability of marker location data was measured over a period of two days to determine the geometric drift error. In this test, the camera was focused on a 4 x 5 matrix of 1 cm square markers located 4.5 m from the camera lens. One hundred frames of data were acquired, using an illumination pulse width of 750 ps and a constant level threshold for the marker detector. Data was taken at both the 30Hz and the 60 Hz sampling rate immediately after the system had been turned on. Data were acquired at 30 min intervals for 6 h, using the same data acquisition procedure. The test was repeated 24 hr later leaving the marker matrix in place. Marker centroid data were analyzed by examining the deviation of marker centroids over the two day period. This test was repeated three times over importance

a period of three weeks. These data indicate that the worst case errors occur prior to one hour after power turn on. After one hour, the camera errors are constant and apparently the system reaches thermal equilibrium. The results of these tests indicate that when the system has been operating for one hour, the worst case error is 1 part in 1000 or 0.1% error. regardless of the sampling rate. Furthermore, many markers have an error of no greater than 1 part in 2000 or 0.05% error. System dynamic accuracy was tested by using the rotating meterstick shown in Fig. 7. Dynamic errors were measured by observing the zero crossings of the yardstick marker trajectories using the software graphic display routines. Since the markers rotate in unison about the center marker (marker located on the motor shaft), each one moves up and down (or from side to side) in a sinusoidal fashion. These sinusoids should cross the vertical (or horizontal) axis at the same point. Any error in the dyamic location detection introduced by the motion measurement system will be evident in a display of vertical position or horizontal position versus frame number or time. This error occurs where the trajectories fail to cross at a common point. Figure 8 shows both the vertical coordinate versus frame number and an expanded view of one of the common crossing points. One vertical unit represents approximately 1 mm. The stick was rotating at 1.92 rev/s so that the markers located at lOcm, 20cm and 50 cm from the yardstick center moved at equivalent linear velocities of 1.2, 3 and 6 m/s, respectively. The illumination/sampling rate was 60 Hz. The results indicate that the error for the marker moving at 1.2 m/s is 5 mm in a 1.85 m square FOV for an error of 0.27% The marker moving at 3 m/s has an error of 1.5 mm or 0.08% and the marker moving at 6 m/s has negligible error. The pronounced error at 1.2 m/s is caused by image lag and the method used in the centroid calculation to eliminate lag-caused errors. As the marker image moves vertically across the scan lines,

VERTICAL COORDINATE, mm EXPANDED VIEW

400-

0

I

20 FRi&

40

NUMiER

60

31.5

32.0 32.5 FRAME NUMBER -

Fig. 8. Dynamic accuracy test results.

33.0

An automated motion measurement system for clinical gait analysis line segments caused by lag are deleted automatically, resulting in essentially error free centroid data. But, if the marker moves horizontally along scan lines, lag causes the marker image to elongate backward toward the location the marker occupied in the previous frame. This type of error is not compensated in the centroid program and produces the 5 mm error. System resolution was tested by placing the meterstick shown in Fig. 7 on a table in the center of the gait path. The ruler was moved incrementally in the horizontally direction and 100 frames of data were recorded, at both 30Hz and 60Hz sampling rates, after the system had reached thermal equilibrium (i.e., 1 h after the power was turned on). The centroid data was then examined and the test was repeated in the vertical direction. The results of these tests indicate that movement as small as 1 part in 2000 can be resolved. The last tests performed on the system involved monitoring of human motion using multiple markers. In these tests the subject was fitted with markers placed on the neck, shoulder, elbow, wrist, waist, knee, ankle, toe and heel of the left and right sides. Front markers were placed on the neck midline, left shoulder and right shoulder. This marker set was used since physicians seem to be primarily interested in hip, pelvic, knee and foot rotations and translations with a secondary interest in arm and torso motion. The elbow, wrist, waist, knee, ankle and toe markers were made of 28 mm diameter balls covered with Scotchlight 7610 tape that were attached to elastic bands and were visible both from the front and the side. The other markers were 17.5 mm square pieces of Scotchlight 7610. All of the human motion data discussed here was acquired at a 30Hz sampling rate with the marker detector in the constant threshold mode. Lateral view data was taken while the subject traversed the gait path. Frontal view data was taken by having the subject walk directly toward the camera. Lateral and frontal view data were obtained for five normal adults, one adult with gait abnormality, three normal children and three children with cerebral palsy. Figure 9 shows the left lateral view stick figures of a

513

normal adult female. Note that the tracking algorithm tracks these data while picking up the hip marker position after the arm no longer obscures the hip marker. The various phases of gait are evident in this sequence though the FOV is only adequate for one gait cycle. The slight back-kneeing evident on the stick figures is caused by knee marker motion relative to the knee. This will be eliminated by attaching the knee marker directly to the knee. Figure 10 is a left lateral view of a pre-teenage female with spastic dyplegia cerebral palsy. These data are also tracked well and the abnormality of the subject is most prominent in the motion of the arms and feet. Specifically due to the amount of arm flexion needed to maintain balance, the waist marker is rarely obscured. Also the child appears to spend a large amount of time in stance phase with minimal knee flexion in swing phase.

CONCLUSIONS

The experimental results indicate that the motion measurement system described is suitable for use in clinical gait analysis. Specifically the experiments have proven that a passive marker system may be used to obtain motion data automatically. This is important since a passive marker system obviates the need for a marker telemetry system or umbilical cord therefore minimizing gait disruption. This project also proved that near i.r. energy can be effectively used for marker illumination. Since this energy is invisible to the eye, the subject is not distracted. Both the use of passive markers and the use of near i.r. energy are supported in the multiple marker tracking results. Future work on the motion measurement system will concentrate on tracking multiple markers, using multiple views, and eliminating lag effects. Other minor improvements such as increasing the FOV and an improved marker set will also be made. Additionally the PDP 11/70 will be replaced with an on-site PDP 11/44 minicomputer. One of the advantages of this system is improved accuracy and resolution when compared to previously described television-based motion measurement sys-

Fig. 9. Tracker motion data from a normal adult-left lateral view.

516

KENNETH D. TAYLOR et al.

Fig. 10. Tracked motion data from a cerebral palsy child-left

terns. This improvement is achieved primarily by calculating the centroid of the marker location rather than using the beginning or the end of the marker location. This feature tends to minimize errors due to intensity variations. The requisite camera data acquisition technique requires additional hardware not present in other television based systems (e.g., the VICON system). However, the reader should be aware that the additional hardware required is primarily an additional off-line memory and appropriate control circuitry for data input/output to that memory. Other circuitry in Fig. 3, such as the SBC bus interface and MST link interface will not be used when the motion measurement system is connected to an on-site computer. In conclusion, the system has demonstrated the capability to acquire human motion data, from multiple markers, with a resolution of 1 part in 2000 and a static accuracy of 1 part in 1000. Additionally, the system has a dynamic accuracy of 1 part in 500 (0.2% error) within a velocity range of 0 to 6 m/s except at velocities between 1.0-1.5 m/s where the error increases to 0.27%. This increased error is due to lag effects and is reducible to a lower level by more advanced lag compensation techniques. The experimental results obtained demonstrate the suitability of this motion measurement system for gait analysis in a clinical environment.

Acknowledgements-The authors thank Mr George R. Wisner and Dr James P. Waters of the United Technologies Research Center for their advice in the design and construction of the motion measurement systems. The authors also thank Dr James R. Gage of Newington Children’s Hospital for his assistance in coordinating the human motion studies and for serving as medical advisor.

lateral view,

REFERENCES

Andriacchi, T. P., Hampton, S. J., Schultz, A. B. and Galante, J. 0. (1979) Threedimensional coordinate data processing in human motion analysis. J. biomech. Engng 101.279-283. Cheng, 1. (1974) Computer television analysis of biped locomotion. Ph.D. dissertation, Ohio State University, Columbus, Ohio. Gage, H. (1964) Accelerographic analysis of human gait. ASME Paper 64-Wa/HUF-8. Jarrett, M. 0. (1976) A television/computer system for human locomotion analysis. Ph.D. dissertation, University of Strathclyde, Glasgow, Scotland. Kasvand, T. and Milner, M. (1972) Pattern recognition applied to measurement of human limb positions during movement. J. Cybernet. 2,66-78. Kettelkamp, D. B., Johnson R. J., Smidt G. L., Chao E. Y. S. and Walker M. (1970) An electrogoniometric study of knee motion in normal gait. .I. Bone Jr Surg. 52, 775-790. Lamoreux, L. W. (1971) Kinematic measurements in the study of human walking. Bull. Prosth. Res. Spring, 10-15. Mitchelson, D. L. (1975) Recording of movement without photography. In Techniques for the Analysis of Human Mooement (Edited by Grieve, D. W., Miller, D. I., Mitchelson, D., Paul, J. P. and Smith, A. J.), pp. 59-65. Lepus Books, London. Simon, S. R., Deutsch, S. D., Nuzzo, R. M., Mansour, J. M., Jackson, L. F. and Rosenthal, R. K. (1978) Genu recurvatum in spastic cerebral palsy. J. Bone Jr Surg. 6OA, 882-894. Sutherland, D. H. and Hagy, J. L. (1972) Measurement ofgait movement from motion picture film. J. Bone Jt Surg. 54A, 787-797. Sutherland, D. H.. Olshen, R., Cooper, L. and Savio, L. Y. W. (1980) Development of mature gait. J. Bone Jr Surg. 62A, 336-353. Wiley, G. R. (1951) Adomced Engineering Mathematics, pp. 527-533. McGraw-Hill, New York. Winter, D. A., Greenlaw, R. K. and Hobson, D. A. (1972) Television computer analysis of kinematics of human gait. Compur. biomed. Res. 5, 498-504.

Woltring, H. J. (1974) New possibilities for human motion studies by real-time light spot position measurement. Biotelemerry 1, 132-146. Woltring. H. J. and Marsolais, E. 8. (1980) Optoelectric (Sclspot) gait measurement in two- and threedimensional space-a preliminary report. Bull. Prosth Res. 17.46-52.