Motion measurement with high-speed video

Motion measurement S. Holzreiter,

with high-speed

video

J. Kastner and P. Wagner

Rehabilitationszentrum ‘Weisser Hof’, Klosterneuburg, Austria

Labor fir

Ganganalyse,

Postfach 36, A-3400

Received June 1990, accepted July 1992

A new kinematic measurementastern based on a high-speedvideo system, combinedwith a computer-assistedevaluation for the analysis ofgaitpatterns, is described.The system allows both a reviewable visual assessmentin slow motion (up to l@_Wfiamess- as well as automatic measurementof the kinematics of body segmenti. Specially developedsofiware, which llsw a pattern search algorithm and an additional subptiel correction,results in a deviation of hss than 0.1% (without consideringthe lens nonlitwarity). For most casesthe recognitionand trackingof temporarily concealedmarkers is also achieved. Pie results of the computer-a&ted high-speed video analysis are being applied in rehabilitation programmes to increasethe objectivity of standard movements, e.g. gait analysti ofpeople with ar+ial limbs. Keywords:

Gait analysis, biomechanics,

kinematic

measurement,

INTRODUCTION With the rapid development of electronic technology, motion analysis has become more and more accessible for clinical practice. One central part of motion analysis is the measurement of the spatial movement of marked points on the e erimentee. This is called kinematic measurement. ? or a long time the only oint-bymethod was a manual, frame-by-frame, point analysis of high-speed tine film’. To 1 a there exist many customary electronic systems worI in in real time or at least much faster than manual tine ! llm analysis2T3. Cine film analysis seems to be out of date but each of the available electronic systems has at least one of the following disadvantages: Video-based systems have a maximum recording frequency of 50 or 60 frames s-l. Most of these only observe the location of the marker points and do not simultaneously record a video image of the subject. There are no systems producing a slow-motion film with the view of the measurement camera as does film analysis. Systems with active infra-red markers require ‘wiring’ of the experimentee and often have trouble with reflections of the infra-red signal off the walls or floor.

COMPUTER ASSISTED ANALYSIS OF HIGH-SI’FXD VIDEO The system presented is based on a high-speed video recorder (Kodak EKTAPRO 1000) with a recording frequency of up to 1000 frames per second and a resolution of 239 X 192 pixels. We have developed a subpixel correction algorithm resulting in high resolution and precision. Correspondence

and reprint requests to: S. Holzreiter

01993 Butterworth-Heinemann 0141-5425/93/02140-03 140

J. Biomed.

for BES

Eng. 1993, Vol. 15, March

high-speed

video, pattern recognition

For recording, circular markers made of prismreflex foil are attached to the experimentee and illuminated by a reflector in the immediate roximity of the camera. The aperture can be aBjusted as in regular recordin : therefore the observation of the video is also visuaBs ly possible. The recording room needs no special arrangements, but objects which look similar to the markers should be covered if they are very close to the paths of the markers. The analysis of the films is done by a computer program running on a Micro-VAX minicomputer. The first frame of the sequence to be analysed is resented on the computer screen and the marker Pocations are indicated manually. The grey-level patterns defined in this way (see Appendix [2]) are searched automatically in the subsequent frames b a pattern search algorithm (Fz’ re 7). This algori x m uses a least-squares method $see A pendix [I]). The location of the marker can normal Py be ex ected in the neighbourhood of the location calculate B from the previous frame (see Appendix [3]). The program tries to shift the search pattern to each possible position and compares the pattern with the image, pixel by pixel. For each comparison a distance function is defined as the sum of the squares of the differences. The marker must occur where the difference function is a minimum. Concealed marker points A specific problem which occurs in gait analysis is the tracking of temporarily concealed markers, for example on extremities further away from the camera. The program must therefore be able to project where the marker will reappear and finally be able to locate it. Any projection of the expected location is approximate and therefore does not

et al.

Motion mearurement: S. Hobeiter

Axis of symmetry

of the

marker

0

1

2

3

4

5

6

7

8

9

30

30

30

30

Pixel number 30

30

30

230

230 Crey

Starting image withthe path of the marker points&rived by pa&em recognition and subpixel correction (without any smoothing). The rings on the right side are the manually located search patterns

Figure

1

always lead to the target. Experience has shown that a linear extra olation with a simultaneous enlargement of the seam R domain is normally sufficient. To decide whether the marker is concealed or not, the program compares the difference function of each position in the search domain. If there is no value significantly lower than the mean then the marker must be concealed (see Appendix [4]). In contrast to a simple examination of the distance function using a fix threshold, this method does not depend on the pattern size or pattern form. The subpixel correction The accuracy of the attem recognition is limited b the ixel size. In tR e specific case of the Kod ax EK TpAPRO system with 293 x 192 pixels this is not sufficient. The solution is an algorithm which calculates a subpixel correction usin grey values in the edges of the markers. The con c!ition for this method is a certain blurring of the image. It is necessary to mention this in order to understand the theoretical background of the subpixel correction. The blurring can easily be achieved by adjusting the camera slightly out of focus. Figure 2 illustrates why the important information using a focused image, is for subpixel correction, missing. A displacement of the marker point of 0.4 pixels shows no change in the detected grey values, considering a single line of the image. On the other hand, Figure 3 shows the grey value relief of an unfocused marker. If one examines the pixel pairs 2-3 and 5-6 in Figure 3, one is able to approximate the flanks of the grey value relief between the respective ixels using a straight line (lines gl and g2 in Figure 4P. Drawing a connecting line (a), for example, at the intensity level of 150 between the two straight lines, one can see that the point of bisection lies, with high accuracy, close to the axis of symmetry. This is the key to the subpixel-correction method. The arbilevel of 150 is calculated in trarily chosen intensi and minimum our program from 3( e maximum intensity levels of the considered image segment. Since the suitable pixel pair values do not all meet the flanks, this may contribute different correction values, therefore several different intensity levels are taken

230 value

Figure 2 A theoretical video camera with no blurring does not supply data for a subpixel correction. If the section illuminated by the marker is shifted by 0.4 pixels there is no change in the detected grey levels 250 J200 z .t z d c

Axis

t

of symmetry

of the

L-marker

150 100

1

‘\

50 -I--L

I 0

1

2

34

5 Pixel

30

33

74

169

67

a9

number

221

186

Crey

value

91

36

30

30

Figure 3 If the recording has some blurring the grey values at the edges of the marker point will change even if it moves a distance smaller than one pixel 250

r

I 0

, Calculated ..-- k....

,

I

I

1

2

3

I,

4

axis

of

symmetry

I

I

I

I

I

5

6

7

8

9

Pixel number Figure 4 For subpixel correction the flanks of the real grey level are estimated by straight lines (gl, g2). The bisection of the connection line (a) is a good approximation for the real centre of the marker

into consideration and the correction value is then averaged. For the algorithm to be usable in practice, it should not be influenced by saturation in the video recording or overload of the A/D converter. In practice, however, this is unavoidable, as the camera does not possess an automatic diaphragm and the marker points reflect with an excessive brightness. Disturbance of the algorithm can be avoided by excluding pixels at saturation level from the evaluation. The subpixel correction raises the accuracy and resolution of the EKTAPRO 1000 system to a level comparable with the highest class of kinematic measurement systems. The level of accuracy achieved and luminance will depend on the surroundin values of the marker points. 8 n average, however, one can estimate that the resolution (not the

J. Biomed. Eng. 1993, Vol. 15, March

141

et al.

Motion meawrement: S. HoLpiier 60 r

REFERENCE!3 1. FumCe EH. TV/Computer Motion Analysis Systems. Delft: Delft University of Technology, 1989; 6. 2. Fumke EH. TV/Computer Motion Analysis Systems. Delft: Delft University of Technology, 1989; 63, 66. 3. Gustafsson L, Lanshammar H. Enoch -An ZntcgratedSystnn for Measurement and Analysis of Human Gait, Uppsala, Sweden: Institute of Technology, University of Uppsala, 1977; 37.

50 40 -

i

30-

N 20 -

10

APPENDIX t

I 0

I

I

I

I

I

I

I

1

lo

20

30

40

50

60

70

80

x (mm)

Figure 5 Drawing by the plotter at original size in comparison to the kinematic results smoothed by a spline algorithm and converted into real coordinates (dashed line). The region scanned by one pixel is about 14 mm in diameter

accuracy!) must be much smaller than one-hundredth of a pixel size.

1. Search

DISCUSSION This system is already being put into use to analyse human motion sequences simultaneous1 with the measurement of ground-reaction forces. TX e speed of the algorithm is sufficient. The time consumption for the transfer of the images from the video recorder to the computer is still unsatisfacto . It may be that the program loses track of one maru er point, if several points are concealed over a long time. If this occurs then it is necessary to indicate manually the position of its reappearance. We regard the possibilit of comparing measurement results with the vi J eo film as a particular advantage. Furthermore, the results can even be inserted into the video images and therefore can be used for animation graphics. On account of the simple requirements of the measurement surroundings, the high recording fre uency and the extend accuracy, this method may 9 so be utilized in areas outside the medical field, e.g. in robot construction, for mobile machinery, production lines, vibration measurements, crash tests and so on.

142 J. Biomed. Eng. l&3, Vol. 15, March

points

The location of the marker point to pixel accuracy is calculated by the search of the minimum of the following distance function:

dxv_Y = kj)CM

tbx+i,y+j

-

bx,y - mi,j)2

E

“,Y

image line, image column of the actual tested location

d XlY

distance function at line X, column y

bX*Y

grey level of the image in line x and column

bX+l,y+J

Y grey level of the image in line x + i and column y +i

EXAMPLE OF RESULTS To demonstrate the recision of the method a plotter with a reflective mar Ker on the pen was filmed from a distance of 3.75m. This distance is usable for fullbody gait analysis. Figure 5 shows the drawing by the plotter in original magnification and over it the sketch of the kinematic results converted into real coordinates (smoothed by a cubic spline function). As one can clearly see, the deviation is only a few millimetres. Under test conditions the usable area was ap roximately 3 x 2.3 m (the recording area is a little Parger but a narrow margin at the edges cannot be used for the analysis). The results are the more impressive considering that one pixel observes a region of about 14mm in diameter.

of the marker

grey level of the search pattern where mo,o is the centre

mi,j

set of indices for a circular with radius r defined by

A4

M=

{(i,j)I(i2+j2)<

r2};

2. Imprinting

of the pattern

mi,j =

-

bx+i,y+j

3. Search S=

search pattern

bx,y for each (i,j)

E M.

domain

{(x, y))(x-“‘)2+(y-y’)2
S

search domain

s

radius of the search domain. Normally 1 or 2 pixels depending on the recording frequency and the maximum acceleration of the marker points

X’,

y’

coordinates of the marker point in the previous frame or as projected by linear regression.

The marker point is detected d X*Y = min(D), 4. Decision

at line x column y if

where D = {d,,,

as to whether

1(x, y) E S}.

the point is concealed

mean distance number of elements in D q=vLzQ)zr q > 0.5 +

marker point concealed.