Head movements and actographic recordings in free-moving animals, using computer analysis of video images

Journal of Neuroscience Methods, 9 (1983) 357-365

357

Elsevier

Head movements and actographic recordings in free-moving animals, using computer analysis of video images J.C. L e c a s a n d G . D u t r i e u x Laboratoire de Physiologie Neroeuse, D~partement de Psychophysiologie, C.N.R.S., 91190 Gif - sur- Yvette (France)

(Received June 9th, 1983) (Revised September 27th, 1983) (Accepted September 29th, 1983)

Key words: video-detection - actographic techniques - head movements - on-line recordings

A method is described to record the X,Y-coordinates of two bright spots on a TV image. These spots are produced by a hght-emitting diodes (LEDs) assembly, which must be affixed on the moving target. The system was developed to record head movements of free-moving animals, chronically implanted to bear a socket LEDs holder, but it can be easily adapted to other applications, such as the measure of limb displacements in Man. Recordings are allowed up to 25 frames per second with an approximate spatial resolution of 255(X)×300(Y). The method, which is based on standard TV equipment, involves a hardware interface, feeding the X and Y counts into a laboratory minicomputer and data acquisition software. A sample record is shown and other applications are discussed in relation to current non-video and video actographic techniques.

Introduction The ability to detect a n d measure a n i m a l locations a n d m o v e m e n t s is often i m p o r t a n t in evaluating behavior. I n m a n y instances, actographic techniques, using optical, ultrasonic, or capacitive detectors seem usable for precise, reliable a n d a u t o m a t e d data collection. However, they all become i n a d e q u a t e when consecutive positions of one b o d y segment m u s t be recorded, which involves the m a p p i n g of two p o i n t s in Cartesian coordinates. This is especially the case when studying gait parameters or a r m reaching m o v e m e n t s in h u m a n s . But this also holds true in experiments using small a n i m a l s s u b m i t t e d to maze or d i s c r i m i n a t i o n training, when the evaluation of head m o v e m e n t s a n d orienting behaviors appears to be a most critical aspect of the analysis of behavior. A p r o m i s i n g technique, the 'light pulse' method, has b e e n p r o m o t e d in the study of limb m o v e m e n t s i n M a n : a pulsing light-emitting diode ( L E D ) is attached to the m o v i n g limb a n d photographed. To make the most of this technique, a significant i m p r o v e m e n t has been made b y substituting TV i m a g e - c o m p u t e r analysis to the

0165-0270/83/$03.00 © 1983 Elsevier Science Publishers B.V.

358 standing camera film records. Analyzing gait parameters in Man, Winter et al. (1972), attached reflecting markers on the limbs of human subjects walking in front of a TV camera. The video signal was digitized at high speed and stored in the memory of a CDC 1700 computer. The authors were then able to calculate the center of the markers and thus the relative joint angles. This method, however, is time-consuming and requires a computer with a large memory. To reduce these requirements, In-Sheng Cheng et al. (1975) have suggested the combination of this method with the light pulse principle. Instead of the whole TV field, they simply recorded the (Xi, Yi) coordinates of LEDs, which were then translated, by the TV tube, into larger signals than those from the darker background. In a very similar way, Dutrieux et al. (1978) and Tanger et al. (1978) have described vertical/horizontal detection devices, which score both the X - Y displacements and rearings of rats, viewed by a TV camera as white spots on a dark floor. More recently, a computerized system with similar characteristics has been commercialized (Videomex, Colombus Instruments). The on-line technique, described in this paper, was aimed at the simultaneous detection of two tracing LEDs on the same image. It has been designed to measure the axis of a behaving animal's head, using chronic procedures to affix a LED assembly on the skull. This method provides a record of the absolute position and orientation of the animal's head with a temporal resolution of 40 ms. Hence, the precise course of fast head movements can be monitored, recorded and later reconstructed during off-line analysis of the behavior exhibited by the free-moving animals. It also becomes possible to know not only the exact position of the animal's head in the test area, but also, by extrapolating the head axis onto the test enclosures, the approximate spot which it is facing. In addition, the method is compatible with the implementation of interactive features, allowing the computer to take decisions about the experiment as a function of the animal's head position and orientation.

Materials and Methods Our design takes advantage of these two basic features, first proposed by In-Sheng Cheng et al. (1975): (a) implanted LEDs on the animal's head give high signal-to-noise ratios at the output of the TV tube, and (b) a coordinate system based on the intrinsic feature of video images: the number of lines from top to bottom represents the Y-coordinate of the bright spots, while a 'horizontal clock' counter gives their X-position. The system involves a LED assembly, composed of two LEDs of different size, a peripheral device, interfacing a standard 'Sony' TV camera to a DEC PdP-11 minicomputer and a data acquisition software.

Interface design The interface device (Fig. 1) is the external synchronizing source for the TV camera. It also generates horizontal clock pulses (f = 6 MHz), used to monitor the X-position. Interlacing of the TV field is discarded, which results in a vertical

359

1 Frame sync

~

I COUNTER[0-9] 1

LEDs CONTROL LEDs

IACtlve Frame

I Per~odic~ty

(

2

a

Line

~

sync WOR~INGAREA Y top Ybottom X le~t X rlght

)

I

Trigger

),

)

End of WorLIng

NEWDATA READY

Area (1)

11Ill GATING

I

,

l CLOC~ XClocL

Y COORDINATE

•

I

60

In Xa ~

Clr ~

ONTROL t"

x. Xb

COUNTER I

. PARALLEL INFU 'T

I L

Xb c0~rE~

,

Fig. 1. Block diagram of video-computer hardware interface device. The End of Working Area information (*) is used to save Y-testings in the input loop. It is especially needed if an interrupt logic, controlled by the NDR, is employed.

resolution of about 300 horizontal lines. Based on the general TV frame timing, the hardware defines a working zone, delimited by a top and a bottom horizontal line and by two vertical boundaries, supplied, from left to tight, by precision monostable delays triggered by the Y (line) sync pulses. These Y-top, Y-bottom, X-left and X-tight margins may be adjusted separately. Within this working area, the Y-sync pulses are directly output, as an external clock, to the computer. Once a bright spot is detected on the video output, a Schmitt trigger fires, cleating the X-counter and simultaneously opening a gate to feed the counter again with horizontal clock pulses, until the X-right boundary is reached. At this time, a New Data Ready pulse ( N D R ) is directed to the computer to initiate reading on a parallel input line. In fact, since two LED markers are used, and since they may be aligned on a same TV horizontal line, the interface includes two parallel X-counters. The first one runs from any first trigger detection up to the end of the current line inside the working area. The second counter is only used when the Schmitt trigger fires for a second spot on the same line. Both counters are simultaneously loaded on a parallel

360

input line of the computer. Using 8-bit counters results in a reduction of the horizontal scale to 255 X-positions, but this is usually sufficient. Current Vidicon camera tubes often have a response time course, which result in an enduring persistency of bright objects over several consecutive images. In order to detect only the actual LED spots, and not the phantom ones, a driver circuit switches the LEDs on only for a short duration, in synchrony with the working zone of 'active images'. Such images are those during which detection and counting is allowed. Their frequency may be adjusted from 5 up to 25 frames per second. In principle, the recording rate could reach 50 Hz, but it depends on the characteristics of the TV tube (see discussion). Fig. 2 illustrates the temporal relationships between LED activation and the active window period, as set up in our application. Software

subsystem

The mainline program is specific to the application. Our system registrates head coordinates and body displacements of hamsters submitted to visual discrimination training through daily sessions of several trials. The F O R T R A N mainline stream accepts session and subject parameters, opens individual data files, updates the index tables and calls for a machine-language acquisition subroutine on each trial. Upon completion of the experiment, it stores the collected data on disk and loops at the beginning to ask the operator for any further subject/session.

Data acquisition subroutine Input loop. This routine continuously tests an input flag, which monitors the N D R from the interface. When set, this flag triggers reading of the two X-counters on the parallel input line. The Y-coordinate is obtained from the computer's real-time clock counter, which keeps track of the line-sync pulses. The completion of scanning of the current TV field is acknowledged by testing the Y-coordinate, which has a known predetermined value at the end of the working area. In this case, the program branches to a data processing routine, which calculates the X - Y coordinates of each individual LED marker. Otherwise, it loops on the input flag test. I

F.,~E SYNC.

Delay

2

I

I

Delay

3

I I(

!

20 msec

I

)|

'1

I ACTIVE FRAME

m !

WORKING AREA

I

,,,Lfl

vIDEO RE~ O N , ~

,L,ll]Itll[ILltlII1[1

I[II~, .....

I LEDs ON | Delay

1

I

Fig. 2. Timing diagram of LEDs switching and video response in relation to the TV frame synchronization. Note the time course. This is the setting for recording at 25 measures per second. Delays 1, 2 and 3 refer to the corresponding block number in Fig. 1.

361

i!iiii!iii!!i!!ii!ii!iiii! L •

.

.

.

.

.

. .

.

.

.

.

o

,

.

•

.

.

•

.

.

•

.

•

.

.

.

.

.

.

.

.

o

. .

.

.

.

.

.

.

.

. .

.

.

.

.

o

.

.

°

.

.

,

.

.

.

.

.

.

. o .

° .

.

.

o

°

.

.

.

°

°

°

. °

.

.

.

, .

.

.

" .

.

" ° " , o o .

.

.

.

•

MIN I

°

'

'

'

.

'

' o

. °

.

o

.

.

" °

• .

.

.

"

.

.

.

'

° .

' .

°

.

,

°

.

.

° .

' .

'

°

j Iq~X I

0 0

o

o

°

.

~

.

.

°

.

.

0

o

°

I

I

.

.

'

,

o

•

.

,

o

•

°

.

, ° . .

°

.

.

,

•

°

.

• ° °

° °

.

•

" . .

.

o

.

" .

.

.

.

i

10 0 0 O ~ - - 0~I[EO 0 0 0 0

.

°

.

.

| i l

.

°

.

.

.

o

.

.

°

.

.

.

.

. .

°

.

.

. . ,

i

.

.

o

.

,

.

.

.

.

' .

.

.

.

.

° .

.

.

.

.

.

o

.

'

.

,

.

.

.

°

.

°

,

.

.

.

o

.

.

.

'

"

.

. . . .

,

.

.

.

.

.

.

.

°

l

. .

" ' . ,

.

.

. .

.

.

J

°

.

.

.

. .

.

.

.

.

.

. .

. .

.

. .

.

.

.

.

.

.

<"-- HAX

. .

0 •

•

°

•

0

0

,

" .

.

,

o

| 1

°

°

.

.

lD O

.

o

° " . . °

"

"

°

''

I .

X

<-...- j <"" HIN

0 0

Y HIST()~q

I

0 0 ~- '~ 0 0

0 0 0 0 0 0l

X H I STOI]R/~q

Fig. 3. Spatial representation of the relationship between LEDs spots and logical X,Y-histograms. By convention, the front LED is the smaller one, labelled '1'. Black dots on the left contour of LEDs represent Schmitt trigger firing.

Coordinates processor. The algorithm used is based on the properties of logical (one or zero) histograms constructed from the individual X - Y data, obtained from scanning each video field (Fig. 3). In each histogram, the routine searches for the MIN and MAX values and for the i and j bounds of the largest zero-range, if any. This 'empty' area is a function of the distance separating the tracing LEDs and of their relative orientation to the X- and Y-axis. For instance, if the LED assembly is parallel to the X-axis, the Y-coordinates cluster onto the same values. Thus the size of this zero-range provides a criterion to choose the most discriminant histogram used to sort the coordinates associated to the front and to the rear LEDs (Fig. 4). Given [MIN(X), i(X)] and [j(X), MAX(X)], the bounds of the logical-one clusters on the X-histogram, the routine scans the input data buffer to obtain the associated Y-values: Ymin, Ymax [MIN(X), i(X)] and Ymin, Ymax [j(X), MAX(X)] Then the individual LED coordinates are obtained from simple geometric means formulae: LED1 X,= 0.5*(MIN(X)+i(X)-I) V , = 0.5 * (Ymin + Vmax) spot size = (i(X) - MIN(X)) * (Ymax -- Ymi,) LED2

X 2= 0.5*(j(X)+MAX(X)-I) Y2 = 0 . 5 * (Ymin + Ymax)

spot size = (MAX(X) - j ( X ) ) * (Ymax - Ymi,)

362

I I,o Xl,

1= 0

~-~

i

'

PROCESS1NPUTBUFFER for I MIN-P~X X, MIN-I~X Yl X and Y LI~ICAI. HIST. I

II

PROCESSX-HIST. obtain iX, jX {rOll largest Zero-Ranqe

'1 PROCESSY-HIST. obtain iY, jY from largest Zero-Range

/

'I

, ,.s

I 11= "5$(MinXH4axl) J yI= .5$(MinY+MaxY) X2= O, Y2= 0

/

i

i= zero~.o0.~l [

I no X-Zero-RanqeLARI~R ?

J

,"sJ in°,

l l Y=n, Y..:%1.X,.l

1

X.,n, *.::I.=.,,.l J

Jl

II

I", .i, ~,. LE0 I I

Iv,,.,

I

1]

I

l I,.,o ,.::,1,x ,xl

x.,o.xXx'i,1,.,,i I

Jl

t

s,.~0

Jl )

I ,

n°,l~

I .,x~,

.I,~

/

J

I RETURN

Fig. 4. Flow chart of LEDs coordinates processing routine. Notations MIN, MAX, i, j, refer to the corresponding ones in Fig. 3. A t the end of this c o m p u t i n g procedure, the sizes of the two L E D s are c o m p a r e d . C o o r d i n a t e s are eventually i n t e r c h a n g e d in o r d e r to be always a d d r e s s e d into m e m o r y a c c o r d i n g to the same convention. ( X ] - Y ] ) refers to the smaller front L E D , a n d ( X 2 - Y 2) to the larger rear one.

363

If the system loses track of one LED, which may happen if the animal's head is in a vertical position, no zero-range is detected in logical histograms and only one coordinates pair is computed from the M I N - M A X X - Y values.

l~esults

Fig. 5 illustrates a data sample obtained with this method during a learning trial of a hamster trained on a visual discrimination task. Acquisition rate was 16.66 frames per second (one active image out of 3), which seemed sufficient to monitor head movements and body displacements in this species. From such data, the kinematics of the animal's run from right to left may be fully analyzed, since they allow for the computation of displacements speed during locomotion (proportional to the distance between two consecutive head vectors), and absolute head positions and orientations at any time. Further analyses involve the time spent stationary at

Fig. 5. Left: a picture of a Hamster's head, bearing the LEDs assembly, affixed on a miniature chronically implanted socket. Right: two sample records obtained during two different trials of a behavioral session. In each subpicture, the computer has drawn the experimental enclosure, as viewed by the TV camera, with the starting box right to the rightmost vertical line, and the two goal boxes at extreme left. The X means that the goal door on this side (the 'wrong' one) is locked. Consecutive positions of the animal's head axis are represented by short line segments, drawn from the front to the rear LEDs X,Y-coordinates. Note the orienting head movements, as fan-shaped clusters of head vectors in the middle of the training apparatus.

364 critical places, such as the choice area in front Of the two goal alleys. This time can be scored and the ongoing behavior of the animal deduced to a certain extent. Grooming, for instance, is associated with a large number of shortened head vectors and detection losses, resulting from the nearly vertical position of the animal's head. On the other hand, orienting and visual exploration produce typical head movement patterns towards specific directions of space.

Discussion Most of the actographic methods described in the literature are using photocells, ultrasonic or capacitive sensors to locate the animal in the test area (see Moross and Kaufman, 1976; Finger, 1972; Tuddenham et al., 1973; Van den Steen and Vanwersh, 1981). These techniques have been improved with the introduction of small laboratory computers, which monitor an increased number of detectors and make available the restitution of activity charts (Opto-Varimex, Automex, Columbus). However, together with former types of TV systems (Dutrieux et al., 1978; Tanger et al., 1978), all of these methods have in common that they only provide the X - Y coordinates of one single spot. When two points are required, so as to define the position of any one body segment, the method described in this paper (which is based on the very same 'light pulse' principle) offers a convenient alternative to the time consuming and fastidious analyses of videotapes or camera films (Wyss and Pollack, 1981). Our technique allows identification and mapping of two LED markers with any orientation onto a background plane. Assuming that a third point has a fixed position, it is possible to draw a second line segment, which makes possible the recording of arm reaching/ pointing movements or simple gait parameters in humans. Used for the measurement of head movements, it makes available many behavioral parameters, impossible to score using any other actographic technique. In addition to the variables which may be derived from a precise chart of the animal's path, the knowledge of what the animal is facing may lead to the definition of direct 'attentional' indices. Provided that the experimental situation allows in some way to distinguish freezing behavioral sequences (for instance, in choice situations, on the basis of the subsequent behavior), the time spent stationary in front of a stimulus may be taken as a relevant index of the animal's reactions to this stimulus. Therefore, most of the animal's runs can be reduced into several segments of specific behaviors. This 'microscopical' approach to behavioral analysis may introduce heuristic developments in animal studies on attention, learning and memory. In our method, the use of standard (low cost) TV equipment and the economy of processing for obtaining LEDs coordinates are features which challenge the digitalization of the whole video field (Winter, 1972; 512 system, Imaging Technology). With regards to the sampling rate, Woltring (1974) has stated that, due to the 20 ms necessary for the line-by-line scanning process, video-based detection methods would be inappropriate for movement studies. But a recent paper by Winter (1982) has shown that a 24-30 Hz sampling rate is largely sufficient for the analysis of the

365

human gait. In our experiments, we routinely recorded well-resolved rodent head movements at 16.66 frames per s, with an old Vidicon TV tube and this recording frequency could be easily increased up to 25/s, with a Newvicon tube TV camera, having shorter response-time characteristics. In fact, our method can be adapted to different needs, when medium performance is required. Whether more than one LED pair is necessary, but slow data rate can be accepted, the LEDs may be pulsed on alternate active frames by the accepting computer. On the other hand, if two markers are sufficient, but fast acquisition is desired, the system can be operated at a 3 0 - 5 0 sampling rate, with a charge-coupled device (CCD) camera tube, which has no persistency effect. Although such a TV camera is much more expensive, it must be noted that the CCD technology is compatible with the highest performance. Using special CCD cameras and sophisticated interfacing, Macellari (1982) has described the CoSTEL system, which maps up to eight alternately flashing LED markers, at 100 Hz, with 3-dimensional capabilities. The method here described was essentially designed as a low-cost, easy to build and medium resolution system, with potential interactive flexibility. This latter feature may be further improved by implementing the input loop as a microcode controlling a microprocesor chip installed into the hardware interface. This would free the accepting computer from the input scanning process. More complex interactive calculations could then be run concurrently with data collection. Circuit diagrams and program listings are available from authors.

References Dutrieux, G., Platel, A. and Deweer, B. (1978) Analyse automatique du comportement exploratoire chez le rat: actographie par utilisation d'un circuit ferm6 de trl~vision, Physiol. Behav., 21: 721-726. Finger, F.W. (1972) Measuring behavioural activity. In R.D. Myers (Ed.), Methods in Psychobiology. Academic Press, New York, pp. 1-19. In-Sheng Cheng, Koozekanani, S.H. and Fatehi, M.T. (1975) A simple computer-television interface system for gait analysis, IEEE Trans. biomed, eng., 3: 259-260. Macellari, V. (1982) CoSTEL: A computer peripheral remote sensing device for 3-dimensional monitoring of human motion, Med. Biol. Eng. Comput., 21: 311-318. Moross, G.G. and Kaufman, G.I. (1976) Activity monitor for small animals, Physiol. Behav., 16: 493-495. Tanger, H.J., Vanwersh, R.A.P. and Woithuis, O.L. (1978) Automated TV-based system for open-field studies: effects of methamphetamine, Pharmacol. Biochem. Behav., 9: 555-557. Tuddenham, A., Perry, G.C. and Brittain, (1973) A meter suitable for long-term activity studies on small animals, Physiol. Behav., 10: 637-639. Van Den Steen, L. and Vanwersh, R.A.P. (1981) Improved capacitive transducer for animal movements, Med. Biol. Eng. Comput., 19: 479-482. Winter, D.A. (1982) Camera speeds for normal and pathological gait analyses, Med. Biol. Eng. Comput., 20: 408-412. Winter, D.A., Greenlow, R.R. and Hobson, D.A. (1972) Television-computer analysis of kinematics of human gait, Comput. Biomed. Res., 5: 498-504. Woltring, H.J. (1974) New possibilities for human motion studies by realtime light spot position measurement, Biotelem., 1: 132-146. Wyss, V.P. and Pollack, V.A. (1981) Kinematic data acquisition system for two or three-dimensional motion analysis, Med. Biol. Eng. Comput., 29: 287-290.