- Email: [email protected]
Eman: A Modular and Iterative Eye-Movement Analysis Program
Theoretical and Applied Aspects of Eye Movement Research AG.Gale and F. Johnson (Editors) 0Elsevier Science Publishers B.V. (North-Holland), 1984
EMAN:
31
A MODULAR AND ITERATIVE EYE-MOVEMENT ANALYSIS PROGRAM
Reinhold M. K l i e g l Max Planck I n s t i t u t e f o r Human Development and Education B e r l i n , West Germany EMAN i s an eye-movement a n a l y s i s program t h a t c o n s i s t s of
f o u r modules. The f i r s t module r e s c a l e s eye p o s i t i o n s t o coordinates of t h e d i s p l a y . The second and t h i r d modu l e s reduce d a t a t o a f i x a t i o n format and i d e n t i f y areas of bad measurement by means of i t e r a t i v e p a s s e s over t h e d a t a . I n t h e f o u r t h module i t e r a t i v e algorithms are employed f o r t h e i d e n t i f i c a t i o n of l i n e numbers and f o r achieving congruence between f i x a t i o n s and d i s p l a y . INTRODUCTION
Videobased systems f o r monitoring eye movements - such as t h e Applied Science Models o r D e m e l ' s DEBIC 80 - generate information about eye p o s i t i o n and p u p i l diameter a t 60 o r 5 0 Hz. The program EMAN was w r i t t e n f o r t h e a n a l y s i s of t h i s eye-position information. EMAN f i r s t r e s c a l e s machine o u t p u t t o c o o r d i n a t e s of t h e d i s p l a y a s described i n K l i e g l and Olson (1981). The r e s c a l e d v a l u e s are then processed by three o t h e r program modu l e s t h a t a r e r e p o r t e d i n t h i s paper. F i r s t , reduction of v a l i d samples t o p o s i t i o n c l u s t e r s and r e d u c t i o n of low-data-quality samples to b l i n k c l u s t e r s i s accomplished i n t h e Module REDUCE. The second module, cLFANl.JP, performs a d d i t i o n a l checks on t h e d a t a - q u a l i t y of p o s i t i o n c l u s t e r s and r e moves saccadic p o s i t i o n c l u s t e r s from t h e d a t a base. F i n a l l y , t h e o u t p u t of CLEANUP, a sequence of f i x a t i o n s , i s matched t o t h e t e x t d i s p l a y by M o d u l e TEXT. T h i s module permits i n t e r a c t i v e changes of parameters t o achieve congruence between f i x a t i o n s and l i n e s of t h e t e x t d i s p l a y . Of course, t h e l a s t module i s o n l y meaningful f o r t h e a n a l y s i s of eye movements t h a t were c o l l e c t e d d u r i n g reading. The program w a s parameterized on t h e b a s i s of t h e following three assumpt i o n s : One, t h e machine r e s o l u t i o n o r t h e d i s p l a y s i z e allow r e l i a b l e ident i f i c a t i o n of f i x a t i o n s a t t h e l e v e l of letters. Two, with a 60 Hz samp l i n g rate, only c l u s t e r s encompassing a t least s i x samples (i.e. l a s t i n g longer than 100 m s ) w i l l be c a l l e d a f i x a t i o n . Three, c l u s t e r s with fewer samples may occur between f i x a t i o n s and i n d i c a t e t h e t i m e necessary t o s h i f t t h e eye. Since it t a k e s up t o s i x samples t o complete a r e t u r n sweep across a l i n e of t e x t of about 20 degrees of v i s u a l angle, the t i m e between two f i x a t i o n s w i t h i n a l i n e should n o t exceed f i v e samples; I n programming EMAN it was n o t assumed t h a t d a t a obtained during a c e r t a i n t r i a l are of e q u a l l y good q u a l i t y . To t h e c o n t r a r y , each program module w a s motivated by a d i f f e r e n t s e t of problems a l l o r some of which may apply d u r i n g a given t r i a l . I n t h e d e s c r i p t i o n of t h e modules, emphasis w i l l be placed on t h e problems they address and how s o l u t i o n s were implemented. I n
32
R,M. Kliegl
g e n e r a l , conservative s o l u t i o n s a r e advocated s i n c e eye-movement researche r s a r e u s u a l l y blessed with enough d a t a t o compensate f o r seqments of lowquality data. MODULE REDUCE
P r i o r t o reduction t h e r e m u s t be a c r i t e r i o n t o d i s t i n g u i s h c l e a r l y i n v a l i d from p o t e n t i a l l y v a l i d measurements. I n videobased systems i n v a l i d measurement most obviously occurs when t h e s u b j e c t b l i n k s o r when t h e i d e n t i f i c a t i o n of p u p i l and corneal r e f l e x i o n f a i l s . Both c o n d i t i o n s lead t o a zero o r g r e a t l y reduced p u p i l diameter. To determine a c r i t e r i o n f o r v a l i d measurement t h e program computes mean and standard d e v i a t i o n of non-zero p u p i l diameters. Based on e i t h e r a m u l t i p l e of standard d e v i a t i o n s o r a c e r t a i n percentage of t h e mean, a lower boundary f o r p u p i l diameter i s s e t . A f t e r determining t h e c r i t e r i o n f o r p o t e n t i a l l y v a l i d measurements, t h e d a t a a r e processed by t h e algorithm displayed i n Figure 1. I f t h e p u p i l Successive diameter i s below t h e c r i t e r i o n , t h e sample i s c a l l e d a BLINK. b l i n k s a r e collapsed i n t o a BLINK CLUSTER. A sequence of v a l i d samples i s combined t o a POSITION CLUSTER i f t h e i r coordinates a r e s u f f i c i e n t l y s i m i l a r . Note, however, t h a t some c l u s t e r s may contain only one sample.
BLINK? YES: -
NO: -
WAS LAST SAMPLE (CLUSTER) BLINK? YES:
UPDATE BLINK CLUSTER
NO:
FINISH POSITION CLUSTER START NEW BLINK CLUSTER
WAS LAST SAMPLE (CLUSTER) BLINK? YES:
NO:
FIGURE 1.
FINISH BLINK CLUSTER START NEW POSITION CLUSTER
I S PFfESENT SAMPLE (CLUSTER) W I T H I N WINDOW? YES:
UPDATE POSITION CLUSTER
NO:
FINISH POSITION CLUSTER START N E W POSITION CLUSTER
Reduction algorithm i n Module REDUCE
I f t h e p u p i l diameter i s below t h e c r i t e r i o n , t h e t o p block of t h e algorithm i s executed. I f t h e l a s t sample w a s a b l i n k a s w e l l , t h e p r e s e n t sample i s simply added t o t h e b l i n k c l u s t e r . I f t h e l a s t sample w a s v a l i d , then t h e previous p o s i t i o n c l u s t e r i s closed and a blink c l u s t e r i s s t a r t e d . I f a sample p a s s e s t h e f i r s t v a l i d i t y t e s t , it w i l l e n t e r t h e second block of t h e algorithm. There, i f t h e l a s t sample was a b l i n k , t h e b l i n k c l u s t e r
Eye-Movemen t Analysis
33
is closed and, s i n c e p o s i t i o n s during b l i n k s are not v a l i d , i t s x- and yv a l u e s are set t o zero. The p r e s e n t sample then becomes t h e seed f o r a new c l u s t e r of v a l i d p o s i t i o n s . I f t h e l a s t sample was v a l i d as w e l l , then, i f t h e c o o r d i n a t e s of t h e p r e s e n t sample a r e within a window around coord i n a t e s based on t h e running mean of a c l u s t e r of previous samples, t h e p r e s e n t sample i s added t o t h i s c l u s t e r . Otherwise, t h e previous c l u s t e r i s f i n i s h e d and t h e p r e s e n t sample becomes t h e seed f o r a new c l u s t e r . The algorithm t h a t decides whether a sample i s within a window was suggested by Mason (1976). A d e t a i l e d d e s c r i p t i o n of t h e p r e s e n t a d a p t a t i o n can be found i n K l i e g l and Olson (1981). Since t h e p o s i t i o n s were r e s c a l e d t o d i s p l a y c o o r d i n a t e s and s i n c e t h e r e s o l u t i o n of t h e eye-tracker under t h e viewing c o n d i t i o n s i n t h e l a b o r a t o r y were p l u s o r minus one c h a r a c t e r , t h e minimal d i s t a n c e between c l u s t e r s w a s set a t one. N e w p o s i t i o n c l u s t e r s are formed i f a s i n g l e sample f a l l s o u t s i d e t h e window. The c l u s t e r r e s u l t i n g from t h i s seed, however, may be within t h e window of t h e preceding p o s i t i o n c l u s t e r . To ensure t h a t successive c l u s t e r s are a t l e a s t t h e minimum d i s t a n c e a p a r t from each o t h e r , t h e clusters thems e l v e s are processed by t h e reduction algorithm. Successive c l u s t e r s a r e c o l l a p s e d i n t o one, i f they a r e c l o s e r than t h e minimum d i s t a n c e . The a l gorithm i s a p p l i e d i t e r a t i v e l y u n t i l no f u r t h e r reduction i n t h e number of c l u s t e r s occurs. Thus, t h e reduction i t e r a t i o n w i l l s t o p , once a l l neighbouring p o s i t i o n c l u s t e r s have a minimum d i s t a n c e from each o t h e r .
Experience h a s shown t h a t o c c a s i o n a l l y two c l u s t e r s with v i r t u a l l y i d e n t i c a l p o s i t i o n s w e r e separated by a s i n g l e c l u s t e r containing mostly one o r two samples. Since it i s impossible t h a t an eye movement can l e a v e and come back t o a c e r t a i n p o s i t i o n without an intervening f i x a t i o n , it i s assumed t h a t t h e s e samples r e f l e c t spurious measurement and t h a t t h e two c l u s t e r s are a c t u a l l y one. To i d e n t i f y and remove o u t l i e r s of t h i s kind a second i t e r a t i v e algorithm compares t r i p l e t s of neighbouring c l u s t e r s . The o u t l i e r i s counted f o r t h e d u r a t i o n , but i t s p o s i t i o n information is ignored f o r t h e computation of c l u s t e r coordinates. The i t e r a t i o n s t o p s once no f u r t h e r r e d u c t i o n i n number of c l u s t e r s occurs. i t e r a t i v e reduction algorithm t h e d a t a are a v a i l a b l e as a sequence of c l u s t e r s . For p o s i t i o n c l u s t e r s we know how many samples they encompass, t h e i r average h o r i z o n t a l and v e r t i c a l posit i o n s and t h e i r average p u p i l diameters. Blink c l u s t e r s have z e r o v a l u e s i n p o s i t i o n and p u p i l diameter v e c t o r s . So f a r no fixation-saccade d i s t i n c t i o n w a s made and no sample was removed from t h e d a t a base. The next module, CLEANUP, s e r v e s t h i s purpose.
A t t h e end of t h i s two-stage
MODULE CLEANUP
The c r i t e r i o n f o r d i s t i n g u i s h i n g v a l i d from i n v a l i d measurements a l o n e i s n o t s u f f i c i e n t . There a r e o t h e r c o n d i t i o n s that y i e l d doubtful d a t a qual i t y , b u t a r e n o t n e c e s s a r i l y r e f l e c t e d i n loss o f , or s u f f i c i e n t l y reduced v a l u e s of p u p i l diameters. For example, p r i o r t o a b l i n k o r loss of del i m i t e r s , t h e p o s i t i o n information i s a l r e a d y contaminated by t h e ensuing loss of measurement. Analogously, low d a t a q u a l i t y may be-observed a f t e r an u n s t a b l e measurement s i t u a t i o n . To ensure data q u a l i t y around b l i n k c l u s t e r s , t h e algorithm shown i n F i g u r e 2 i s used. The d a t a base i s searched for b l i n k clusters. Once a b l i n k c l u s t e r is encountered, t h e program engages i n a backward s e a r c h u n t i l two consecutive p o s i t i o n c l u s t e r s w i t h similar v e r t i c a l p o s i t i o n are found. P o s i t i o n c l u s t e r s too discrep-
R.M. Kliegl
34
a n t from t h e previous ones a r e recoded a s blink c l u s t e r s . rithm i s used f o r a forward search a f t e r a b l i n k c l u s t e r .
A s i m i l a r alqo-
I S CLUSTER N A B L I N K CLUSTER? NO:
N: = N
-b
1
YES: N I : = N FLAG I S "FALSE" U N T I L FLAG I S TRUE:
ITERATE:
ITERATE
HAVE P O S I T I O N CLUSTERS ( N I VERTICAL POSITION?
-
1) AND ( N I - 2) S I M I L A R
Y E S : FLAG I S "TRUE" N = N + I NO:
RECODE P O S I T I O N CLUSTER ( N I 1 NI = NI
-
- 1) AS BLINK CLUSTER
END OF I T E R A T E
F I G U R E 2.
Ensuring s t a b l e measurement around b l i n k clusters
I n t h i s c a s e , only v e r t i c a l information i s used s i n c e unstable measurement mainly occurs a t t h e end or beginning of l i n e s . A t t h e s e l o c a t i o n s horiz o n t a l p o s i t i o n s are of course maximally d i s c r e p a n t . Since unstable measurement u s u a l l y r e s u l t s i n a b i g v a r i a t i o n of v e r t i c a l p o s i t i o n s , t h e s i m i l a r i t y between two consecutive p o s i t i o n c l u s t e r s can be s e t f a i r l y l i b e r a l , e . g . , a t two o r t h r e e l i n e s i n a reading s i t u a t i o n . The next s t e p towards a c l e a n sequence of f i x a t i o n s is t o remove b l i n k and s a c c a d i c p o s i t i o n c l u s t e r s ( i . e . c l u s t e r s with fewer than s i x samples) from t h e d a t a base. While d e l e t i n g t h e s e c l u s t e r s , t h e program records how many samples a r e removed between two f i x a t i o n s . This permits t h e i d e n t i f i c a t i o n of a r e a s of low d a t a q u a l i t y t h a t never r e s u l t e d i n a n a c t u a l l o s s of del i m i t e r s . Usually a sequence of p o s i t i o n c l u s t e r s with few samples per c l u s t e r i s i n d i c a t i v e of a condition i n which t h e t i m e between two f i x a t i o n s might well exceed e s t i m a t e s of d u r a t i o n s of saccadic movements. A s mentioned previously, t h e maximum time between f i x a t i o n s within a l i n e should be s h o r t e r than t h e time f o r a linesweep. I f t h e t i m e i s longer than f i v e samples, it may be i n f e r r e d t h a t t h e two f i x a t i o n s d e l i m i t an a r e a of bad measurement. To r e t a i n information about a r e a s of b l i n k c l u s t e r s and about a r e a s of bad measurement, t h e f i x a t i o n s d e l i m i t i n g t h e s e conditions a r e flagged i n t h e v e c t o r coding t h e p u p i l diameter. The f i x a t i o n preceding t h e bad a r e a cont a i n s t h e number of samples t h a t were l o s t ; t h e f i x a t i o n followinq t h e bad a r e a codes whether it w a s a b l i n k c l u s t e r o r unstable measurement t h a t caused t h e loss. T h i s information i s q u i t e important i n f u t u r e analyses
Eye-Movement Analysis
35
s i n c e it prevents wrong i n f e r e n c e s about information processing between t h e two f i x a t i o n s . For example, we most c e r t a i n l y must not conclude t h a t t h i s a r e a was skipped with a l a r g e saccade. MODULE TEXT
For eye movements c o l l e c t e d during reading, t h e described d a t a processing w a s o f t e n n o t s u f f i c i e n t t o achieve congruence between f i x a t i o n s and t h e a p p r o p r i a t e l i n e s of t h e t e x t paragraph. Reasons f o r t h e f a i l u r e could have been a temporarily i n v a l i d c a l i b r a t i o n t a b l e , g r e a t e r machine noise In a s s o c i a t e d with t h e v e r t i c a l p o s i t i o n , or a s u b j e c t ' s f i x a t i o n h a b i t s . g e n e r a l , even t h e presence of only a small systematic d r i f t may cause a g r e a t s i m i l a r i t y i n v e r t i c a l p o s i t i o n s of f i x a t i o n s a t opposite ends of successive l i n e s and may l e a d t o misplaced f i x a t i o n s . The TEXT module w a s w r i t t e n t o remove systematic d r i f t i n v e r t i c a l posit i o n within a l i n e ( s e e Figure 3 ) . The program compares successive p a i r s of f i x a t i o n s . I f t h e i r v e r t i c a l and h o r i z o n t a l p o s i t i o n s are within a c e r t a i n range t h e y a r e considered t o be from t h e same l i n e . I n t h i s c a s e , i f t h e i r v e r t i c a l p o s i t i o n s are n o t c l o s e t o i d e n t i c a l , t h e v e r t i c a l p o s i t i o n of t h e f i r s t f i x a t i o n i s replaced with t h e average of both. An unint e r u p t e d sequence of f i x a t i o n s judged t o be within t h e same l i n e w i l l be c a l l e d a c l u s t e r of f i x a t i o n s within a l i n e (FWL). The program i t e r a t i v e l y p r o c e s s e s t h e d a t a base u n t i l a l l f i x a t i o n s within FWL c l u s t e r s have v i r tually identical positions.
FLAG IS "TRUE" WHILE FLAG I S "TRUE" DO:
FLAG IS "FALSE" FOR N -
= 1 TO (NUMBER OF FIX -1) Do:
ARE N I AND (N
+
1) FIXATION ON THE SAME LINE?
YES: ARE VERTICAL POSITIONS IDENTICAt? YES: N = N NO:
+
1
VERT. POS. OF FIX. N = AVERAGE OF VERT. POS. OF FIX. N AND FIX. ( N + 1) FLAG = "TRUE"
N = N + l NO: END OF
END OF
FIGURE 3 .
N = N + I
FOR LOOP MxlP
Determining v e r t i c a l p o s i t i o n f o r f i x a t i o n s w i t h i n a l i n e
36
R.M. Kliegl
Once t h e i t e r a t i o n s t o p s it i s l i k e l y t h a t more FWL c l u s t e r s were ident i f i e d than t h e t e x t contains l i n e s . I f two f i x a t i o n s within a l i n e were separated by a l a r g e saccade o r i f t h e s u b j e c t jumped back t o re-read p a r t of t h e t e x t , sequences of f i x a t i o n s within one l i n e w i l l y i e l d s e p a r a t e FWL c l u s t e r s . To a r r i v e a t t h e a p p r o p r i a t e s e t , FWL c l u s t e r s i n d i c a t i n g t h e same l i n e s need t o be collapsed. Under d e f a u l t parameterization t h e program assumes a f i x e d minimum d i s tance between " t r u e " l i n e s and combines FWL c l u s t e r s t h a t a r e within t h i s margin. I d e a l l y , t h e number of l i n e s i d e n t i f i e d by t h e program should correspond t o t h e number of l i n e s t h e s u b j e c t read. A n immediate evaluat i o n of t h e q u a l i t y of t h e computations i s p o s s i b l e by inspecting tF. o l o t of f i x a t i o n s and t e x t which t h e program generates on t h e terminal. I? t h i s p l o t it i s immediately obvious i f t h e program found too many o ii: few l i n e s . I f t h e p l o t i s bad, t h e user has t h e option of t e l l i n g :..,' program t h e number of l i n e s i t should f i n d . This w i l l invoke another i t e r a t i v e procedure. A t t h e beginning, t h e minimum d i s t a n c e between l i n e s i s z e r o , b u t w i l l be slowly increased with every i t e r a t i o n u n t i l t h e d i s t a n c e has been i d e n t i f i e d t h a t g e n e r a t e s t h e d e s i r e d number of l i n e s . Again, a terminal p l o t may be used f o r e v a l u a t i v e purposes. I f t h e p l o t i s s t i l l not adequate, t h e u s e r has t h e o p t i o n of changing parameters t h a t determine whether f i x a t i o n p a i r s should be a l l o c a t e d t o a FWL c l u s t e r . A change i n t h e s e parameters w i l l r e s u l t i n a d i f f e r e n t s e t of FWL c l u s t e r s and, consequently, i n d i f f e r e n t l i n e values. I f t h e program f a i l s again, one should probably d e l e t e p a r t s of t h e t e x t f o r which congruence could not be achieved - or improve on t h e algorithms proposed i n t h i s paper. TECHNICAL INFORMATION EMAN w a s w r i t t e n i n VAX/VMS F o r t r a n 77 and i s implemented on DEC VAX 11/780 machines. Input t o t h e program a r e x,y-coordinates and p u p i l diameters c o l l e c t e d with a f i x e d sampling r a t e . The program reads a l l t h e records of a t r i a l i n t o t h r e e v e c t o r s . Two a d d i t i o n a l v e c t o r s s t o r e the number of samples i n a c l u s t e r and t h e record number of t h e f i r s t sample i n t h e clust e r . A t t h e beginning, t h e f i r s t vector i s a u n i t v e c t o r ; t h e second vect o r simply c o n t a i n s t h e record numbers. These f i v e v e c t o r s form a workspace matrix common t o a l l modules. A l l computations a r e c a r r i e d o u t on workspace elements with r e s u l t s r e p l a c i n g o r i g i n a l values. I n Module RESCALE X and y coordinates i n machine u n i t s a r e replaced with X and y coo r d i n a t e s of d i s p l a y dimensions. Modules REDUCE and CLEANUP e f f e c t e l e ments of a l l v e c t o r s . Whenever samples a r e combined i n a c l u s t e r , t h e number of samples and t h e i r average p o s i t i o n and p u p i l diameter a r e s t o r e d i n corresponding elements of t h e f i r s t sample of t h e c l u s t e r . Then, t h e s a m p l e s following t h e c l u s t e r a r e s h i f t e d up t h e a p p r o p r i a t e number of e l e ments. Thus, with every i t e r a t i o n t h e l e n g t h of t h e workspace v e c t o r s i s reduced. I t i s p o s s i b l e t o o b t a i n output a f t e r every i t e r a t i o n t o monitor t h e reduction and "cleaning" process. A f t e r d a t a were processed i n TEXT, t h e workspace v e c t o r s c o n t a i n number of samples, h o r i z o n t a l d i s p l a y posit i o n , l i n e number, p u p i l diameter ( o r a f l a g i n d i c a t i n g l o s s o r unstable measurement following/preceding t h e f i x a t i o n ) , and t h e record number of t h e f i r s t sample of t h i s f i x a t i o n . The l a t t e r permits immediate i d e n t i f i c a t i o n of t h e f i x a t i o n i n t h e raw d a t a . The o u t p u t i s s u i t a b l e f o r i n p u t t o programs FIXPLT and MATCH described i n K l i e g l (1981). S u b s t i t u t i o n of t h e TEXT module with algorithms designed f o r analyses of o t h e r than paragraphreading d a t a i s p o s s i b l e . Modules a v a i l a b l e are f o r analyses of trackinqa - l i g h t t a s k ( P a v l i d i s (1981), Olson, K l i e g l , & Davidson ( i n p r e s s ) ) and
Eye-Movement Analysis
31
t h e a n a l y s e s of single-word reading d a t a . Despite t h e l a r g e number of ite r a t i v e procedures t h e a n a l y s i s i s q u i t e f a s t . For example, an a n a l y s i s of eye movements c o l l e c t e d during reading of a 20-line long t e x t paragraph (6538 r e c o r d s , reduced t o 354 f i x a t i o n s ) was accomplished within one minute. The program is a v a i l a b l e upon r e q u e s t . REFERENCES K l i e g l , R . , Automated and i n t e r a c t i v e a n a l y s i s of eye f i x a t i o n d a t a in reading, Behavior Research Methods and Instrumentation 13 (1981) 115-1 20. K l i e g l , R . and Olson, R . K . , Reduction and c a l i b r a t i o n of eye monitor d a t a , Behavior Research Methods and Instrumentation 13 (1981) 107-111. Mason, R. L . , D i g i t a l computer e s t i m a t i o n of eye f i x a t i o n , Behavior R e s e a r c h Methods and Instrumentation 8 (1976) 185-188. Olson, R . K . , K l i e g l , R. and Davidson, B. J . , Dyslexic and normal r e a d e r ' s eye movements, J o u r n a l of Experimental Psychology: Human Perception and Performance ( i n p r e s s ) . P a v l i d i s , G. Th., Do eye movements hold t h e key t o d y s l e x i a ? , Neuropsychol o g i a 19 (1981) 57-64.
EMAN:
31
A MODULAR AND ITERATIVE EYE-MOVEMENT ANALYSIS PROGRAM
Reinhold M. K l i e g l Max Planck I n s t i t u t e f o r Human Development and Education B e r l i n , West Germany EMAN i s an eye-movement a n a l y s i s program t h a t c o n s i s t s of
f o u r modules. The f i r s t module r e s c a l e s eye p o s i t i o n s t o coordinates of t h e d i s p l a y . The second and t h i r d modu l e s reduce d a t a t o a f i x a t i o n format and i d e n t i f y areas of bad measurement by means of i t e r a t i v e p a s s e s over t h e d a t a . I n t h e f o u r t h module i t e r a t i v e algorithms are employed f o r t h e i d e n t i f i c a t i o n of l i n e numbers and f o r achieving congruence between f i x a t i o n s and d i s p l a y . INTRODUCTION
Videobased systems f o r monitoring eye movements - such as t h e Applied Science Models o r D e m e l ' s DEBIC 80 - generate information about eye p o s i t i o n and p u p i l diameter a t 60 o r 5 0 Hz. The program EMAN was w r i t t e n f o r t h e a n a l y s i s of t h i s eye-position information. EMAN f i r s t r e s c a l e s machine o u t p u t t o c o o r d i n a t e s of t h e d i s p l a y a s described i n K l i e g l and Olson (1981). The r e s c a l e d v a l u e s are then processed by three o t h e r program modu l e s t h a t a r e r e p o r t e d i n t h i s paper. F i r s t , reduction of v a l i d samples t o p o s i t i o n c l u s t e r s and r e d u c t i o n of low-data-quality samples to b l i n k c l u s t e r s i s accomplished i n t h e Module REDUCE. The second module, cLFANl.JP, performs a d d i t i o n a l checks on t h e d a t a - q u a l i t y of p o s i t i o n c l u s t e r s and r e moves saccadic p o s i t i o n c l u s t e r s from t h e d a t a base. F i n a l l y , t h e o u t p u t of CLEANUP, a sequence of f i x a t i o n s , i s matched t o t h e t e x t d i s p l a y by M o d u l e TEXT. T h i s module permits i n t e r a c t i v e changes of parameters t o achieve congruence between f i x a t i o n s and l i n e s of t h e t e x t d i s p l a y . Of course, t h e l a s t module i s o n l y meaningful f o r t h e a n a l y s i s of eye movements t h a t were c o l l e c t e d d u r i n g reading. The program w a s parameterized on t h e b a s i s of t h e following three assumpt i o n s : One, t h e machine r e s o l u t i o n o r t h e d i s p l a y s i z e allow r e l i a b l e ident i f i c a t i o n of f i x a t i o n s a t t h e l e v e l of letters. Two, with a 60 Hz samp l i n g rate, only c l u s t e r s encompassing a t least s i x samples (i.e. l a s t i n g longer than 100 m s ) w i l l be c a l l e d a f i x a t i o n . Three, c l u s t e r s with fewer samples may occur between f i x a t i o n s and i n d i c a t e t h e t i m e necessary t o s h i f t t h e eye. Since it t a k e s up t o s i x samples t o complete a r e t u r n sweep across a l i n e of t e x t of about 20 degrees of v i s u a l angle, the t i m e between two f i x a t i o n s w i t h i n a l i n e should n o t exceed f i v e samples; I n programming EMAN it was n o t assumed t h a t d a t a obtained during a c e r t a i n t r i a l are of e q u a l l y good q u a l i t y . To t h e c o n t r a r y , each program module w a s motivated by a d i f f e r e n t s e t of problems a l l o r some of which may apply d u r i n g a given t r i a l . I n t h e d e s c r i p t i o n of t h e modules, emphasis w i l l be placed on t h e problems they address and how s o l u t i o n s were implemented. I n
32
R,M. Kliegl
g e n e r a l , conservative s o l u t i o n s a r e advocated s i n c e eye-movement researche r s a r e u s u a l l y blessed with enough d a t a t o compensate f o r seqments of lowquality data. MODULE REDUCE
P r i o r t o reduction t h e r e m u s t be a c r i t e r i o n t o d i s t i n g u i s h c l e a r l y i n v a l i d from p o t e n t i a l l y v a l i d measurements. I n videobased systems i n v a l i d measurement most obviously occurs when t h e s u b j e c t b l i n k s o r when t h e i d e n t i f i c a t i o n of p u p i l and corneal r e f l e x i o n f a i l s . Both c o n d i t i o n s lead t o a zero o r g r e a t l y reduced p u p i l diameter. To determine a c r i t e r i o n f o r v a l i d measurement t h e program computes mean and standard d e v i a t i o n of non-zero p u p i l diameters. Based on e i t h e r a m u l t i p l e of standard d e v i a t i o n s o r a c e r t a i n percentage of t h e mean, a lower boundary f o r p u p i l diameter i s s e t . A f t e r determining t h e c r i t e r i o n f o r p o t e n t i a l l y v a l i d measurements, t h e d a t a a r e processed by t h e algorithm displayed i n Figure 1. I f t h e p u p i l Successive diameter i s below t h e c r i t e r i o n , t h e sample i s c a l l e d a BLINK. b l i n k s a r e collapsed i n t o a BLINK CLUSTER. A sequence of v a l i d samples i s combined t o a POSITION CLUSTER i f t h e i r coordinates a r e s u f f i c i e n t l y s i m i l a r . Note, however, t h a t some c l u s t e r s may contain only one sample.
BLINK? YES: -
NO: -
WAS LAST SAMPLE (CLUSTER) BLINK? YES:
UPDATE BLINK CLUSTER
NO:
FINISH POSITION CLUSTER START NEW BLINK CLUSTER
WAS LAST SAMPLE (CLUSTER) BLINK? YES:
NO:
FIGURE 1.
FINISH BLINK CLUSTER START NEW POSITION CLUSTER
I S PFfESENT SAMPLE (CLUSTER) W I T H I N WINDOW? YES:
UPDATE POSITION CLUSTER
NO:
FINISH POSITION CLUSTER START N E W POSITION CLUSTER
Reduction algorithm i n Module REDUCE
I f t h e p u p i l diameter i s below t h e c r i t e r i o n , t h e t o p block of t h e algorithm i s executed. I f t h e l a s t sample w a s a b l i n k a s w e l l , t h e p r e s e n t sample i s simply added t o t h e b l i n k c l u s t e r . I f t h e l a s t sample w a s v a l i d , then t h e previous p o s i t i o n c l u s t e r i s closed and a blink c l u s t e r i s s t a r t e d . I f a sample p a s s e s t h e f i r s t v a l i d i t y t e s t , it w i l l e n t e r t h e second block of t h e algorithm. There, i f t h e l a s t sample was a b l i n k , t h e b l i n k c l u s t e r
Eye-Movemen t Analysis
33
is closed and, s i n c e p o s i t i o n s during b l i n k s are not v a l i d , i t s x- and yv a l u e s are set t o zero. The p r e s e n t sample then becomes t h e seed f o r a new c l u s t e r of v a l i d p o s i t i o n s . I f t h e l a s t sample was v a l i d as w e l l , then, i f t h e c o o r d i n a t e s of t h e p r e s e n t sample a r e within a window around coord i n a t e s based on t h e running mean of a c l u s t e r of previous samples, t h e p r e s e n t sample i s added t o t h i s c l u s t e r . Otherwise, t h e previous c l u s t e r i s f i n i s h e d and t h e p r e s e n t sample becomes t h e seed f o r a new c l u s t e r . The algorithm t h a t decides whether a sample i s within a window was suggested by Mason (1976). A d e t a i l e d d e s c r i p t i o n of t h e p r e s e n t a d a p t a t i o n can be found i n K l i e g l and Olson (1981). Since t h e p o s i t i o n s were r e s c a l e d t o d i s p l a y c o o r d i n a t e s and s i n c e t h e r e s o l u t i o n of t h e eye-tracker under t h e viewing c o n d i t i o n s i n t h e l a b o r a t o r y were p l u s o r minus one c h a r a c t e r , t h e minimal d i s t a n c e between c l u s t e r s w a s set a t one. N e w p o s i t i o n c l u s t e r s are formed i f a s i n g l e sample f a l l s o u t s i d e t h e window. The c l u s t e r r e s u l t i n g from t h i s seed, however, may be within t h e window of t h e preceding p o s i t i o n c l u s t e r . To ensure t h a t successive c l u s t e r s are a t l e a s t t h e minimum d i s t a n c e a p a r t from each o t h e r , t h e clusters thems e l v e s are processed by t h e reduction algorithm. Successive c l u s t e r s a r e c o l l a p s e d i n t o one, i f they a r e c l o s e r than t h e minimum d i s t a n c e . The a l gorithm i s a p p l i e d i t e r a t i v e l y u n t i l no f u r t h e r reduction i n t h e number of c l u s t e r s occurs. Thus, t h e reduction i t e r a t i o n w i l l s t o p , once a l l neighbouring p o s i t i o n c l u s t e r s have a minimum d i s t a n c e from each o t h e r .
Experience h a s shown t h a t o c c a s i o n a l l y two c l u s t e r s with v i r t u a l l y i d e n t i c a l p o s i t i o n s w e r e separated by a s i n g l e c l u s t e r containing mostly one o r two samples. Since it i s impossible t h a t an eye movement can l e a v e and come back t o a c e r t a i n p o s i t i o n without an intervening f i x a t i o n , it i s assumed t h a t t h e s e samples r e f l e c t spurious measurement and t h a t t h e two c l u s t e r s are a c t u a l l y one. To i d e n t i f y and remove o u t l i e r s of t h i s kind a second i t e r a t i v e algorithm compares t r i p l e t s of neighbouring c l u s t e r s . The o u t l i e r i s counted f o r t h e d u r a t i o n , but i t s p o s i t i o n information is ignored f o r t h e computation of c l u s t e r coordinates. The i t e r a t i o n s t o p s once no f u r t h e r r e d u c t i o n i n number of c l u s t e r s occurs. i t e r a t i v e reduction algorithm t h e d a t a are a v a i l a b l e as a sequence of c l u s t e r s . For p o s i t i o n c l u s t e r s we know how many samples they encompass, t h e i r average h o r i z o n t a l and v e r t i c a l posit i o n s and t h e i r average p u p i l diameters. Blink c l u s t e r s have z e r o v a l u e s i n p o s i t i o n and p u p i l diameter v e c t o r s . So f a r no fixation-saccade d i s t i n c t i o n w a s made and no sample was removed from t h e d a t a base. The next module, CLEANUP, s e r v e s t h i s purpose.
A t t h e end of t h i s two-stage
MODULE CLEANUP
The c r i t e r i o n f o r d i s t i n g u i s h i n g v a l i d from i n v a l i d measurements a l o n e i s n o t s u f f i c i e n t . There a r e o t h e r c o n d i t i o n s that y i e l d doubtful d a t a qual i t y , b u t a r e n o t n e c e s s a r i l y r e f l e c t e d i n loss o f , or s u f f i c i e n t l y reduced v a l u e s of p u p i l diameters. For example, p r i o r t o a b l i n k o r loss of del i m i t e r s , t h e p o s i t i o n information i s a l r e a d y contaminated by t h e ensuing loss of measurement. Analogously, low d a t a q u a l i t y may be-observed a f t e r an u n s t a b l e measurement s i t u a t i o n . To ensure data q u a l i t y around b l i n k c l u s t e r s , t h e algorithm shown i n F i g u r e 2 i s used. The d a t a base i s searched for b l i n k clusters. Once a b l i n k c l u s t e r is encountered, t h e program engages i n a backward s e a r c h u n t i l two consecutive p o s i t i o n c l u s t e r s w i t h similar v e r t i c a l p o s i t i o n are found. P o s i t i o n c l u s t e r s too discrep-
R.M. Kliegl
34
a n t from t h e previous ones a r e recoded a s blink c l u s t e r s . rithm i s used f o r a forward search a f t e r a b l i n k c l u s t e r .
A s i m i l a r alqo-
I S CLUSTER N A B L I N K CLUSTER? NO:
N: = N
-b
1
YES: N I : = N FLAG I S "FALSE" U N T I L FLAG I S TRUE:
ITERATE:
ITERATE
HAVE P O S I T I O N CLUSTERS ( N I VERTICAL POSITION?
-
1) AND ( N I - 2) S I M I L A R
Y E S : FLAG I S "TRUE" N = N + I NO:
RECODE P O S I T I O N CLUSTER ( N I 1 NI = NI
-
- 1) AS BLINK CLUSTER
END OF I T E R A T E
F I G U R E 2.
Ensuring s t a b l e measurement around b l i n k clusters
I n t h i s c a s e , only v e r t i c a l information i s used s i n c e unstable measurement mainly occurs a t t h e end or beginning of l i n e s . A t t h e s e l o c a t i o n s horiz o n t a l p o s i t i o n s are of course maximally d i s c r e p a n t . Since unstable measurement u s u a l l y r e s u l t s i n a b i g v a r i a t i o n of v e r t i c a l p o s i t i o n s , t h e s i m i l a r i t y between two consecutive p o s i t i o n c l u s t e r s can be s e t f a i r l y l i b e r a l , e . g . , a t two o r t h r e e l i n e s i n a reading s i t u a t i o n . The next s t e p towards a c l e a n sequence of f i x a t i o n s is t o remove b l i n k and s a c c a d i c p o s i t i o n c l u s t e r s ( i . e . c l u s t e r s with fewer than s i x samples) from t h e d a t a base. While d e l e t i n g t h e s e c l u s t e r s , t h e program records how many samples a r e removed between two f i x a t i o n s . This permits t h e i d e n t i f i c a t i o n of a r e a s of low d a t a q u a l i t y t h a t never r e s u l t e d i n a n a c t u a l l o s s of del i m i t e r s . Usually a sequence of p o s i t i o n c l u s t e r s with few samples per c l u s t e r i s i n d i c a t i v e of a condition i n which t h e t i m e between two f i x a t i o n s might well exceed e s t i m a t e s of d u r a t i o n s of saccadic movements. A s mentioned previously, t h e maximum time between f i x a t i o n s within a l i n e should be s h o r t e r than t h e time f o r a linesweep. I f t h e t i m e i s longer than f i v e samples, it may be i n f e r r e d t h a t t h e two f i x a t i o n s d e l i m i t an a r e a of bad measurement. To r e t a i n information about a r e a s of b l i n k c l u s t e r s and about a r e a s of bad measurement, t h e f i x a t i o n s d e l i m i t i n g t h e s e conditions a r e flagged i n t h e v e c t o r coding t h e p u p i l diameter. The f i x a t i o n preceding t h e bad a r e a cont a i n s t h e number of samples t h a t were l o s t ; t h e f i x a t i o n followinq t h e bad a r e a codes whether it w a s a b l i n k c l u s t e r o r unstable measurement t h a t caused t h e loss. T h i s information i s q u i t e important i n f u t u r e analyses
Eye-Movement Analysis
35
s i n c e it prevents wrong i n f e r e n c e s about information processing between t h e two f i x a t i o n s . For example, we most c e r t a i n l y must not conclude t h a t t h i s a r e a was skipped with a l a r g e saccade. MODULE TEXT
For eye movements c o l l e c t e d during reading, t h e described d a t a processing w a s o f t e n n o t s u f f i c i e n t t o achieve congruence between f i x a t i o n s and t h e a p p r o p r i a t e l i n e s of t h e t e x t paragraph. Reasons f o r t h e f a i l u r e could have been a temporarily i n v a l i d c a l i b r a t i o n t a b l e , g r e a t e r machine noise In a s s o c i a t e d with t h e v e r t i c a l p o s i t i o n , or a s u b j e c t ' s f i x a t i o n h a b i t s . g e n e r a l , even t h e presence of only a small systematic d r i f t may cause a g r e a t s i m i l a r i t y i n v e r t i c a l p o s i t i o n s of f i x a t i o n s a t opposite ends of successive l i n e s and may l e a d t o misplaced f i x a t i o n s . The TEXT module w a s w r i t t e n t o remove systematic d r i f t i n v e r t i c a l posit i o n within a l i n e ( s e e Figure 3 ) . The program compares successive p a i r s of f i x a t i o n s . I f t h e i r v e r t i c a l and h o r i z o n t a l p o s i t i o n s are within a c e r t a i n range t h e y a r e considered t o be from t h e same l i n e . I n t h i s c a s e , i f t h e i r v e r t i c a l p o s i t i o n s are n o t c l o s e t o i d e n t i c a l , t h e v e r t i c a l p o s i t i o n of t h e f i r s t f i x a t i o n i s replaced with t h e average of both. An unint e r u p t e d sequence of f i x a t i o n s judged t o be within t h e same l i n e w i l l be c a l l e d a c l u s t e r of f i x a t i o n s within a l i n e (FWL). The program i t e r a t i v e l y p r o c e s s e s t h e d a t a base u n t i l a l l f i x a t i o n s within FWL c l u s t e r s have v i r tually identical positions.
FLAG IS "TRUE" WHILE FLAG I S "TRUE" DO:
FLAG IS "FALSE" FOR N -
= 1 TO (NUMBER OF FIX -1) Do:
ARE N I AND (N
+
1) FIXATION ON THE SAME LINE?
YES: ARE VERTICAL POSITIONS IDENTICAt? YES: N = N NO:
+
1
VERT. POS. OF FIX. N = AVERAGE OF VERT. POS. OF FIX. N AND FIX. ( N + 1) FLAG = "TRUE"
N = N + l NO: END OF
END OF
FIGURE 3 .
N = N + I
FOR LOOP MxlP
Determining v e r t i c a l p o s i t i o n f o r f i x a t i o n s w i t h i n a l i n e
36
R.M. Kliegl
Once t h e i t e r a t i o n s t o p s it i s l i k e l y t h a t more FWL c l u s t e r s were ident i f i e d than t h e t e x t contains l i n e s . I f two f i x a t i o n s within a l i n e were separated by a l a r g e saccade o r i f t h e s u b j e c t jumped back t o re-read p a r t of t h e t e x t , sequences of f i x a t i o n s within one l i n e w i l l y i e l d s e p a r a t e FWL c l u s t e r s . To a r r i v e a t t h e a p p r o p r i a t e s e t , FWL c l u s t e r s i n d i c a t i n g t h e same l i n e s need t o be collapsed. Under d e f a u l t parameterization t h e program assumes a f i x e d minimum d i s tance between " t r u e " l i n e s and combines FWL c l u s t e r s t h a t a r e within t h i s margin. I d e a l l y , t h e number of l i n e s i d e n t i f i e d by t h e program should correspond t o t h e number of l i n e s t h e s u b j e c t read. A n immediate evaluat i o n of t h e q u a l i t y of t h e computations i s p o s s i b l e by inspecting tF. o l o t of f i x a t i o n s and t e x t which t h e program generates on t h e terminal. I? t h i s p l o t it i s immediately obvious i f t h e program found too many o ii: few l i n e s . I f t h e p l o t i s bad, t h e user has t h e option of t e l l i n g :..,' program t h e number of l i n e s i t should f i n d . This w i l l invoke another i t e r a t i v e procedure. A t t h e beginning, t h e minimum d i s t a n c e between l i n e s i s z e r o , b u t w i l l be slowly increased with every i t e r a t i o n u n t i l t h e d i s t a n c e has been i d e n t i f i e d t h a t g e n e r a t e s t h e d e s i r e d number of l i n e s . Again, a terminal p l o t may be used f o r e v a l u a t i v e purposes. I f t h e p l o t i s s t i l l not adequate, t h e u s e r has t h e o p t i o n of changing parameters t h a t determine whether f i x a t i o n p a i r s should be a l l o c a t e d t o a FWL c l u s t e r . A change i n t h e s e parameters w i l l r e s u l t i n a d i f f e r e n t s e t of FWL c l u s t e r s and, consequently, i n d i f f e r e n t l i n e values. I f t h e program f a i l s again, one should probably d e l e t e p a r t s of t h e t e x t f o r which congruence could not be achieved - or improve on t h e algorithms proposed i n t h i s paper. TECHNICAL INFORMATION EMAN w a s w r i t t e n i n VAX/VMS F o r t r a n 77 and i s implemented on DEC VAX 11/780 machines. Input t o t h e program a r e x,y-coordinates and p u p i l diameters c o l l e c t e d with a f i x e d sampling r a t e . The program reads a l l t h e records of a t r i a l i n t o t h r e e v e c t o r s . Two a d d i t i o n a l v e c t o r s s t o r e the number of samples i n a c l u s t e r and t h e record number of t h e f i r s t sample i n t h e clust e r . A t t h e beginning, t h e f i r s t vector i s a u n i t v e c t o r ; t h e second vect o r simply c o n t a i n s t h e record numbers. These f i v e v e c t o r s form a workspace matrix common t o a l l modules. A l l computations a r e c a r r i e d o u t on workspace elements with r e s u l t s r e p l a c i n g o r i g i n a l values. I n Module RESCALE X and y coordinates i n machine u n i t s a r e replaced with X and y coo r d i n a t e s of d i s p l a y dimensions. Modules REDUCE and CLEANUP e f f e c t e l e ments of a l l v e c t o r s . Whenever samples a r e combined i n a c l u s t e r , t h e number of samples and t h e i r average p o s i t i o n and p u p i l diameter a r e s t o r e d i n corresponding elements of t h e f i r s t sample of t h e c l u s t e r . Then, t h e s a m p l e s following t h e c l u s t e r a r e s h i f t e d up t h e a p p r o p r i a t e number of e l e ments. Thus, with every i t e r a t i o n t h e l e n g t h of t h e workspace v e c t o r s i s reduced. I t i s p o s s i b l e t o o b t a i n output a f t e r every i t e r a t i o n t o monitor t h e reduction and "cleaning" process. A f t e r d a t a were processed i n TEXT, t h e workspace v e c t o r s c o n t a i n number of samples, h o r i z o n t a l d i s p l a y posit i o n , l i n e number, p u p i l diameter ( o r a f l a g i n d i c a t i n g l o s s o r unstable measurement following/preceding t h e f i x a t i o n ) , and t h e record number of t h e f i r s t sample of t h i s f i x a t i o n . The l a t t e r permits immediate i d e n t i f i c a t i o n of t h e f i x a t i o n i n t h e raw d a t a . The o u t p u t i s s u i t a b l e f o r i n p u t t o programs FIXPLT and MATCH described i n K l i e g l (1981). S u b s t i t u t i o n of t h e TEXT module with algorithms designed f o r analyses of o t h e r than paragraphreading d a t a i s p o s s i b l e . Modules a v a i l a b l e are f o r analyses of trackinqa - l i g h t t a s k ( P a v l i d i s (1981), Olson, K l i e g l , & Davidson ( i n p r e s s ) ) and
Eye-Movement Analysis
31
t h e a n a l y s e s of single-word reading d a t a . Despite t h e l a r g e number of ite r a t i v e procedures t h e a n a l y s i s i s q u i t e f a s t . For example, an a n a l y s i s of eye movements c o l l e c t e d during reading of a 20-line long t e x t paragraph (6538 r e c o r d s , reduced t o 354 f i x a t i o n s ) was accomplished within one minute. The program is a v a i l a b l e upon r e q u e s t . REFERENCES K l i e g l , R . , Automated and i n t e r a c t i v e a n a l y s i s of eye f i x a t i o n d a t a in reading, Behavior Research Methods and Instrumentation 13 (1981) 115-1 20. K l i e g l , R . and Olson, R . K . , Reduction and c a l i b r a t i o n of eye monitor d a t a , Behavior Research Methods and Instrumentation 13 (1981) 107-111. Mason, R. L . , D i g i t a l computer e s t i m a t i o n of eye f i x a t i o n , Behavior R e s e a r c h Methods and Instrumentation 8 (1976) 185-188. Olson, R . K . , K l i e g l , R. and Davidson, B. J . , Dyslexic and normal r e a d e r ' s eye movements, J o u r n a l of Experimental Psychology: Human Perception and Performance ( i n p r e s s ) . P a v l i d i s , G. Th., Do eye movements hold t h e key t o d y s l e x i a ? , Neuropsychol o g i a 19 (1981) 57-64.