- Email: [email protected]
Use of the Nac Eye Mark Recorder to Study Visual Strategies of Military Aircraft Pilots
Theoretical and Applied Aspects of Eye Movement Research A.G. Gale and F. Johnson (Editors) 0 E h i e r Science PublishersB.V. (North-Holland),1984
367
USE OF THE NAC EYE MARK RECORDER TO STUDY VISUAL STRATEGIES OF MILITARY AIRCRAFT PILOTS
Jean-Paul PAPIN
Centre d'Etudes e t de Recherches de Medecine Aerospatiale
5 bis avenue de l a Porte de Sevres 75731 PARIS CEDEX 15
SUMMARY Recording of gaze direction of experienced a i r c r a f t pilots with the NAC EYE MARK RECORDER permits describing an optimal visual behavior i n instrument and visual f l i g h t . Results lead t o suggestions for the design and correction ergonomics of new visual information display modes. An analysis of visual behaviors of p i l o t students shows that the acquisition of the optimal behavior i s progressive and unconscious. To improve the training of these students, i t i s suggested t o use gaze direction recordings in two ways : one is t o show students the optimal behavior of expert p i l o t s , the other i s t o show students t h e i r own behavior.
INTRODUCTION The f i r s t recordings of visual behavior of a i r c r a f t p i l o t s were made by Tiff i n and Bromer as early as 1942, and many studies followed, b o t h 2n a i r c r a f t (Milton, 1952 ; Llewellyn Thomas, 1963 ; Spady, 1977) and on helicopter (Barnes, 1972 ; Frezell e t Hofmann, 1975). However, t h i s type of study remains of current i n t e r e s t . Indeed, the use of color TV screens or electronic head-up o r down displays t o provide modern a i r c r a f t p i l o t s w i t h the information he needs to f l y raised a double question : an ergonomic question : how t o provide the crew with only t h a t i n formation needed a t a given moment of the f l i g h t and i n the best possible form. a pedagogical question : how t o teach users e f f i c i e n t information pick-up and processing. To reach these goals i t i s necessary t o know : the nature of visual information really used d u r i n g the various f l i g h t phases ;
.
. .
. Whether
there are prefential visual strategies to pick-up and process t h i s information ; f i n a l l y , whether i t is possible t o describe, explain and teach these strategies i f they e x i s t . To answer these questions i t i s necessary t o know, a t every instant, what a p i l o t looks a t and perceives i n his surrounding environment. This i s made possible by the continuous recording of the projection of the gaze axis on the image of the visual environment of a subject. In the f i e l d of aeronaut i c s , one of the preferred techniques i s the use of a photooculograph derived from Mackworth and Mackworth's studies (1958) : the NAC EYE MARK RECORDER. This device, now well known, was used by the Ce&e d ' E & ~ d u c?,t de Rechached
.
J.-P.Papin
368
de MEdecine A E h o o p d d e (Aerospace M e d i c i n e r e s e a r c h C e n t e r ) i n P a r i s t o
conduct s e v e r a l experiments. Some o f these experiments a r e r e p o r t e d i n t h i s paper. I
-
ANALYSIS OF VISUAL STRATEGIES USED BY CONFIRMED PILOTS 1-1.
-
Instrument f l y i n g
A s t u d y conducted i n 1980 by Papin e,t d ,i n a f i g h t e r a i r c r a f t simulator consisted i n recording the v i s u a l behavior o f twelve p i l o t s r e p r e s e n t a t i v e o f a reconnaissance squadron. A l l p i l o t s had f l o w n 500 t o 2500 hours. They had t o r e p e a t f i v e t i m e s a v e r y s t e r e o t y p e d t a s k : a ground c o n t r o l approach. R e s u l t s show t h a t p i l o t s who a r e v e r y f a m i l i a r w i t h t h i s t a s k have good p e r formance scores t h r o u g h o u t a1 1 t e s t s . The a n a l y s i s o f e x p l o r a t o r y v i s u a l b e h a v i o r s shows t h a t t h e r e a r e no s i g n i ficant inter o r i n t r a - i n d i v i d u a l d i f f e r e n c e s . There i s a common v i s u a l b e h a v i o r and t h e s t r a t e g y o f v i s u a l e x p l o r a t i o n may be c o n s i d e r e d as o p t i mal, even i f a b e t t e r one may e x i s t , s i n c e t h e performance l e v e l remains s a t is f a c t o ry W i s b e h a v i o r i s c h a r a c t e r i z e d b y t h e f o l l o w i n g f a c t s . The i n s t r u m e n t panel i s explored i n a s t a r - l i k e p a t t e r n from a prefered instrument : p o s i t i o n i n d i c a t o r w h i c h p r o v i d e s i n f o r m a t i o n about t h e a i r c r a f t p o s i t i o n i n space and heading. The p i l o t ' s gaze s t o p s on t h e p o s i t i o n i n d i c a t o r , s h i f t s t o a n o t h e r i n s t r u m e n t , t h e n comes back t o t h e p o s i t i o n i n d i c a t o r s h i f t i n g a g a i n t o a n o t h e r i n d i c a t o r . Sometimes, gaze s t o p s on an i n s t r u m e n t c l o s e t o t h e i n s trument which was j u s t checked b e f o r e s h i f t i n g back t o t h e p o s i t i o n i n d i c a t o r . Whatever t h e case may be, gaze never l e a v e s t h e p o s i t i o n i n d i c a t o r more t h a n t h r e e seconds, and i n 70 % o f cases more t h a n one second. However, gaze may s h i f t f r o m motor i n d i c a t o r s f o r s e v e r a l seconds. The t o t a l amount o f t i m e s p e n t c h e c k i n g t h e p o s i t i o n i n d i c a t o r accounts f o r 64 % o f t h e t o t a l t e s t t i m e . It i s a l s o p o s s i b l e t o c l a s s i f y i n t r u m e n t s as a f u n c t i o n o f t h e amount o f t i m e s p e n t checking them o r o f t h e number o f times t h e y were checked. We w i l l c a l l these " l o o k - f i x a t i o n s " f o r two reasons. First;tly, because when d a t a i s analysed, what i s c o n s i d e r e d t h e sum o f a l l s m a l l s h i f t s occ u r i n g when gaze s h i f t s back t o a d i a l r a t h e r t h a n each i n d i v i d u a l " f i x a t i o n " . Second, because i n most cases, these l o o k - f i x a t i o n s a r e l i k e l y t o be a s s o c i a t e d w i t h t h e p i c k - u p o f i n f o r m a t i o n , o r a t l e a s t , i n f o r m a t i o n p i c k - u p was p o s s i b l e . I n a d d i t i o n , as t h e r e i s a s t r o n g c o r r e l a t i o n between t h e d u r a t i o n and t h e number o f f i x a t i o n s , i t becomes p o s s i b l e t o express r e s u l t s as l o o k - f i x a t i o n r a t e s f o r v a r i o u s i n s t r u m e n t s . I n t h e p r e s e n t case, i t i s p o s s i b l e t o e s t a b l i s h a hierarchy f o r the various instruments : p o s i t i o n i n d i c a t o r , airspeed, a l t i m e t e r , v e r t i c a l speed, motor i n d i c a t o r s and stand-by h o r i z o n . Another i m p o r t a n t f e a t u r e o f t h e s e r e s u l t s i s t h e average d u r a t i o n o f t h e s e l o o k - f i x a t i o n s . An a n a l y s i s b y i n t e r v i e w s evidenced t h a t t h e y depend on t h e n a t u r e o f t h e i n f o r m a t i o n t o be p i c k e d up. A l o o k - f i x a t i o n o f l e s s t h a n 100-150 m i l l i s e c o n d s i s n o t l o n g enough t o r e a d a d i a l . I n a 100-150 m i l l i second l o o k - f i x a t i o n t h e p i l o t can check t h a t a hand has n o t moved. He needs 150-200 m i 11iseconds t o p e r c e i v e a move and 200-250 m i 11iseconds t o assess t h e a n g u l a r v a l u e o f t h i s move. F i n a l l y , a p i l o t needs a p p r o x i m a t e l y 400 m i l l i s e c o n d s t o r e a d a d i g i t value. These r e s u l t s c o n f i r m t h a t t o be c o r r e c t , t h e p i c k - u p o f i n f o r m a t i o n on an i n s t r u m e n t panel must be a s s o c i a t e d w i t h s y s t e m a t i c and c o n t i n u o u s e x p l o r a t i o n ( i n f o r m a t i o n on d i a l s which a r e n o t checked i s n o t perceived).An e f f i c i e n t method i s t o e x p l o r e an area i n a s t a r - l i k e p a t t e r n f r o m a p r e f e r e d s i t e t o which gaze comes back f r e q u e n t l y . F i n a l l y , i n f o r m a t i o n i s more r a p i d l y p i c k e d up on symbol d i s p l a y s t h a n on alphanumeric d i s p l a y s .
-
.
Visual Strategies of Pilots
369
1-2. Visual f l y i n g
The v i s u a l s t r a t e g y o f confirmed p i l o t s i n v sual f l y i n g was analysed during a research study (Papin eA d.,1981 designed t o assess the p o s s i b i l i t y of f l y i n g a t a c t i c a l mission on a he i c o p t e r w i t h reduced v i s u a l f i e l d and monocular vision. Such f l y i n g - c o n d i t i o n s are encountered when v i s u a l aids are used i n n i g h t f l i g h t . The image o f the landscape r e corded by an i n f r a r e d camera placed outside o f the c o c k p i t appears on a m i n i TV screen attached t o the p i l o t ' s eyes. I n order t o simulate the r e duced v i s u a l f i e l d , a s e r i e s o f masks i s used which provides e i t h e r binocul a r o r monocular v i s i o n w i t h v i s u a l f i e l d s o f 60", 40" o r 20". This system p l u s the NAC were used t o monitor the gaze d i r e c t i o n o f seven p i l o t s w i t h a good experience o f t a c t i c a l f l i g h t during a r e a l f l i g h t during daytime. The 20-minute mission consisted i n searching a p o t e n t i a l ennemy i n a h i l l y t e r r a i n w h i l e keeping away from the enewy's s i g h t and blows. Recordedgaze s h i f t s were analysed e i t h e r as a f u n c t i o n o f objects looked a t outside the c o c k p i t o r as a f u n c t i o n o f the p o s i t i o n o f gaze p r o j e c t i o n on the he1i c o p t e r cockpit. Results show t h a t a l l p i l o t s have a nearly i d e n t i c a l v i s u a l behavior f o r a given c o n d i t i o n o f v i s i o n and f o r a f l i g h t over a p a r t i c u l a r type o f t e r r a i n . One o f the most remarkable f a c t s i s t h a t whatever the conditions may be, v i s u a l e x p l o r a t i o n i s organized i n space r e l a t i v e t o the canopy. Gaze s h i f t s from one area o f the cockpit t o another, i n a starshaped p a t t e r n from the area which corresponds t o gaze p r o j e c t i o n on the canopy when the p i l o t i s a t r e s t and looks s t r a i g h t i n f r o n t o f him. The only case i n which gaze s h i f t s depend on the type o f o b j e c t looked a t i s when the eye encounters a t r e e o r the border o f a f o r e s t . I n t h i s case, i t t r a v e l s back and f o r t h several times from the t r e e top t o the bottom. Another important i t e m i s t h a t most l o o k - f i x a t i o n s tend t o concentrate i n a r e l a t i v e l y small area around the area which i s again and again looked a t under normal v i s u a l conditions. The area where 80 % o f l o o k - f i x a t i o n s are concentrated on the canopy corresponds t o the base o f a cone whose p o i n t i s the p i l o t ' s head and which has 30" aperture angle. When the v i s u a l f i e l d shrinks t o concent r a t e 80 % o f l o o k - f i x a t i o n s , the angle o f t h i s cone increases. It reaches 60" when the f l i g h t t e s t i s conducted i n monocular v i s i o n w i t h a 40" v i s u a l f i e l d . Concurrently, an increase i n the number o f l o o k - f i x a t i o n s on the lower h a l f o f the c o c k p i t and on the instruments i s observed. This change i n behavior suggests t h a t p i l o t s search i n foveal v i s i o n i n f o r m a t i o n which they u s u a l l y capture i n peripheral v i s i o n and which g i v e them i n f o r m a t i o n on t h e i r o r i e n t a t i o n , t h e i r a l t i t u d e , and airspeed. Actually, t o compensate t h i s l a c k o f information, they observe more f r e q u e n t l y the instruments which provide information on these parameters. Such observations, i.e. pick-up o f information outside the c o c k p i t clues and increased checking o f c e r t a i n parameters i n monocular vision, had i m mediate ergonomic a p p l i c a t i o n .
-
Ergonomic a p p l i c a t i o n s During the f i r s t f l i g h t s flown w i t h the prototype system where the image o f the landscape recorded by a camera placed outside the c o c k p i t appears on a TV screen w i t h o u t c o c k p i t clues, the p i l o t seemed t o be conf r o n t e d w i t h s p a t i a l o r i e n t a t i o n , speed, a l t i t u d e and distance evaluation d i f f i c u l t i e s . It was therefore recommended t o show on the image seen by the p i l o t bars corresponding t o the v e r t i c a l and h o r i z o n t a l frame o f the canopy and a l s o t o superimpose on the image heading, airspeed, a l t i t u d e and changing c l i m and descent speeds. These changes added during experiments seemed t o t a l l y satisfactory. 1-3.
370
I1
-
L-R Papin ANALYSIS OF VISUAL STRATEGIES USED BY TRAINING PILOTS
The r e p o r t e d example concerns t h e f o l l o w - u p o f t h e e v o l u t i o n o f t h e v i s u a l b e h a v i o r o f s i x m i l i t a r y a i r c a r r i e r s t u d e n t p i l o t s . Gaze d i r e c t i o n o f these p i l o t s was r e c o r d e d a t r e g u l a r i n t e r v a l s d u r i n g t h e i r 18-month t r a i n i n g program, d u r i n g r e a l o r s i m u l a t e d f l i g h t s , b u t always d u r i n g t h e same "Rodeotacan" t a s k . T h i s t a s k of a p p r o x i m a tely 17 minutes encompasses a l l p o s s i b l e f l i g h t phases. Resu lt s were compared among each o t h e r and a l s o w i t h those o f a c o n t r o l group o f f l i g h t i n s t r u c t o r s who performed t h e same t a s k . R e s u l t s show t h a t a l l s t u d e n t s d i s p l a y a common c h a r c t e r i s t i c b e h avior a t each t r a i n i n g phase. A t t h e begin n i n g o f t h e t r a i n i n g programm, s t u d e n t p i l o t s e x p l o r e d t h e i n s t r u m e n t panel i n a s t a r - l i k e p a t t e r n s t a r t i n g from t h e p i t c h i n d i c a t o r which corresponds t o t h e p o s i t i o n i n d i c a t o r i n combat a i r c r a f t . However they o f t e n lo ok ed back and f o r t h between two i n st rument s, s t o p p i n g f o r a p r a c t i c a l l y i d e n t i c a l l y s h o r t t i m e on each one. Some omissions were a l s o recorded. As t h e s t u d e n t s progressed, t h e number o f back and f o r t h gaze s h i f t s decreased w h i l e t h e d u r a t i o n o f l o o k - f i x a t i o n s increased. Omissions g r a d u a l l y disappeared and l o o k - f i x a t i o n on t h e most i m p o r t a n t i n d i c a t o r s f o r a g i v e n f l i g h t phase became l o n g e r . T h i s f i n a l behavior i s s p e c i f i c t o f l i g h t i n s t r u c t o r s . E v i d e n t l y , t h e a c q u i s i t i o n o f t h i s behavior i s a slow, t o t a l l y unconscious process. The q u e s t i o n i s t o know whether i t may be p o s s i b l e t o a c t i v e l y i n f l u e n c e t h i s a c q u i s i t i o n process. I11
- USE OF GAZE DIRECTION RECORDINGS I N PILOT TRAINING PROGRAMS
To i n f l u e n c e a c t i v e l y t h e a c q u i s i t i o n o f o p t i m a l v i s u a l e x p l o r a t i o n s t r a t e g i e s , two axes o f r e s e a r c h a r e s i m u l t a n e o u s l y f l o l l o w e d . The f i r s t c o n s i s t s i n c r e a t i n g t e a c h i n g m a t e r i a l which shows and e x p l a i n s these behaviors. I n o r d e r t o do t h i s , once t h e v i s u a l behavior o f e x p e r t p i l o t s has been re c o r d e d d u r i n g r e a l o r s i m u l a t e d f l i g h t , a document i s prepared which a1 t e r n a t e s dynamic p r e s e n t a t i o n o f these behaviors, e x p l a n a t o r y sketches and consequences o f most f r e q u e n t inadequate behaviors , e x h i b i t e d bu s t u d e n t p i l o t s . Such documents have been made, i n c o l o r , f o r t h e v a r i o u s phases o f h e l i c o p t e r f l i g h t i n s t r u c t i o n , b o t h o v e r c o u n t r y s i d e and mountainous t e r r a i n . However, t hey have n o t been d i s t r i b u t e d t o s t u d e n t p i l o t s s i n c e t h e r e s u l t s o b t a i n e d r e c e n t l y i n r a d i o l o g y ( P m a l b e r t i e l at, 1983) and a v i a t i o n s t u d i e s (Spady, 1982) show t h a t alone, v i e w i n g t h e o p t i m a l behavior i s n o t e f f i c i e n t . What i s e f f i c i e n t i s e i t h e r t o show t h e s t u d e n t s t h e i r own behavior, o r t o l e t them p a r t i c i p a t e a c t i v e l y when t h e f i l m i s shown. To o b t a i n such a c t i v e p a r t i c i p a t i o n d u r i n g f i l m viewing, s p e c i a l scenes must be prepared and we a r e c u r r e n t l y p r e p a r i n g them f o r h e l i c o p t e r f l i g h t i n s t r u c t i o n . The second a x i s f o l l o w e d i n t h i s t e a c h i n g r e s e a rch i s t o l e t s t u d e n t s object i v a t e t h e i r b ehav i o r . Such an approach i s p r o m i s i n g b u t r a i s e s numerous problems. Indeed, i n i t s p r e s e n t c o n f i g u r a t i o n , t h e NAC i s n o t adapted f o r stude n t s who a r e s t a t i n g t h e i r f l i g h t i n s t r u c t i o n o r f o r s i n g l e - s e a t e r comb a t a i r c r a f t s t u den t s . However, t h e method can be used i n a s i m u l a t o r t o h e l p st udent s a c q u i r e emergengy procedures as we demonstrated i t i n a s t u d y on h e l i c o p t e r simul a t o r o r as Spady (1982) showed i t on a Boeing 737 s i m u l a t o r . T h i s method cans a l s o be used i n r e a l f l i g h t when a p i l o t i s t r a n s f e r r e d t o a new h e l i c o p t e r o r a new a i r c r a f t . We c o u l d t h u s h e l p tremendously an a i r c a r r i e r p i l o t . I n s t r u c t o r s were c o n f r o n t e d w i t h t h e a l t e r n a t i v e t o e i t h e r dismiss t h e s t u d e n t o r make him r e p e a t h i s courses. An i n f l i g h t r e c o r d i n g helped t h e
Visual Strategies of Pilots
37 1
p i l o t i d e n t i f y h i s e r r o r s and t r u s t h i s i n s t r u c t o r s again. He could corr e c t h i s e r r o r s q u i c k l y and catch up on what he had missed. This c l i n i c a l observation i s very i n t e r e s t i n g b u t i t i s d i f f i c u l t t o make a generalizat i o n since actions must be punctual and r e q u i r e g r e a t a v a i l a b i l i t y . NOW, what should be done i s t o go from research t o p r a c t i c a l application. 'Ib ck t h i s s p e c i a l i s t s must be t r a i n e d and special equipment must be acquired. I t would c e r t a i n l y be a long and expensive process. CONCLUSION The NAC EYE MARK Recorder i s a valuable t o o l f o r the study o f the v i s u a l behavior o f confirmed p i l o t s i n v i s u a l f l i g h t on h e l i c o p t e r o r a i r c a r r i e r and a l l a i r c r a f t types i n simulators. Such studies h e l p s o l v i n g ergonomic c o r r e c t i o n and design problems. The NAC can be used e f f i c i e n t l y t o prepare t r a i n i n g m a t e r i a l f o r p i l o t s t u dents, and t o help c e r t a i n categories o f f a i l i n g students. However, i t should n o t be used t o analyse p i l o t ' s work i n a s i n g l e seater f i g h t e r a i r c r a f t i n r e a l f l i g h t nor t o help beginner students. I n order t o go f u r ther i n t h i s type o f approach i n the f i e l d o f a v i a t i o n , i t w i l l be necessary t o w a i t f o r the development o f recording equipment b e t t e r adapted t o t h i s task.
-
REFERENCES
-
BARNES J.A. : "Analysis o f p i l o t s eye movements during h e l i c o p t e r f l i g h t " Tech. memo. 11-72, Human engineering laboratory, a p r i l 1972.
-
FREZELL T.L., HOFMANN M.A. : Comparison o f v i s u a l performance o f monocul a r and binocular v i s i o n during VHR h e l i c o p t e r f l i g h t . Aerospace Medic a l pane1,Agard conference pr0ceedingG.B. Val CP 1982, n04 10/1975.
-
MACKWORTH Y.F., MACKWORTH N.M. : Eye f i x a t i o n s recorded on changing v i s u a l scenes by the t e l e v i s i o n eye-marker. J. opt. SOC. Amer., 1958.
-
MILTON J.L. : Analysis o f p i l o t s eye movements i n f l i g h t . A v i a t i o n Medecine, February 1952.
-
P A P I N J.P., NAUREILS P., SANTUCCI G. :Pickup o f v i s u a l i n f o r m a t i o n by the p i l o t during a ground c o n t r o l approach i n a f i g h t e r a i r c r a f t simulator. A v i a t i o n space, and environnemental medicine, may 1980.
-
-
-
PAPIN J.P., WIATZ A., GANGLOFF B. : La d i r e c t i o n du regard e t des alarmes v i s u e l l e s sur simulateur PUMA 5A 330, Rapport CERMA 80-12, J u i n 80. PAPIN J.P., MENU J.P., SANTUCCI G. : Vision monoculaire en vol t a c t i q u e s u r h 6 l i c o p t e r e . Agard CP 312, mai 1981.
.
SPADY A., AMOS : A i r l i n e p i l o t scanning behavior during approaches and landing i n Boeing 737 simulator, Agard CP 240, A p r i l 1978. SPADY A., JONIS D., COATES G., KIRBY R. : Effectiveness o f using r e a l time eye scanning information f o r p i l o t s t r a i n i n g . Proceding HF Society 1982.
THOMAS E., LLEWELLYN, The eye movements of a p i l o t during a i r c r a f t Landing. Aerospace Medicine, May 1963. TIFFIN J., BROMES : Analysis o f eye f i x a t i o n s and p a t t e r n s o f eye mowements i n landing a Piper cub J 3 a i r plane. Report no 10 d i v i s i o n o f research c i v i l aeronautics adminstration, Washington DC, February 1943.
367
USE OF THE NAC EYE MARK RECORDER TO STUDY VISUAL STRATEGIES OF MILITARY AIRCRAFT PILOTS
Jean-Paul PAPIN
Centre d'Etudes e t de Recherches de Medecine Aerospatiale
5 bis avenue de l a Porte de Sevres 75731 PARIS CEDEX 15
SUMMARY Recording of gaze direction of experienced a i r c r a f t pilots with the NAC EYE MARK RECORDER permits describing an optimal visual behavior i n instrument and visual f l i g h t . Results lead t o suggestions for the design and correction ergonomics of new visual information display modes. An analysis of visual behaviors of p i l o t students shows that the acquisition of the optimal behavior i s progressive and unconscious. To improve the training of these students, i t i s suggested t o use gaze direction recordings in two ways : one is t o show students the optimal behavior of expert p i l o t s , the other i s t o show students t h e i r own behavior.
INTRODUCTION The f i r s t recordings of visual behavior of a i r c r a f t p i l o t s were made by Tiff i n and Bromer as early as 1942, and many studies followed, b o t h 2n a i r c r a f t (Milton, 1952 ; Llewellyn Thomas, 1963 ; Spady, 1977) and on helicopter (Barnes, 1972 ; Frezell e t Hofmann, 1975). However, t h i s type of study remains of current i n t e r e s t . Indeed, the use of color TV screens or electronic head-up o r down displays t o provide modern a i r c r a f t p i l o t s w i t h the information he needs to f l y raised a double question : an ergonomic question : how t o provide the crew with only t h a t i n formation needed a t a given moment of the f l i g h t and i n the best possible form. a pedagogical question : how t o teach users e f f i c i e n t information pick-up and processing. To reach these goals i t i s necessary t o know : the nature of visual information really used d u r i n g the various f l i g h t phases ;
.
. .
. Whether
there are prefential visual strategies to pick-up and process t h i s information ; f i n a l l y , whether i t is possible t o describe, explain and teach these strategies i f they e x i s t . To answer these questions i t i s necessary t o know, a t every instant, what a p i l o t looks a t and perceives i n his surrounding environment. This i s made possible by the continuous recording of the projection of the gaze axis on the image of the visual environment of a subject. In the f i e l d of aeronaut i c s , one of the preferred techniques i s the use of a photooculograph derived from Mackworth and Mackworth's studies (1958) : the NAC EYE MARK RECORDER. This device, now well known, was used by the Ce&e d ' E & ~ d u c?,t de Rechached
.
J.-P.Papin
368
de MEdecine A E h o o p d d e (Aerospace M e d i c i n e r e s e a r c h C e n t e r ) i n P a r i s t o
conduct s e v e r a l experiments. Some o f these experiments a r e r e p o r t e d i n t h i s paper. I
-
ANALYSIS OF VISUAL STRATEGIES USED BY CONFIRMED PILOTS 1-1.
-
Instrument f l y i n g
A s t u d y conducted i n 1980 by Papin e,t d ,i n a f i g h t e r a i r c r a f t simulator consisted i n recording the v i s u a l behavior o f twelve p i l o t s r e p r e s e n t a t i v e o f a reconnaissance squadron. A l l p i l o t s had f l o w n 500 t o 2500 hours. They had t o r e p e a t f i v e t i m e s a v e r y s t e r e o t y p e d t a s k : a ground c o n t r o l approach. R e s u l t s show t h a t p i l o t s who a r e v e r y f a m i l i a r w i t h t h i s t a s k have good p e r formance scores t h r o u g h o u t a1 1 t e s t s . The a n a l y s i s o f e x p l o r a t o r y v i s u a l b e h a v i o r s shows t h a t t h e r e a r e no s i g n i ficant inter o r i n t r a - i n d i v i d u a l d i f f e r e n c e s . There i s a common v i s u a l b e h a v i o r and t h e s t r a t e g y o f v i s u a l e x p l o r a t i o n may be c o n s i d e r e d as o p t i mal, even i f a b e t t e r one may e x i s t , s i n c e t h e performance l e v e l remains s a t is f a c t o ry W i s b e h a v i o r i s c h a r a c t e r i z e d b y t h e f o l l o w i n g f a c t s . The i n s t r u m e n t panel i s explored i n a s t a r - l i k e p a t t e r n from a prefered instrument : p o s i t i o n i n d i c a t o r w h i c h p r o v i d e s i n f o r m a t i o n about t h e a i r c r a f t p o s i t i o n i n space and heading. The p i l o t ' s gaze s t o p s on t h e p o s i t i o n i n d i c a t o r , s h i f t s t o a n o t h e r i n s t r u m e n t , t h e n comes back t o t h e p o s i t i o n i n d i c a t o r s h i f t i n g a g a i n t o a n o t h e r i n d i c a t o r . Sometimes, gaze s t o p s on an i n s t r u m e n t c l o s e t o t h e i n s trument which was j u s t checked b e f o r e s h i f t i n g back t o t h e p o s i t i o n i n d i c a t o r . Whatever t h e case may be, gaze never l e a v e s t h e p o s i t i o n i n d i c a t o r more t h a n t h r e e seconds, and i n 70 % o f cases more t h a n one second. However, gaze may s h i f t f r o m motor i n d i c a t o r s f o r s e v e r a l seconds. The t o t a l amount o f t i m e s p e n t c h e c k i n g t h e p o s i t i o n i n d i c a t o r accounts f o r 64 % o f t h e t o t a l t e s t t i m e . It i s a l s o p o s s i b l e t o c l a s s i f y i n t r u m e n t s as a f u n c t i o n o f t h e amount o f t i m e s p e n t checking them o r o f t h e number o f times t h e y were checked. We w i l l c a l l these " l o o k - f i x a t i o n s " f o r two reasons. First;tly, because when d a t a i s analysed, what i s c o n s i d e r e d t h e sum o f a l l s m a l l s h i f t s occ u r i n g when gaze s h i f t s back t o a d i a l r a t h e r t h a n each i n d i v i d u a l " f i x a t i o n " . Second, because i n most cases, these l o o k - f i x a t i o n s a r e l i k e l y t o be a s s o c i a t e d w i t h t h e p i c k - u p o f i n f o r m a t i o n , o r a t l e a s t , i n f o r m a t i o n p i c k - u p was p o s s i b l e . I n a d d i t i o n , as t h e r e i s a s t r o n g c o r r e l a t i o n between t h e d u r a t i o n and t h e number o f f i x a t i o n s , i t becomes p o s s i b l e t o express r e s u l t s as l o o k - f i x a t i o n r a t e s f o r v a r i o u s i n s t r u m e n t s . I n t h e p r e s e n t case, i t i s p o s s i b l e t o e s t a b l i s h a hierarchy f o r the various instruments : p o s i t i o n i n d i c a t o r , airspeed, a l t i m e t e r , v e r t i c a l speed, motor i n d i c a t o r s and stand-by h o r i z o n . Another i m p o r t a n t f e a t u r e o f t h e s e r e s u l t s i s t h e average d u r a t i o n o f t h e s e l o o k - f i x a t i o n s . An a n a l y s i s b y i n t e r v i e w s evidenced t h a t t h e y depend on t h e n a t u r e o f t h e i n f o r m a t i o n t o be p i c k e d up. A l o o k - f i x a t i o n o f l e s s t h a n 100-150 m i l l i s e c o n d s i s n o t l o n g enough t o r e a d a d i a l . I n a 100-150 m i l l i second l o o k - f i x a t i o n t h e p i l o t can check t h a t a hand has n o t moved. He needs 150-200 m i 11iseconds t o p e r c e i v e a move and 200-250 m i 11iseconds t o assess t h e a n g u l a r v a l u e o f t h i s move. F i n a l l y , a p i l o t needs a p p r o x i m a t e l y 400 m i l l i s e c o n d s t o r e a d a d i g i t value. These r e s u l t s c o n f i r m t h a t t o be c o r r e c t , t h e p i c k - u p o f i n f o r m a t i o n on an i n s t r u m e n t panel must be a s s o c i a t e d w i t h s y s t e m a t i c and c o n t i n u o u s e x p l o r a t i o n ( i n f o r m a t i o n on d i a l s which a r e n o t checked i s n o t perceived).An e f f i c i e n t method i s t o e x p l o r e an area i n a s t a r - l i k e p a t t e r n f r o m a p r e f e r e d s i t e t o which gaze comes back f r e q u e n t l y . F i n a l l y , i n f o r m a t i o n i s more r a p i d l y p i c k e d up on symbol d i s p l a y s t h a n on alphanumeric d i s p l a y s .
-
.
Visual Strategies of Pilots
369
1-2. Visual f l y i n g
The v i s u a l s t r a t e g y o f confirmed p i l o t s i n v sual f l y i n g was analysed during a research study (Papin eA d.,1981 designed t o assess the p o s s i b i l i t y of f l y i n g a t a c t i c a l mission on a he i c o p t e r w i t h reduced v i s u a l f i e l d and monocular vision. Such f l y i n g - c o n d i t i o n s are encountered when v i s u a l aids are used i n n i g h t f l i g h t . The image o f the landscape r e corded by an i n f r a r e d camera placed outside o f the c o c k p i t appears on a m i n i TV screen attached t o the p i l o t ' s eyes. I n order t o simulate the r e duced v i s u a l f i e l d , a s e r i e s o f masks i s used which provides e i t h e r binocul a r o r monocular v i s i o n w i t h v i s u a l f i e l d s o f 60", 40" o r 20". This system p l u s the NAC were used t o monitor the gaze d i r e c t i o n o f seven p i l o t s w i t h a good experience o f t a c t i c a l f l i g h t during a r e a l f l i g h t during daytime. The 20-minute mission consisted i n searching a p o t e n t i a l ennemy i n a h i l l y t e r r a i n w h i l e keeping away from the enewy's s i g h t and blows. Recordedgaze s h i f t s were analysed e i t h e r as a f u n c t i o n o f objects looked a t outside the c o c k p i t o r as a f u n c t i o n o f the p o s i t i o n o f gaze p r o j e c t i o n on the he1i c o p t e r cockpit. Results show t h a t a l l p i l o t s have a nearly i d e n t i c a l v i s u a l behavior f o r a given c o n d i t i o n o f v i s i o n and f o r a f l i g h t over a p a r t i c u l a r type o f t e r r a i n . One o f the most remarkable f a c t s i s t h a t whatever the conditions may be, v i s u a l e x p l o r a t i o n i s organized i n space r e l a t i v e t o the canopy. Gaze s h i f t s from one area o f the cockpit t o another, i n a starshaped p a t t e r n from the area which corresponds t o gaze p r o j e c t i o n on the canopy when the p i l o t i s a t r e s t and looks s t r a i g h t i n f r o n t o f him. The only case i n which gaze s h i f t s depend on the type o f o b j e c t looked a t i s when the eye encounters a t r e e o r the border o f a f o r e s t . I n t h i s case, i t t r a v e l s back and f o r t h several times from the t r e e top t o the bottom. Another important i t e m i s t h a t most l o o k - f i x a t i o n s tend t o concentrate i n a r e l a t i v e l y small area around the area which i s again and again looked a t under normal v i s u a l conditions. The area where 80 % o f l o o k - f i x a t i o n s are concentrated on the canopy corresponds t o the base o f a cone whose p o i n t i s the p i l o t ' s head and which has 30" aperture angle. When the v i s u a l f i e l d shrinks t o concent r a t e 80 % o f l o o k - f i x a t i o n s , the angle o f t h i s cone increases. It reaches 60" when the f l i g h t t e s t i s conducted i n monocular v i s i o n w i t h a 40" v i s u a l f i e l d . Concurrently, an increase i n the number o f l o o k - f i x a t i o n s on the lower h a l f o f the c o c k p i t and on the instruments i s observed. This change i n behavior suggests t h a t p i l o t s search i n foveal v i s i o n i n f o r m a t i o n which they u s u a l l y capture i n peripheral v i s i o n and which g i v e them i n f o r m a t i o n on t h e i r o r i e n t a t i o n , t h e i r a l t i t u d e , and airspeed. Actually, t o compensate t h i s l a c k o f information, they observe more f r e q u e n t l y the instruments which provide information on these parameters. Such observations, i.e. pick-up o f information outside the c o c k p i t clues and increased checking o f c e r t a i n parameters i n monocular vision, had i m mediate ergonomic a p p l i c a t i o n .
-
Ergonomic a p p l i c a t i o n s During the f i r s t f l i g h t s flown w i t h the prototype system where the image o f the landscape recorded by a camera placed outside the c o c k p i t appears on a TV screen w i t h o u t c o c k p i t clues, the p i l o t seemed t o be conf r o n t e d w i t h s p a t i a l o r i e n t a t i o n , speed, a l t i t u d e and distance evaluation d i f f i c u l t i e s . It was therefore recommended t o show on the image seen by the p i l o t bars corresponding t o the v e r t i c a l and h o r i z o n t a l frame o f the canopy and a l s o t o superimpose on the image heading, airspeed, a l t i t u d e and changing c l i m and descent speeds. These changes added during experiments seemed t o t a l l y satisfactory. 1-3.
370
I1
-
L-R Papin ANALYSIS OF VISUAL STRATEGIES USED BY TRAINING PILOTS
The r e p o r t e d example concerns t h e f o l l o w - u p o f t h e e v o l u t i o n o f t h e v i s u a l b e h a v i o r o f s i x m i l i t a r y a i r c a r r i e r s t u d e n t p i l o t s . Gaze d i r e c t i o n o f these p i l o t s was r e c o r d e d a t r e g u l a r i n t e r v a l s d u r i n g t h e i r 18-month t r a i n i n g program, d u r i n g r e a l o r s i m u l a t e d f l i g h t s , b u t always d u r i n g t h e same "Rodeotacan" t a s k . T h i s t a s k of a p p r o x i m a tely 17 minutes encompasses a l l p o s s i b l e f l i g h t phases. Resu lt s were compared among each o t h e r and a l s o w i t h those o f a c o n t r o l group o f f l i g h t i n s t r u c t o r s who performed t h e same t a s k . R e s u l t s show t h a t a l l s t u d e n t s d i s p l a y a common c h a r c t e r i s t i c b e h avior a t each t r a i n i n g phase. A t t h e begin n i n g o f t h e t r a i n i n g programm, s t u d e n t p i l o t s e x p l o r e d t h e i n s t r u m e n t panel i n a s t a r - l i k e p a t t e r n s t a r t i n g from t h e p i t c h i n d i c a t o r which corresponds t o t h e p o s i t i o n i n d i c a t o r i n combat a i r c r a f t . However they o f t e n lo ok ed back and f o r t h between two i n st rument s, s t o p p i n g f o r a p r a c t i c a l l y i d e n t i c a l l y s h o r t t i m e on each one. Some omissions were a l s o recorded. As t h e s t u d e n t s progressed, t h e number o f back and f o r t h gaze s h i f t s decreased w h i l e t h e d u r a t i o n o f l o o k - f i x a t i o n s increased. Omissions g r a d u a l l y disappeared and l o o k - f i x a t i o n on t h e most i m p o r t a n t i n d i c a t o r s f o r a g i v e n f l i g h t phase became l o n g e r . T h i s f i n a l behavior i s s p e c i f i c t o f l i g h t i n s t r u c t o r s . E v i d e n t l y , t h e a c q u i s i t i o n o f t h i s behavior i s a slow, t o t a l l y unconscious process. The q u e s t i o n i s t o know whether i t may be p o s s i b l e t o a c t i v e l y i n f l u e n c e t h i s a c q u i s i t i o n process. I11
- USE OF GAZE DIRECTION RECORDINGS I N PILOT TRAINING PROGRAMS
To i n f l u e n c e a c t i v e l y t h e a c q u i s i t i o n o f o p t i m a l v i s u a l e x p l o r a t i o n s t r a t e g i e s , two axes o f r e s e a r c h a r e s i m u l t a n e o u s l y f l o l l o w e d . The f i r s t c o n s i s t s i n c r e a t i n g t e a c h i n g m a t e r i a l which shows and e x p l a i n s these behaviors. I n o r d e r t o do t h i s , once t h e v i s u a l behavior o f e x p e r t p i l o t s has been re c o r d e d d u r i n g r e a l o r s i m u l a t e d f l i g h t , a document i s prepared which a1 t e r n a t e s dynamic p r e s e n t a t i o n o f these behaviors, e x p l a n a t o r y sketches and consequences o f most f r e q u e n t inadequate behaviors , e x h i b i t e d bu s t u d e n t p i l o t s . Such documents have been made, i n c o l o r , f o r t h e v a r i o u s phases o f h e l i c o p t e r f l i g h t i n s t r u c t i o n , b o t h o v e r c o u n t r y s i d e and mountainous t e r r a i n . However, t hey have n o t been d i s t r i b u t e d t o s t u d e n t p i l o t s s i n c e t h e r e s u l t s o b t a i n e d r e c e n t l y i n r a d i o l o g y ( P m a l b e r t i e l at, 1983) and a v i a t i o n s t u d i e s (Spady, 1982) show t h a t alone, v i e w i n g t h e o p t i m a l behavior i s n o t e f f i c i e n t . What i s e f f i c i e n t i s e i t h e r t o show t h e s t u d e n t s t h e i r own behavior, o r t o l e t them p a r t i c i p a t e a c t i v e l y when t h e f i l m i s shown. To o b t a i n such a c t i v e p a r t i c i p a t i o n d u r i n g f i l m viewing, s p e c i a l scenes must be prepared and we a r e c u r r e n t l y p r e p a r i n g them f o r h e l i c o p t e r f l i g h t i n s t r u c t i o n . The second a x i s f o l l o w e d i n t h i s t e a c h i n g r e s e a rch i s t o l e t s t u d e n t s object i v a t e t h e i r b ehav i o r . Such an approach i s p r o m i s i n g b u t r a i s e s numerous problems. Indeed, i n i t s p r e s e n t c o n f i g u r a t i o n , t h e NAC i s n o t adapted f o r stude n t s who a r e s t a t i n g t h e i r f l i g h t i n s t r u c t i o n o r f o r s i n g l e - s e a t e r comb a t a i r c r a f t s t u den t s . However, t h e method can be used i n a s i m u l a t o r t o h e l p st udent s a c q u i r e emergengy procedures as we demonstrated i t i n a s t u d y on h e l i c o p t e r simul a t o r o r as Spady (1982) showed i t on a Boeing 737 s i m u l a t o r . T h i s method cans a l s o be used i n r e a l f l i g h t when a p i l o t i s t r a n s f e r r e d t o a new h e l i c o p t e r o r a new a i r c r a f t . We c o u l d t h u s h e l p tremendously an a i r c a r r i e r p i l o t . I n s t r u c t o r s were c o n f r o n t e d w i t h t h e a l t e r n a t i v e t o e i t h e r dismiss t h e s t u d e n t o r make him r e p e a t h i s courses. An i n f l i g h t r e c o r d i n g helped t h e
Visual Strategies of Pilots
37 1
p i l o t i d e n t i f y h i s e r r o r s and t r u s t h i s i n s t r u c t o r s again. He could corr e c t h i s e r r o r s q u i c k l y and catch up on what he had missed. This c l i n i c a l observation i s very i n t e r e s t i n g b u t i t i s d i f f i c u l t t o make a generalizat i o n since actions must be punctual and r e q u i r e g r e a t a v a i l a b i l i t y . NOW, what should be done i s t o go from research t o p r a c t i c a l application. 'Ib ck t h i s s p e c i a l i s t s must be t r a i n e d and special equipment must be acquired. I t would c e r t a i n l y be a long and expensive process. CONCLUSION The NAC EYE MARK Recorder i s a valuable t o o l f o r the study o f the v i s u a l behavior o f confirmed p i l o t s i n v i s u a l f l i g h t on h e l i c o p t e r o r a i r c a r r i e r and a l l a i r c r a f t types i n simulators. Such studies h e l p s o l v i n g ergonomic c o r r e c t i o n and design problems. The NAC can be used e f f i c i e n t l y t o prepare t r a i n i n g m a t e r i a l f o r p i l o t s t u dents, and t o help c e r t a i n categories o f f a i l i n g students. However, i t should n o t be used t o analyse p i l o t ' s work i n a s i n g l e seater f i g h t e r a i r c r a f t i n r e a l f l i g h t nor t o help beginner students. I n order t o go f u r ther i n t h i s type o f approach i n the f i e l d o f a v i a t i o n , i t w i l l be necessary t o w a i t f o r the development o f recording equipment b e t t e r adapted t o t h i s task.
-
REFERENCES
-
BARNES J.A. : "Analysis o f p i l o t s eye movements during h e l i c o p t e r f l i g h t " Tech. memo. 11-72, Human engineering laboratory, a p r i l 1972.
-
FREZELL T.L., HOFMANN M.A. : Comparison o f v i s u a l performance o f monocul a r and binocular v i s i o n during VHR h e l i c o p t e r f l i g h t . Aerospace Medic a l pane1,Agard conference pr0ceedingG.B. Val CP 1982, n04 10/1975.
-
MACKWORTH Y.F., MACKWORTH N.M. : Eye f i x a t i o n s recorded on changing v i s u a l scenes by the t e l e v i s i o n eye-marker. J. opt. SOC. Amer., 1958.
-
MILTON J.L. : Analysis o f p i l o t s eye movements i n f l i g h t . A v i a t i o n Medecine, February 1952.
-
P A P I N J.P., NAUREILS P., SANTUCCI G. :Pickup o f v i s u a l i n f o r m a t i o n by the p i l o t during a ground c o n t r o l approach i n a f i g h t e r a i r c r a f t simulator. A v i a t i o n space, and environnemental medicine, may 1980.
-
-
-
PAPIN J.P., WIATZ A., GANGLOFF B. : La d i r e c t i o n du regard e t des alarmes v i s u e l l e s sur simulateur PUMA 5A 330, Rapport CERMA 80-12, J u i n 80. PAPIN J.P., MENU J.P., SANTUCCI G. : Vision monoculaire en vol t a c t i q u e s u r h 6 l i c o p t e r e . Agard CP 312, mai 1981.
.
SPADY A., AMOS : A i r l i n e p i l o t scanning behavior during approaches and landing i n Boeing 737 simulator, Agard CP 240, A p r i l 1978. SPADY A., JONIS D., COATES G., KIRBY R. : Effectiveness o f using r e a l time eye scanning information f o r p i l o t s t r a i n i n g . Proceding HF Society 1982.
THOMAS E., LLEWELLYN, The eye movements of a p i l o t during a i r c r a f t Landing. Aerospace Medicine, May 1963. TIFFIN J., BROMES : Analysis o f eye f i x a t i o n s and p a t t e r n s o f eye mowements i n landing a Piper cub J 3 a i r plane. Report no 10 d i v i s i o n o f research c i v i l aeronautics adminstration, Washington DC, February 1943.