- Email: [email protected]
Eye movement indices of mental workload
Acta Psychologica North-Holland
75 (1990) 75-89
EVE MOVEMENT
75
INDICES OF MENTAL
WORKLOAD
*
James G. MAY University of New Orleans, USA
Robert
S. KENNEDY
Essex Corporation,
Orlando, USA
Mary C. WILLIAMS University of New Orleans, USA
William P. DUNLAP Tulane University, New Orleans, USA
Julie R. BRANNAN University of New Orleans, USA Accepted
October
1989
Four investigations were carried out to assess the feasibility of using eye movement measures as indices of mental workload. In the first experiment, saccadic extent was measured during free viewing while subjects performed low, moderate and high complexity, auditory tone counting as the workload tasks. The range of saccadic extent decreased significantly as tone counting complexity (workload) was increased. In the second experiment the range of spontaneous saccades was measured under three levels of counting complexity with a visual task that did not require fixation or tracking. The results indicated that the extent of saccadic eye movements was significantly restricted as counting complexity increased. In the third experiment, the effects of practice were examined and decreased saccadic range under high tone counting complexity was observed even when significant increases in performance occurred with practice. Finally, in experiment 4, the first experiment was repeated with additional optokinetic stimulation and the saccadic range was again observed to decrease with tone counting complexity. The results indicated that the extent of spontaneous and elicited eye movements was significantly restricted as counting complexity increased. We conclude that this measure may provide a valuable index of mental workload. * Research sponsored by the Air Force Office of Scientific Research (AFOSR) under Contract F49620-87-C-0002. Requests for reprints should be sent to J.G. May, Dept. of Psychology, University of New Orleans, Lakefront, New Orleans, LA 70148, USA.
OOOl-6918/90/$03.50
0 1990 - Elsevier Science Publishers
B.V. (North-Holland)
76
J.G. May et al. / Workload and eye movements
It is generally accepted that humans are limited capacity information processors, and that human performance is a function of both individual processing capabilities and of task demands (Kahneman 1973; Moray 1967; Wickens 1984). It is inevitable, therefore, that in certain situations, human performers reach the upper limits of their ability to cope with task demands, and performance is jeopardized when these limits are approached or exceeded. For this reason, a need has arisen for the development of reliable, accurate, and non-intrusive measures of mental workload. Workload refers to demands imposed on a human operator by a given task, and workload measurement involves an attempt to assess performance limitations of humans and to characterize conditions under which task demands can or cannot be met by the performer (Gopher and Braune 1984). Because it has been suggested that there may be little or no deterioration in performance until the point of failure is closely approached (Schmidt 1978), sensitive measures of workload are of vital importance. Workload assessment can be used not only to evaluate performance requirements, but also to conform system design to human processing limitations and to predict workload changes with system modification. Additionally, by assessing workload impact on individuals for tasks of constant difficulty, workload indices can be used to determine individual differences in capability and thereby aid in personnel selection. One approach to the assessment of workload that has recently received a great deal of attention involves the measurement of an observer’s functional field of view. Macworth (1965) has described the functional field of view as an area around central fixation from which information is actively processed during performance of a visual task. The functional field of view has been found to be sensitive to cognitive load in that it changes in size according to processing demands. As processing demands increase, a shrinkage of the functional field of view is typically observed. Macworth (1965) has referred to this constriction as ‘tunnel vision’, and has proposed that it serves to prevent an overloading of the processing system when more information is available than can be processed. Others (Bursil 1958; Teichner 1968) have also discussed a shrinking of attentional fields as greater processing demands are imposed. Such modulation of the functional field of view with task demands has been reported in single task paradigms such as visual search (Edwards and Goolkasian 1974), and matching (Williams
J.G. May et al. / Workload and eye movements
71
and Lefton 1981), as well as in dual-task paradigms where attention is divided between a fovea1 and peripheral task (Holmes et al. 1977; Williams, 1982, 1985). These studies employed direct behavioral (performance) measures of cognitive load, where an evaluation of an observer’s overt task behavior (e.g. speed or accuracy of performance) is made. An alternative approach to the measurement of cognitive load would involve direct physiological measures. Although such measures have the advantage of eliminating the possibility of subjective distortion and generally do not interfere with task performance, Kahneman (1973) has suggested that to be valid, physiological indicators of workload must meet three essential criteria. First, they must be sensitive to variations in task demands produced by changes in task parameters. Second, they must reflect between-task differences in processing load elicited by qualitatively different cognitive operations. Finally, they must be sensitive to individual differences in processing load as individuals of different abilities perform different tasks. A number of eye movement indices of mental workload have been investigated. One of the first was pupillary changes during mental activity. Pupil diameter has been shown to increase during mental arithmetic (Schaefer et al. 1968; Payne et al. 1968; Kahneman et al. 1968), short-term memorization (Kahneman and Beatty 1966, Kahneman et al. 1968; Kahneman and Wright 1971), pitch discrimination (Kahneman and Beatty 1967), decision-making (Simpson and Hale 1969), and letter matching (Beatty and Wagoner 1978). In all cases the degree of pupillary dilation was directly related to task difficulty. In a recent review (Beatty 1982) it was concluded that this index was a valid physiological indicant of mental effort. Another index which has received consideration is the number of eye blinks during mental effort. These results are somewhat equivocal, in that some researchers have found increased rate of blinking with mental effort (Holland and Tarlow 1972, 1975), while others have found the opposite trend (Crammon and Schuri 1980; Schuri and Crammon 1981; Wood and Hasset 1950). Stern and Skelly (1984) have reported decreased eye blink rate for individuals in control of aircraft and during difficult maneuvers. In the present studies, we have examined eye movement characteristics associated with a range of cognitive workloads as a physiological measure to determine if this measure meets Kahneman’s criteria for a valid index of workload. We measured the extent of visual saccades
78
J.G. May et al. / Workload and eye movements
under various levels of complex tone counting (Jerison 1956). Although previous reports of attentional constriction did not involve eye movement recording, the possibility that attentional demands might be associated with changes in eye movement patterns seemed a distinct possibility.
Experiment 1: The effects of auditory taskload on the extent of saccadic eye movement Methods
Subjects
Five subjects were used. There were two males and three females ranged from 24 to 45 years.
and their ages
Apparatus
The infrared eye-tracking instrument (Eye Trac, Model 106) was used to record eye movements from the left eye. Head stabilization was accomplished with a bite bar. Signals from the eye tracker were applied to the modulation input of a voltage-controlled frequency generator (Wavetek, Model 148), the output of which was fed into a signal processor (Nicolet, Model 1072) which was programmed to accumulate a time-interval histogram. In this fashion, eye-movement extent was coded in terms of frequency modulation and depicted as a frequency histogram wherein the left half contained the frequency of leftward eye movements and the right side contained the frequency of rightward eye movements. The resultant histograms were plotted on an X-Y plotter (Hewlett-Packard, Model 7044A). The dependent variable was the range of eye movements, right-most extent of the histogram minus left-most extent of the llistogram. Tone-counting tasks were administered with a microprocessor (NEC, Model 8201A) which was programmed to present a random series of 36 low tones, 28 medium tones and 24 high tones. Tone durations were 0.5 seconds, and although the same temporal distribution was repeated every 60 seconds, the subjects did not identify a pattern. Responses were entered and cataloged on the microprocessor and scoring included the number correct, incorrect, and missed. Three tone-counting tasks were used. Task one required a response after each fourth low tone (low task load). Task two required a response after each fourth low tone and each fourth middle tone (medium task load), counted separately and kept track of separately. Task three required a response after each fourth low, medium, and high tone (high task load). Three separate keys were used to indicate the three different tone counts. Scoring was always reset in the event of an incorrect response and reset after the first response following a miss. Procedure
Each subject was allowed one practice run on each tone-counting task prior to data collection. Each session began with a fixation condition in which the subject fixated a
J.G. May et al. / Workload and eye movements
19
small cross (subtending 10 min of visual angle) for five minutes during which eyemovements were recorded. This measure was taken to provide a measure of fixation stability and served as a benchmark for comparison with the experimental measures. Next they performed an alternating fixation task which required 20 degree saccades at an aperiodic rate (0.2 Hz) for five minutes while eye movements were recorded. This measure was used to calibrate the equipment for each subject so that the experimental measures could later be expressed in degrees of visual angle. Following this they were allowed to freely move their eyes for five minutes during which eye movements were recorded. This measure served as a no task baseline against which to compare the experimental measures. After this baseline condition, they were asked to perform the one-, two-, and three-channel counting tasks under free viewing for five minutes each, while eye movements were being recorded. Results Performance
data
Presented in fig. 1 are the percent correct performance scores on the Jerison counting task as a function of workload (number of channels monitored). ‘As can be seen, under the low and medium workload condition performance was high (86% and 89%, respectively) whereas under high workload condition a substantial drop was seen. Although the main effect of workload failed to reach significance (F(2, 8) = 3.63; p = 0.75) the quadratic trend was significant F(1, 4) = 19.23; p = 0.012). Eye movement
data
The histograms obtained under all six experimental conditions for a single subject are presented in fig. 2. The results for the other four subjects were similar and are
o--o h
EXPERIMENT
1
o--o I
40 I
: 2
m
al 30.(---Fv
z g z
20-
d 0 ii 2
10--(---F 60
t LOW
MEDIUM
TASK
HIGH
150
DIFFICULTY
Fig. 1. Mean performance (closed circles - right ordinate) and saccadic extent measures (open circles - left ordinate) as a function of workload on an auditory task in experiment 1. F= Mean for fixation condition; FV = Free viewingcondition.
J.G. May et al. / Workload and eye movements
80
ALTERNATE FIXATION
FIXATION
30 20 10 0
10 20 30
30 20 10 0
sAccA~lc
FREE VIEWING
10 20 30
EXTENT
3020
10
0
102030
(DEG.)
Fig. 2. Histogramsdepicting the range and frequency of saccadic extent under various control and workloadconditions for one subject. The small vertical arrows in each panel depict how saccadic ranges were measured.
omitted. It may be seen that under conditions of steady fixation (panel a) the distribution of frequency modulation was quite narrow, indicating that the extent of leftward or rightward movement was quite small. Under conditions of 20 degree alternate fixation (panel b), the distribution of frequency modulation is bimodal, indicating that the extent of leftward and rightward eye movements was quite extensive. These data were used to calibrate the abscissa (saccadic extent) in degrees of visual angle. Under conditions of free viewing (panel c), the distribution of frequency modulation was greater than fixation and alternating fixation, indicating that the range of eye movements during this condition was more extensive than both steady fixation and saccades of 20 degrees. The effects of tone counting are depicted in panels d through f for one-, two-, and three-channel counting. It is evident that as task load increased, the extent of frequency modulation decreased. The recording procedure was contaminated by eye blinks and the artifacts render a measure of the standard deviation of the histogram inappropriate. However, measurement of eye blink signals and the modulation that resulted from them indicated that they could not have contributed to or contaminated a measure of the range of the histogram. Thus, the index of interest is a measure of the extent of eye movements under these different conditions of workload. For this purpose, the range of the histogram was computed and transformed to degrees of visual angle for the saccades and was further normalized by dividing each range by the range obtained under alternating fixation and then multiplying by 20. This was done because there were substantial overall differences in both fixated and spontaneous eye movements between subjects.
J.G. May et al. / Workload and eye movements
81
Average normalized spontaneous saccadic range as a function of workload is presented in fig. 1. The average saccadic ranges for free viewing and fixation are indicated by arrows on the left ordinate. Although the main effect of workload only approached significance (F(2, 8) = 4.22; p = 0.056), it is clear that saccadic extent decreased as a function of workload (F(1, 4) = 9.74, p = 0.035, for the linear component). To further substantiate the relation of saccadic extent to workload correlation coefficients between saccadic extent and workload (1, 2, 3) were computed for each subject, which averaged r = -0.69, and ranged from -0.33 to -0.99, the greater the workload, the less the range of eye movements for each subject. Discussion It is clear from the performance data of this experiment that the modified Jerison counting task offers considerable control of task workload and provides an excellent behavioral index of that parameter. Although the performance scores varied on average from 64 to 96 percent in the present study, this technique can be modified by the inclusion of more tone categories and increased rate of tone presentation to expand the range of workloads investigated. Such a manipulation might well improve the correlation obtained between saccadic and behavioral measures of workload. The eye movement results are encouraging in that the extent of spontaneous saccades was significantly restricted as task difficulty increased. These measures could be obtained easily in many situations that require dynamic information processing, but a number of potential problems must be addressed. The extent to which these procedures might be used in situations where visual information is to be processed is another important consideration. It may be that a decrease in saccadic extent is also observed with increased workload in situations where visual monitoring of events is necessary. The degree to which this relationship exists may depend on whether the primary visual task requires precise fixation or tracking performance, but many visual activities do not. In experiment 2 this relationship was re-examined with a visual counting task that is analogous to the previously used auditory counting task. Although all correlations of eye movement range with workload were negative, the predicted direction, the analysis of variance results were only marginally significant; therefore, in subsequent experiments larger sample sizes were used.
Experiment 2: The effects of visual taskload on the extent of saccadic eye movement Method Subjects and apparatus Ten subjects were used in this experiment. Four were male and five were female. They ranged in age from 19 to 26 years. They were asked to perform low, medium, and high difficulty levels of a visual counting task while eye movements were recorded using the same methodology reported in experiment 1. The same microcomputer was used to administer the task, but the microprocessor was reprogrammed to present a
J.G. May et al. / Workload and eye movements
82
EXPERIMENT
2
o--o
40 35 ---100
z
30 --90
25 --
2 p 0 m
20 -8o
15 -f---F
10-l
3
r70
LOW
MEDIUM
TASK Fig. 3. Mean performance
: m
HIGH
DlFFlCbLTY
and saccadic extent measures as a function of workload in experiment 2. Conventions as in fig. 1.
on a visual task
series of three dark rectangles (1 cm X 2 cm) on the face of an LCD screen. The rectangles were arrayed horizontally in three channels (left, center, and right) with the left and right rectangles located ten degrees to each side of the central one. The screen was located 18 inches in front of the subject’s bite bar. Each rectangle was presented for one second and their order of occurrence was aperiodic with the average rate of occurrence being 0.2 Hz. Procedure Under the three counting conditions the subjects: (1) counted each occurrence of the left rectangle and depressed a key after each fourth occurrence; (2) counted the occurrence of each left and middle rectangle and depressed different keys after the fourth occurrence of each one, and (3) counted the occurrence of each left, middle, and right rectangle and depressed different keys after the fourth occurrence of each. Fixation, alternating fixation, and free-viewing control conditions were run as described in experiment 1. Results Measures of rectangle counting performance and normalized saccadic range were computed for each condition. The mean saccadic extent and counting performance scores have been presented in fig. 3 for each counting condition. The performance with increasing task load declined, but overall performance was quite high relative to that for tone counting in experiment 1. Decreasing saccadic extent was, again, associated with increases in task difficulty. Analysis of variance revealed significant main effects for performance (F(2, 18) = 6.21; p < 0.009) and saccadic extent (F(2, 18 = 16.06;
J.G. May et al. / Workload and eye mouements
83
p -C 0.0001). Trend analysis showed both significant linear and quadratic components of the performance data (p = 0.0355 and p = 00312, respectively), but a highly significant linear trend for saccadic range (F(1, 18) = 16.06; p = 0.0001) and no quadratic trend. Correlations between workload and saccadic extent for each subject ranged between - 0.66 and - 1.00, with a mean of - 0.92. Discussion These data indicate that the saccadic extent index of taskload is valid under conditions of visual monitoring. Thus, it appears as though this technique may be generalized to many visual performance tasks which do not require precise fixation or tracking. Furthermore, the correlations indicate that the range of eye movements was a strong predictor for individual subjects. Both the two previous experiments were carried out with subjects that had little practice on the tone counting task. The possibility exists that the relationship between saccadic extent and workload may deteriorate with practice. For this reason we examined the effects of practice with the three channel (high workload) task.
Experiment 3: The effects of practice on saccadic extent index of mental workload
Method
Subjects Ten subjects were employed in this experiment. Five were male and five were female. They ranged in age from 20 to 32 years. Each participated in 10 eye movement recording sessions occurring over a period of 10 successive days. Each subject was paid $100.00 at the completion of the experiment. Apparatus The same eye-movement techniques used in the previous experiments were used here. The high workload (three channel) version of the tone counting task was administered as in experiment 1. Procedure Subjects participated in 10 sessions. In the first session, each subject performed the auditory tone counting task once prior to recording eye movements. The control conditions, fixation, alternate fixation, and free viewing, were then performed for five minutes each as in the previous experiment. Following the control conditions, eye movements were recorded while subjects performed the tone counting task for five minutes. The procedure was repeated in the subsequent nine sessions except that subjects were not given pretest practice.
84
J.G. May et al. /
o-o
r\
Workload and eye movements
EXPERIMENT
o--o
3
100
40,
-90 : -- 80
z
-- 70
2
6o
P 0 rl
-- 50 2 -- 40
1
2
3
4
5
6
7
8
9
10
SESSIONS Fig. 4. Mean performance and saccadic extent measures as a function of repeated tests at high workloadin experiment3. Conventions as in fig. 1.
Results
Data reduction involved normalizing the range of saccades for fixation, free-viewing and task-related saccades by dividing these measures by the range for alternate fixation. These three measures and the performance scores (mean percent correct) for each session were submitted to an analysis of variance. For eye movement data, a condition by sessions design was employed. For the tone counting performance measure, a simple repeated measured design was employed. The mean tone counting performance as a function of practice sessions is depicted in fig. 4. It is apparent in the figure that performance increased considerably with practice. This is supported by the results of the analysis of variance which revealed a significant main effect for sessions (F(9, 18) = 9.13; p < 0.01). Mean normalized saccadic extent across testing sessions are also depicted in fig. 4 for each condition. The mean ranges for the fixation condition are consistently lowest across sessions, while those for free-viewing are highest. The ranges obtained in the task condition fell between those obtained for the free-viewing and fixation conditions. Analysis of variance revealed that the saccadic extent measures did not change systematically with practice on the tone counting task. These conclusions are supported by the results of the analysis of variance which revealed a significant main effect for conditions (F(1, 2) = 10.1; p < 0.01) but no other significant effects. Discussion
The results show: (1) that the average saccadic extent obtained while subjects perform a difficult tone counting task is restricted compared to free-viewing conditions,
85
J.G. May et al. / Workload and eye movements
and (2) that percent correct performance on the tone counting task continued to improve across ten testing sessions while saccadic extent was not modified as a function of practice. The first finding reinforces the contention that saccadic extent may be used to measure mental workload. The second finding runs contrary to the expectation that the performance would become automated with practice and that saccadic extent would increase. It may be hypothesized that, although subjects were getting better at the tone counting task, it remained a ‘high’ workload task which was reflected in saccadic extent which remained constant across sessions. If this hypothesis were true, then it could be expected that if these subjects were administered low and medium difficulty levels of the tone counting task, saccadic extent should reflect the decrease in workload but performance should remain high. This expectation was tested and confirmed with three subjects used in experiment 3. In experiment 2, we determined that the range of spontaneous saccades decreased with increasing workload when a visual task, that did not require fixation or controlled eye movements, was used. The possibility exists that this potential index of taskload might be disrupted by reflexive eye movements. To test this notion, we measured the range of saccadic extent under different taskloads during optokinetic stimulation.
Experiment 4: The effects of auditory taskload on the extent movements obtained under low, medium, and high workloads
of optokinetic
eye
Method Subjects and apparatus Ten new subjects were used. Six were male and four were female and they ranged in age from 23 to 34 years. The same eye-movement techniques and tone counting task used in experiment 1 was used in this experiment. In addition, optokinetic stimulation was provided by a 20 degree field of vertical stripes displayed on a color monitor (Tektronix, Model 690SR). The stripes were red and green and had spatial frequency of 1.2 c/d. The stripes appeared to drift to the right for 10 seconds and then to drift to the left for 10 seconds. This cycle was repeated for 5 minutes. Procedure Subjects were required to participate in the control conditions as in experiment 1 (fixation and alternate fixation), and to undergo 5 minutes of optokinetic stimulation while eye movements were recorded. Next, they were required to perform the one-tone, two-tone, and three-tone counting tasks. Each of the tone counting tasks were performed for five minutes under conditions of optokinetic stimulation while eye movements were recorded. Results The data were reduced and analyzed as in experiment 1. The mean performance scores (percent correct) and the mean normalized saccadic extent are presented in fig. 5
86
J.G. May et al. / Workload and eye movements
EXPERIMENT
o-o
A
20~
: E E
18--
-0 V
8
16--
2
14--
4
o--o 100 -- 90
(---OKN
A,_ I
. . 1.
P\l
-- 80
T
%
-0 1‘1,
\
0 3
12-- C---F
0 :
109
:
3:
-- 70
E
-60
z m
-50 pa
3 u
i r40
LOW
MEDIUM
TASK
HIGH
DIFFICULTY
Fig. 5. Mean performance and saccadic extent measures as a function of workload in the presence of optokinetic stimulation in experiment 4. Conventions as in fig. 1.
for each tone counting condition. As is apparent in the figure, tone counting performance declined significantly with task load, as expected. Interestingly, saccadic extent decreased with increased task difficulty, even when optokinetic stimulation was used. Analysis of variance showed a highly significant main effect and linear component for performance (F(2, 18) = 18.00; p = 0.0001 and F(1, 9) = 38.34; p = 0.0002, respectively), but no quadratic component. Analysis of the saccadic range data showed a similar pattern of significance for workload main effect I;(2, 18) = 5.49; p = 0.026) and for the linear component (F(1, 9) = 7.11; p = 0.026). Correlations between saccadic extent and workload for individual subjects ranged from 0.87 to -0.95; although correlations for three subjects were positive, the mean correlation was - 0.49, reflecting the fact that the majority of subjects showed the predicted behavior.
Discussion
Saccadic extent measured in subjects who performed tasks of low, medium, and high difficulty reflected changes in mental workload even under conditions of optokinetic nystagmus. This finding indicates that the saccadic extent index of workload is a valid measure even when a reflexive visual forcing function is employed. The correlational analysis did show that three of the ten subjects were disrupted and did not show the predicted relationship, so that in terms of individual differences, the saccadic measure may not be as reliable in the presence of optokinetic stimulation or other oculo-motor stimuli.
J. G. May et al. / Workload and eye movements
87
General discussion The major finding of this investigation is that the range of spontaneous eye movements decreases as cognitive workload increases. This is the case when an auditory counting task is used and also when a visual counting task, that does not involve fixation or tracking, is employed. The results of experiments 3 indicated that the saccadic extent measure is not affected by extended practice under difficult task conditions. Thus, under extended practice on a task that induces high workload, saccadic extent reflected workload while performance increased. This finding is important given the previous use of performance indices to measure workload. Finally, the results of the last experiment suggest that saccadic extent remains a valid measure of the mental effort required under low, medium and high task difficulty even in the presence of a reflexive visual forcing function. This finding is important given the increasing concern with temporal factors involved in workload (e.g., Matthews 1986) and the possible effects of practice on workload indices reported in other research (Wilson et al. 1986). The techniques employed in the present investigation included two serious limitations. First, the eye tracking apparatus was insensitive to eye movements in non-horizontal meridians. It may be possible to use similar techniques with instruments that track vertical as well as horizontal eye movement signals. Second, the apparatus employed did not allow for the exclusion of eye-blinks. As Stern and Skelly (1984) have shown, eye-blinks change under conditions of cognitive workload, but our present findings cannot be attributed to changes in rate of eye blink. ’ However, it may well be that a system which measures both frequency of eye blinks and saccadic range uncontaminated by the former would allow a more sensitive measure of eye movement variance (e.g. the standard deviation) to be appropriate and thereby offer a more sensitive index of taskload. A final caveat concerns the applicability of this technique to real world settings. Although the apparatus used in the present study required a bite bar for head stabilization, other less restrictive methods are available and need only involve head mounted eye monitoring equipment. ’ The change in the signal level resulting from an eye blink was only 6% of the signal associated with 20” saccades. Thus, eye blinks could produce artifactual measures of no greater than 1.2O. These artifacts were considerably less than the minimum saccadic range observed in any subject in any of the experiments.
88
J. G. May et al. / Workload and eye mouements
These findings raise interesting questions regarding previous reports of attentional constriction during conditions of high levels of mental workload. If this phenomenon is associated with an attempt to restrict information processing’ to a limited portion of the central visual field, then it stands to reason that a concomitant restriction of eye movements would facilitate such a process. Future research aimed at assessing attentional constriction and eye movement extent under varying levels of cognitive workload would further elucidate this relationship.
References Beatty, J., 1982. Task-evoked pupillary responses, processing load, and the structure of processing resources. Psychological Bulletin, 91, 276-292. Beatty, J. and B.L. Wagoner 1978. Pupillometric signs of brain activation vary with level of cognitive processing. Science 199, 1216-1218. Bursill, A.E., 1958. The restriction of peripheral vision during exposure to hot and humid conditions. Quarterly Journal of Experimental Psychology 10, 113-129. Crammon, D.V. and U. Schuri, 1980. Blink frequency and speech motor activity. Neuropsychologia 18, 603-606. Edwards, D.C. and P.A. Goolkasian, 1974. Peripheral vision location and kinds of complex processing. Journal of Experimental Psychology 102, 244249. Gopher, D. and R. Braune, 1984. On the psychophysics of workload: Why bother with subjective measures? Human Factors 26, 519-532. Holland, M.K. and G. Tarlow, 1972. Blinking and mental load. Psychological Reports 31, 119-127. Holland, M.K. and G. Tarlow, 1975. Blinking and thinking. Perceptual, and Motor Skills, 41, 403-406. Holmes, D.L., K.M. Cohen, M.W. Haith and F.J. Morreson, 1977. Peripheral visual processing. Perception and Psychophysics 22, 571-577. Jerison, H., 1956. Effect of a combination of noise and fatigue on a complex counting task (WADC TR-55-360). Dayton, OH: Wright Air Development Center, Wright-Patterson AFB. Kahneman, D., 1973. Attention and effort. Englewood Cliffs, NJ: Prentice-Hall. Kahneman, D. and J. Beatty, 1966. Pupillary diameter and load on memory. Science 154, 1583-1585. Kahneman, D. and J. Beatty, 1967. Pupillary responses in a pitch discrimination task. Perception and Psychophysics 2, 101-105. Kahneman, D. and P. Wright, 1971. Changes in pupil size and rehearsal strategies in a short-term memory task. Quarterly Journal of Experimental Psychology 23, 187-196. Kahneman, D., W.S. Peavler and L. Onuska, 1968. Effects of verbalization and incentive on pupil response to mental activity. Canadian Journal of Psychology 22, 186-196. Macworth, N.H., 1965, Visual noise causes tunnel vision. Psychonomic Science 3, 67-68. Matthews, M.L., 1986. The influence of visual taskload history on visual performance. Human Factors 28, 623-632. Moray, N., 1967. Where is attention limited? A survey and a model. Acta Psychologica 27, 84-92. Payne, D.T., M.E. Parry and S.J. Harasymiw, 1968. Percentage of pupillary dilation as a measure of item difficulty. Perception and Psychophysics 4, 139-143.
J.G. May et al. / Workload and eye movements
89
Schaefer, T. Jr., J.B. Ferguson, J.A. Klein and E.B. Rawson, 1968. Pupillary responses during mental activities. Psychonomic Science 12, 137-138. Schmidt, D., 1978. A queueing analysis of the air traffic controllers’ workload. IEEE Transactions on Systems, Man, and Cybernetics, SMC-8, 492-498. Schuri, U. and D.V. Cammon, 1981. Heart rate and blink rate responses during mental arithmetic with and without continuous verbalization of results. Psychophysiology 18, 650-653. Simpson, H.M. and S.M. Hale, 1969. Pupillary changes during a decision-making task. Perceptual and Motor Skills 29, 495-498. Stem, J.A. and J.J. Skelly, 1984. The eye blink and workload considerations. Proceedings of the 28th Human Factors Society Meeting, San Antonio, CA, Santa Monica, CA, Human Factors Society. Teichner, W.H., 1968. Interaction of behavioral and physiological stress reactions. Psychological Review 75, 271-291. Wickens, C., 1984. Engineering psychology and human performance. Columbus, OH: Charles E. Merrill. Williams, L.J., 1982. Cognitive load and the functional field of view. Human Factors 24, 683-692. Williams, L.J., 1985. Tunnel vision induced by a fovea1 load manipulation. Human Factors 27, 221-227. Williams, L.J. and L.A. Lefton, 1981. Processing of alphabetic information presented in the fovea or the periphery: Functional field of view and cognitive workload. Perception 10, 645-650. Wilson, G., K. McCloskey and I. Davis, 1986. Linguistic processing: Physiological & subjective correlates. Proceedings of the 30th Human Factors Society Meeting, Dayton, OH, Santa Monica, CA, Human Factors Society. Wood, C.L. and J. Hasset 1950. Eye blinking during problem solving: The effect of problem difficulty and internally vs externally directed attention. Psychphysiology, 20, 18-20.
75 (1990) 75-89
EVE MOVEMENT
75
INDICES OF MENTAL
WORKLOAD
*
James G. MAY University of New Orleans, USA
Robert
S. KENNEDY
Essex Corporation,
Orlando, USA
Mary C. WILLIAMS University of New Orleans, USA
William P. DUNLAP Tulane University, New Orleans, USA
Julie R. BRANNAN University of New Orleans, USA Accepted
October
1989
Four investigations were carried out to assess the feasibility of using eye movement measures as indices of mental workload. In the first experiment, saccadic extent was measured during free viewing while subjects performed low, moderate and high complexity, auditory tone counting as the workload tasks. The range of saccadic extent decreased significantly as tone counting complexity (workload) was increased. In the second experiment the range of spontaneous saccades was measured under three levels of counting complexity with a visual task that did not require fixation or tracking. The results indicated that the extent of saccadic eye movements was significantly restricted as counting complexity increased. In the third experiment, the effects of practice were examined and decreased saccadic range under high tone counting complexity was observed even when significant increases in performance occurred with practice. Finally, in experiment 4, the first experiment was repeated with additional optokinetic stimulation and the saccadic range was again observed to decrease with tone counting complexity. The results indicated that the extent of spontaneous and elicited eye movements was significantly restricted as counting complexity increased. We conclude that this measure may provide a valuable index of mental workload. * Research sponsored by the Air Force Office of Scientific Research (AFOSR) under Contract F49620-87-C-0002. Requests for reprints should be sent to J.G. May, Dept. of Psychology, University of New Orleans, Lakefront, New Orleans, LA 70148, USA.
OOOl-6918/90/$03.50
0 1990 - Elsevier Science Publishers
B.V. (North-Holland)
76
J.G. May et al. / Workload and eye movements
It is generally accepted that humans are limited capacity information processors, and that human performance is a function of both individual processing capabilities and of task demands (Kahneman 1973; Moray 1967; Wickens 1984). It is inevitable, therefore, that in certain situations, human performers reach the upper limits of their ability to cope with task demands, and performance is jeopardized when these limits are approached or exceeded. For this reason, a need has arisen for the development of reliable, accurate, and non-intrusive measures of mental workload. Workload refers to demands imposed on a human operator by a given task, and workload measurement involves an attempt to assess performance limitations of humans and to characterize conditions under which task demands can or cannot be met by the performer (Gopher and Braune 1984). Because it has been suggested that there may be little or no deterioration in performance until the point of failure is closely approached (Schmidt 1978), sensitive measures of workload are of vital importance. Workload assessment can be used not only to evaluate performance requirements, but also to conform system design to human processing limitations and to predict workload changes with system modification. Additionally, by assessing workload impact on individuals for tasks of constant difficulty, workload indices can be used to determine individual differences in capability and thereby aid in personnel selection. One approach to the assessment of workload that has recently received a great deal of attention involves the measurement of an observer’s functional field of view. Macworth (1965) has described the functional field of view as an area around central fixation from which information is actively processed during performance of a visual task. The functional field of view has been found to be sensitive to cognitive load in that it changes in size according to processing demands. As processing demands increase, a shrinkage of the functional field of view is typically observed. Macworth (1965) has referred to this constriction as ‘tunnel vision’, and has proposed that it serves to prevent an overloading of the processing system when more information is available than can be processed. Others (Bursil 1958; Teichner 1968) have also discussed a shrinking of attentional fields as greater processing demands are imposed. Such modulation of the functional field of view with task demands has been reported in single task paradigms such as visual search (Edwards and Goolkasian 1974), and matching (Williams
J.G. May et al. / Workload and eye movements
71
and Lefton 1981), as well as in dual-task paradigms where attention is divided between a fovea1 and peripheral task (Holmes et al. 1977; Williams, 1982, 1985). These studies employed direct behavioral (performance) measures of cognitive load, where an evaluation of an observer’s overt task behavior (e.g. speed or accuracy of performance) is made. An alternative approach to the measurement of cognitive load would involve direct physiological measures. Although such measures have the advantage of eliminating the possibility of subjective distortion and generally do not interfere with task performance, Kahneman (1973) has suggested that to be valid, physiological indicators of workload must meet three essential criteria. First, they must be sensitive to variations in task demands produced by changes in task parameters. Second, they must reflect between-task differences in processing load elicited by qualitatively different cognitive operations. Finally, they must be sensitive to individual differences in processing load as individuals of different abilities perform different tasks. A number of eye movement indices of mental workload have been investigated. One of the first was pupillary changes during mental activity. Pupil diameter has been shown to increase during mental arithmetic (Schaefer et al. 1968; Payne et al. 1968; Kahneman et al. 1968), short-term memorization (Kahneman and Beatty 1966, Kahneman et al. 1968; Kahneman and Wright 1971), pitch discrimination (Kahneman and Beatty 1967), decision-making (Simpson and Hale 1969), and letter matching (Beatty and Wagoner 1978). In all cases the degree of pupillary dilation was directly related to task difficulty. In a recent review (Beatty 1982) it was concluded that this index was a valid physiological indicant of mental effort. Another index which has received consideration is the number of eye blinks during mental effort. These results are somewhat equivocal, in that some researchers have found increased rate of blinking with mental effort (Holland and Tarlow 1972, 1975), while others have found the opposite trend (Crammon and Schuri 1980; Schuri and Crammon 1981; Wood and Hasset 1950). Stern and Skelly (1984) have reported decreased eye blink rate for individuals in control of aircraft and during difficult maneuvers. In the present studies, we have examined eye movement characteristics associated with a range of cognitive workloads as a physiological measure to determine if this measure meets Kahneman’s criteria for a valid index of workload. We measured the extent of visual saccades
78
J.G. May et al. / Workload and eye movements
under various levels of complex tone counting (Jerison 1956). Although previous reports of attentional constriction did not involve eye movement recording, the possibility that attentional demands might be associated with changes in eye movement patterns seemed a distinct possibility.
Experiment 1: The effects of auditory taskload on the extent of saccadic eye movement Methods
Subjects
Five subjects were used. There were two males and three females ranged from 24 to 45 years.
and their ages
Apparatus
The infrared eye-tracking instrument (Eye Trac, Model 106) was used to record eye movements from the left eye. Head stabilization was accomplished with a bite bar. Signals from the eye tracker were applied to the modulation input of a voltage-controlled frequency generator (Wavetek, Model 148), the output of which was fed into a signal processor (Nicolet, Model 1072) which was programmed to accumulate a time-interval histogram. In this fashion, eye-movement extent was coded in terms of frequency modulation and depicted as a frequency histogram wherein the left half contained the frequency of leftward eye movements and the right side contained the frequency of rightward eye movements. The resultant histograms were plotted on an X-Y plotter (Hewlett-Packard, Model 7044A). The dependent variable was the range of eye movements, right-most extent of the histogram minus left-most extent of the llistogram. Tone-counting tasks were administered with a microprocessor (NEC, Model 8201A) which was programmed to present a random series of 36 low tones, 28 medium tones and 24 high tones. Tone durations were 0.5 seconds, and although the same temporal distribution was repeated every 60 seconds, the subjects did not identify a pattern. Responses were entered and cataloged on the microprocessor and scoring included the number correct, incorrect, and missed. Three tone-counting tasks were used. Task one required a response after each fourth low tone (low task load). Task two required a response after each fourth low tone and each fourth middle tone (medium task load), counted separately and kept track of separately. Task three required a response after each fourth low, medium, and high tone (high task load). Three separate keys were used to indicate the three different tone counts. Scoring was always reset in the event of an incorrect response and reset after the first response following a miss. Procedure
Each subject was allowed one practice run on each tone-counting task prior to data collection. Each session began with a fixation condition in which the subject fixated a
J.G. May et al. / Workload and eye movements
19
small cross (subtending 10 min of visual angle) for five minutes during which eyemovements were recorded. This measure was taken to provide a measure of fixation stability and served as a benchmark for comparison with the experimental measures. Next they performed an alternating fixation task which required 20 degree saccades at an aperiodic rate (0.2 Hz) for five minutes while eye movements were recorded. This measure was used to calibrate the equipment for each subject so that the experimental measures could later be expressed in degrees of visual angle. Following this they were allowed to freely move their eyes for five minutes during which eye movements were recorded. This measure served as a no task baseline against which to compare the experimental measures. After this baseline condition, they were asked to perform the one-, two-, and three-channel counting tasks under free viewing for five minutes each, while eye movements were being recorded. Results Performance
data
Presented in fig. 1 are the percent correct performance scores on the Jerison counting task as a function of workload (number of channels monitored). ‘As can be seen, under the low and medium workload condition performance was high (86% and 89%, respectively) whereas under high workload condition a substantial drop was seen. Although the main effect of workload failed to reach significance (F(2, 8) = 3.63; p = 0.75) the quadratic trend was significant F(1, 4) = 19.23; p = 0.012). Eye movement
data
The histograms obtained under all six experimental conditions for a single subject are presented in fig. 2. The results for the other four subjects were similar and are
o--o h
EXPERIMENT
1
o--o I
40 I
: 2
m
al 30.(---Fv
z g z
20-
d 0 ii 2
10--(---F 60
t LOW
MEDIUM
TASK
HIGH
150
DIFFICULTY
Fig. 1. Mean performance (closed circles - right ordinate) and saccadic extent measures (open circles - left ordinate) as a function of workload on an auditory task in experiment 1. F= Mean for fixation condition; FV = Free viewingcondition.
J.G. May et al. / Workload and eye movements
80
ALTERNATE FIXATION
FIXATION
30 20 10 0
10 20 30
30 20 10 0
sAccA~lc
FREE VIEWING
10 20 30
EXTENT
3020
10
0
102030
(DEG.)
Fig. 2. Histogramsdepicting the range and frequency of saccadic extent under various control and workloadconditions for one subject. The small vertical arrows in each panel depict how saccadic ranges were measured.
omitted. It may be seen that under conditions of steady fixation (panel a) the distribution of frequency modulation was quite narrow, indicating that the extent of leftward or rightward movement was quite small. Under conditions of 20 degree alternate fixation (panel b), the distribution of frequency modulation is bimodal, indicating that the extent of leftward and rightward eye movements was quite extensive. These data were used to calibrate the abscissa (saccadic extent) in degrees of visual angle. Under conditions of free viewing (panel c), the distribution of frequency modulation was greater than fixation and alternating fixation, indicating that the range of eye movements during this condition was more extensive than both steady fixation and saccades of 20 degrees. The effects of tone counting are depicted in panels d through f for one-, two-, and three-channel counting. It is evident that as task load increased, the extent of frequency modulation decreased. The recording procedure was contaminated by eye blinks and the artifacts render a measure of the standard deviation of the histogram inappropriate. However, measurement of eye blink signals and the modulation that resulted from them indicated that they could not have contributed to or contaminated a measure of the range of the histogram. Thus, the index of interest is a measure of the extent of eye movements under these different conditions of workload. For this purpose, the range of the histogram was computed and transformed to degrees of visual angle for the saccades and was further normalized by dividing each range by the range obtained under alternating fixation and then multiplying by 20. This was done because there were substantial overall differences in both fixated and spontaneous eye movements between subjects.
J.G. May et al. / Workload and eye movements
81
Average normalized spontaneous saccadic range as a function of workload is presented in fig. 1. The average saccadic ranges for free viewing and fixation are indicated by arrows on the left ordinate. Although the main effect of workload only approached significance (F(2, 8) = 4.22; p = 0.056), it is clear that saccadic extent decreased as a function of workload (F(1, 4) = 9.74, p = 0.035, for the linear component). To further substantiate the relation of saccadic extent to workload correlation coefficients between saccadic extent and workload (1, 2, 3) were computed for each subject, which averaged r = -0.69, and ranged from -0.33 to -0.99, the greater the workload, the less the range of eye movements for each subject. Discussion It is clear from the performance data of this experiment that the modified Jerison counting task offers considerable control of task workload and provides an excellent behavioral index of that parameter. Although the performance scores varied on average from 64 to 96 percent in the present study, this technique can be modified by the inclusion of more tone categories and increased rate of tone presentation to expand the range of workloads investigated. Such a manipulation might well improve the correlation obtained between saccadic and behavioral measures of workload. The eye movement results are encouraging in that the extent of spontaneous saccades was significantly restricted as task difficulty increased. These measures could be obtained easily in many situations that require dynamic information processing, but a number of potential problems must be addressed. The extent to which these procedures might be used in situations where visual information is to be processed is another important consideration. It may be that a decrease in saccadic extent is also observed with increased workload in situations where visual monitoring of events is necessary. The degree to which this relationship exists may depend on whether the primary visual task requires precise fixation or tracking performance, but many visual activities do not. In experiment 2 this relationship was re-examined with a visual counting task that is analogous to the previously used auditory counting task. Although all correlations of eye movement range with workload were negative, the predicted direction, the analysis of variance results were only marginally significant; therefore, in subsequent experiments larger sample sizes were used.
Experiment 2: The effects of visual taskload on the extent of saccadic eye movement Method Subjects and apparatus Ten subjects were used in this experiment. Four were male and five were female. They ranged in age from 19 to 26 years. They were asked to perform low, medium, and high difficulty levels of a visual counting task while eye movements were recorded using the same methodology reported in experiment 1. The same microcomputer was used to administer the task, but the microprocessor was reprogrammed to present a
J.G. May et al. / Workload and eye movements
82
EXPERIMENT
2
o--o
40 35 ---100
z
30 --90
25 --
2 p 0 m
20 -8o
15 -f---F
10-l
3
r70
LOW
MEDIUM
TASK Fig. 3. Mean performance
: m
HIGH
DlFFlCbLTY
and saccadic extent measures as a function of workload in experiment 2. Conventions as in fig. 1.
on a visual task
series of three dark rectangles (1 cm X 2 cm) on the face of an LCD screen. The rectangles were arrayed horizontally in three channels (left, center, and right) with the left and right rectangles located ten degrees to each side of the central one. The screen was located 18 inches in front of the subject’s bite bar. Each rectangle was presented for one second and their order of occurrence was aperiodic with the average rate of occurrence being 0.2 Hz. Procedure Under the three counting conditions the subjects: (1) counted each occurrence of the left rectangle and depressed a key after each fourth occurrence; (2) counted the occurrence of each left and middle rectangle and depressed different keys after the fourth occurrence of each one, and (3) counted the occurrence of each left, middle, and right rectangle and depressed different keys after the fourth occurrence of each. Fixation, alternating fixation, and free-viewing control conditions were run as described in experiment 1. Results Measures of rectangle counting performance and normalized saccadic range were computed for each condition. The mean saccadic extent and counting performance scores have been presented in fig. 3 for each counting condition. The performance with increasing task load declined, but overall performance was quite high relative to that for tone counting in experiment 1. Decreasing saccadic extent was, again, associated with increases in task difficulty. Analysis of variance revealed significant main effects for performance (F(2, 18) = 6.21; p < 0.009) and saccadic extent (F(2, 18 = 16.06;
J.G. May et al. / Workload and eye mouements
83
p -C 0.0001). Trend analysis showed both significant linear and quadratic components of the performance data (p = 0.0355 and p = 00312, respectively), but a highly significant linear trend for saccadic range (F(1, 18) = 16.06; p = 0.0001) and no quadratic trend. Correlations between workload and saccadic extent for each subject ranged between - 0.66 and - 1.00, with a mean of - 0.92. Discussion These data indicate that the saccadic extent index of taskload is valid under conditions of visual monitoring. Thus, it appears as though this technique may be generalized to many visual performance tasks which do not require precise fixation or tracking. Furthermore, the correlations indicate that the range of eye movements was a strong predictor for individual subjects. Both the two previous experiments were carried out with subjects that had little practice on the tone counting task. The possibility exists that the relationship between saccadic extent and workload may deteriorate with practice. For this reason we examined the effects of practice with the three channel (high workload) task.
Experiment 3: The effects of practice on saccadic extent index of mental workload
Method
Subjects Ten subjects were employed in this experiment. Five were male and five were female. They ranged in age from 20 to 32 years. Each participated in 10 eye movement recording sessions occurring over a period of 10 successive days. Each subject was paid $100.00 at the completion of the experiment. Apparatus The same eye-movement techniques used in the previous experiments were used here. The high workload (three channel) version of the tone counting task was administered as in experiment 1. Procedure Subjects participated in 10 sessions. In the first session, each subject performed the auditory tone counting task once prior to recording eye movements. The control conditions, fixation, alternate fixation, and free viewing, were then performed for five minutes each as in the previous experiment. Following the control conditions, eye movements were recorded while subjects performed the tone counting task for five minutes. The procedure was repeated in the subsequent nine sessions except that subjects were not given pretest practice.
84
J.G. May et al. /
o-o
r\
Workload and eye movements
EXPERIMENT
o--o
3
100
40,
-90 : -- 80
z
-- 70
2
6o
P 0 rl
-- 50 2 -- 40
1
2
3
4
5
6
7
8
9
10
SESSIONS Fig. 4. Mean performance and saccadic extent measures as a function of repeated tests at high workloadin experiment3. Conventions as in fig. 1.
Results
Data reduction involved normalizing the range of saccades for fixation, free-viewing and task-related saccades by dividing these measures by the range for alternate fixation. These three measures and the performance scores (mean percent correct) for each session were submitted to an analysis of variance. For eye movement data, a condition by sessions design was employed. For the tone counting performance measure, a simple repeated measured design was employed. The mean tone counting performance as a function of practice sessions is depicted in fig. 4. It is apparent in the figure that performance increased considerably with practice. This is supported by the results of the analysis of variance which revealed a significant main effect for sessions (F(9, 18) = 9.13; p < 0.01). Mean normalized saccadic extent across testing sessions are also depicted in fig. 4 for each condition. The mean ranges for the fixation condition are consistently lowest across sessions, while those for free-viewing are highest. The ranges obtained in the task condition fell between those obtained for the free-viewing and fixation conditions. Analysis of variance revealed that the saccadic extent measures did not change systematically with practice on the tone counting task. These conclusions are supported by the results of the analysis of variance which revealed a significant main effect for conditions (F(1, 2) = 10.1; p < 0.01) but no other significant effects. Discussion
The results show: (1) that the average saccadic extent obtained while subjects perform a difficult tone counting task is restricted compared to free-viewing conditions,
85
J.G. May et al. / Workload and eye movements
and (2) that percent correct performance on the tone counting task continued to improve across ten testing sessions while saccadic extent was not modified as a function of practice. The first finding reinforces the contention that saccadic extent may be used to measure mental workload. The second finding runs contrary to the expectation that the performance would become automated with practice and that saccadic extent would increase. It may be hypothesized that, although subjects were getting better at the tone counting task, it remained a ‘high’ workload task which was reflected in saccadic extent which remained constant across sessions. If this hypothesis were true, then it could be expected that if these subjects were administered low and medium difficulty levels of the tone counting task, saccadic extent should reflect the decrease in workload but performance should remain high. This expectation was tested and confirmed with three subjects used in experiment 3. In experiment 2, we determined that the range of spontaneous saccades decreased with increasing workload when a visual task, that did not require fixation or controlled eye movements, was used. The possibility exists that this potential index of taskload might be disrupted by reflexive eye movements. To test this notion, we measured the range of saccadic extent under different taskloads during optokinetic stimulation.
Experiment 4: The effects of auditory taskload on the extent movements obtained under low, medium, and high workloads
of optokinetic
eye
Method Subjects and apparatus Ten new subjects were used. Six were male and four were female and they ranged in age from 23 to 34 years. The same eye-movement techniques and tone counting task used in experiment 1 was used in this experiment. In addition, optokinetic stimulation was provided by a 20 degree field of vertical stripes displayed on a color monitor (Tektronix, Model 690SR). The stripes were red and green and had spatial frequency of 1.2 c/d. The stripes appeared to drift to the right for 10 seconds and then to drift to the left for 10 seconds. This cycle was repeated for 5 minutes. Procedure Subjects were required to participate in the control conditions as in experiment 1 (fixation and alternate fixation), and to undergo 5 minutes of optokinetic stimulation while eye movements were recorded. Next, they were required to perform the one-tone, two-tone, and three-tone counting tasks. Each of the tone counting tasks were performed for five minutes under conditions of optokinetic stimulation while eye movements were recorded. Results The data were reduced and analyzed as in experiment 1. The mean performance scores (percent correct) and the mean normalized saccadic extent are presented in fig. 5
86
J.G. May et al. / Workload and eye movements
EXPERIMENT
o-o
A
20~
: E E
18--
-0 V
8
16--
2
14--
4
o--o 100 -- 90
(---OKN
A,_ I
. . 1.
P\l
-- 80
T
%
-0 1‘1,
\
0 3
12-- C---F
0 :
109
:
3:
-- 70
E
-60
z m
-50 pa
3 u
i r40
LOW
MEDIUM
TASK
HIGH
DIFFICULTY
Fig. 5. Mean performance and saccadic extent measures as a function of workload in the presence of optokinetic stimulation in experiment 4. Conventions as in fig. 1.
for each tone counting condition. As is apparent in the figure, tone counting performance declined significantly with task load, as expected. Interestingly, saccadic extent decreased with increased task difficulty, even when optokinetic stimulation was used. Analysis of variance showed a highly significant main effect and linear component for performance (F(2, 18) = 18.00; p = 0.0001 and F(1, 9) = 38.34; p = 0.0002, respectively), but no quadratic component. Analysis of the saccadic range data showed a similar pattern of significance for workload main effect I;(2, 18) = 5.49; p = 0.026) and for the linear component (F(1, 9) = 7.11; p = 0.026). Correlations between saccadic extent and workload for individual subjects ranged from 0.87 to -0.95; although correlations for three subjects were positive, the mean correlation was - 0.49, reflecting the fact that the majority of subjects showed the predicted behavior.
Discussion
Saccadic extent measured in subjects who performed tasks of low, medium, and high difficulty reflected changes in mental workload even under conditions of optokinetic nystagmus. This finding indicates that the saccadic extent index of workload is a valid measure even when a reflexive visual forcing function is employed. The correlational analysis did show that three of the ten subjects were disrupted and did not show the predicted relationship, so that in terms of individual differences, the saccadic measure may not be as reliable in the presence of optokinetic stimulation or other oculo-motor stimuli.
J. G. May et al. / Workload and eye movements
87
General discussion The major finding of this investigation is that the range of spontaneous eye movements decreases as cognitive workload increases. This is the case when an auditory counting task is used and also when a visual counting task, that does not involve fixation or tracking, is employed. The results of experiments 3 indicated that the saccadic extent measure is not affected by extended practice under difficult task conditions. Thus, under extended practice on a task that induces high workload, saccadic extent reflected workload while performance increased. This finding is important given the previous use of performance indices to measure workload. Finally, the results of the last experiment suggest that saccadic extent remains a valid measure of the mental effort required under low, medium and high task difficulty even in the presence of a reflexive visual forcing function. This finding is important given the increasing concern with temporal factors involved in workload (e.g., Matthews 1986) and the possible effects of practice on workload indices reported in other research (Wilson et al. 1986). The techniques employed in the present investigation included two serious limitations. First, the eye tracking apparatus was insensitive to eye movements in non-horizontal meridians. It may be possible to use similar techniques with instruments that track vertical as well as horizontal eye movement signals. Second, the apparatus employed did not allow for the exclusion of eye-blinks. As Stern and Skelly (1984) have shown, eye-blinks change under conditions of cognitive workload, but our present findings cannot be attributed to changes in rate of eye blink. ’ However, it may well be that a system which measures both frequency of eye blinks and saccadic range uncontaminated by the former would allow a more sensitive measure of eye movement variance (e.g. the standard deviation) to be appropriate and thereby offer a more sensitive index of taskload. A final caveat concerns the applicability of this technique to real world settings. Although the apparatus used in the present study required a bite bar for head stabilization, other less restrictive methods are available and need only involve head mounted eye monitoring equipment. ’ The change in the signal level resulting from an eye blink was only 6% of the signal associated with 20” saccades. Thus, eye blinks could produce artifactual measures of no greater than 1.2O. These artifacts were considerably less than the minimum saccadic range observed in any subject in any of the experiments.
88
J. G. May et al. / Workload and eye mouements
These findings raise interesting questions regarding previous reports of attentional constriction during conditions of high levels of mental workload. If this phenomenon is associated with an attempt to restrict information processing’ to a limited portion of the central visual field, then it stands to reason that a concomitant restriction of eye movements would facilitate such a process. Future research aimed at assessing attentional constriction and eye movement extent under varying levels of cognitive workload would further elucidate this relationship.
References Beatty, J., 1982. Task-evoked pupillary responses, processing load, and the structure of processing resources. Psychological Bulletin, 91, 276-292. Beatty, J. and B.L. Wagoner 1978. Pupillometric signs of brain activation vary with level of cognitive processing. Science 199, 1216-1218. Bursill, A.E., 1958. The restriction of peripheral vision during exposure to hot and humid conditions. Quarterly Journal of Experimental Psychology 10, 113-129. Crammon, D.V. and U. Schuri, 1980. Blink frequency and speech motor activity. Neuropsychologia 18, 603-606. Edwards, D.C. and P.A. Goolkasian, 1974. Peripheral vision location and kinds of complex processing. Journal of Experimental Psychology 102, 244249. Gopher, D. and R. Braune, 1984. On the psychophysics of workload: Why bother with subjective measures? Human Factors 26, 519-532. Holland, M.K. and G. Tarlow, 1972. Blinking and mental load. Psychological Reports 31, 119-127. Holland, M.K. and G. Tarlow, 1975. Blinking and thinking. Perceptual, and Motor Skills, 41, 403-406. Holmes, D.L., K.M. Cohen, M.W. Haith and F.J. Morreson, 1977. Peripheral visual processing. Perception and Psychophysics 22, 571-577. Jerison, H., 1956. Effect of a combination of noise and fatigue on a complex counting task (WADC TR-55-360). Dayton, OH: Wright Air Development Center, Wright-Patterson AFB. Kahneman, D., 1973. Attention and effort. Englewood Cliffs, NJ: Prentice-Hall. Kahneman, D. and J. Beatty, 1966. Pupillary diameter and load on memory. Science 154, 1583-1585. Kahneman, D. and J. Beatty, 1967. Pupillary responses in a pitch discrimination task. Perception and Psychophysics 2, 101-105. Kahneman, D. and P. Wright, 1971. Changes in pupil size and rehearsal strategies in a short-term memory task. Quarterly Journal of Experimental Psychology 23, 187-196. Kahneman, D., W.S. Peavler and L. Onuska, 1968. Effects of verbalization and incentive on pupil response to mental activity. Canadian Journal of Psychology 22, 186-196. Macworth, N.H., 1965, Visual noise causes tunnel vision. Psychonomic Science 3, 67-68. Matthews, M.L., 1986. The influence of visual taskload history on visual performance. Human Factors 28, 623-632. Moray, N., 1967. Where is attention limited? A survey and a model. Acta Psychologica 27, 84-92. Payne, D.T., M.E. Parry and S.J. Harasymiw, 1968. Percentage of pupillary dilation as a measure of item difficulty. Perception and Psychophysics 4, 139-143.
J.G. May et al. / Workload and eye movements
89
Schaefer, T. Jr., J.B. Ferguson, J.A. Klein and E.B. Rawson, 1968. Pupillary responses during mental activities. Psychonomic Science 12, 137-138. Schmidt, D., 1978. A queueing analysis of the air traffic controllers’ workload. IEEE Transactions on Systems, Man, and Cybernetics, SMC-8, 492-498. Schuri, U. and D.V. Cammon, 1981. Heart rate and blink rate responses during mental arithmetic with and without continuous verbalization of results. Psychophysiology 18, 650-653. Simpson, H.M. and S.M. Hale, 1969. Pupillary changes during a decision-making task. Perceptual and Motor Skills 29, 495-498. Stem, J.A. and J.J. Skelly, 1984. The eye blink and workload considerations. Proceedings of the 28th Human Factors Society Meeting, San Antonio, CA, Santa Monica, CA, Human Factors Society. Teichner, W.H., 1968. Interaction of behavioral and physiological stress reactions. Psychological Review 75, 271-291. Wickens, C., 1984. Engineering psychology and human performance. Columbus, OH: Charles E. Merrill. Williams, L.J., 1982. Cognitive load and the functional field of view. Human Factors 24, 683-692. Williams, L.J., 1985. Tunnel vision induced by a fovea1 load manipulation. Human Factors 27, 221-227. Williams, L.J. and L.A. Lefton, 1981. Processing of alphabetic information presented in the fovea or the periphery: Functional field of view and cognitive workload. Perception 10, 645-650. Wilson, G., K. McCloskey and I. Davis, 1986. Linguistic processing: Physiological & subjective correlates. Proceedings of the 30th Human Factors Society Meeting, Dayton, OH, Santa Monica, CA, Human Factors Society. Wood, C.L. and J. Hasset 1950. Eye blinking during problem solving: The effect of problem difficulty and internally vs externally directed attention. Psychphysiology, 20, 18-20.