- Email: [email protected]
Pupil detection and tracking for analysis of fixational eye micromovements
Optik 123 (2012) 11–15
Contents lists available at ScienceDirect
Optik journal homepage: www.elsevier.de/ijleo
Pupil detection and tracking for analysis of fixational eye micromovements Ana B. Roig a , Marta Morales b , Julian Espinosa a , Jorge Perez a , David Mas a,∗ , Carlos Illueca a a b
Dep. Optica, University of Alicante, PO Box 99, 03080 Alicante, Spain Dep. Fisica Aplicada, University of Alicante, PO Box 99, 03080 Alicante, Spain
a r t i c l e
i n f o
Article history: Received 4 May 2010 Accepted 15 October 2010
Keywords: Eye moments target tracking Miccrosaccades ocular drifts image processing
a b s t r a c t Even with fixed gaze, the human eye is under steady motion (tremor micosaccades and drifts). Detection of these movements requires of invasive techniques or expensive devices with sophisticated detection methods. In this paper we present a technique for pupil segmentation and contour analysis which will provide valuable information about fixational eye movements. The method is based on fitting an ellipse to the pupil contour. Pupillary hippus, microsaccades and drifts are obtained as well as cyclotorsional movements. The method is simple and the experimental requirements are easily available since just a biomicroscope and a digital videocamera are required. © 2011 Elsevier GmbH. All rights reserved.
1. Introduction The human eye is constantly moving even when it is maintaining a stable fixation point. Usually, these movements are too small for being appreciated without specific instrumentation but they play a fundamental role in eye physiology and, unfortunately, increase the complexity of the eye with respect to conventional optical systems [1]. Stable images on the retina rapidly saturate the photoreceptors and fade. The visual system needs of constant stimulation to which it adapts. These constant changes are achieved by fixational micromovements which consists of microtremors, drifts and microsaccades [2–5]. Microtremors are constant high frequency (80 Hz) low amplitude (120–2500 nm) eye tremors. Their role in vision is to avoid image fading due to saturation of retinal photoreceptors. Drifts are smooth pursuit movements associated to the instability of the oculomotor system and microsaccades are sudden jerk like movements of the eye of amplitude varying from 2 to 120 arcmin and around 25 ms of duration. Their possible role is to correct displacements in eye position produced by drifts. The equipment for eye movement detection has significantly advanced in recent decades from rudimentary and invasive methods, like the electrooculogram [6] or scleral search coils [7–9], to non-invasive systems as the video-oculography [10]. The principle of the “search coil” technique is based on the induction of an electrical field on as small coil [7]. Although it is highly invasive, this method is actually considered the “gold stan-
∗ Corresponding author. E-mail address: [email protected] (D. Mas). 0030-4026/$ – see front matter © 2011 Elsevier GmbH. All rights reserved. doi:10.1016/j.ijleo.2010.10.049
dard” for eye movement measurements. The induction coil is inside a flexible ring of silicone rubber which adheres to the limbus of the human eye concentric with the cornea, through a contact lens [8,11]. The head of the subject is placed inside magnetic fields. Induced voltages in the coil provide data about horizontal and vertical positions of the eyes, as well as cyclotorsions. Pupil detection has been used for accurate calculation of eye orientation [12]. It is usually assumed that the optical axis (the line through the pupil center and the eye rotation center) coincides with the direction of gaze. Thus, accurate determination of pupil center is crucial for precise eye tracking [13]. Pupil center is usually determined by detecting its border and assuming a particular geometrical shape. The simplest approximation consists on assuming that the pupil is rotationally symmetric [14], however elliptical shape seems to be a better choice [13]. The determination of the pupil shape parameters is also of great interest for non-invasive early diagnosis of the central nervous system response to environmental stimuli [15]. Pupil geometrical features such as area, semiaxis, centroid and orientation can be obtained by image segmentation. The aim of this work is to present a simple non-invasive optical method for detecting and measuring fixational micromovements. A video camera has been attached to a biomicroscope and the anterior segment of the eye has been captured. The contour of the pupil has been detected and fitted to an ellipse. Through its geometrical parameters we have studied the spontaneous pupil dilation, microsaccades and drifts and also cyclotorsions. The presented technique is simple and rather inexpensive since the experimental requirements are available in all optometric and ophthalmic faculties and can be easily developed by a graduate student. The principles here explained can be also extended to the
12
A.B. Roig et al. / Optik 123 (2012) 11–15
Fig. 1. Slit-lamp and digital video camera used to capture the images of frontal corneal.
analysis of more complex effects, either using a high speed camera or adapting the setup for binocular observation. 2. Subjects and methods Images of the frontal cornea were obtained by illuminating the eye with a Haag–Street style slit-lamp (SL-990). Video sequences were captured with a digital video camera working at 63 fps and a spatial resolution of 800 × 560 px attached to the lamp (see Fig. 1). Diffused illumination was used to obtain uniform illumination on the whole cornea and a red filter was additionally used to avoid discomfort to the subject. Five healthy subjects (3 women, 2 men, aged 22–40) among the staff of the Optics Department of the University of Alicante were asked to participate in this experience. We adhered to the tenets of the Declaration of Helsinki during this study. All participants were informed about the nature and purpose of the study and all of them provided informed consent. Three different sequences were registered for each subject’s right eye with the left eye occluded. Measurements were taken in total darkness to avoid reflections and discomfort to the subjects. During the measurements a red LED was used as a fixation point. Residual head movements were restrained by the use of a dental bite bar and firmly fastening the head to the chinrest frame. Finally, the chinrest structure was reinforced with lateral supports, as can be appreciated in Fig. 1. In order to further minimize the head movements the subjects were asked to not breathe during the measuring time, which lasted 10 s. Obtained images are analyzed off-line using MATLAB. Edge detection algorithms were applied to determine the pupil contour which is fitted to an ellipse. Position of the center of the ellipse gives information about slow (drifts) and sudden eye movements (microsacaddes), while rotation of the axes describes spontaneous cyclo-torsional movements. Furthermore, variation of the pupil area provides information about spontaneous changes in pupil aperture, known as “hippus” [16]. 3. Image processing algorithms The method here applied consists of detecting the pupil contour in all frames and calculating the best ellipse fitting to it. First, a threshold value is selected for hard clipping of the image thus obtaining a black region of interest which corresponds to the pupil. A morphological closing operation is then performed in order to
Fig. 2. (a) Image representing the pupil contour in the presence of sparkle, (b) adjustment of contour data, and (c) image representing the pupil contour and adjustment.
eliminate noise in the image that could distort the pupil contour [17]. Edge detection determines the border of the black area and thus traces the pupil contour. In many occasions, reflections from the illumination system may appear inside the region of interest (see Fig. 2). Isolated reflections inside the pupil area will not disturb the detection process but those close to the border may interfere in the obtained contour. In order to eliminate this interference the size of the sparkle is delimited by selecting all connected saturated pixels in the image. If the area so obtained intersects with the pupil
A.B. Roig et al. / Optik 123 (2012) 11–15
13
Fig. 3. Variation of the pupil diameter for the subject AR.
Fig. 5. Representation of the gaze tracking where can see normal movements (drifts and microssacades).
Fig. 4. Eccentricity of the ellipse for the same subject (subject AR).
contour these data are removed from the set to be fitted to an ellipse. The excluded data are a small percentage of the points delimiting the pupil and they not affect to the goodness of the fit. The coordinates (x, y) of the pixels forming the border of the pupil are fitted to the ellipse general equation: A · x2 + B · xy + C · y2 + D · x + E · y + F = 0
(1)
In Annex 1 we show the obtention of the geometrical parameters of the ellipse (semi-axes, center and orientation) from the polinomial parameters in expression (1). The MATLAB procedure for such adjustment can be found in [18]. The correspondence between pixels and real distances is obtained by capturing a pattern with known size. Thus, we obtained a resolution of 40 px/mm, being the accuracy of the measurement of 2.5 m. If we consider that the rotation center of the eye is situated at around 15 mm from the corneal apex [19] the angular resolution of the image is 10.5 px/◦ . 4. Results 4.1. Pupil diameter We have studied the spontaneous variations of the pupil diameter under constant illumination or hippus [16,20]. In Fig. 3 we show the variation of the major and minor axes with respect their mean values for a typical observer. Coincidence of variations in both axis shows that there is not asymmetric deformation of the pupil during is activity. In Fig. 4, we show the eccentricity of the ellipse for the Table 1 Pupil area and the eccentricity obtained for all subjects. Subject
Area (mm2 )
AR MM JE DM BD
18.55 8.70 12.13 10.58 8.04
± ± ± ± ±
1.18 0.47 0.49 0.11 0.37
Eccentricity 0.14 0.20 0.27 0.23 0.24
± ± ± ± ±
0.01 0.02 0.01 0.03 0.01
same subject (subject AR). From both Figs. 3 and 4 we can appreciate that, in general the pupil maintains its shape throughout the measurement time. In Table 1 we show the eccentricity obtained for all the observers, together with the pupil area variation. We can see there that variation of the pupil area due to hippus is around 5% while variation in eccentricity is around 8%. Results of area variation are in agreement with the literature [21] which reports area oscillations larger than 3%. Although pupil shape is assumed to be elliptic [15], no information about average eccentricity and its variation was found, since vast majority of pupilometers used in ophthalmology assume circular pupils for healthy subjects. A statistical study with a larger number of subjects will determine the mean values and variation of the pupil eccentricity. 4.2. Eye movements Eye movement has been analyzed by tracking the center of the ellipse (X0 , Y0 ). Since variation in both minor and major axis is equivalent, displacement of the pupil center may not to correspond to shape variations. It has been described in the bibliography [22] that pupil can displace depending whether the eye is looking to close or far objects. In this case, since the accommodation distance is fixed, we can assume that registered displacements are due to eye movements. In Fig. 5 we show the gaze tracking of one typical subject (subject JE). We would like to underline that observed dynamics correspond to normal movements (drifts and microsaccades) while the sight is fixed on a target. Microsaccades can be recognized as long and sudden movements of the eye in the horizontal direction, while drifts are smooth movements that happen between two successive microsaccades [23,24]. Separate representation of vertical and horizontal components provides better insight about the eye movement structure. In Fig. 6 we represent those components after removing a linear trend. Microsaccades can be clearly identified there as sudden jumps. These movements are considered small saccades that occur during fixation to keep the image on the fovea and/or to correct displacements in eye position produced by drifts. Their duration is of about 25 ms (1–2 frames) and occur 1 or 2 per second along a straight path mainly in the horizontal direction [12,23–26] as can be seen from the figure. Amplitude of microsaccades can vary between 1 arcmin to 1◦ . In our case we roughly calculate a mean value of 30 arcmin. In some measurements and for some subjects appeared peaks of more than 5◦ which probably represent big saccades due to loss of fixation. In any case, the estimation has been done by hand and an automatic procedure has to be design in order to recognize and properly describe microsaccadic movements. As we described above, drifts can be seen as a low frequency variation in the position of the eye. Fine variations of the eye
14
A.B. Roig et al. / Optik 123 (2012) 11–15
method here developed is very dependant on accurate determination of geometrical parameters in the ellipse, and errors in their determination can distort results from indirect measurements like the angle. Other source of error may come from the loss of fixation. Change of fixation point may also produce an adjustment of the binocular vergence and thus modify the angle of the ellipse. Unfortunately the method does not allow distinguishing between the different possibilities. 5. Conclusion
Fig. 6. (a) Representation of horizontal component for subject AR where microssacades can be clear identified and (b) representation of vertical component for the same subject.
Applied method for detecting pupil contour allows tracking the fixational eye movements. With a simple setup and software application, we are able to detect small microsaccades and drift movements, as well as cyclotorsional movements. Microtremors are movements of very small amplitude and very high frequency so their detection is beyond the capabilities of our system. Obtained results are in agreement with those found in the bibliography. Identification of microsaccades and drifts is clear, and it is also possible to detect pupilar hippus. Cyclotorsions are also detected but results are not clear and some improvement is needed. Information reported here may be enough to reproduce the experiment by any postgraduate student. The setup can be also modified in order to analyze binocular correlations, pupil movement associated to accommodation or analyze whether these movements respond to any clear pattern on they are chaotic. By adapting the illumination system to have more light, one could even increase the temporal resolution of the camera and analyze eye dynamics before and after microsaccade episodes or detect and analyze microtremors. Acknowledgement This work has been supported by the Spanish Ministerio de Ciencia e Innovacion through the project FIS2009-05639-E. Annex 1. An ellipse is determined by
Fig. 7. Representation the ellipse orientation for the subject MM.
position around the drift path may come from eye pulsations or under sampled tremors, which have a frequency around 100 Hz. Our system samples at 63 Hz, so movements above 32 Hz will not be correctly registered. The specific values of the ellipse orientation, which can be obtained from expression (A.14) are not an important parameter, but it is their variations, since they will inform about cyclotorsional eye movements. In Fig. 7 we show this parameter for the subject here analyzed. In Table 2 we show the mean ± SD values obtained for each subject. By considering the standard deviation as a measure of variation, we can see in the table that values differ very much between subjects. Values found in the literature are around 2◦ [27] although cyclotorsions of 9.5◦ during surgical interventions have been reported [28]. For some of the observers, values are close to those reported, while for others, discrepancy is really high. The
2
Ax + Bxy + Cy2 + Dx + Ey + F = 0
(A.1)
First, we study the ellipse’s orientation relative to coordinate axes. The coefficient B equal to zero indicates that the ellipse’s axes are parallel to the coordinate axes. In this case, if C > A the ellipse presents the major axis parallel to the horizontal one. On the contrary, major axis is parallel to the vertical one. The center position and the length of the axes of the ellipse can be determined as follows Ax2 + Cy2 + Dx + Ey + F = 0
A x+
D 2
2A
D2
−
4A
+C y+
E 2 2C
2
(A.2) −
E2 4C
+F =0
(A.3)
2
E2 (x + (D/(2A))) (y + (E/(2C))) D2 + −F + = 4A 4C 1/A 1/C 2
(A.4)
2
(x + (D/(2A))) (y + (E/(2C))) + =1 ((D2 /(4A2 )) + (E 2 /(4AC)) − (F/A)) ((D2 /(4AC)) + (E 2 /(4C 2 )) − (F/C))
(A.5)
Thus the ellipse is centered in the point Table 2 Angular oscillation for all subjects.
(xc , yc ) =
Subject
Angle (◦ )
AR MM JE DM BD
−11.752 9.604 2.983 −0.507 −9.760
± ± ± ± ±
14.068 7.904 21.031 3.646 3.194
D −
2A
,−
E 2C
(A.6)
Major semiaxis, a, minor semiaxis, b, and the eccentricity, e, are given by
a = max
E2 F D2 + − 2 4AC A 4A
,
F E2 D2 − + 4AC C 4C 2
(A.7)
A.B. Roig et al. / Optik 123 (2012) 11–15
b = min
e=
1−
D2 E2 F + − 2 4AC A 4A
,
D2 F E2 − + 4AC C 4C 2
b 2
(A.8)
(A.9)
a
In case of being B different to zero in (1), we redirect the ellipse by making a rotation of angle − (angle between the major axis with the horizontal one) in order to set ellipse’s axes parallel to coordinate ones. The new coordinates (x , y ) are
x y
=
cos(−) sin(−)
− sin(−) cos(−)
x y
(A.10)
Now, the ellipse can be expressed as 2
A(x cos + y sin ) + B(x cos + y sin )(−x sin + y cos ) 2
+ C(−x sin + y cos ) + D(x cos + y sin ) + E(−x sin + y cos ) + F = 0
(A.11)
Grouping terms we get 2
A x + B xy + C y2 + D x + E y + F = 0
(A.12)
with A = A cos2 + C sin2 − B sin cos B = 2(A − C) sin cos + B cos2 − B sin2 C = A sin2 + C cos2 + B sin cos D = D cos − E sin E = D sin + E cos F = F
(A.13)
The principal axes of this ellipse must be parallel to the Cartesian axes, so B must be zero. From this condition (B = 0) the angle results =
1 arctan 2
B C −A
(A.14)
Now, from the resulting ellipse, we can obtain the center position and the length of the axes applying Eqs. (A.7)–(A.9) with the parameters in (A.13). References [1] H. Kasprzak, D.R. Iskander, T. Bajraszewski, A. Kowalczyk, W. NowakSzczepanowska, High accuracy measurements of spectral characteristics of movement of the eye elements, Opt. Pura Apl. 40 (2007) 7–11.
15
[2] R.W. Ditchburn, J.A. Foley-Fisher, Assembled data on eyemovements, Opt. Acta 14 (1967) 113–118. [3] A. Yarbus, Eye Movements and Vision, Plenum Press, New York, 1967. [4] R.H.S. Carpenter, Movements of the Eyes, 2nd ed., Pion, London, 1988. [5] S. Martinez-Conde, S.L. Macknik, D.H. Hubel, The role of fixational eye movements in visual perception, Nat. Rev. Neurosci. 5 (2004) 229–240. [6] E. Marg, Development of electro-oculography; standing potential of the eye in registration of eye movement, AMA Arch. Ophthalmol. 45 (1951) 169–185. [7] D.A. Robinson, A method of measuring eye movement using a scleral search coil in a magnetic field, IEEE Trans. Biomed. Eng. 10 (1963) 137–145. [8] H. Collewijn, F. Van der Mark, T.C. Jansen, Precise recording of human eye movements, Vision Res. 14 (1975) 447–450. [9] L. Ferman, H. Collewijn, T.C. Jansen, A.V. Van den Berg, Human gaze stability in the horizontal, vertical and torsional direction during voluntary head movements, evaluated with a three-dimensional scleral induction coil technique, Vision Res. 27 (1987) 811–828. [10] H. Scherer, W. Teiwes, A.H. Clarke, Measuring three dimensions of eye movement in dynamic situations by means of videooulography, Acta Otolaryngol. 111 (1991) 182–187. [11] R.T. Kenyon, A soft contact lens search coil for measuring eye movements, Vision Res. 25 (1985) 1629–1633. [12] J.R. Liang, S. Moshel, A.Z. Zivotofsky, A. Caspi, R. Engbert, R. Kliegl, S. Havlin, Scaling of horizontal and vertical fixational eyen movements, Am. Phys. Soc. 71 (2005) 1–6. [13] D. Zhu, S.T. Moore, T. Raphan, Robust pupil detection using a curvature algorithm, Comput. Methods Programs Biomed. 59 (1999) 145–157. [14] D.R. Iskander, S. Mioschek, M. Trunk, W. Werth, Detecting eyes in digital images, in: Proceedings of the Seventh International Symposium Signal Processing and its Applications, vol. II, 2003, pp. 21–24. [15] A. De Santis, D. Iacovelllo, Optimal segmentation of pupillometric images for estimating pupil shape parameters, Comput. Methods Programs Biomed. 84 (2006) 174–187. [16] H. Bouma, L.C.J. Baghius, Hippus of the pupil: periods of slow oscillations of unknown origin, Vision Res. 11 (1971) 1345–1351. [17] J. Serra, Image Analysis and Mathematical Morphology, Academic Press, Inc., Orlando, FL, 1983. [18] O. Gal, http://www.mathworks.com/matlabcentral/fileexchange/3215fitellipse (last seen in 04.27.2010). [19] G.A. Fry, The center of rotation of the eye, Am. J. Optom. Arch. Am. Acad. Optom. 39 (1962) 581–595. [20] H.J. Wyatt, The form of the human pupil, Vision Res. 35 (1995) 2021–2036. [21] S.Z. Li (Ed.), Encyclopedia of Biometrics, Springer, New York, 2009. [22] L.S. Gray, B. Winn, B. Gilmartin, Accomodative microfluctuations and pupil diameter, Vision Res. 33 (15) (1993) 2083–2090. [23] J.R. Charlier, M. Behague, C. Buquet, Shift of the pupil center with pupil constriction, Invest. Ophthalmol. Vis. Sci. 35 (4) (1994) 118. [24] F. Censi, G. Calcagnini, F. De Pasquale, S. Lino, S. Cerutti, Baroceptorsenstive fluctuations of human pupil diameter, Comput. Cardiol. 26 (1999) 233–236. [25] G. Calcagnini, P. Giovannelli, F. Censi, P. Bartolini, V. Barbaro, Baroreceptorsensitive fluctuations of heart rate and pupil diameter, in: Proc. 23rd Annual EMBS Int. Conf., 25–28, Instanbul, Turkey, 2001, pp. 600–603. [26] D.R. Iskander, M.J. Collins, S. Mioschek, M. Trunk, Autonomic pupillometry from digital images, IEEE Trans. Biomed. Eng. 51 (9) (2004) 1132–1137. [27] J. Porter, G. Yoon, S. MacRae, G. Pan, T. Twietmeyer, I.G. Cox, D.R. Williams, Surgeon offsets and dynamic eye movements in laser refractive surgery, J. Cataract Refract. Surg. 31 (2005) 2058–2066. [28] D.A. Chernyak, Cyclotorsional eye motion occurring between wavefront measurement and refractive surgery, J. Cataract Refract. Surg. 30 (2004) 633–639.
Contents lists available at ScienceDirect
Optik journal homepage: www.elsevier.de/ijleo
Pupil detection and tracking for analysis of fixational eye micromovements Ana B. Roig a , Marta Morales b , Julian Espinosa a , Jorge Perez a , David Mas a,∗ , Carlos Illueca a a b
Dep. Optica, University of Alicante, PO Box 99, 03080 Alicante, Spain Dep. Fisica Aplicada, University of Alicante, PO Box 99, 03080 Alicante, Spain
a r t i c l e
i n f o
Article history: Received 4 May 2010 Accepted 15 October 2010
Keywords: Eye moments target tracking Miccrosaccades ocular drifts image processing
a b s t r a c t Even with fixed gaze, the human eye is under steady motion (tremor micosaccades and drifts). Detection of these movements requires of invasive techniques or expensive devices with sophisticated detection methods. In this paper we present a technique for pupil segmentation and contour analysis which will provide valuable information about fixational eye movements. The method is based on fitting an ellipse to the pupil contour. Pupillary hippus, microsaccades and drifts are obtained as well as cyclotorsional movements. The method is simple and the experimental requirements are easily available since just a biomicroscope and a digital videocamera are required. © 2011 Elsevier GmbH. All rights reserved.
1. Introduction The human eye is constantly moving even when it is maintaining a stable fixation point. Usually, these movements are too small for being appreciated without specific instrumentation but they play a fundamental role in eye physiology and, unfortunately, increase the complexity of the eye with respect to conventional optical systems [1]. Stable images on the retina rapidly saturate the photoreceptors and fade. The visual system needs of constant stimulation to which it adapts. These constant changes are achieved by fixational micromovements which consists of microtremors, drifts and microsaccades [2–5]. Microtremors are constant high frequency (80 Hz) low amplitude (120–2500 nm) eye tremors. Their role in vision is to avoid image fading due to saturation of retinal photoreceptors. Drifts are smooth pursuit movements associated to the instability of the oculomotor system and microsaccades are sudden jerk like movements of the eye of amplitude varying from 2 to 120 arcmin and around 25 ms of duration. Their possible role is to correct displacements in eye position produced by drifts. The equipment for eye movement detection has significantly advanced in recent decades from rudimentary and invasive methods, like the electrooculogram [6] or scleral search coils [7–9], to non-invasive systems as the video-oculography [10]. The principle of the “search coil” technique is based on the induction of an electrical field on as small coil [7]. Although it is highly invasive, this method is actually considered the “gold stan-
∗ Corresponding author. E-mail address: [email protected] (D. Mas). 0030-4026/$ – see front matter © 2011 Elsevier GmbH. All rights reserved. doi:10.1016/j.ijleo.2010.10.049
dard” for eye movement measurements. The induction coil is inside a flexible ring of silicone rubber which adheres to the limbus of the human eye concentric with the cornea, through a contact lens [8,11]. The head of the subject is placed inside magnetic fields. Induced voltages in the coil provide data about horizontal and vertical positions of the eyes, as well as cyclotorsions. Pupil detection has been used for accurate calculation of eye orientation [12]. It is usually assumed that the optical axis (the line through the pupil center and the eye rotation center) coincides with the direction of gaze. Thus, accurate determination of pupil center is crucial for precise eye tracking [13]. Pupil center is usually determined by detecting its border and assuming a particular geometrical shape. The simplest approximation consists on assuming that the pupil is rotationally symmetric [14], however elliptical shape seems to be a better choice [13]. The determination of the pupil shape parameters is also of great interest for non-invasive early diagnosis of the central nervous system response to environmental stimuli [15]. Pupil geometrical features such as area, semiaxis, centroid and orientation can be obtained by image segmentation. The aim of this work is to present a simple non-invasive optical method for detecting and measuring fixational micromovements. A video camera has been attached to a biomicroscope and the anterior segment of the eye has been captured. The contour of the pupil has been detected and fitted to an ellipse. Through its geometrical parameters we have studied the spontaneous pupil dilation, microsaccades and drifts and also cyclotorsions. The presented technique is simple and rather inexpensive since the experimental requirements are available in all optometric and ophthalmic faculties and can be easily developed by a graduate student. The principles here explained can be also extended to the
12
A.B. Roig et al. / Optik 123 (2012) 11–15
Fig. 1. Slit-lamp and digital video camera used to capture the images of frontal corneal.
analysis of more complex effects, either using a high speed camera or adapting the setup for binocular observation. 2. Subjects and methods Images of the frontal cornea were obtained by illuminating the eye with a Haag–Street style slit-lamp (SL-990). Video sequences were captured with a digital video camera working at 63 fps and a spatial resolution of 800 × 560 px attached to the lamp (see Fig. 1). Diffused illumination was used to obtain uniform illumination on the whole cornea and a red filter was additionally used to avoid discomfort to the subject. Five healthy subjects (3 women, 2 men, aged 22–40) among the staff of the Optics Department of the University of Alicante were asked to participate in this experience. We adhered to the tenets of the Declaration of Helsinki during this study. All participants were informed about the nature and purpose of the study and all of them provided informed consent. Three different sequences were registered for each subject’s right eye with the left eye occluded. Measurements were taken in total darkness to avoid reflections and discomfort to the subjects. During the measurements a red LED was used as a fixation point. Residual head movements were restrained by the use of a dental bite bar and firmly fastening the head to the chinrest frame. Finally, the chinrest structure was reinforced with lateral supports, as can be appreciated in Fig. 1. In order to further minimize the head movements the subjects were asked to not breathe during the measuring time, which lasted 10 s. Obtained images are analyzed off-line using MATLAB. Edge detection algorithms were applied to determine the pupil contour which is fitted to an ellipse. Position of the center of the ellipse gives information about slow (drifts) and sudden eye movements (microsacaddes), while rotation of the axes describes spontaneous cyclo-torsional movements. Furthermore, variation of the pupil area provides information about spontaneous changes in pupil aperture, known as “hippus” [16]. 3. Image processing algorithms The method here applied consists of detecting the pupil contour in all frames and calculating the best ellipse fitting to it. First, a threshold value is selected for hard clipping of the image thus obtaining a black region of interest which corresponds to the pupil. A morphological closing operation is then performed in order to
Fig. 2. (a) Image representing the pupil contour in the presence of sparkle, (b) adjustment of contour data, and (c) image representing the pupil contour and adjustment.
eliminate noise in the image that could distort the pupil contour [17]. Edge detection determines the border of the black area and thus traces the pupil contour. In many occasions, reflections from the illumination system may appear inside the region of interest (see Fig. 2). Isolated reflections inside the pupil area will not disturb the detection process but those close to the border may interfere in the obtained contour. In order to eliminate this interference the size of the sparkle is delimited by selecting all connected saturated pixels in the image. If the area so obtained intersects with the pupil
A.B. Roig et al. / Optik 123 (2012) 11–15
13
Fig. 3. Variation of the pupil diameter for the subject AR.
Fig. 5. Representation of the gaze tracking where can see normal movements (drifts and microssacades).
Fig. 4. Eccentricity of the ellipse for the same subject (subject AR).
contour these data are removed from the set to be fitted to an ellipse. The excluded data are a small percentage of the points delimiting the pupil and they not affect to the goodness of the fit. The coordinates (x, y) of the pixels forming the border of the pupil are fitted to the ellipse general equation: A · x2 + B · xy + C · y2 + D · x + E · y + F = 0
(1)
In Annex 1 we show the obtention of the geometrical parameters of the ellipse (semi-axes, center and orientation) from the polinomial parameters in expression (1). The MATLAB procedure for such adjustment can be found in [18]. The correspondence between pixels and real distances is obtained by capturing a pattern with known size. Thus, we obtained a resolution of 40 px/mm, being the accuracy of the measurement of 2.5 m. If we consider that the rotation center of the eye is situated at around 15 mm from the corneal apex [19] the angular resolution of the image is 10.5 px/◦ . 4. Results 4.1. Pupil diameter We have studied the spontaneous variations of the pupil diameter under constant illumination or hippus [16,20]. In Fig. 3 we show the variation of the major and minor axes with respect their mean values for a typical observer. Coincidence of variations in both axis shows that there is not asymmetric deformation of the pupil during is activity. In Fig. 4, we show the eccentricity of the ellipse for the Table 1 Pupil area and the eccentricity obtained for all subjects. Subject
Area (mm2 )
AR MM JE DM BD
18.55 8.70 12.13 10.58 8.04
± ± ± ± ±
1.18 0.47 0.49 0.11 0.37
Eccentricity 0.14 0.20 0.27 0.23 0.24
± ± ± ± ±
0.01 0.02 0.01 0.03 0.01
same subject (subject AR). From both Figs. 3 and 4 we can appreciate that, in general the pupil maintains its shape throughout the measurement time. In Table 1 we show the eccentricity obtained for all the observers, together with the pupil area variation. We can see there that variation of the pupil area due to hippus is around 5% while variation in eccentricity is around 8%. Results of area variation are in agreement with the literature [21] which reports area oscillations larger than 3%. Although pupil shape is assumed to be elliptic [15], no information about average eccentricity and its variation was found, since vast majority of pupilometers used in ophthalmology assume circular pupils for healthy subjects. A statistical study with a larger number of subjects will determine the mean values and variation of the pupil eccentricity. 4.2. Eye movements Eye movement has been analyzed by tracking the center of the ellipse (X0 , Y0 ). Since variation in both minor and major axis is equivalent, displacement of the pupil center may not to correspond to shape variations. It has been described in the bibliography [22] that pupil can displace depending whether the eye is looking to close or far objects. In this case, since the accommodation distance is fixed, we can assume that registered displacements are due to eye movements. In Fig. 5 we show the gaze tracking of one typical subject (subject JE). We would like to underline that observed dynamics correspond to normal movements (drifts and microsaccades) while the sight is fixed on a target. Microsaccades can be recognized as long and sudden movements of the eye in the horizontal direction, while drifts are smooth movements that happen between two successive microsaccades [23,24]. Separate representation of vertical and horizontal components provides better insight about the eye movement structure. In Fig. 6 we represent those components after removing a linear trend. Microsaccades can be clearly identified there as sudden jumps. These movements are considered small saccades that occur during fixation to keep the image on the fovea and/or to correct displacements in eye position produced by drifts. Their duration is of about 25 ms (1–2 frames) and occur 1 or 2 per second along a straight path mainly in the horizontal direction [12,23–26] as can be seen from the figure. Amplitude of microsaccades can vary between 1 arcmin to 1◦ . In our case we roughly calculate a mean value of 30 arcmin. In some measurements and for some subjects appeared peaks of more than 5◦ which probably represent big saccades due to loss of fixation. In any case, the estimation has been done by hand and an automatic procedure has to be design in order to recognize and properly describe microsaccadic movements. As we described above, drifts can be seen as a low frequency variation in the position of the eye. Fine variations of the eye
14
A.B. Roig et al. / Optik 123 (2012) 11–15
method here developed is very dependant on accurate determination of geometrical parameters in the ellipse, and errors in their determination can distort results from indirect measurements like the angle. Other source of error may come from the loss of fixation. Change of fixation point may also produce an adjustment of the binocular vergence and thus modify the angle of the ellipse. Unfortunately the method does not allow distinguishing between the different possibilities. 5. Conclusion
Fig. 6. (a) Representation of horizontal component for subject AR where microssacades can be clear identified and (b) representation of vertical component for the same subject.
Applied method for detecting pupil contour allows tracking the fixational eye movements. With a simple setup and software application, we are able to detect small microsaccades and drift movements, as well as cyclotorsional movements. Microtremors are movements of very small amplitude and very high frequency so their detection is beyond the capabilities of our system. Obtained results are in agreement with those found in the bibliography. Identification of microsaccades and drifts is clear, and it is also possible to detect pupilar hippus. Cyclotorsions are also detected but results are not clear and some improvement is needed. Information reported here may be enough to reproduce the experiment by any postgraduate student. The setup can be also modified in order to analyze binocular correlations, pupil movement associated to accommodation or analyze whether these movements respond to any clear pattern on they are chaotic. By adapting the illumination system to have more light, one could even increase the temporal resolution of the camera and analyze eye dynamics before and after microsaccade episodes or detect and analyze microtremors. Acknowledgement This work has been supported by the Spanish Ministerio de Ciencia e Innovacion through the project FIS2009-05639-E. Annex 1. An ellipse is determined by
Fig. 7. Representation the ellipse orientation for the subject MM.
position around the drift path may come from eye pulsations or under sampled tremors, which have a frequency around 100 Hz. Our system samples at 63 Hz, so movements above 32 Hz will not be correctly registered. The specific values of the ellipse orientation, which can be obtained from expression (A.14) are not an important parameter, but it is their variations, since they will inform about cyclotorsional eye movements. In Fig. 7 we show this parameter for the subject here analyzed. In Table 2 we show the mean ± SD values obtained for each subject. By considering the standard deviation as a measure of variation, we can see in the table that values differ very much between subjects. Values found in the literature are around 2◦ [27] although cyclotorsions of 9.5◦ during surgical interventions have been reported [28]. For some of the observers, values are close to those reported, while for others, discrepancy is really high. The
2
Ax + Bxy + Cy2 + Dx + Ey + F = 0
(A.1)
First, we study the ellipse’s orientation relative to coordinate axes. The coefficient B equal to zero indicates that the ellipse’s axes are parallel to the coordinate axes. In this case, if C > A the ellipse presents the major axis parallel to the horizontal one. On the contrary, major axis is parallel to the vertical one. The center position and the length of the axes of the ellipse can be determined as follows Ax2 + Cy2 + Dx + Ey + F = 0
A x+
D 2
2A
D2
−
4A
+C y+
E 2 2C
2
(A.2) −
E2 4C
+F =0
(A.3)
2
E2 (x + (D/(2A))) (y + (E/(2C))) D2 + −F + = 4A 4C 1/A 1/C 2
(A.4)
2
(x + (D/(2A))) (y + (E/(2C))) + =1 ((D2 /(4A2 )) + (E 2 /(4AC)) − (F/A)) ((D2 /(4AC)) + (E 2 /(4C 2 )) − (F/C))
(A.5)
Thus the ellipse is centered in the point Table 2 Angular oscillation for all subjects.
(xc , yc ) =
Subject
Angle (◦ )
AR MM JE DM BD
−11.752 9.604 2.983 −0.507 −9.760
± ± ± ± ±
14.068 7.904 21.031 3.646 3.194
D −
2A
,−
E 2C
(A.6)
Major semiaxis, a, minor semiaxis, b, and the eccentricity, e, are given by
a = max
E2 F D2 + − 2 4AC A 4A
,
F E2 D2 − + 4AC C 4C 2
(A.7)
A.B. Roig et al. / Optik 123 (2012) 11–15
b = min
e=
1−
D2 E2 F + − 2 4AC A 4A
,
D2 F E2 − + 4AC C 4C 2
b 2
(A.8)
(A.9)
a
In case of being B different to zero in (1), we redirect the ellipse by making a rotation of angle − (angle between the major axis with the horizontal one) in order to set ellipse’s axes parallel to coordinate ones. The new coordinates (x , y ) are
x y
=
cos(−) sin(−)
− sin(−) cos(−)
x y
(A.10)
Now, the ellipse can be expressed as 2
A(x cos + y sin ) + B(x cos + y sin )(−x sin + y cos ) 2
+ C(−x sin + y cos ) + D(x cos + y sin ) + E(−x sin + y cos ) + F = 0
(A.11)
Grouping terms we get 2
A x + B xy + C y2 + D x + E y + F = 0
(A.12)
with A = A cos2 + C sin2 − B sin cos B = 2(A − C) sin cos + B cos2 − B sin2 C = A sin2 + C cos2 + B sin cos D = D cos − E sin E = D sin + E cos F = F
(A.13)
The principal axes of this ellipse must be parallel to the Cartesian axes, so B must be zero. From this condition (B = 0) the angle results =
1 arctan 2
B C −A
(A.14)
Now, from the resulting ellipse, we can obtain the center position and the length of the axes applying Eqs. (A.7)–(A.9) with the parameters in (A.13). References [1] H. Kasprzak, D.R. Iskander, T. Bajraszewski, A. Kowalczyk, W. NowakSzczepanowska, High accuracy measurements of spectral characteristics of movement of the eye elements, Opt. Pura Apl. 40 (2007) 7–11.
15
[2] R.W. Ditchburn, J.A. Foley-Fisher, Assembled data on eyemovements, Opt. Acta 14 (1967) 113–118. [3] A. Yarbus, Eye Movements and Vision, Plenum Press, New York, 1967. [4] R.H.S. Carpenter, Movements of the Eyes, 2nd ed., Pion, London, 1988. [5] S. Martinez-Conde, S.L. Macknik, D.H. Hubel, The role of fixational eye movements in visual perception, Nat. Rev. Neurosci. 5 (2004) 229–240. [6] E. Marg, Development of electro-oculography; standing potential of the eye in registration of eye movement, AMA Arch. Ophthalmol. 45 (1951) 169–185. [7] D.A. Robinson, A method of measuring eye movement using a scleral search coil in a magnetic field, IEEE Trans. Biomed. Eng. 10 (1963) 137–145. [8] H. Collewijn, F. Van der Mark, T.C. Jansen, Precise recording of human eye movements, Vision Res. 14 (1975) 447–450. [9] L. Ferman, H. Collewijn, T.C. Jansen, A.V. Van den Berg, Human gaze stability in the horizontal, vertical and torsional direction during voluntary head movements, evaluated with a three-dimensional scleral induction coil technique, Vision Res. 27 (1987) 811–828. [10] H. Scherer, W. Teiwes, A.H. Clarke, Measuring three dimensions of eye movement in dynamic situations by means of videooulography, Acta Otolaryngol. 111 (1991) 182–187. [11] R.T. Kenyon, A soft contact lens search coil for measuring eye movements, Vision Res. 25 (1985) 1629–1633. [12] J.R. Liang, S. Moshel, A.Z. Zivotofsky, A. Caspi, R. Engbert, R. Kliegl, S. Havlin, Scaling of horizontal and vertical fixational eyen movements, Am. Phys. Soc. 71 (2005) 1–6. [13] D. Zhu, S.T. Moore, T. Raphan, Robust pupil detection using a curvature algorithm, Comput. Methods Programs Biomed. 59 (1999) 145–157. [14] D.R. Iskander, S. Mioschek, M. Trunk, W. Werth, Detecting eyes in digital images, in: Proceedings of the Seventh International Symposium Signal Processing and its Applications, vol. II, 2003, pp. 21–24. [15] A. De Santis, D. Iacovelllo, Optimal segmentation of pupillometric images for estimating pupil shape parameters, Comput. Methods Programs Biomed. 84 (2006) 174–187. [16] H. Bouma, L.C.J. Baghius, Hippus of the pupil: periods of slow oscillations of unknown origin, Vision Res. 11 (1971) 1345–1351. [17] J. Serra, Image Analysis and Mathematical Morphology, Academic Press, Inc., Orlando, FL, 1983. [18] O. Gal, http://www.mathworks.com/matlabcentral/fileexchange/3215fitellipse (last seen in 04.27.2010). [19] G.A. Fry, The center of rotation of the eye, Am. J. Optom. Arch. Am. Acad. Optom. 39 (1962) 581–595. [20] H.J. Wyatt, The form of the human pupil, Vision Res. 35 (1995) 2021–2036. [21] S.Z. Li (Ed.), Encyclopedia of Biometrics, Springer, New York, 2009. [22] L.S. Gray, B. Winn, B. Gilmartin, Accomodative microfluctuations and pupil diameter, Vision Res. 33 (15) (1993) 2083–2090. [23] J.R. Charlier, M. Behague, C. Buquet, Shift of the pupil center with pupil constriction, Invest. Ophthalmol. Vis. Sci. 35 (4) (1994) 118. [24] F. Censi, G. Calcagnini, F. De Pasquale, S. Lino, S. Cerutti, Baroceptorsenstive fluctuations of human pupil diameter, Comput. Cardiol. 26 (1999) 233–236. [25] G. Calcagnini, P. Giovannelli, F. Censi, P. Bartolini, V. Barbaro, Baroreceptorsensitive fluctuations of heart rate and pupil diameter, in: Proc. 23rd Annual EMBS Int. Conf., 25–28, Instanbul, Turkey, 2001, pp. 600–603. [26] D.R. Iskander, M.J. Collins, S. Mioschek, M. Trunk, Autonomic pupillometry from digital images, IEEE Trans. Biomed. Eng. 51 (9) (2004) 1132–1137. [27] J. Porter, G. Yoon, S. MacRae, G. Pan, T. Twietmeyer, I.G. Cox, D.R. Williams, Surgeon offsets and dynamic eye movements in laser refractive surgery, J. Cataract Refract. Surg. 31 (2005) 2058–2066. [28] D.A. Chernyak, Cyclotorsional eye motion occurring between wavefront measurement and refractive surgery, J. Cataract Refract. Surg. 30 (2004) 633–639.