- Email: [email protected]
Pseudo-loss of Fixation in Automated Perimetry
Pseudo-loss of Fixation in Automated Perimetry OLGA SANABRIA, WILLIAM J. FEUER, MS, DOUGLAS R. ANDERSON, MD
Abstract: During automated perimetry with the Humphrey Visual Field Analyzer, field examinations are labeled unreliable whenever the reported rate of fixation loss is 20% or more. The reported rate of fixation loss results in part from times when the patient's gaze drifts from the fixation point during the examination, but also in part from technical artifacts such as faulty initial localization of the blind spot or false-positive responses by the patient. It was found that technical artifacts caused nearly half of the instances in which a field examination had a reported fixation loss rate of greater than 20%. It was also found that the perimetrist can prevent the artifacts, with the result that the frequency of field examinations labeled as having excessive fixation loss fell from 26% to 14%. Ophthalmology 1991; 98:76-78
Mapping of the visual field depends on knowing the direction in which the eye is pointed. The Humphrey Visual Field Analyzer i ,2 attempts to estimate the proportion of time a patient's gaze is not directed at the fixation target as an index to identify the individual visual field tests that are unreliable. The estimate of fixation loss (FL) rate depends on the fact that, at any instant when the patient is looking at the fixation target, a stimulus presented at the location of the physiologic blind spot will not be seen. If the patient's gaze has wandered (FL), a stimulus projected onto that location will be seen. On this basis, the steadiness of the patient's fixation during a particular examination is estimated by the strategy of presenting the stimulus in the location of the physiologic blind spot from time to time and recording as the FL rate the proportion of such presentations to which the patient responds. If the reported rate of FL is 20% or greater, the field test is considered unreliable. This cut-off rate was established early in the development of the software because at higher rates of FL the sensitivity and specificity of the test began to deteriorate (Patella M, Heijl A, personal communications). On this same basis, it was determined that results of the visual field examination become unOriginally received: June 13, 1990. Revision accepted: August 30, 1990. From the Bascom Palmer Eye Institute, Department of Ophthalmology, University of Miami School of Medicine, Miami. Supported in part by National Glaucoma Research, a program of the American .Health Assistance Foundation, Rockville, and in part by a Senior Scientific Investigators Award to Dr. Anderson by Research to Prevent Blindness, Inc, New York. None of the authors or any members of their families has a proprietary interest in the Allergan-Humphrey Visual Field Analyzer or its software. Reprint requests to Douglas R. Anderson, MD, Bascom Palmer Eye Institute, PO Box 016880, Miami, FL 33101.
76
reliable when the false-positive or false-negative rates are 33% or greater. Of particular concern is that when specificity deteriorates, a normal field may falsely be called abnormal. We became concerned on reading reports that one fifth to one third of field tests performed fail to meet the reliability standards, usually because of a high rate of FL. 3- 5 It would be a disappointment if only two thirds of the field tests used clinically were dependable. We believed that a much higher proportion of visual field tests were reliable. We noted particularly that while participating in a multi-center collaborative clinical trial, which requires that reliability standards be met, we rarely had to repeat a field test because it failed to meet the reliability standard. Therefore, we decided to study the frequency of unreliable field tests in our institution. In the course of doing so, we discovered a higher rate of FL than we expected, were able to document that the high rate is partly artifactual, and demonstrated a method to overcome the problem.
MATERIALS AND METHODS After obtaining approval from the University of Miami Institutional Review Board (IRB), we reviewed results of all Humphrey Visual Field Analyzer program 30-2 visual field examinations6 performed during a 2-week period at the Bascom Palmer Eye Institute in the Staff Clinic and in the private Glaucoma Consultation Service of the faculty. We accomplished this by printing out all examinations for that interval from the back-up disk of each perimeter. For each field examination, the clinic in which the field was performed and the name of the technician were noted. After we found a high percentage of examinations were designated unreliable by virtue of having a high FL rate,
SANABRIA et al
•
LOSS OF FIXATION IN PERIMETRY
the importance of fixation monitoring was discussed at one of the ophthalmic technician's weekly in-service meetings. In particular, the perimetrists were asked to monitor the reports of FL as the test proceeded. They were instructed to interrupt ("pause") the program and have the machine relocate the blind spot if the patient responded to any two of the first several (up to 10) of the presentations to test fixation . (Some of our perimetrists, including the one responsible for visual fields in the multicenter collaborative study, were already using this procedure, while others were not.) As always, eye position was observed on the monitor, and the patient was encouraged to hold his or her fixation steady if prone to tendency to look around. After this session, another 2week sampling of visual field examinations was obtained for study.
RESULTS Of all fields performed, II were eliminated from analysis because the fixation monitoring had been turned off by the technician (as a dense defect adjoining the blind spot prevented the machine from defining the boundary of the blind spot). There remained 421 field tests for analysis, as presented in Table 1. In the original sampling, before in-service instruction had been given, 57 (26%) of 219 visual field tests exceeded the accepted rate of FL, but after the training session, only 29 (14%) of 202 visual field tests failed to meet the standard criterion. This was similar to the 13% rate reported by Hardage and Stamper. 7 By Rosner's multiple logistic regression model for correlated data, 8 which we used because frequently data for both the right and left eye of patients were included, the difference before and after the technical instruction was statistically significant (P = 0.0083). There was no statistical significance to the difference in the rates between the staff and consultation clinics, nor was the effect of technical instruction different between the two clinics. For further analysis, we examined the mean FL rate in each subgroup (Table 2), rather than the frequency of exceeding the standard criterion. The square root of the FL rate was used to eliminate inequality of variance among groups. A two-factor analysis of variance, performed with Rosner's analysis for correlated data,9 again showed a statistically significant effect of instruction (P = 0.022). Although the change seemed to be more prominent in the consultation clinic, a test to detect a difference in improvement between the two clinics failed to achieve statistical significance. The apparent difference between the two clinics in the benefit of instruction could be real, but the observed trend could also have been due to chance. Thirteen technicians had performed visual field examinations during the study. The assignment of some individuals included other forms of clinical assistance, so that only eight had performed field examinations during both of the sampled times. The changes in FL rates are shown in Figure 1. There were substantial decreases in the FL rates for the five technicians that started out with
Table 1. Frequency of Unacceptable Fixation Loss Rate
Clinic Staff Consultation
Instruction Status Before After Before After
Fixation Loss Rate ~20%
<20%
Total
21 (27%) 16 (20%) 36 (25%) 13 (11%)
57 (73%) 63 (80%) 105 (75%) 110 (89%)
78 79 141 123
Table 2. Fixation Loss Rate* (n) Clinic
Before Instruction
After Instruction
Staff Consultation
13.0 ± 13.8 (78) 13.6 ± 15.6 (141)
12.8 ± 14.8 (79) 8.2 ± 8.8 (123)
* Rates multiplied by 100 to give rates in percent. Rates are given as the raw percentage and not the square-root transform used in the analYSis of variance. Mean ± standard deviation.
average FL rates greater than 12% in the patients that they tested. There were only marginal differences among results of the other three technicians. Although large postinstruction changes existed in some and not in others, the data were inadequate to show a statistically significant difference among technicians. Only one technician performed reasonable numbers of examinations in both clinics before and after instruction; for this person, the postinstruction drop in FL rates was similar in both clinics. The correlation between FL and false-positive rates was statistically significant both before and after instruction. The correlation was similar in both groups and, although statistically significant, it was small (r2 = 0.06 and 0.08, respectively).
DISCUSSION When interpreting perimetry, it is important to know whether the field test is reliable. Steady fixation is an important factor in reliability, and is an important trait of the examination to monitor. The Heijl-Krakau method 10 of recording how often the patient responds to stimuli projected into the position of the blind spot will reveal occasions when the eye is not in position. Responses will also occur, however, if the patient has a tendency for false-positive responses, or if, despite proper fixation, the physiologic blind spot is not where the stimulus is presented. The latter can occur, for example, if the patient's blind spot is not in the average normal position because of anatomic variation, or if the head is tilted slightly. False-positive responses will falsely raise the reported rate of FL, for if a patient responds when there is no stimulus at all, they will also respond to a stimulus that they do not see in the physiologic blind spot. This potential source offalsely reported FL has been previously noted. 4 It accounts for the statistically significant correlation between the false-positive rate and the reported FL rate in our data. However, this is not the main reason for an 77
OPHTHALMOLOGY
•
JANUARY 1991
0.20 0.18
EI
Before
•
After
EI
0.16 EI
0.14
Fixation Loss Rate
0.12 0.10
:1-
0.08
n~17
0.06
11 •
n~69
•
n~13
n~48
0.04
EI
•
-I<
n~16
0.02 0.00 0
10
15
Technician Number
Fig 1. Change in fixation loss rate among field examinations by eight technicians. The designated numbers (n) are the combined numbers of examinations before and after instruction for each technician.
exaggerated report of the rate ofFL. Our data suggest that the rate of false-positive responses accounts for only 6 to 8% of the variance in the reported FL rate. Few patients have many false-positive responses, and it is logical that for only a few examinations is a reported unacceptable FL rate accompanied by a high rate of false-positive responses as the apparent cause. A far more frequent and troublesome artifact is when the blind spot of the properly fixing eye is not in the location where the test stimulus was presented. It is important for the perimetrist to recognize this potential artifact. We have shown that the problem is greatly reduced when the blind spot is relocated selectively in those examinations in which false reports ofFL are observed early in the test. This maneuver does not change the actual reliability of the test results, but does help in the interpretation by allowing the examination to be recognized as reliable. The perimetrist should act promptly if two FLs are recorded early in the test. If three FLs have already been recorded, it will be difficult to remain below the 20% limit for FL (or impossible if four are recorded and 19 or fewer presentations in the blind spot are made in the course of a typical examination). Even when two false reports of FL are already on record, there can be only one or two further false responses during the remainder of the examination if the reliability criterion is to be met. To be specific, on any given field examination, a person with a true FL rate of 10% has only a 4% chance of producing a test unacceptable because of a high FL rate. If they see the first 2 of 20 presentations to the "blind spot" because it has not been properly located, even after correction, the chance of producing an unacceptable test is raised to 27%. By watching the FL rate and the eye position on the monitor, the perimetrist may also recognize truly unsteady fixation (or a high rate of false-positive responses) and encourage better performance by the patient, thereby improving the actual reliability of the test. In the recent software, approximately 5% of the stimulus presentations are in the blind spot, but the fixation checks are more frequent at the beginning of the test. In the case of true FL, there-
78
•
VOLUME 98
•
NUMBER 1
fore, the perimetrist can recognize the problem very early in the test, take corrective measures, and make the test more reliable before the test has gone very far. In the case of a continued high reported rate of FL when fixation observed on the monitor is, in fact, steady, the perimetrist can assist the interpreter of the field by noting on the print-out that fixation was steady despite the computer's message to the contrary. The clinician interpreting the field diagram will be helped by understanding the potential artifact when the reported rate of FL is high. In the few cases with high false-positive rates, the source of the unreliability is identified. If, except for the FL rate, there are no other evidences of unreliability, and, for example a low value for the short-term fluctuation is recorded, the clinician may suspect that there was faulty localization of the blind spot. Indeed, in such instances it would be possible simply to ignore the reported high frequency of FL, thus avoiding the need to declare a high percentage of the fields unreliable. Otherwise it might be necessary to repeat the field test. However, accepting fields with a high FL rate will reduce the sensitivity and specificity of the test (because some individuals with high reported FL will, in fact, have unsteady fixation). It is better as the test is performed to make the effort to reduce the artifact, and thus allow more accurate judgments regarding which examinations are reliable and which are unreliable. Thus, although the interpreter can cautiously override the computerized declaration of unreliability for a field examination already performed, he should instruct the perimetrist to prevent false reports of FL if cases with high FL rate occur too often in his practice.
REFERENCES 1. Heijl A. The Humphrey field analyzer, construction and concepts. In: Heijl A, Greve EL, eds. Sixth Intemationall Visual Field Symposium: Santa Margherita Ligure, May 1984. Oordrecht: Dr. W. Junk, 1985 (Doc Ophthalmol Proc Ser; 42). 2. Heijl A. The Humphrey field analyzer, concepts and clinical results. In: Greve EL, Leydhecker W, Raitta C, eds. Second European Glau· coma Symposium: Heisinki, May 1984. Oordrecht: Dr W. Junk, 1985; 55-64 (Doc Ophthalmol Proc Ser; 43). 3. Nelson-Quigg JM, Twelker JO, Johnson CA. Response properties of normal observers and patients during automated perimetry. Arch Ophthalmol1989; 107:1612-5. 4. Katz J, Sommer A. Reliability indexes of automated peri metric tests. Arch Ophthalmol1988; 106:1252-4. 5. Bickler-Bluth M, Trick GL, Kolker AE, Cooper OG. Assessing the utility of reliability indices for automated visual fields. Testing ocular hypertensives. Ophthalmology 1989; 96:616-9. 6. Heijl A. Humphrey Field Analyzer. In: Orance SM, Anderson DR, eds. Automatic Perimetry in Glaucoma: A Practical Guide. Orlando: Grune & Stratton, 1985; 129-40. 7. Hardage L, Stamper RL. Reliability indices for automated visual fields [Letter]. Ophthalmology 1989; 96:1810. 8. Rosner B, Milton R. Significance testing for correlated binary outcome data. Biometrics 1988; 44:505-12. 9. Rosner B. Multivariate methods in ophthalmology with application to other paired data situations. Biometrics 1984; 40:1025-35. 10. Heijl A, Krakau CET. An automatic static perimeter, design and pilot study. Acta Ophthalmol 1975; 53:293-310.
Abstract: During automated perimetry with the Humphrey Visual Field Analyzer, field examinations are labeled unreliable whenever the reported rate of fixation loss is 20% or more. The reported rate of fixation loss results in part from times when the patient's gaze drifts from the fixation point during the examination, but also in part from technical artifacts such as faulty initial localization of the blind spot or false-positive responses by the patient. It was found that technical artifacts caused nearly half of the instances in which a field examination had a reported fixation loss rate of greater than 20%. It was also found that the perimetrist can prevent the artifacts, with the result that the frequency of field examinations labeled as having excessive fixation loss fell from 26% to 14%. Ophthalmology 1991; 98:76-78
Mapping of the visual field depends on knowing the direction in which the eye is pointed. The Humphrey Visual Field Analyzer i ,2 attempts to estimate the proportion of time a patient's gaze is not directed at the fixation target as an index to identify the individual visual field tests that are unreliable. The estimate of fixation loss (FL) rate depends on the fact that, at any instant when the patient is looking at the fixation target, a stimulus presented at the location of the physiologic blind spot will not be seen. If the patient's gaze has wandered (FL), a stimulus projected onto that location will be seen. On this basis, the steadiness of the patient's fixation during a particular examination is estimated by the strategy of presenting the stimulus in the location of the physiologic blind spot from time to time and recording as the FL rate the proportion of such presentations to which the patient responds. If the reported rate of FL is 20% or greater, the field test is considered unreliable. This cut-off rate was established early in the development of the software because at higher rates of FL the sensitivity and specificity of the test began to deteriorate (Patella M, Heijl A, personal communications). On this same basis, it was determined that results of the visual field examination become unOriginally received: June 13, 1990. Revision accepted: August 30, 1990. From the Bascom Palmer Eye Institute, Department of Ophthalmology, University of Miami School of Medicine, Miami. Supported in part by National Glaucoma Research, a program of the American .Health Assistance Foundation, Rockville, and in part by a Senior Scientific Investigators Award to Dr. Anderson by Research to Prevent Blindness, Inc, New York. None of the authors or any members of their families has a proprietary interest in the Allergan-Humphrey Visual Field Analyzer or its software. Reprint requests to Douglas R. Anderson, MD, Bascom Palmer Eye Institute, PO Box 016880, Miami, FL 33101.
76
reliable when the false-positive or false-negative rates are 33% or greater. Of particular concern is that when specificity deteriorates, a normal field may falsely be called abnormal. We became concerned on reading reports that one fifth to one third of field tests performed fail to meet the reliability standards, usually because of a high rate of FL. 3- 5 It would be a disappointment if only two thirds of the field tests used clinically were dependable. We believed that a much higher proportion of visual field tests were reliable. We noted particularly that while participating in a multi-center collaborative clinical trial, which requires that reliability standards be met, we rarely had to repeat a field test because it failed to meet the reliability standard. Therefore, we decided to study the frequency of unreliable field tests in our institution. In the course of doing so, we discovered a higher rate of FL than we expected, were able to document that the high rate is partly artifactual, and demonstrated a method to overcome the problem.
MATERIALS AND METHODS After obtaining approval from the University of Miami Institutional Review Board (IRB), we reviewed results of all Humphrey Visual Field Analyzer program 30-2 visual field examinations6 performed during a 2-week period at the Bascom Palmer Eye Institute in the Staff Clinic and in the private Glaucoma Consultation Service of the faculty. We accomplished this by printing out all examinations for that interval from the back-up disk of each perimeter. For each field examination, the clinic in which the field was performed and the name of the technician were noted. After we found a high percentage of examinations were designated unreliable by virtue of having a high FL rate,
SANABRIA et al
•
LOSS OF FIXATION IN PERIMETRY
the importance of fixation monitoring was discussed at one of the ophthalmic technician's weekly in-service meetings. In particular, the perimetrists were asked to monitor the reports of FL as the test proceeded. They were instructed to interrupt ("pause") the program and have the machine relocate the blind spot if the patient responded to any two of the first several (up to 10) of the presentations to test fixation . (Some of our perimetrists, including the one responsible for visual fields in the multicenter collaborative study, were already using this procedure, while others were not.) As always, eye position was observed on the monitor, and the patient was encouraged to hold his or her fixation steady if prone to tendency to look around. After this session, another 2week sampling of visual field examinations was obtained for study.
RESULTS Of all fields performed, II were eliminated from analysis because the fixation monitoring had been turned off by the technician (as a dense defect adjoining the blind spot prevented the machine from defining the boundary of the blind spot). There remained 421 field tests for analysis, as presented in Table 1. In the original sampling, before in-service instruction had been given, 57 (26%) of 219 visual field tests exceeded the accepted rate of FL, but after the training session, only 29 (14%) of 202 visual field tests failed to meet the standard criterion. This was similar to the 13% rate reported by Hardage and Stamper. 7 By Rosner's multiple logistic regression model for correlated data, 8 which we used because frequently data for both the right and left eye of patients were included, the difference before and after the technical instruction was statistically significant (P = 0.0083). There was no statistical significance to the difference in the rates between the staff and consultation clinics, nor was the effect of technical instruction different between the two clinics. For further analysis, we examined the mean FL rate in each subgroup (Table 2), rather than the frequency of exceeding the standard criterion. The square root of the FL rate was used to eliminate inequality of variance among groups. A two-factor analysis of variance, performed with Rosner's analysis for correlated data,9 again showed a statistically significant effect of instruction (P = 0.022). Although the change seemed to be more prominent in the consultation clinic, a test to detect a difference in improvement between the two clinics failed to achieve statistical significance. The apparent difference between the two clinics in the benefit of instruction could be real, but the observed trend could also have been due to chance. Thirteen technicians had performed visual field examinations during the study. The assignment of some individuals included other forms of clinical assistance, so that only eight had performed field examinations during both of the sampled times. The changes in FL rates are shown in Figure 1. There were substantial decreases in the FL rates for the five technicians that started out with
Table 1. Frequency of Unacceptable Fixation Loss Rate
Clinic Staff Consultation
Instruction Status Before After Before After
Fixation Loss Rate ~20%
<20%
Total
21 (27%) 16 (20%) 36 (25%) 13 (11%)
57 (73%) 63 (80%) 105 (75%) 110 (89%)
78 79 141 123
Table 2. Fixation Loss Rate* (n) Clinic
Before Instruction
After Instruction
Staff Consultation
13.0 ± 13.8 (78) 13.6 ± 15.6 (141)
12.8 ± 14.8 (79) 8.2 ± 8.8 (123)
* Rates multiplied by 100 to give rates in percent. Rates are given as the raw percentage and not the square-root transform used in the analYSis of variance. Mean ± standard deviation.
average FL rates greater than 12% in the patients that they tested. There were only marginal differences among results of the other three technicians. Although large postinstruction changes existed in some and not in others, the data were inadequate to show a statistically significant difference among technicians. Only one technician performed reasonable numbers of examinations in both clinics before and after instruction; for this person, the postinstruction drop in FL rates was similar in both clinics. The correlation between FL and false-positive rates was statistically significant both before and after instruction. The correlation was similar in both groups and, although statistically significant, it was small (r2 = 0.06 and 0.08, respectively).
DISCUSSION When interpreting perimetry, it is important to know whether the field test is reliable. Steady fixation is an important factor in reliability, and is an important trait of the examination to monitor. The Heijl-Krakau method 10 of recording how often the patient responds to stimuli projected into the position of the blind spot will reveal occasions when the eye is not in position. Responses will also occur, however, if the patient has a tendency for false-positive responses, or if, despite proper fixation, the physiologic blind spot is not where the stimulus is presented. The latter can occur, for example, if the patient's blind spot is not in the average normal position because of anatomic variation, or if the head is tilted slightly. False-positive responses will falsely raise the reported rate of FL, for if a patient responds when there is no stimulus at all, they will also respond to a stimulus that they do not see in the physiologic blind spot. This potential source offalsely reported FL has been previously noted. 4 It accounts for the statistically significant correlation between the false-positive rate and the reported FL rate in our data. However, this is not the main reason for an 77
OPHTHALMOLOGY
•
JANUARY 1991
0.20 0.18
EI
Before
•
After
EI
0.16 EI
0.14
Fixation Loss Rate
0.12 0.10
:1-
0.08
n~17
0.06
11 •
n~69
•
n~13
n~48
0.04
EI
•
-I<
n~16
0.02 0.00 0
10
15
Technician Number
Fig 1. Change in fixation loss rate among field examinations by eight technicians. The designated numbers (n) are the combined numbers of examinations before and after instruction for each technician.
exaggerated report of the rate ofFL. Our data suggest that the rate of false-positive responses accounts for only 6 to 8% of the variance in the reported FL rate. Few patients have many false-positive responses, and it is logical that for only a few examinations is a reported unacceptable FL rate accompanied by a high rate of false-positive responses as the apparent cause. A far more frequent and troublesome artifact is when the blind spot of the properly fixing eye is not in the location where the test stimulus was presented. It is important for the perimetrist to recognize this potential artifact. We have shown that the problem is greatly reduced when the blind spot is relocated selectively in those examinations in which false reports ofFL are observed early in the test. This maneuver does not change the actual reliability of the test results, but does help in the interpretation by allowing the examination to be recognized as reliable. The perimetrist should act promptly if two FLs are recorded early in the test. If three FLs have already been recorded, it will be difficult to remain below the 20% limit for FL (or impossible if four are recorded and 19 or fewer presentations in the blind spot are made in the course of a typical examination). Even when two false reports of FL are already on record, there can be only one or two further false responses during the remainder of the examination if the reliability criterion is to be met. To be specific, on any given field examination, a person with a true FL rate of 10% has only a 4% chance of producing a test unacceptable because of a high FL rate. If they see the first 2 of 20 presentations to the "blind spot" because it has not been properly located, even after correction, the chance of producing an unacceptable test is raised to 27%. By watching the FL rate and the eye position on the monitor, the perimetrist may also recognize truly unsteady fixation (or a high rate of false-positive responses) and encourage better performance by the patient, thereby improving the actual reliability of the test. In the recent software, approximately 5% of the stimulus presentations are in the blind spot, but the fixation checks are more frequent at the beginning of the test. In the case of true FL, there-
78
•
VOLUME 98
•
NUMBER 1
fore, the perimetrist can recognize the problem very early in the test, take corrective measures, and make the test more reliable before the test has gone very far. In the case of a continued high reported rate of FL when fixation observed on the monitor is, in fact, steady, the perimetrist can assist the interpreter of the field by noting on the print-out that fixation was steady despite the computer's message to the contrary. The clinician interpreting the field diagram will be helped by understanding the potential artifact when the reported rate of FL is high. In the few cases with high false-positive rates, the source of the unreliability is identified. If, except for the FL rate, there are no other evidences of unreliability, and, for example a low value for the short-term fluctuation is recorded, the clinician may suspect that there was faulty localization of the blind spot. Indeed, in such instances it would be possible simply to ignore the reported high frequency of FL, thus avoiding the need to declare a high percentage of the fields unreliable. Otherwise it might be necessary to repeat the field test. However, accepting fields with a high FL rate will reduce the sensitivity and specificity of the test (because some individuals with high reported FL will, in fact, have unsteady fixation). It is better as the test is performed to make the effort to reduce the artifact, and thus allow more accurate judgments regarding which examinations are reliable and which are unreliable. Thus, although the interpreter can cautiously override the computerized declaration of unreliability for a field examination already performed, he should instruct the perimetrist to prevent false reports of FL if cases with high FL rate occur too often in his practice.
REFERENCES 1. Heijl A. The Humphrey field analyzer, construction and concepts. In: Heijl A, Greve EL, eds. Sixth Intemationall Visual Field Symposium: Santa Margherita Ligure, May 1984. Oordrecht: Dr. W. Junk, 1985 (Doc Ophthalmol Proc Ser; 42). 2. Heijl A. The Humphrey field analyzer, concepts and clinical results. In: Greve EL, Leydhecker W, Raitta C, eds. Second European Glau· coma Symposium: Heisinki, May 1984. Oordrecht: Dr W. Junk, 1985; 55-64 (Doc Ophthalmol Proc Ser; 43). 3. Nelson-Quigg JM, Twelker JO, Johnson CA. Response properties of normal observers and patients during automated perimetry. Arch Ophthalmol1989; 107:1612-5. 4. Katz J, Sommer A. Reliability indexes of automated peri metric tests. Arch Ophthalmol1988; 106:1252-4. 5. Bickler-Bluth M, Trick GL, Kolker AE, Cooper OG. Assessing the utility of reliability indices for automated visual fields. Testing ocular hypertensives. Ophthalmology 1989; 96:616-9. 6. Heijl A. Humphrey Field Analyzer. In: Orance SM, Anderson DR, eds. Automatic Perimetry in Glaucoma: A Practical Guide. Orlando: Grune & Stratton, 1985; 129-40. 7. Hardage L, Stamper RL. Reliability indices for automated visual fields [Letter]. Ophthalmology 1989; 96:1810. 8. Rosner B, Milton R. Significance testing for correlated binary outcome data. Biometrics 1988; 44:505-12. 9. Rosner B. Multivariate methods in ophthalmology with application to other paired data situations. Biometrics 1984; 40:1025-35. 10. Heijl A, Krakau CET. An automatic static perimeter, design and pilot study. Acta Ophthalmol 1975; 53:293-310.