- Email: [email protected]
Validation of Spot screening device for amblyopia risk factors
Validation of Spot screening device for amblyopia risk factors Glynnis A. Garry, BA, and Sean P. Donahue, MD, PhD PURPOSE
To validate the Spot Vision Screener, a handheld digital screening device that evaluates children for amblyopia risk factors as defined by 2013 criteria of the American Association for Pediatric Ophthalmology and Strabismus (AAPOS), in the setting of a controlled pediatric ophthalmology clinic.
METHODS
During a 3-month period, children 2-9 years of age were screened using Spot in a pediatric ophthalmology clinic before receiving a gold standard eye examination. Gold standard examinations were evaluated using the 2013 AAPOS Vision Screening Committee guidelines and compared with results from Spot, which were evaluated using two different manufacturer referral criteria: v1.0.3 and v1.1.51. The specificity and sensitivity for each set of referral criteria to detect both amblyopia risk factors and amblyopia were calculated.
RESULTS
A total of 233 children were included. Of these, 155 were successfully screened and analyzed according to two different referral criteria. Spot screeing revealed ambyopia risk factors in 109 patients; examination confirmed amblyopia in 64. Using the original manufacturer’s criteria (v1.0.3), Spot was 89% sensitive and 71% specific in detecting amblyopia risk factors. The updated referral criteria (v1.1.51) were applied to the same 155 patients, and specificity improved to 88% (P \ 0.02); sensitivity remained minimally affected, at 85% (P \ 0.05). Spot-v1.0.3 was 92% sensitive and 41% specific in detecting amblyopia, whereas Spot-v1.1.51 was 89% sensitive and 53% specific for detecting amblyopia.
CONCLUSIONS
The Spot-v1.0.3 had high sensitivity but overreferred for suspected myopia and strabismus; Spot-v1.1.51 maintained high sensitivity and improved specificity. The original referral criteria has a high sensitivity to detect amblyopia risk factors but low specificty; v1.1.51 criteria increases specificity with minimal impact on sensitivity. ( J AAPOS 2014;18: 476-480)
A
mblyopia is the leading cause of monocular visual impairment in childhood and confers significant risk for bilateral visual impairment in adulthood.1 It is readily treatable when detected early; therefore, effective screening methods are critical to ensuring good outcomes.2 Numerous automated screening instruments have been designed with the aim of detecting amblyopia risk factors (ARFs): strabismus, anisometropia, hyperopia, astigmatism, myopia, and media opacities. However, the referral criteria used by many automated devices have Author affiliations: Vanderbilt Eye Institute, Vanderbilt University Medical Center, Nashville, Tennessee Financial Support: Research to Prevent Blindness, NY, NY, Unrestricted grant to the Department of Ophthalmology, Vanderbilt University. Presented as a poster at the 39th Annual Meeting of the American Association for Pediatric Ophthalmology and Strabismus, Boston, Massachusetts, April 3-7, 2013. Submitted November 8, 2013. Revision accepted July 16, 2014. Published online September 27, 2014. Correspondence: Sean P. Donahue, MD, PhD, Department of Ophthalmology and Visual Sciences, Vanderbilt University, Nashville TN 37232 (email: [email protected]). Copyright Ó 2014 by the American Association for Pediatric Ophthalmology and Strabismus. 1091-8531/$36.00 http://dx.doi.org/10.1016/j.jaapos.2014.07.156
476
low specificity, resulting in a high rate of false positive and unnecessary referrals.3,4 The Spot Vision Screener (PediaVision, Lake Mary, FL) is a handheld, digital photoscreener that uses corneal and retinal reflections to screen for ARFs. Spot estimates refractive error within 7.50 D (spherical equivalent) and compares the estimated value with preprogrammed, ageadjusted criteria for six amblyopia risk factors and generates a “pass” or “refer” result according to manufacturer specifications (Table 1). The device is battery-operated, portable, and Wi-Fi enabled, with a touch screen and electronic data storage and transfer. Previous studies by Arnold and Armitage5 have compared Spot with other vision screeners using the 2003 guidelines of the American Association for Pediatric Ophthalmology and Strabismus (AAPOS) for detecting amblyopia risk factors and found that all 4 screening instruments are capable of achieving a high degree of accuracy. It has become increasingly apparent that the manipulation of referral criteria is key to improving the specificity of these instruments. The purpose of this study is to determine the specificity and sensitivity of the Spot Vision Screener using two different referral criteria to detect amblyopia and its
Journal of AAPOS
Volume 18 Number 5 / October 2014
Garry and Donahue
477
Table 1. Manufacturer criteria for Spot v1.0.3 and v1.1.51 Gaze Age, months v1.0.3 6-12 12-36 36-72 72-240 240-1200 v1.1.51 6-12 12-36 36-72 72-240 240-1200
Anisometropia, D
Astigmatism, D
Myopia, D
Hyperopia, D
Anisocoria, mm
Vertical
Nasal
Temporal
Asymmetry
$1.5 $1 $1 $1 $1
$2.25 $2 $1.75 $1.5 $1.5
$ -2 $ -2 $ -1.25 $ -0.75 $ -0.75
$3.5 $3 $2.5 $2.5 $1.5
$1 $1 $1 $1 $1
$5 $5 $5 $5 $5
$5 $5 $5 $5 $5
$8 $8 $8 $8 $8
$6 $6 $6 $6 $6
$1.5 $1 $1 $1 $1
$2.25 $2 $1.75 $1.5 $1.5
$ -2 $ -2 $ -1 $ -1 $ -0.75
$3.5 $3 $2.5 $2.5 $1.5
$1 $1 $1 $1 $1
$8 $8 $8 $8 $8
$5 $5 $5 $5 $5
$8 $8 $8 $8 $8
$8 $8 $8 $8 $8
risk factors as defined by 2013 AAPOS criteria in a pediatric ophthalmology clinic setting.
Table 2. 2013 AAPOS Vision Screening Committee amblyopia risk factor criteria Age, months Anisometropia Hyperopia Astigmatism
Methods Approval from the Institutional Research Board of Vanderbilt University Medical Center was obtained for this study, and conformed to the requirements of the US Health Insurance Portability and Accountability Act of 1996. Informed consent was obtained for all participants, all of whom were patients at the Vanderbilt Eye Institute. Patients were recruited from the practices of four pediatric ophthalmologists. A single investigator (GG) performed all screenings. Screenings were performed using manufacturer recommendations and instructions.6 Inclusion criteria consisted of children 2-9 years of age who were to receive a gold standard eye examination on the same day as their screening with Spot. Those whose eyes had already been dilated prior to screening or who had strabismus surgery within the past 3 months were excluded from the study. The device was held at eye-level, 3 feet away from the seated child in a dimly lit room, to capture data capture. The output from the screening instrument consists of 7 output values for each eye, including estimates of spherical and cylindrical refractive error, axis, and gaze vector. Suspected reasons for referral are listed, and if the estimates exceed preprogrammed, age-adjusted values, the affected eye(s) are indicated. The device produces one of two recommendationsE—“Complete Eye Exam Recommended” or “All Measurements in Range.” Participants then received a gold standard pediatric ophthalmic examination, consisting of an assessment of visual acuity, strabismus, and refractive error using cycloplegic retinoscopy. The examining ophthalmologist was masked to the results of the Spot screening. To determine the “primary diagnosis” from the gold standard eye examination, amblyopia risk factors were ranked in the following order: (1) media opacity, (2) manifest strabismus, (3) anisometropia, (4) hyperopia, (5) astigmatism, and (6) myopia. The revised 2013 American Association of Pediatric Ophthalmology and Strabismus (AAPOS) Vision Screening guidelines were used to determine whether an amblyopia risk factor was present (Table 2).7 The specificity and sensitivity of
Journal of AAPOS
12-30 31-48 .48 All ages
.2.5 D .2.0 D .1.5 D
.4.5 D .2.0 D .4.0 D .2.0 D .3.5 D .1.5 D Manifest strabismus .8 PD Media opacity .1 mm
Myopia . 3.5 D . 3.0 D . 1.5 D
D, diopter; PD, prism diopter. the device to detect risk factors for amblyopia and amblyopia itself were determined.
Results During a 3-month period, 233 children were screened; 18 screenings (7%) were unsuccessful due to poor cooperation (n 5 6), congenital ptosis (n 5 6), aphakia (n 5 3), congenital nystagmus (n 5 2), and congenital aniridia (n 5 1). Due to accidental file corruption, 60 random screenings were lost. Of the 233, 155 children 24 months to 9 years of age (mean age, 5.16 years) were successfully screened with the Spot Vision Screener and received a gold standard ophthalmic examination. Of these, 64 children (41%) had amblyopia, defined by a difference of 2 lines of best spectacle-corrected visual acuity, and for preverbal children unmaintained fixation using fixation preference testing. At the time of the screening, 109 (70%) had ARFs as defined by the 2013 AAPOS criteria, and 46 (30%) were normal. Primary diagnoses of this population are enumerated in Table 3. The manufacturer’s original referral criteria (v1.0.3) had a sensitivity of 89% and a specificity of 71% to detect amblyopia risk factors (Table 4). Specificity was low because 12 children were falsely referred (Table 5). These normal children were referred for suspected strabismus (n 5 6), myopia (n 5 5), and moderate symmetric astigmatism (n 5 1). Three of these children had an intermittent strabismus, which is no longer considered an amblyopia risk factor; 2 were referred for myopia and the third for strabismus. One child with Brown syndrome was referred
478
Garry and Donahue
Volume 18 Number 5 / October 2014
Table 3. Primary diagnoses of screened population Amblyopia risk factors Media opacity Strabismus Anisometropia Hypermetropia Astigmatism Myopia
Table 4. Sensitivity and specificity of various referral criteria to detect amblyopia risk factors Number
Criteria
Sample size (n)
Sensitivity
Specificity
7 62 21 13 6 0
Spot v1.0.3 Spot v1.1.51
155 155
89% 85%
71% 88%
for suspected strabismus and although this child has a manifest strabismus, the strabismus present in either Brown and Duane syndrome is no longer considered a risk factor for amblyopia; this child was referred by Spot and therefore considered a false positive.4 There were 12 false negative screens (Table 6), which occurred in 10 out of 62 cases of strabismus (including 2 children with a 25D esotropia and one with a 40D exotropia), 1 of 6 cases of astigmatism, and 1 of 7 cases of media opacity. After the screenings were completed, PediaVision released a software update, Spot v1.1.51, designed with the intent of improving strabismus and myopia detection as well as enhancing quality of image capture. With Spot v1.1.51, thresholds for gaze (strabismus) and myopia referral have been increased. Using the video files captured by the device for each screening, the software update was applied retroactively to the 155 completed screenings in a masked manner, a new result of “Complete Eye Exam Recommended” or “All Measurements in Range” was produced, and sensitivity and specificity of the device was reevaluated. Only screening results with available data for both software versions were analyzed. In screening for amblyopia risk factors using v1.1.51, the specificity (88%) increased significantly (X2 5 6.125 [P \ 0.02]), whereas the sensitivity (85%) did not change significantly (X2 5 4.166 [P . 0.05]; Table 4).8 When evaluated using the new v1.1.51 software, 7 of the 12 initially false positive screens were reevaluated as true negative screens (gaze, n 5 3; astigmatism, n 5 1; myopia, n 5 3; Table 5), leaving only 5 false positives. In an effort to improve the specificity of Spot and minimize unnecessary referrals, v1.1.51 decreased the sensitivity of the device by failing to refer 5 children who were appropriately referred with the previous software (Table 6), including the patient with a 40D exotropia, and 2 children with $5.25 D hyperopia. While this device is designed with the purpose of detecting risk factors for amblyopia, it is notable that Spot v1.0.3 was 92% sensitive and 41% specific in detecting amblyopia. Spot v1.1.51 was 89% sensitive and 53% specific in detecting amblyopia.
Discussion In developing a useful screening instrument, the manufacturer must balance sensitivity and specificity. Low device specificity produces a low positive predictive value and an excess of false positive referrals, contributing to increased
health care costs, undue parental anxiety, and a lack of confidence in the screening process itself by physicians and other users. This may minimize device utilization or attention to appropriate follow-up after referral.5 In the case of screening for amblyopia risk factors, which have a relatively low prevalence, a high positive predictive value is desirable since many children with amblyopia risk factors do not develop amblyopia. Additionally, a short delay in diagnosis typically does not adversely affect outcomes. While a high sensitivity is desirable in any screening process, an optimized specificity is preferable in such preschool field screening settings where the majority of participants do not have risk factors. A previous study by Armitage and Arnold5 demonstrated that Spot v1.1.51 has a sensitivity of 80% and specificity of 85%. Aside from population size and demographic, the most notable difference in methods of the two studies was our application of the 2013 AAPOS guidelines as opposed to their utilization of the 2003 AAPOS guidelines. In optimizing the specificity of the Spot Vision Screener, the device has become a more useful field-screening instrument. By decreasing the referral criteria vertical gaze threshold, the device eliminated 3 of the 7 false referrals for gaze (strabismus); however it missed 1 child with a 40D exotropia and 2 with $5.25 D spherical hyperopia. Likewise, altering the myopia referral criteria in the 3672 month and 72-240 month age groups decreased myopia false positives. Astigmatism criteria remained unaltered between the two sets of criteria, however one case of suspected astigmatism was reevaluated as a true negative using v1.1.51 criteria. We suggest further refinement of criteria to detect myopia, strabismus, and astigmatism, and decreasing the referral criteria for suspected hypermetropia. Alternatively, if the hypermetropic children that received a pass result from Spot are accommodating when focusing upon the device, implementing a shorter image capture time could be considered. While our study had access to same-day gold standard eye examinations for every patient, one limitation to our design involved using an ophthalmology clinic patient population as opposed to a more typical screening population in the field with high numbers of normal children. Because of the substantial difference in disease prevalence between these populations, we were unable to calculate a positive predictive value. A large-scale field study using a more appropriate screening population is currently ongoing. An additional drawback includes the high prevalence of strabismus in our screening population (40%). Historically, the inability to assess for strabismus has been a point of
Journal of AAPOS
Volume 18 Number 5 / October 2014
Garry and Donahue
479
Table 5. Children with false positive screens using Spot v1.0.3 Gold standard examination Right eye 1 2a 3a 4a 5a 6 7a 8 9b 10 11a 12a
Age, years
Strabismus, PD
8 3 8 5 7 8 3 5 5 4 9 3
0 0 0 0 18X(T) 0 0 0 17ET 8X(T) 18 X(T) 18X(T)
Sphere, D
Left eye
Cylinder, D
Axis
0 1.5 0 0.75 0 0 1.5 0.25 0.25 0.75 0 0
0 90 0 90 0 0 90 90 90 90 0 0
1 1 1.25 0.5 0.75 1.5 2.5 3 0.25 1 0.5 1
Sphere, D 1 1 1 0 0.75 1.5 2.5 2 0.75 1 0.5 1
Cylinder, D
Axis
Referral reason
0 1.5 0.5 0 0 0.25 1.5 0.25 0 0.75 0 0
0 90 50 0 0 180 90 90 0 90 0 0
Myopia Gaze Myopia Gaze Myopia Myopia Astigmatism Anisometropia Gaze Gaze Myopia Gaze
D, diopters; ET, esotropia; PD, prism diopters; X(T), intermittent exotropia. a False positive upon screening with Spot v1.0.3, reevaluated as true negative with Spot v1.1.51. b Brown syndrome (no longer considered an amblyopia risk factor).
Table 6. Children with false negative screens using Spot v1.0.3 Gold standard examination Right eye 1 2 3c 4 5 6 7 8c 9b 10 11 12c 13a 14a 15a 16a 17a
Age, years
Strabismus, PD
9 5 5 5 7 8 2 2 4 6 5 9 5 3 7 6 8
12ET 20E(T) 10ET 16E(T) 4ET 25ET 25ET 18E(T) 0 18XT 2ET 10ET 0 4XT 0 DVD 40XT
Sphere, D 2.5 0.25 0.5 3 1.75 1.25 2 5 3.75 2.25 1.25 1.5 6 5.25 1 2.25 0.75
Left eye
Cylinder, D
Axis
1.25 0.75 0.5 0.75 0.5 0 0 1.5 0.5 0 0.5 0 1.5 0.25 0 1.75 1.25
95 90 90 90 90 0 0 45 20 0 180 0 95 180 0 95 85
Sphere, D 2.75 0.25 0.75 2.5 1 1.25 2 5 2.5 2.25 1.75 1.5 6 3.75 0.25 3.5 0.25
Cylinder, D
Axis
1.25 1 0 0.5 0 0 0 0 0.75 0 0.5 0 1.5 1.75 2 2 0.5
90 90 0 90 0 0 0 0 180 0 80 0 90 110 90 85 110
D, diopter; ET, esotropia; E(T), intermittent esotropia; PD, prism diopter; XT, exotropia. True positive upon screening with Spot v1.0.3 reevaluated as false negative with Spot v1.1.51. b Cataract .1 mm present upon examination. c History of strabismus surgery. a
weakness in photoscreening technologies. While Spot assesses for strabismus and accurately identified the majority of affected patients, strabismus accounted for the large majority of false negative exams and a substantial number of false positive examinations. Several of the patients who were referred were found not to have amblyopia risk factors but had other abnormal findings, such as nystagmus with head turn or ocular coloboma. These cases were characterized as false positives. Similarly, some of the children in this study had a refractive error that would have been considered an amblyopia risk factors in the previous AAPOS
Journal of AAPOS
guidelines, but are no longer considered as such because of the age-adjusted criteria. With a much higher prevalence of ocular pathology in a clinic-based study, it is likely that this study substantially underestimates the specificity of the instrument.
References 1. Holmes JM, Clarke MP. Amblyopia. Lancet 2006;367:1343-51. 2. Bradfield YS. Identification and treatment of amblyopia. American Family Physician 2013;87:348-52.
480
Garry and Donahue
3. Silverstein E, Lorenz S, Emmons K, Donahue SP. Limits on improving the positive predictive value of the Welch Allyn SureSight for preschool vision screening. J AAPOS 2009;13:45-50. 4. Nathan NR, Donahue SP. Modification of Plusoptix referral criteria to enhance sensitivity and specificity during pediatric vision screening. J AAPOS 2011;15:551-5. 5. Arnold RW, Armitage MD. Performance of four new photoscreeners on pediatric patients with high risk amblyopia. J Pediatr Ophthalmol Strabismus 2014;51:46-52.
Volume 18 Number 5 / October 2014 6. Spot Instructions for Use. Model VS100. SPOT Vision Screener. PediaVision Holdings. Lake Mary, FL: LLC; 2011. 7. Donahue SP, Arthur B, Neely DE, Arnold RW, Silbert D, Ruben JB. AAPOS Vision Screening Committee. Guidelines for automated preschool vision screening: a 10-year, evidence-based update. J AAPOS 2013;17:4-8. 8. Hawass NED. Comparing the sensitivities and specificities of two diagnostic procedures performed on the same group of patients. Brit J Radiol 1997;70:360.
Novel use of the retinoscope in visualization of the anterior segment Examination of the anterior segment with proper magnification is often difficult in a young child, and slit-beam delineation of anterior segment structures is usually limited to the use of a portable slit-lamp. While providing adequate magnification with a pivoted slit-beam, the narrow field of view and short focal distance make estimating the proximity to the child’s eye difficult, rendering the examination challenging—and potentially hazardous. We propose a new method of examining the anterior segment in children that combines a 20 D indirect ophthalmoscopy lens with a streak retinoscope in the “sleeve-up” position with a narrowed beam. The 20 D lens is held approximately 3–5 cm from the patient’s eye, while the retinoscope streak is projected through the lens onto the anterior segment obliquely. This technique allows a magnified view of the anterior segment structures with slit-beam delineation and is particularly useful in determining anterior chamber depth in a child intolerant of slit-lamp examination, as when evaluating for iris bombe after anterior segment surgery or characterizing the tube position of a glaucoma drainage device. —Ta C. Chang, MD, and Kara M. Cavuoto, MD, Bascom Palmer Eye Institute, University of Miami Miller School of Medicine, Miami, Florida The authors thank Drs. Elizabeth Dale and Michael Smith for their assistance.
Journal of AAPOS
To validate the Spot Vision Screener, a handheld digital screening device that evaluates children for amblyopia risk factors as defined by 2013 criteria of the American Association for Pediatric Ophthalmology and Strabismus (AAPOS), in the setting of a controlled pediatric ophthalmology clinic.
METHODS
During a 3-month period, children 2-9 years of age were screened using Spot in a pediatric ophthalmology clinic before receiving a gold standard eye examination. Gold standard examinations were evaluated using the 2013 AAPOS Vision Screening Committee guidelines and compared with results from Spot, which were evaluated using two different manufacturer referral criteria: v1.0.3 and v1.1.51. The specificity and sensitivity for each set of referral criteria to detect both amblyopia risk factors and amblyopia were calculated.
RESULTS
A total of 233 children were included. Of these, 155 were successfully screened and analyzed according to two different referral criteria. Spot screeing revealed ambyopia risk factors in 109 patients; examination confirmed amblyopia in 64. Using the original manufacturer’s criteria (v1.0.3), Spot was 89% sensitive and 71% specific in detecting amblyopia risk factors. The updated referral criteria (v1.1.51) were applied to the same 155 patients, and specificity improved to 88% (P \ 0.02); sensitivity remained minimally affected, at 85% (P \ 0.05). Spot-v1.0.3 was 92% sensitive and 41% specific in detecting amblyopia, whereas Spot-v1.1.51 was 89% sensitive and 53% specific for detecting amblyopia.
CONCLUSIONS
The Spot-v1.0.3 had high sensitivity but overreferred for suspected myopia and strabismus; Spot-v1.1.51 maintained high sensitivity and improved specificity. The original referral criteria has a high sensitivity to detect amblyopia risk factors but low specificty; v1.1.51 criteria increases specificity with minimal impact on sensitivity. ( J AAPOS 2014;18: 476-480)
A
mblyopia is the leading cause of monocular visual impairment in childhood and confers significant risk for bilateral visual impairment in adulthood.1 It is readily treatable when detected early; therefore, effective screening methods are critical to ensuring good outcomes.2 Numerous automated screening instruments have been designed with the aim of detecting amblyopia risk factors (ARFs): strabismus, anisometropia, hyperopia, astigmatism, myopia, and media opacities. However, the referral criteria used by many automated devices have Author affiliations: Vanderbilt Eye Institute, Vanderbilt University Medical Center, Nashville, Tennessee Financial Support: Research to Prevent Blindness, NY, NY, Unrestricted grant to the Department of Ophthalmology, Vanderbilt University. Presented as a poster at the 39th Annual Meeting of the American Association for Pediatric Ophthalmology and Strabismus, Boston, Massachusetts, April 3-7, 2013. Submitted November 8, 2013. Revision accepted July 16, 2014. Published online September 27, 2014. Correspondence: Sean P. Donahue, MD, PhD, Department of Ophthalmology and Visual Sciences, Vanderbilt University, Nashville TN 37232 (email: [email protected]). Copyright Ó 2014 by the American Association for Pediatric Ophthalmology and Strabismus. 1091-8531/$36.00 http://dx.doi.org/10.1016/j.jaapos.2014.07.156
476
low specificity, resulting in a high rate of false positive and unnecessary referrals.3,4 The Spot Vision Screener (PediaVision, Lake Mary, FL) is a handheld, digital photoscreener that uses corneal and retinal reflections to screen for ARFs. Spot estimates refractive error within 7.50 D (spherical equivalent) and compares the estimated value with preprogrammed, ageadjusted criteria for six amblyopia risk factors and generates a “pass” or “refer” result according to manufacturer specifications (Table 1). The device is battery-operated, portable, and Wi-Fi enabled, with a touch screen and electronic data storage and transfer. Previous studies by Arnold and Armitage5 have compared Spot with other vision screeners using the 2003 guidelines of the American Association for Pediatric Ophthalmology and Strabismus (AAPOS) for detecting amblyopia risk factors and found that all 4 screening instruments are capable of achieving a high degree of accuracy. It has become increasingly apparent that the manipulation of referral criteria is key to improving the specificity of these instruments. The purpose of this study is to determine the specificity and sensitivity of the Spot Vision Screener using two different referral criteria to detect amblyopia and its
Journal of AAPOS
Volume 18 Number 5 / October 2014
Garry and Donahue
477
Table 1. Manufacturer criteria for Spot v1.0.3 and v1.1.51 Gaze Age, months v1.0.3 6-12 12-36 36-72 72-240 240-1200 v1.1.51 6-12 12-36 36-72 72-240 240-1200
Anisometropia, D
Astigmatism, D
Myopia, D
Hyperopia, D
Anisocoria, mm
Vertical
Nasal
Temporal
Asymmetry
$1.5 $1 $1 $1 $1
$2.25 $2 $1.75 $1.5 $1.5
$ -2 $ -2 $ -1.25 $ -0.75 $ -0.75
$3.5 $3 $2.5 $2.5 $1.5
$1 $1 $1 $1 $1
$5 $5 $5 $5 $5
$5 $5 $5 $5 $5
$8 $8 $8 $8 $8
$6 $6 $6 $6 $6
$1.5 $1 $1 $1 $1
$2.25 $2 $1.75 $1.5 $1.5
$ -2 $ -2 $ -1 $ -1 $ -0.75
$3.5 $3 $2.5 $2.5 $1.5
$1 $1 $1 $1 $1
$8 $8 $8 $8 $8
$5 $5 $5 $5 $5
$8 $8 $8 $8 $8
$8 $8 $8 $8 $8
risk factors as defined by 2013 AAPOS criteria in a pediatric ophthalmology clinic setting.
Table 2. 2013 AAPOS Vision Screening Committee amblyopia risk factor criteria Age, months Anisometropia Hyperopia Astigmatism
Methods Approval from the Institutional Research Board of Vanderbilt University Medical Center was obtained for this study, and conformed to the requirements of the US Health Insurance Portability and Accountability Act of 1996. Informed consent was obtained for all participants, all of whom were patients at the Vanderbilt Eye Institute. Patients were recruited from the practices of four pediatric ophthalmologists. A single investigator (GG) performed all screenings. Screenings were performed using manufacturer recommendations and instructions.6 Inclusion criteria consisted of children 2-9 years of age who were to receive a gold standard eye examination on the same day as their screening with Spot. Those whose eyes had already been dilated prior to screening or who had strabismus surgery within the past 3 months were excluded from the study. The device was held at eye-level, 3 feet away from the seated child in a dimly lit room, to capture data capture. The output from the screening instrument consists of 7 output values for each eye, including estimates of spherical and cylindrical refractive error, axis, and gaze vector. Suspected reasons for referral are listed, and if the estimates exceed preprogrammed, age-adjusted values, the affected eye(s) are indicated. The device produces one of two recommendationsE—“Complete Eye Exam Recommended” or “All Measurements in Range.” Participants then received a gold standard pediatric ophthalmic examination, consisting of an assessment of visual acuity, strabismus, and refractive error using cycloplegic retinoscopy. The examining ophthalmologist was masked to the results of the Spot screening. To determine the “primary diagnosis” from the gold standard eye examination, amblyopia risk factors were ranked in the following order: (1) media opacity, (2) manifest strabismus, (3) anisometropia, (4) hyperopia, (5) astigmatism, and (6) myopia. The revised 2013 American Association of Pediatric Ophthalmology and Strabismus (AAPOS) Vision Screening guidelines were used to determine whether an amblyopia risk factor was present (Table 2).7 The specificity and sensitivity of
Journal of AAPOS
12-30 31-48 .48 All ages
.2.5 D .2.0 D .1.5 D
.4.5 D .2.0 D .4.0 D .2.0 D .3.5 D .1.5 D Manifest strabismus .8 PD Media opacity .1 mm
Myopia . 3.5 D . 3.0 D . 1.5 D
D, diopter; PD, prism diopter. the device to detect risk factors for amblyopia and amblyopia itself were determined.
Results During a 3-month period, 233 children were screened; 18 screenings (7%) were unsuccessful due to poor cooperation (n 5 6), congenital ptosis (n 5 6), aphakia (n 5 3), congenital nystagmus (n 5 2), and congenital aniridia (n 5 1). Due to accidental file corruption, 60 random screenings were lost. Of the 233, 155 children 24 months to 9 years of age (mean age, 5.16 years) were successfully screened with the Spot Vision Screener and received a gold standard ophthalmic examination. Of these, 64 children (41%) had amblyopia, defined by a difference of 2 lines of best spectacle-corrected visual acuity, and for preverbal children unmaintained fixation using fixation preference testing. At the time of the screening, 109 (70%) had ARFs as defined by the 2013 AAPOS criteria, and 46 (30%) were normal. Primary diagnoses of this population are enumerated in Table 3. The manufacturer’s original referral criteria (v1.0.3) had a sensitivity of 89% and a specificity of 71% to detect amblyopia risk factors (Table 4). Specificity was low because 12 children were falsely referred (Table 5). These normal children were referred for suspected strabismus (n 5 6), myopia (n 5 5), and moderate symmetric astigmatism (n 5 1). Three of these children had an intermittent strabismus, which is no longer considered an amblyopia risk factor; 2 were referred for myopia and the third for strabismus. One child with Brown syndrome was referred
478
Garry and Donahue
Volume 18 Number 5 / October 2014
Table 3. Primary diagnoses of screened population Amblyopia risk factors Media opacity Strabismus Anisometropia Hypermetropia Astigmatism Myopia
Table 4. Sensitivity and specificity of various referral criteria to detect amblyopia risk factors Number
Criteria
Sample size (n)
Sensitivity
Specificity
7 62 21 13 6 0
Spot v1.0.3 Spot v1.1.51
155 155
89% 85%
71% 88%
for suspected strabismus and although this child has a manifest strabismus, the strabismus present in either Brown and Duane syndrome is no longer considered a risk factor for amblyopia; this child was referred by Spot and therefore considered a false positive.4 There were 12 false negative screens (Table 6), which occurred in 10 out of 62 cases of strabismus (including 2 children with a 25D esotropia and one with a 40D exotropia), 1 of 6 cases of astigmatism, and 1 of 7 cases of media opacity. After the screenings were completed, PediaVision released a software update, Spot v1.1.51, designed with the intent of improving strabismus and myopia detection as well as enhancing quality of image capture. With Spot v1.1.51, thresholds for gaze (strabismus) and myopia referral have been increased. Using the video files captured by the device for each screening, the software update was applied retroactively to the 155 completed screenings in a masked manner, a new result of “Complete Eye Exam Recommended” or “All Measurements in Range” was produced, and sensitivity and specificity of the device was reevaluated. Only screening results with available data for both software versions were analyzed. In screening for amblyopia risk factors using v1.1.51, the specificity (88%) increased significantly (X2 5 6.125 [P \ 0.02]), whereas the sensitivity (85%) did not change significantly (X2 5 4.166 [P . 0.05]; Table 4).8 When evaluated using the new v1.1.51 software, 7 of the 12 initially false positive screens were reevaluated as true negative screens (gaze, n 5 3; astigmatism, n 5 1; myopia, n 5 3; Table 5), leaving only 5 false positives. In an effort to improve the specificity of Spot and minimize unnecessary referrals, v1.1.51 decreased the sensitivity of the device by failing to refer 5 children who were appropriately referred with the previous software (Table 6), including the patient with a 40D exotropia, and 2 children with $5.25 D hyperopia. While this device is designed with the purpose of detecting risk factors for amblyopia, it is notable that Spot v1.0.3 was 92% sensitive and 41% specific in detecting amblyopia. Spot v1.1.51 was 89% sensitive and 53% specific in detecting amblyopia.
Discussion In developing a useful screening instrument, the manufacturer must balance sensitivity and specificity. Low device specificity produces a low positive predictive value and an excess of false positive referrals, contributing to increased
health care costs, undue parental anxiety, and a lack of confidence in the screening process itself by physicians and other users. This may minimize device utilization or attention to appropriate follow-up after referral.5 In the case of screening for amblyopia risk factors, which have a relatively low prevalence, a high positive predictive value is desirable since many children with amblyopia risk factors do not develop amblyopia. Additionally, a short delay in diagnosis typically does not adversely affect outcomes. While a high sensitivity is desirable in any screening process, an optimized specificity is preferable in such preschool field screening settings where the majority of participants do not have risk factors. A previous study by Armitage and Arnold5 demonstrated that Spot v1.1.51 has a sensitivity of 80% and specificity of 85%. Aside from population size and demographic, the most notable difference in methods of the two studies was our application of the 2013 AAPOS guidelines as opposed to their utilization of the 2003 AAPOS guidelines. In optimizing the specificity of the Spot Vision Screener, the device has become a more useful field-screening instrument. By decreasing the referral criteria vertical gaze threshold, the device eliminated 3 of the 7 false referrals for gaze (strabismus); however it missed 1 child with a 40D exotropia and 2 with $5.25 D spherical hyperopia. Likewise, altering the myopia referral criteria in the 3672 month and 72-240 month age groups decreased myopia false positives. Astigmatism criteria remained unaltered between the two sets of criteria, however one case of suspected astigmatism was reevaluated as a true negative using v1.1.51 criteria. We suggest further refinement of criteria to detect myopia, strabismus, and astigmatism, and decreasing the referral criteria for suspected hypermetropia. Alternatively, if the hypermetropic children that received a pass result from Spot are accommodating when focusing upon the device, implementing a shorter image capture time could be considered. While our study had access to same-day gold standard eye examinations for every patient, one limitation to our design involved using an ophthalmology clinic patient population as opposed to a more typical screening population in the field with high numbers of normal children. Because of the substantial difference in disease prevalence between these populations, we were unable to calculate a positive predictive value. A large-scale field study using a more appropriate screening population is currently ongoing. An additional drawback includes the high prevalence of strabismus in our screening population (40%). Historically, the inability to assess for strabismus has been a point of
Journal of AAPOS
Volume 18 Number 5 / October 2014
Garry and Donahue
479
Table 5. Children with false positive screens using Spot v1.0.3 Gold standard examination Right eye 1 2a 3a 4a 5a 6 7a 8 9b 10 11a 12a
Age, years
Strabismus, PD
8 3 8 5 7 8 3 5 5 4 9 3
0 0 0 0 18X(T) 0 0 0 17ET 8X(T) 18 X(T) 18X(T)
Sphere, D
Left eye
Cylinder, D
Axis
0 1.5 0 0.75 0 0 1.5 0.25 0.25 0.75 0 0
0 90 0 90 0 0 90 90 90 90 0 0
1 1 1.25 0.5 0.75 1.5 2.5 3 0.25 1 0.5 1
Sphere, D 1 1 1 0 0.75 1.5 2.5 2 0.75 1 0.5 1
Cylinder, D
Axis
Referral reason
0 1.5 0.5 0 0 0.25 1.5 0.25 0 0.75 0 0
0 90 50 0 0 180 90 90 0 90 0 0
Myopia Gaze Myopia Gaze Myopia Myopia Astigmatism Anisometropia Gaze Gaze Myopia Gaze
D, diopters; ET, esotropia; PD, prism diopters; X(T), intermittent exotropia. a False positive upon screening with Spot v1.0.3, reevaluated as true negative with Spot v1.1.51. b Brown syndrome (no longer considered an amblyopia risk factor).
Table 6. Children with false negative screens using Spot v1.0.3 Gold standard examination Right eye 1 2 3c 4 5 6 7 8c 9b 10 11 12c 13a 14a 15a 16a 17a
Age, years
Strabismus, PD
9 5 5 5 7 8 2 2 4 6 5 9 5 3 7 6 8
12ET 20E(T) 10ET 16E(T) 4ET 25ET 25ET 18E(T) 0 18XT 2ET 10ET 0 4XT 0 DVD 40XT
Sphere, D 2.5 0.25 0.5 3 1.75 1.25 2 5 3.75 2.25 1.25 1.5 6 5.25 1 2.25 0.75
Left eye
Cylinder, D
Axis
1.25 0.75 0.5 0.75 0.5 0 0 1.5 0.5 0 0.5 0 1.5 0.25 0 1.75 1.25
95 90 90 90 90 0 0 45 20 0 180 0 95 180 0 95 85
Sphere, D 2.75 0.25 0.75 2.5 1 1.25 2 5 2.5 2.25 1.75 1.5 6 3.75 0.25 3.5 0.25
Cylinder, D
Axis
1.25 1 0 0.5 0 0 0 0 0.75 0 0.5 0 1.5 1.75 2 2 0.5
90 90 0 90 0 0 0 0 180 0 80 0 90 110 90 85 110
D, diopter; ET, esotropia; E(T), intermittent esotropia; PD, prism diopter; XT, exotropia. True positive upon screening with Spot v1.0.3 reevaluated as false negative with Spot v1.1.51. b Cataract .1 mm present upon examination. c History of strabismus surgery. a
weakness in photoscreening technologies. While Spot assesses for strabismus and accurately identified the majority of affected patients, strabismus accounted for the large majority of false negative exams and a substantial number of false positive examinations. Several of the patients who were referred were found not to have amblyopia risk factors but had other abnormal findings, such as nystagmus with head turn or ocular coloboma. These cases were characterized as false positives. Similarly, some of the children in this study had a refractive error that would have been considered an amblyopia risk factors in the previous AAPOS
Journal of AAPOS
guidelines, but are no longer considered as such because of the age-adjusted criteria. With a much higher prevalence of ocular pathology in a clinic-based study, it is likely that this study substantially underestimates the specificity of the instrument.
References 1. Holmes JM, Clarke MP. Amblyopia. Lancet 2006;367:1343-51. 2. Bradfield YS. Identification and treatment of amblyopia. American Family Physician 2013;87:348-52.
480
Garry and Donahue
3. Silverstein E, Lorenz S, Emmons K, Donahue SP. Limits on improving the positive predictive value of the Welch Allyn SureSight for preschool vision screening. J AAPOS 2009;13:45-50. 4. Nathan NR, Donahue SP. Modification of Plusoptix referral criteria to enhance sensitivity and specificity during pediatric vision screening. J AAPOS 2011;15:551-5. 5. Arnold RW, Armitage MD. Performance of four new photoscreeners on pediatric patients with high risk amblyopia. J Pediatr Ophthalmol Strabismus 2014;51:46-52.
Volume 18 Number 5 / October 2014 6. Spot Instructions for Use. Model VS100. SPOT Vision Screener. PediaVision Holdings. Lake Mary, FL: LLC; 2011. 7. Donahue SP, Arthur B, Neely DE, Arnold RW, Silbert D, Ruben JB. AAPOS Vision Screening Committee. Guidelines for automated preschool vision screening: a 10-year, evidence-based update. J AAPOS 2013;17:4-8. 8. Hawass NED. Comparing the sensitivities and specificities of two diagnostic procedures performed on the same group of patients. Brit J Radiol 1997;70:360.
Novel use of the retinoscope in visualization of the anterior segment Examination of the anterior segment with proper magnification is often difficult in a young child, and slit-beam delineation of anterior segment structures is usually limited to the use of a portable slit-lamp. While providing adequate magnification with a pivoted slit-beam, the narrow field of view and short focal distance make estimating the proximity to the child’s eye difficult, rendering the examination challenging—and potentially hazardous. We propose a new method of examining the anterior segment in children that combines a 20 D indirect ophthalmoscopy lens with a streak retinoscope in the “sleeve-up” position with a narrowed beam. The 20 D lens is held approximately 3–5 cm from the patient’s eye, while the retinoscope streak is projected through the lens onto the anterior segment obliquely. This technique allows a magnified view of the anterior segment structures with slit-beam delineation and is particularly useful in determining anterior chamber depth in a child intolerant of slit-lamp examination, as when evaluating for iris bombe after anterior segment surgery or characterizing the tube position of a glaucoma drainage device. —Ta C. Chang, MD, and Kara M. Cavuoto, MD, Bascom Palmer Eye Institute, University of Miami Miller School of Medicine, Miami, Florida The authors thank Drs. Elizabeth Dale and Michael Smith for their assistance.
Journal of AAPOS