Analysis of competency in phacoemulsification trainees using the cumulative sum analysis

358

ARTICLE

Analysis of competency in phacoemulsification trainees using the cumulative sum analysis Nicole Shu-Wen Chan, MB BS, Aliza Jap, FRCOphth(UK), FRCS(Edin), FRCS(G), MB BS, Younian Yang, MPH, Shu-Pei Tan, BSc(Hons), Soon-Phaik Chee, FRCOphth, FRCS(G), FRCS(Edin), MMed(S'pore)

Purpose: To determine the number of cases required to achieve competency in phacoemulsification in terms of the posterior capsule rupture rate. Setting: Singapore National Eye Centre, Singapore. Design: Retrospective cohort study. Methods: The posterior capsule rupture rate of the first 300 phacoemulsification cases performed by each trainee in the Singapore National Eye Centre (2004 to 2012) was charted using cumulative sum graphs. Competency was primarily defined as a posterior capsule rupture rate of 2% or lower and secondarily as 5% or lower.

L

earning and performing safe cataract surgery are integral parts of all ophthalmology training programs. In the developed world, phacoemulsification is the technique of choice because it enables rapid visual rehabilitation. However, the phacoemulsification learning curve is steep because it requires good hand–eye coordination under an operating microscope and good foot control. As Singapore's largest provider of ophthalmic care, the Singapore National Eye Centre (SNEC) is also the main teaching institution for ophthalmology trainees. Its cataract training program comprises an initial induction via a didactic teaching course, wet-lab sessions involving animal eyes, and live surgery demonstrations. This is followed by 1-to-1 teaching in which the trainees are brought through the procedure 1 step at a time, starting from wound construction and so forth, until they can complete the entire case satisfactorily. A posterior capsule rupture is the most common serious complication of phacoemulsification and is

Results: Twenty trainees fulfilled the inclusion criteria. Four trainees (20%) and 19 trainees (95%) achieved a posterior capsule rupture rate of 2% and 5%, respectively. At least 41 cases were required to achieve a 5% posterior capsule rupture rate (mean 106, median 83). Conclusions: There was considerable variation in the trainees’ abilities. Only 20% of the trainees achieved a posterior capsule rupture rate of 2% or lower and required at least 212 cases. J Cataract Refract Surg 2017; 43:358–363 Q 2017 ASCRS and ESCRS

frequently used as a surrogate marker of a surgeon's safety record and competence.1 Although competency measured by surgical independence, efficiency, and a low complication rate can be achieved after 75 cases,2 Randleman et al.3 showed that the learning curve measured by the posterior capsule rupture rate might also extend beyond the first 200 cases. The requisite curriculum and minimum number of procedures required to achieve competency have not been well defined and to date, no studies have analyzed the phacoemulsification learning curve of trainees. The cumulative sum (CUSUM) chart is a statistical quality-control tool that provides an objective assessment of the outcomes of consecutive performances with reference to an agreed-on standard. It enables prompt detection of adverse events4 and monitors competency in surgical,5,6 anesthesia,7,8 and diagnostic procedures.9,10 However, there are few publications on the CUSUM chart in the ophthalmic literature.

Submitted: October 14, 2016 | Final revision submitted: December 8, 2016 | Accepted: December 10, 2016 From Singapore General Hospital (Chan), Division of Ophthalmology (Jap), Changi General Hospital, Singapore National Eye Centre (Jap, Yang, Chee), Singapore Eye Research Institute (Tan, Chee), the Department of Ophthalmology (Chee), Yong Loo Lin School of Medicine, National University of Singapore, and Duke-National University of Singapore Graduate Medical School Singapore (Chee), Singapore. Nguyen Duc Quang, Singapore Eye Research Institute, formatted the cumulative sum analysis graphs. Presented at the 20th Congress of the European Society of Ophthalmology, Vienna, Austria, June 2015. Corresponding author: Soon-Phaik Chee, FRCOphth, FRCS(G), FRCS(Edin), MMed(S'pore), Singapore National Eye Centre, 11 Third Hospital Avenue, Singapore 168751. E-mail: [email protected]. Q 2017 ASCRS and ESCRS Published by Elsevier Inc.

0886-3350/$ - see frontmatter http://dx.doi.org/10.1016/j.jcrs.2016.12.026

359

COMPETENCY IN PHACOEMULSIFICATION TRAINEES USING CUSUM ANALYSIS

Table 1. Analysis of trainees' performance in terms of number of cases required to achieve acceptable posterior capsule rupture rate and cumulative posterior capsule rupture rate. Acceptable PCR Rate p0 Z 2% Group

Trainee Performance

Cumulative PCR Rate

p0 Z 5%

Median

Range

Median

Range

Range (%)

1st (n Z 4)

Achieved an acceptable PCR rate

228

212, 247

55

55, 110

1.33, 2.33

2nd (n Z 5)

Initially unacceptable but showed improvement

NA

NA

138

55, 179

3.00, 3.67

3rd (n Z 7)

Required more observation

NA

NA

83

41, 96

2.00, 3.33

4th (n Z 4)

Consistently unacceptable

NA

NA

207

138, 248

5.00, 6.67

NA Z not applicable; p0 Z acceptable failure rate; PCR Z posterior capsule rupture

This study aimed to determine the number of cases trainees had to perform to achieve competency, measured by the posterior capsule rupture rate, using CUSUM analysis. MATERIALS AND METHODS This retrospective cohort study was performed at the SNEC from 2004 to 2012 for trainees who had training at the SNEC. Only the first 300 cases performed after the trainee had been taught the entire procedure satisfactorily were included in the analysis. Trainees who were rotated to other hospitals for training were excluded. The data of all cataract surgery complications and outcomes were obtained as part of an ongoing clinical audit of the cataract education program. This study was performed in accordance with the principles of the Declaration of Helsinki. The Singapore Health Services Centralized Institutional Review Board waived the requirement for approval of this study of audit data. The main outcome measure analyzed was posterior capsule rupture. Competency in phacoemulsification was defined as achieving an acceptable posterior capsule rupture rate of 2% or less. Cumulative Sum Analysis Because the average cumulative posterior capsule rupture rate at SNEC's consultant faculty based on the clinical audit over the study period was 1.4%,11 the acceptable (p0) and unacceptable (p1) failure rates were specified at 2% and 4%, respectively. Each trainee's performance was represented by a line graph of the cumulative failures (CUSUM score) against the sequential case number.8 The p0 and p1 values were used to calculate a value s (ie, indicating success [number of cases without posterior capsule rupture]), by which the CUSUM score decreases for each case without posterior capsule rupture and the graph slopes downward. Conversely, for each case in which posterior capsule rupture occurs, the CUSUM score increases by 1
s and the chart slopes upward.4 The p0 and p1 values are also used to determine the decision intervals of the acceptable (h0) and unacceptable (h1) control lines. The specified type I error (a) and type II errors (b) were

both set at 0.10,12 resulting in equal magnitudes for the h0 and h1 decision intervals.13 If a trainee had consecutive cases without a posterior capsule rupture, the CUSUM chart sloped downward to a point below h0 and the null hypothesis that the true failure rate was not significantly different from the acceptable posterior capsule rupture failure rate of 2% was accepted with a type II error of 0.10. If a trainee had consecutive cases with posterior capsule ruptures, the upward sloping chart eventually exceeded h1. The null hypothesis was rejected, and the alternative hypothesis that the true failure rate was statistically significantly higher than the acceptable failure rate of 2% was accepted with a type I error of 0.10. After exceeding h1, the CUSUM chart was restarted with h1 as the new x-axis and a new decision interval set at 2h1. If the CUSUM graph remained between h0 and h1 without crossing either line, further observation was needed before any statistical inference could be made. The sample size required was calculated using the above parameters.14 The expected number of attempted procedures to cross h0 and h1 was 274 and 224, respectively. Hence, the first 300 consecutive phacoemulsification cases performed by each trainee were included in this study. Because the rate of posterior capsule rupture for trainees in a literature review was between 4.4% and 9.9%,15–17 an acceptable and an unacceptable failure rate were also secondarily specified at 5% and 10%, respectively. The CUSUM scores4 were calculated using R software (version 3.1.0),A and the CUSUM graphs were obtained using Stata software (version 11.1, Statacorp LLC).

RESULTS Twenty trainees fulfilled the inclusion criteria. The mean cumulative posterior capsule rupture rates for the first 75, 100, and 300 cases performed by all trainees were 4.80% G 2.81% (SD), 4.50% G 2.50%, and 3.33% G 1.54%, respectively.

Figure 1. Cumulative sum graph of a trainee in the first group showing that the trainee achieved an acceptable posterior capsule rupture rate of 2% (left) and 5% (right) when the graph crossed h0 after 212 (A) and 55 (B) cases, respectively. The cumulative posterior capsule rupture rate of this trainee was 1.33% (CUSUM Z cumulative sum).

Volume 43 Issue 3 March 2017

360

COMPETENCY IN PHACOEMULSIFICATION TRAINEES USING CUSUM ANALYSIS

Figure 2. Cumulative sum graph of a trainee in the second group showing that the trainee had an initially high posterior capsule rupture rate higher than 2% (left) with a subsequent downward trend that approached but did not reach h0 for a posterior capsule rupture rate of 2%. This trainee achieved an acceptable posterior capsule rupture rate of 5% when the graph crossed h0 after 179 cases (right; A). The cumulative posterior capsule rupture rate of this trainee was 3.33% (CUSUM Z cumulative sum).

Using an acceptable and unacceptable rate of 5% and 10%, respectively, 19 (95%) of 20 trainees achieved a posterior capsule rupture rate not significantly different from 5% by the 300th case. At least 41 cases were required, with a mean and median of 106 cases and 83 cases, respectively. Analysis of the CUSUM graphs using acceptable and unacceptable posterior capsule rupture rates of 2% and 4%, respectively, showed that the trainees could be classified into 4 groups (Table 1) according to competency, defined as posterior capsule rupture rate of 2% or lower. The first group comprised 4 trainees (20%) whose CUSUM graphs sloped downward early and decreased to a point below the h0 control line (Figure 1, left). These trainees also achieved an acceptable posterior capsule rupture rate of 5% (Figure 1, right). The second group comprised 5 trainees (25%) whose CUSUM graph initially exceeded h1 but thereafter sloped downward (Figure 2, left). Although their graphs did not decrease to below h0, the downward trend suggests that they were likely to achieve a posterior capsule rupture rate not significantly different from 2% beyond 300 cases. These trainees achieved an acceptable posterior capsule rupture rate of 5% (Figure 2, right). No statistical inference could be made for the 7 trainees (35%) in the third group, whose CUSUM graphs crossed neither h0 nor h1 (Figure 3, left). These trainees reached an acceptable posterior capsule rupture rate of 5% (Figure 3, right). Further observation of subsequent cases would be required to determine whether these trainees could also achieve a posterior capsule rupture rate not significantly different from 2%. The fourth group of 4 trainees (20%) had CUSUM graphs that exceeded h1 multiple times (Figure 4, left) with no subsequent improvement in performance. Only 3 of these trainees reached an acceptable posterior capsule rupture rate of 5% (Figure 4, right). The cumulative posterior capsule rupture rates of these trainees were much higher than those of the trainees in the first and second groups. DISCUSSION Complications that occur during the trainees’ learning process are of major concern with regard to patient safety.18 In general, more complications are expected to occur, especially in the first few cases,19,20 even in the setting of a structured training program and careful case selection. Volume 43 Issue 3 March 2017

In our study, almost all trainees (95%) achieved a posterior capsule rupture rate not significantly different from 5%, which is consistent with the posterior capsule rupture rates of 4.4% to 9.9% quoted for trainees in a literature review.15–17 At least 41 cases were required to achieve this posterior capsule rupture rate. The mean cumulative posterior capsule rupture rate for the first 75 cases was 4.80% G 2.81%. This is consistent with results in previous studies2,3 that found 75 to 80 cases as a milestone after which trainees are able to perform phacoemulsification more independently and efficiently with significantly lower vitreous loss rates. In comparison, at least 212 cases were required for trainees to achieve a posterior capsule rupture rate not significantly different from 2%, and only 4 (25%) of 20 trainees were able to achieve this. This suggests that although the majority of trainees can achieve a standard expected of trainees by the end of their training, only a small proportion excel and achieve a posterior capsule rupture rate comparable to that achieved by consultants. In practice, it is difficult to expect all trainees to perform at standards similar to consultants by the end of their training. A substantial amount of further training and a considerably greater caseload would be required for most trainees to achieve such a standard. Individual variation in surgical skill between trainees is a common problem in phacoemulsification training.21 Up to 10% of the trainees might encounter problems in developing the necessary surgical skills.22 Because trainees acquire skills at different rates, the emphasis should be on the quantity of cases performed as well as on optimizing the training by early identification of those with persistent difficulties. Cumulative posterior capsule rupture rates require at least 100 cases for meaningful analysis, rendering it difficult to determine which trainees require more attention early on, especially for trainees who are performing neither extremely well nor very poorly. The CUSUM analysis is more sensitive to a change in performance than the cumulative probability plot; hence, the CUSUM would be more useful as a tool for continuous quality improvement in trainees.23,24 This continuous monitoring of performance enables the early detection of trainees with difficulties and allows the training program to be tailored to the individual's capabilities. In our cohort, analysis of the CUSUM graphs using an acceptable posterior capsule rupture rate of 2% or less

COMPETENCY IN PHACOEMULSIFICATION TRAINEES USING CUSUM ANALYSIS

361

Figure 3. Cumulative sum graph of a trainee in the third group whose graph crossed neither h0 nor h1 when using an acceptable posterior capsule rupture rate of 2% (left), although an acceptable posterior capsule rupture rate of 5% was achieved after 83 cases (right; A). The cumulative posterior capsule rupture rate of this trainee was 2.33% (CUSUM Z cumulative sum).

allowed the surgical competency of trainees to be classified into 4 recognizable patterns. The first group comprised trainees who were able to achieve a posterior capsule rupture rate not significantly different from 2%, a complication rate nearly comparable to that of consultants at a median number of 228 cases, with a low cumulative posterior capsule rupture rate ranging from 1.33% to 2.33%. Three of the 4 trainees in this group also achieved a posterior capsule rupture rate not significantly different from 5% by the 55th case. These trainees were likely to be the more surgically talented individuals who achieved competency consistent with that of a consultant. The second group comprised trainees who initially had an unacceptable posterior capsule rupture rate of at least 4% but improved thereafter and, given time, could possibly attain a posterior capsule rupture rate not significantly different from 2%. Their cumulative posterior capsule rupture rate was slightly higher, ranging from 3.00% to 3.67%. These trainees required a median of 138 cases to reach a posterior capsule rupture rate of 5% or less. This group would reflect the average trainee who encounters difficulties in the initial cases but subsequently improves with practice. They require a higher volume of cases to achieve competency, described as a posterior capsule rupture rate of 2% or lower. The third group comprised trainees whose graphs did not cross either of the decision intervals for a 2% posterior capsule rupture rate but were able to achieve a 5% or lower posterior capsule rupture rate at a median of 83 cases. Their cumulative posterior capsule rupture rate ranged from 2.00% to 3.33%, suggesting that they could also eventually achieve a posterior capsule rupture rate of 2% or lower, similar to the trainees in the second group.

The fourth group comprised trainees who had difficulty achieving a posterior capsule rupture rate of 5% or less by a median of 207 cases, with the exception of 1 trainee who failed to do so. Not unexpectedly, these trainees did not achieve a posterior capsule rupture rate of 2%, and they had a cumulative posterior capsule rupture rate ranging from 5.00% to 6.67%. These trainees might be less surgically talented and require more intensive coaching. The CUSUM graphs also show that early performance outcomes are a good indicator of the trainee's potential because the trainees in the first group were able to achieve a posterior capsule rupture rate of less than 5% by 55 cases compared with those in the third and fourth groups, who achieved posterior capsule rupture rates not significantly different from 5% after more than 100 cases. Tutors can therefore use the CUSUM graph to identify which group each trainee is in early in the course of training (after 55 cases). Early identification of trainees with persistent difficulties will enable tutors to prioritize which trainees require more urgent intervention. Once tutors have identified trainees who require intervention, appropriate measures, such as more intensive training targeted at each individual0 s specific areas of weakness, can then be undertaken in a timely manner. The CUSUM analysis therefore helps tutors prioritize and allocate limited training resources more effectively, ensuring that the weaker trainees achieve competency by the end of their training. This study shows the usefulness of the CUSUM analysis and provides insight into the various types of trainees, an important aspect that is seldom emphasized in other phacoemulsification learning curve studies that analyze pooled group data. Although this study only examined the

Figure 4. Cumulative sum graph of a trainee in the fourth group showing a posterior capsule rupture rate consistently higher than 2% (left) and crossing h0 to achieve an acceptable posterior capsule rupture rate of 5% after 248 cases (right; A). The cumulative posterior capsule rupture rate of this trainee was 6.00% (CUSUM Z cumulative sum).

Volume 43 Issue 3 March 2017

362

COMPETENCY IN PHACOEMULSIFICATION TRAINEES USING CUSUM ANALYSIS

usefulness of the CUSUM analysis in identifying trainees who might need further training using just 1 factordthe posterior capsule rupture ratedit can similarly be used to analyze other equally important aspects of their surgical skills in phacoemulsification, such as the ability to recognize and manage a posterior capsule rupture. This study provides a basis for future studies involving larger cohorts or trainees in different institutions to achieve a greater understanding of how to optimize training programs. It can guide tutors in prioritizing the allocation of limited training resources to optimize the training outcomes while maintaining patient safety during the learning curve. It can further be used to show the efficacy of various teaching techniques, including the use of a surgery simulator. The limitations of this study include the small sample of 20 trainees, which limits the generalizability of results to all phacoemulsification trainees, especially because the selection process of trainees varies between institutions. This study solely used the posterior capsule rupture rate as a marker of competence. However, because the training cases were carefully selected to exclude complex cases, such as those with small pupils, zonular weakness, and dense brunescent or white cataracts, these are unlikely to be confounding factors in the posterior capsule rupture rate. Another limitation of this study was the lack of stratification by tutors because the posterior capsule rupture rate might be modified by the tutor's active or passive attitude in intervening.25 Other useful indicators of surgical competency, such as the surgical time, stage at which the posterior capsule rupture occurred, and visual outcome, were also not included in this study. In conclusion, trainees required at least 41 cases and 212 cases to achieve an acceptable posterior capsule rupture rate of 5% and 2%, respectively. Because trainees have varying abilities, the emphasis should not only be on completing a specified quantity of cases but also on tailoring the training to the individual's performance. The CUSUM analysis is a useful tool that can be implemented in training programs to continuously monitor each trainee's progress, enabling early identification of trainees who require more intensive training.

WHAT WAS KNOWN  Trainees' surgical competency in phacoemulsification as measured by surgical independence and efficiency might be achieved after 75 cases; however, the learning curve in terms of the posterior capsule rupture rate might extend beyond the first 200 cases.

WHAT THIS PAPER ADDS  At least 200 cases might be required for trainees to achieve competency in terms of a posterior capsule rupture rate of 2% or less.  There was considerable variability in learning capacity, and the CUSUM analysis was useful in tailoring the training program for the individual trainee.

Volume 43 Issue 3 March 2017

REFERENCES 1. Johnston RL, Taylor H, Smith R, Sparrow JM. The Cataract National Dataset electronic multi-centre audit of 55 567 operations: variation in posterior capsule rupture rates between surgeons. Eye 2010; 24:888–893. Available at: http://www.nature.com/eye/journal/v24/n5/pdf/eye2009195a.pdf. Accessed January 12, 2017 2. Taravella MJ, Davidson R, Erlanger M, Guiton G, Gregory D. Characterizing the learning curve in phacoemulsification. J Cataract Refract Surg 2011; 37:1069–1075 3. Randleman JB, Wolfe JD, Woodward M, Lynn MJ, Cherwek DH, Srivastava SK. The resident surgeon phacoemulsification learning curve. Arch Ophthalmol 2007; 125:1215–1219. Available at: http://archopht. jamanetwork.com/article.aspx?articleidZ419796. Accessed January 12, 2017 4. Salowi MA, Choong YF, Goh PP, Ismail M, Lim TO. CUSUM: a dynamic tool for monitoring competency in cataract surgery performance. Br J Ophthalmol 2010; 94:445–449. Available at: http://bjo.bmj.com/content/94/4/ 445.full.pdf. Accessed January 12, 2017 5. Jaffer U, Cameron AEP. Laparoscopic appendectomy: a junior trainee’s learning curve. JSLS 2008; 12:288–291. Available at: https://www.ncbi. nlm.nih.gov/pmc/articles/PMC3015879/pdf/jsls-12-3-288.pdf. Accessed January 12, 2017 6. Choi DH, Jeong WK, Lim SW, Chung TS, Park JI, Lim SB, Choi HS, Nam BH, Chang HJ, Jeong SY. Learning curves for laparoscopic sigmoidectomy used to manage curable sigmoid colon cancer: single-institute, three-surgeon experience. Surg Endosc 2009; 23:622–628 7. Young A, Miller JP, Azarow K. Establishing learning curves for surgical residents using Cumulative Summation (CUSUM) Analysis. Curr Surg 2005; 62:330–334 8. Kestin IG. A statistical approach to measuring the competence of anaesthetic trainees at practical procedures. Br J Anaesth 1995; 75:805–809. Available at: http://bja.oxfordjournals.org/content/75/6/805.full.pdf. Accessed January 12, 2017 9. Schlup MM, Williams SM, Barbezat GO. ERCP: a review of technical competency and workload in a small unit. Gastrointest Endosc 1997; 46:48–52 10. Molloy M, Bower RH, Hasselgren PO, Dalton BJ. Cholangiography during laparoscopic cholecystectomydcumulative sum analysis of an institutional learning curve. J Gastrointest Surg 1999; 3:185–188 11. Ti SE, Yang YN, Lang SS, Chee SP. A 5-year audit of cataract surgery outcomes after posterior capsule rupture and risk factors affecting visual acuity. Am J Ophthalmol 2014; 157:180–185 12. Yap CH, Colson ME, Watters DA. Cumulative sum techniques for surgeons: a brief review. ANZ J Surg 2007; 77:583–586 13. Wani S, Hall M, Keswani RN, Aslanian HR, Casey B, Burbridge R, Chak A, Chen AM, Cote G, Edmundowicz SA, Faulx AL, Hollander TG, Lee LS, Mullady D, Murad F, Muthusamy VR, Pfau PR, Scheiman JM, Tokar J, Wagh MS, Watson R, Early D. Variation in aptitude of trainees in endoscopic ultrasonography, based on Cumulative Sum Analysis. Clin Gastroenterol Hepatol 2015; 13:1318–1325 14. Rujirojindakul P, McNeil E, Rueangchira-urai R, Siripunt N. Learning curves of Macintosh laryngoscope in nurse anesthetist trainees using cumulative sum method. Anesthesiol Res Pract 2014; article ID 850731. Available at: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3944791/pdf/ARP2014850731.pdf. Accessed January 12, 2017 15. Cruz OA, Wallace GW, Gay CA, Matoba AY, Koch DD. Visual results and complications of phacoemulsification with intraocular lens implantation performed by ophthalmology residents. Ophthalmology 1992; 99:448–452 16. Carricondo PC, Morais Fortes ACF, de Carvalho Mour~ao P, Hajnal M, Kara Jose N. Senior resident phacoemulsification learning curve (corrected from cure). Arq Bras Oftalmol 2010; 73:66–69. Available at: http://www.scielo. br/pdf/abo/v73n1/v73n1a12.pdf. Accessed January 12, 2017 17. Tabandeh H, Smeets B, Teimory M, Seward H. Learning phacoemulsification: the surgeon-in-training. Eye 1994; 8:475–477. Available at: http:// www.nature.com/eye/journal/v8/n4/pdf/eye1994113a.pdf. Accessed January 12, 2017 18. Dooley IJ, O’Brien PD. Subjective difficulty of each stage of phacoemulsification cataract surgery performed by basic surgical trainees. J Cataract Refract Surg 2006; 32:604–608 19. Allinson RW, Palmer ML, Fante R, Stanko M. Vitreous loss during phacoemulsification by residents [letter]. Ophthalmology 1992; 99:1181. Available at: http://www.aaojournal.org/article/S0161-6420(92)31830-5/pdf. Accessed January 12, 2017 20. Allinson RW, Metrikin DC, Fante RG. Incidence of vitreous loss among third year residents performing phacoemulsification. Ophthalmology 1992; 99:726–730 21. Corey RP, Olson RJ. Surgical outcomes of cataract extractions performed by residents using phacoemulsification. J Cataract Refract Surg 1998; 24:66–72

363

COMPETENCY IN PHACOEMULSIFICATION TRAINEES USING CUSUM ANALYSIS

22. Binenbaum G, Volpe NJ. Ophthalmology resident surgical competency; a national survey. Ophthalmology 2006; 113:1237–1244 23. Grunkemeier GL, Jin R, Wu Y. Cumulative sum curves and their prediction limits. Ann Thorac Surg 2009; 87:361–364. Available at: http://www.annal sthoracicsurgery.org/article/S0003-4975(08)02278-9/pdf. Accessed January 12, 2017 24. Sibanda T, Sibanda N. The CUSUM chart method as a tool for continuous monitoring of clinical outcomes using routinely collected data. BMC Med Res Methodol 2007; 7:46. Available at: https://www.ncbi.nlm.nih.gov/ pmc/articles/PMC2204022/pdf/1471-2288-7-46.pdf. Accessed January 12, 2017 25. Lee JS, Hou CH, Yang ML, Kuo JZ, Lin KK. A different approach to assess resident phacoemulsification learning curve: Analysis of both completion and complication rates. Eye 2009; 23:683–687. Available at: http://www.nature. com/eye/journal/v23/n3/pdf/6703103a.pdf. Accessed January 12, 2017

http://cran.r-project.org/doc/manuals/refman.pdf. Accessed January 12, 2017

Disclosures: None of the authors has a financial or proprietary interest in any material or method mentioned.

First author: Nicole Shu-Wen Chan, MB BS Singapore General Hospital, Singapore

OTHER CITED MATERIAL A. R Development Core Team. The R Project for Statistical Computing. Vienna, Austria, R Foundation for Statistical Computing. Available at:

Volume 43 Issue 3 March 2017