- Email: [email protected]
Standardized reporting of appendicitis-related findings improves reliability of ultrasound in diagnosing appendicitis in children
Journal of Pediatric Surgery 53 (2018) 984–987
Contents lists available at ScienceDirect
Journal of Pediatric Surgery journal homepage: www.elsevier.com/locate/jpedsurg
Standardized reporting of appendicitis-related findings improves reliability of ultrasound in diagnosing appendicitis in children☆ Richard Sola Jr. a, Stephanie B. Theut b, Kelly A. Sinclair c, Doug C. Rivard b, Kathy M. Johnson a, Huirong Zhu d, Shawn D. St. Peter a, Sohail R. Shah d,⁎ a
Department of Surgery, The Center for Prospective Clinical Trials, The Children's Mercy Hospital, Kansas City, MO, USA Department of Radiology, The Children's Mercy Hospital and Clinics, Kansas City, MO, USA Department of Emergency Medicine, The Children's Mercy Hospital and Clinics, Kansas City, MO, USA d Division of Pediatric Surgery, Texas Children's Hospital/Baylor College of Medicine, Houston, TX, USA b c
a r t i c l e
i n f o
Article history: Received 22 January 2018 Accepted 1 February 2018 Key words: Appendicitis Children Ultrasound Pediatric surgery
a b s t r a c t Purpose: Our objective was to increase ultrasound reliability for diagnosing appendicitis in an academic children's hospital emergency department (ED) through a multidisciplinary quality improvement initiative. Methods: A retrospective review of ultrasound use in patients diagnosed with appendicitis in our ED from 1/1/ 2011 to 6/30/2014 established a baseline cohort. From 8/1/2014 to 7/31/2015 a diagnostic algorithm that prioritized ultrasound over CT was used in our ED, and a standardized template was implemented for the reporting of appendicitis-related ultrasound findings by our radiologists. Results: Of 627 patients diagnosed with appendicitis in the ED during the retrospective review, 46.1% (n = 289) had an ultrasound. After implementation of the diagnostic algorithm and standardized ultrasound report, 88.4% (n = 236) of 267 patients diagnosed with appendicitis had an ultrasound (p b 0.01). The frequency of indeterminate results decreased from 44.3% to 13.1%, and positive results increased from 46.4% to 66.1% in patients with appendicitis (p b 0.01). The sensitivity of ultrasound (indeterminate counted as negative) increased from 50.6% to 69.2% (p b 0.01). Conclusions: Ultrasound reliability for the diagnosis of appendicitis in children can be improved through standardized results reporting. However, these changes should be made as part of a multidisciplinary quality improvement initiative to account for the initial learning curve necessary to increase experience. Level of Evidence: Level II, Study of Diagnostic Test. © 2018 Elsevier Inc. All rights reserved.
Acute appendicitis (AA) is the most common surgical emergency in children, accounting for approximately 72,000 hospitalizations annually in the United States [1]. However, the diagnosis of appendicitis remains challenging and often relies upon radiographic adjuncts to confirm diagnosis. Computed tomography (CT) scans has been shown to have a high sensitivity and specificity for the diagnosis of appendicitis and this has resulted in their increased use across the country [2,3]. However, concerns about radiation exposure from CT resulting in an increased risk of malignancy in children have resulted in greater efforts to utilize ultrasonography (US) [4–6]. Current recommendations by the American College of Radiology state that US should be the initial imaging modality for the evaluation of suspected appendicitis in children [7,8]. The limitation of US is its dependence on individual operator experience, and this has resulted ☆ All authors have no conflicts of interest to disclose regarding this study. ⁎ Corresponding author at: Texas Children's Hospital / Baylor College of Medicine, 6701 Fannin Suite 1210, Houston, TX 77030. Tel.: +1 832 822 3135; fax: +1 832 825 3141. E-mail address: [email protected] (S.R. Shah). https://doi.org/10.1016/j.jpedsurg.2018.02.028 0022-3468/© 2018 Elsevier Inc. All rights reserved.
in wide variation in its diagnostic accuracy. Reported US sensitivities range from 44% to 94% and specificities range from 47% to 95% [2,3,9–11]. At some centers up to 74% of USs for acute appendicitis are nondiagnostic, which limit the utilization of US and ultimately lead to continued overutilization of CT for the diagnosis of appendicitis [12–19]. The objective of this study was to increase US reliability for diagnosing acute appendicitis in an academic children's hospital emergency department (ED) as part of a multidisciplinary quality improvement initiative. 1. Methods 1.1. Study design In February 2014, as part of a quality improvement (QI) initiative, we established a multidisciplinary team including physician representatives from pediatric surgery, emergency medicine, and radiology at Children's Mercy Hospital (CMH) in Kansas City, Missouri. The project was approved as a QI project intended to improve practice within
R. Sola Jr. et al. / Journal of Pediatric Surgery 53 (2018) 984–987
CMH and was deemed nonhuman subjects research by the CMH Office of Research Integrity. As part of the QI initiative the team created a diagnostic algorithm to formalize the evaluation of patients with suspected appendicitis that would allow for earlier surgical consultation and prioritize US imaging over CT scans. The results of our algorithm implementation have been previously published [20]. In addition to the diagnostic algorithm, the team created a standard template for the reporting of appendicitis-related ultrasound findings by our radiologists. 1.1.1. Retrospective review and data collection Our team initially set out to establish a baseline understanding of the current state of the diagnostic workup for appendicitis in our ED. This was conducted as a retrospective review of all patients diagnosed with appendicitis in the ED from January 1, 2011 to June 30, 2014. Inclusion criteria were all patients seen at CMH with acute appendicitis as identified by ICD-9 codes (540, 540.0, 540.1, 540.9, or 541) or CPT codes (44,950, 44,960, or 44,970). Exclusion criteria were any patient not seen in the CMH ED during the same visit or with outside imaging
985
already diagnostic of acute appendicitis. Data collected included age, gender, body mass index (BMI), information related to diagnostic imaging if performed, treatment of appendicitis, intraoperative findings, and pathology. US reports were also reviewed and categorized as: negative, positive, or indeterminate. 1.1.2. Creation of a standardized ultrasonography reporting template In addition to creation of the new diagnostic algorithm (Fig. 1), our team created a standardized template for the reporting of US findings when evaluating patients with suspected appendicitis. The diagnostic algorithm prioritized the use of US over CT, and included pathways where the ED and/or pediatric surgical providers could request a US for further workup. The standardized US report consisted of four categories: Category 1—negative for appendicitis, Category 2—appendix not visualized without any secondary signs of appendicitis, Category 3—appendix not visualized with secondary signs of appendicitis, Category 4— findings consistent with appendicitis. With this standardized US report, treating providers were instructed that Categories 1 & 2 should be
Fig. 1. Diagnostic algorithm utilized for suspected appendicitis. Alt indicates alternative.
986
R. Sola Jr. et al. / Journal of Pediatric Surgery 53 (2018) 984–987
interpreted as negative for appendicitis, while Categories 3 & 4 should be interpreted as positive for appendicitis. Before implementation of the standardized US report, it was presented at departmental/divisional meetings in pediatric surgery, emergency medicine, and radiology for consensus approval. 1.1.3. Prospective evaluation and data collection After approval, the diagnostic algorithm and standardized US report were implemented on August 1, 2014 and prospectively followed until July 31, 2015. During the 1-year prospective evaluation period, data were collected. The inclusion criteria for prospective follow-up were any patient evaluated in the CMH ED with a suspicion of appendicitis. Exclusion criteria included any patient that had outside imaging already diagnostic of appendicitis. Prospectively followed patients were captured by several methods: (i) The pediatric appendicitis score (PAS), which was utilized as part of the diagnostic algorithm for the initial evaluation, was built into the electronic medical record (EMR). Once the emergency medicine provider selected the individual components of the PAS, a cumulative score was calculated and stored in the patient's record. A daily and weekly report of PAS entries into the EMR was autoforwarded to a surgical research nurse (KMJ) and the principal investigator (PI) (SRS). (ii) A weekly report of all patients discharged home from the ED with a diagnosis of “abdominal pain” in the EMR was autoforwarded to the PI. (iii) A weekly report of all patients that underwent a US or CT scan for “appendicitis,” “appy,” “right lower quadrant,” or “RLQ” in the ED was also autoforwarded to the PI. (iv) Finally, a weekly report of all patients that had an appendectomy (identified by CPT codes 44,950, 44,960, and 44,970) at CMH was autoforwarded to the PI. For every patient on these reports, the EMR was reviewed. If the subject met inclusion criteria, then subject data were entered by the research nurse into a secured electronic spreadsheet for analysis. Each subject's medical record was independently reviewed by the PI for validation of data entry. Data collected during prospective follow-up included age, gender, BMI, information related to diagnostic imaging used, disposition from ED, treatment of appendicitis, intraoperative findings, pathology, and adherence to the standardized US report. In addition, any patient included that was discharged from the ED or admitted to the hospital with a diagnosis other than appendicitis received a follow-up telephone call by ED personnel or our research nurse between 24 h and 1 week from their ED visit to confirm resolution of symptoms. During the 1-year prospective evaluation period, we continued with ongoing education and feedback for emergency medicine, radiology, and pediatric surgery providers to increase awareness and adherence to the standardized US report. This was conducted as reminders at departmental/divisional meetings and as individualized feedback to providers that showed variance from utilization of the standard US report. 1.2. Statistics Descriptive statistics including counts and percentages were analyzed. Pearson's chi-square and Fisher's exact tests were used for categorical variables, and the frequencies were reported as a percentage of the group of origin. Statistical significance was set at p b 0.05, and all reported p values are two-tailed. 2. Results 2.1. Comparison of ultrasonography results before and after implementation of standardized ultrasonography reporting There were 627 patients diagnosed with appendicitis in the ED during the retrospective review period (January 1, 2011 to June 30, 2014), and 289 (46.1%) of them had a US as part of their workup. There were 840 patients seen in the ED for suspected appendicitis during the prospective evaluation period after implementation of the
diagnostic algorithm and standardized US reporting (August 1, 2014 to July 31, 2015). Of these prospectively followed patients, there were 267 (31.7%) diagnosed with appendicitis and 236 (88.4%) of them had a US as part of their workup. The utilization of US was significantly higher in patients diagnosed with appendicitis after implementation of the diagnostic algorithm and standardized US reporting (88.4% vs. 46.1%, p b 0.01). The preimplementation appendicitis cohort that had a US (n = 289) and the postimplementation appendicitis cohort that had a US (n = 236) had very similar demographics. There were no statistically significant differences in age, gender distribution, or BMI between the two cohorts (Table 1). However, the frequency of indeterminate US reports decreased significantly after implementation of standardized US reporting for appendicitis. After implementation, indeterminate US results decreased from 44.3% to 13.1% (p b 0.01). Additionally, the frequency of US reports read as positive increased from 46.4% to 66.1% (p b 0.01), and US reports read as negative increased from 9.3% to 20.8% (p b 0.01) (Table 2). These changes were realized after implementation of standardized US reporting without adversely affecting clinical outcomes. The negative appendectomy rate, based on pathology evaluation, was 9.0% in the preimplementation cohort compared to 9.3% in the postimplementation cohort (p = 1.0) (Table 2). 2.2. Effects of standardized ultrasonography reporting on all patients with suspected appendicitis After implementation of the diagnostic algorithm and standardized US reporting for appendicitis 766 (91.2%) of all 840 patients with suspected appendicitis in the ED were evaluated with US. The 74 patients that were not evaluated with a US were discharged based on history and physical alone, taken to the operating room without imaging, or underwent evaluation with CT without prior US. Of these 766 patients, the US was reported positive for appendicitis in 177 (23%), negative for appendicitis in 511 (67%), and indeterminate in 78 (10%). This resulted in a sensitivity of 69.2% and specificity of 94.0% for US in patients with suspected appendicitis after implementation of the standardized reporting (indeterminate results were counted as negative). This was a significant increase in sensitivity when compared to the US results in the retrospectively reviewed appendicitis cohort (69.2% vs. 50.6%, p b 0.01). The missed appendectomy rate was 0.8% after implementation. The overall compliance with utilization of standardized US reporting by radiology was 85%. Additionally, the telephone follow-up rate for patients discharged from the ED was 86% during the prospective evaluation period. 3. Discussion The reliance on CT scans for the diagnosis of appendicitis in children has significantly increased over the years [20–22]. Current recommendations state that US should be the initial imaging modality for the evaluation of appendicitis in children [8]. At CMH, we were concerned about our higher utilization of CT and lower reliance upon US for the diagnosis of appendicitis as compared to our peers. In an effort to decrease CT utilization and increase US utilization through a QI initiative, we established a multidisciplinary team to evaluate and address this concern. The retrospective review data confirmed our suspicions and revealed a CT utilization rate of 75.4% and only 24.4% utilization of US prior to CT for the diagnosis of appendicitis in our ED. Through our multidisciplinary team, we then created a diagnostic algorithm that used PAS for risk stratification of patients with suspected appendicitis, focused on early surgical consultation, and prioritized US utilization before CT for diagnostic imaging [20]. However, one additional concern was our prior experience with utilizing US for the evaluation of appendicitis and the large number of US reports with an indeterminate result. In an effort to address this concern, we created a standardized US reporting template and implemented it at the same time as the newly created diagnostic algorithm. We also established a detailed workflow that allowed us to
R. Sola Jr. et al. / Journal of Pediatric Surgery 53 (2018) 984–987 Table 1 Demographics of patients with appendicitis before and after implementation of standardized ultrasonography reporting.
Age (years) Gender (% female) BMI
Prior to Diagnostic Algorithm & Standardized US Report (N = 289)
After Diagnostic Algorithm & Standardized US Report (N = 236)
p value
10.4 45.2% 20.5
10.2 41.1% 19.9
NS NS NS
identify patients evaluated for suspected appendicitis in the ED and prospectively follow them. In addition, we formalized a plan for telephone follow-up for all patients that were not diagnosed with appendicitis during their ED visit so we could accurately capture any patient with missed appendicitis. The simultaneous implementation of a complete diagnostic algorithm and prospective evaluation with telephone follow-up was an important factor that allowed providers to feel comfortable during the evaluation period to comply with the standardized US reporting and interpretation. In addition, it allowed us the ability to track the missed appendicitis rate and accurately calculate the specificity of the standardized US reporting, which differentiates this project from most others. After aggressive education efforts within pediatric surgery, radiology, and emergency medicine, the standardized US reporting was implemented. During the 1-year prospective evaluation period, we continued with ongoing education and feedback to our radiologists to increase adherence to the standardized US report. This was conducted as reminders at departmental/divisional meetings and as individualized feedback to providers that showed variance from utilization of the standard US report. These ongoing efforts resulted in an increase in compliance from 76% in the first quarter to 86%–88% during the last three quarters of our evaluation period. Our comparative analysis of patients diagnosed with appendicitis in the ED before and after the implementation of the standardized US reporting demonstrated impressive results. The frequency of indeterminate US result reporting was cut by two-thirds (44.3% to 13.1%). In addition, we significantly increased the sensitivity of US for children with suspected appendicitis at our institution from 50.6% to 69.2%. The specificity of US with the standardized reporting was 94.0%. This was all accomplished without any significant change in the negative appendectomy rate. During the prospective follow-up of all patients with suspected appendicitis in the ED, we were able to reach 86% of patients that were not diagnosed with appendicitis for telephone followup. The missed appendectomy rate was 0.8%. While the overall sensitivity of US in our experience is still low at 69.2% compared to the literature, the diagnostic algorithm as a whole had a sensitivity of 98.6% and a specificity of 94.4% emphasizing the importance of the role of an algorithm as a whole compared to its individual parts. Additionally, these findings demonstrate the importance of making significant practice changes such as this as part of a larger
Table 2 Comparison of patients with appendicitis before and after implementation of standardized ultrasonography reporting.
US Results Positive Negative Indeterminate Negative appendectomy rate
Prior to Diagnostic Algorithm & Standardized US Report (N = 289)
After Diagnostic Algorithm & Standardized US Report (N = 236)
134 (46.4%) 27 (9.3%) 128 (44.3%) 26 (9.0%)
156 (66.1%) 49 (20.8%) 31 (13.1%) 22 (9.3%)
p value
b0.01
NS
987
multidisciplinary effort to ensure maintaining high quality of care for patients. We expect as the experience with US increases at our hospital over time, the sensitivity and specificity of US will also increase. In an effort to maintain continuous quality improvement, there are continued educational and training efforts for our US technologists, who perform the studies with both resident and attending radiology coverage 24 h a day. These efforts should result in a lower incidence of missed appendicitis, a lower negative appendectomy rate, and an even greater shift from utilization of CT to US for the diagnosis of appendicitis at our hospital. One limitation of this project is that during the prospective followup we were unable to reach 14% of patients discharged from the ED. 4. Conclusions Our data suggest that implementation of standardized ultrasonography reporting template for the evaluation of pediatric appendicitis significantly decreases the number of indeterminate results. We recommend that this type of change in reporting be implemented as part of a multidisciplinary quality improvement initiative that also relies upon a comprehensive diagnostic algorithm to account for the initial learning curve necessary to increase experience. References [1] Witt WP, Weiss AJ, Elixhauser A. Overview of hospital stays for children in the United States, 2012. https://www.hcup-us.ahrq.gov/reports/statbriefs/sb187Hospital-Stays-Children-2012.jsp; 2014, Accessed date: 7 December 2016. [2] Doria AS, Moineddin R, Kellenberger CJ, et al. US or CT for diagnosis of appendicitis in children and adults? A meta-analysis. Radiology 2006;241(1):83–94. [3] Sivit CJ. Imaging the child with right lower quadrant pain and suspected appendicitis: current concepts. Pediatr Radiol 2004;34(6):447–53. [4] Pearce MS, Salotti JA, Little MP, et al. Radiation exposure from CT scans in childhood and subsequent risk of leukaemia and brain tumours: a retrospective cohort study. Lancet 2012;380(9840):499–505. [5] Brenner D, Elliston C, Hall E, et al. Estimated risks of radiation-induced fatal cancer from pediatric CT. AJR Am J Roentgenol 2001;176(2):289–96. [6] Brenner DJ, Hall EJ. Computed tomography—an increasing source of radiation exposure. N Engl J Med 2007;357(22):2277–84. [7] http://www.choosingwisely.org/clinician-lists/american-college-radiology-ct-toevaluate-appendicitis-in-children/, Accessed date: 12 June 2017. [8] Smith MP, Katz DS, Lalani T, et al. ACR Appropriateness Criteria(R) right lower quadrant pain—suspected appendicitis. Ultrasound Q 2015;31(2):85–91. [9] Yu YR, Shah SR. Can the diagnosis of appendicitis be made without a computed tomography scan? Adv Surg 2017;51(1):11–28. [10] Fallon SC, Orth RC, Guillerman RP, et al. Development and validation of an ultrasound scoring system for children with suspected acute appendicitis. Pediatr Radiol 2015;45(13):1945–52. [11] Ramachandran P, Sivit CJ, Newman KD, et al. Ultrasonography as an adjunct in the diagnosis of acute appendicitis: a 4-year experience. J Pediatr Surg 1996;31(1): 164–7 [discussion 7-9]. [12] Otero HJ, Crowder L. Imaging utilization for the diagnosis of appendicitis in standalone children's hospitals in the United States: trends and costs. J Am Coll Radiol 2017;14(5):603–8. [13] Cohen B, Bowling J, Midulla P, et al. The non-diagnostic ultrasound in appendicitis: is a non-visualized appendix the same as a negative study? J Pediatr Surg 2015;50(6): 923–7. [14] Alter SM, Walsh B, Lenehan PJ, et al. Ultrasound for diagnosis of appendicitis in a community hospital emergency department has a high rate of nondiagnostic studies. J Emerg Med 2017;52(6):833–8. [15] Ross MJ, Liu H, Netherton SJ, et al. Outcomes of children with suspected appendicitis and incompletely visualized appendix on ultrasound. Acad Emerg Med 2014;21(5): 538–42. [16] Schuh S, Man C, Cheng A, et al. Predictors of non-diagnostic ultrasound scanning in children with suspected appendicitis. J Pediatr 2011;158(1):112–8. [17] Kotagal M, Richards MK, Chapman T, et al. Improving ultrasound quality to reduce computed tomography use in pediatric appendicitis: the Safe and Sound campaign. Am J Surg 2015;209(5):896–900 [discussion]. [18] Bachur RG, Hennelly K, Callahan MJ, et al. Advanced radiologic imaging for pediatric appendicitis, 2005-2009: trends and outcomes. J Pediatr 2012;160(6):1034–8. [19] Ramarajan N, Krishnamoorthi R, Gharahbaghian L, et al. Clinical correlation needed: what do emergency physicians do after an equivocal ultrasound for pediatric acute appendicitis? J Clin Ultrasound 2014;42(7):385–94. [20] Shah SR, Sinclair KA, Theut SB, et al. Computed tomography utilization for the diagnosis of acute appendicitis in children decreases with a diagnostic algorithm. Ann Surg 2016;264(3):474–81. [21] Sivit CJ. Controversies in emergency radiology: acute appendicitis in children—the case for CT. Emerg Radiol 2004;10(5):238–40. [22] Fahimi J, Herring A, Harries A, et al. Computed tomography use among children presenting to emergency departments with abdominal pain. Pediatrics 2012;130(5):e1069-5.
Contents lists available at ScienceDirect
Journal of Pediatric Surgery journal homepage: www.elsevier.com/locate/jpedsurg
Standardized reporting of appendicitis-related findings improves reliability of ultrasound in diagnosing appendicitis in children☆ Richard Sola Jr. a, Stephanie B. Theut b, Kelly A. Sinclair c, Doug C. Rivard b, Kathy M. Johnson a, Huirong Zhu d, Shawn D. St. Peter a, Sohail R. Shah d,⁎ a
Department of Surgery, The Center for Prospective Clinical Trials, The Children's Mercy Hospital, Kansas City, MO, USA Department of Radiology, The Children's Mercy Hospital and Clinics, Kansas City, MO, USA Department of Emergency Medicine, The Children's Mercy Hospital and Clinics, Kansas City, MO, USA d Division of Pediatric Surgery, Texas Children's Hospital/Baylor College of Medicine, Houston, TX, USA b c
a r t i c l e
i n f o
Article history: Received 22 January 2018 Accepted 1 February 2018 Key words: Appendicitis Children Ultrasound Pediatric surgery
a b s t r a c t Purpose: Our objective was to increase ultrasound reliability for diagnosing appendicitis in an academic children's hospital emergency department (ED) through a multidisciplinary quality improvement initiative. Methods: A retrospective review of ultrasound use in patients diagnosed with appendicitis in our ED from 1/1/ 2011 to 6/30/2014 established a baseline cohort. From 8/1/2014 to 7/31/2015 a diagnostic algorithm that prioritized ultrasound over CT was used in our ED, and a standardized template was implemented for the reporting of appendicitis-related ultrasound findings by our radiologists. Results: Of 627 patients diagnosed with appendicitis in the ED during the retrospective review, 46.1% (n = 289) had an ultrasound. After implementation of the diagnostic algorithm and standardized ultrasound report, 88.4% (n = 236) of 267 patients diagnosed with appendicitis had an ultrasound (p b 0.01). The frequency of indeterminate results decreased from 44.3% to 13.1%, and positive results increased from 46.4% to 66.1% in patients with appendicitis (p b 0.01). The sensitivity of ultrasound (indeterminate counted as negative) increased from 50.6% to 69.2% (p b 0.01). Conclusions: Ultrasound reliability for the diagnosis of appendicitis in children can be improved through standardized results reporting. However, these changes should be made as part of a multidisciplinary quality improvement initiative to account for the initial learning curve necessary to increase experience. Level of Evidence: Level II, Study of Diagnostic Test. © 2018 Elsevier Inc. All rights reserved.
Acute appendicitis (AA) is the most common surgical emergency in children, accounting for approximately 72,000 hospitalizations annually in the United States [1]. However, the diagnosis of appendicitis remains challenging and often relies upon radiographic adjuncts to confirm diagnosis. Computed tomography (CT) scans has been shown to have a high sensitivity and specificity for the diagnosis of appendicitis and this has resulted in their increased use across the country [2,3]. However, concerns about radiation exposure from CT resulting in an increased risk of malignancy in children have resulted in greater efforts to utilize ultrasonography (US) [4–6]. Current recommendations by the American College of Radiology state that US should be the initial imaging modality for the evaluation of suspected appendicitis in children [7,8]. The limitation of US is its dependence on individual operator experience, and this has resulted ☆ All authors have no conflicts of interest to disclose regarding this study. ⁎ Corresponding author at: Texas Children's Hospital / Baylor College of Medicine, 6701 Fannin Suite 1210, Houston, TX 77030. Tel.: +1 832 822 3135; fax: +1 832 825 3141. E-mail address: [email protected] (S.R. Shah). https://doi.org/10.1016/j.jpedsurg.2018.02.028 0022-3468/© 2018 Elsevier Inc. All rights reserved.
in wide variation in its diagnostic accuracy. Reported US sensitivities range from 44% to 94% and specificities range from 47% to 95% [2,3,9–11]. At some centers up to 74% of USs for acute appendicitis are nondiagnostic, which limit the utilization of US and ultimately lead to continued overutilization of CT for the diagnosis of appendicitis [12–19]. The objective of this study was to increase US reliability for diagnosing acute appendicitis in an academic children's hospital emergency department (ED) as part of a multidisciplinary quality improvement initiative. 1. Methods 1.1. Study design In February 2014, as part of a quality improvement (QI) initiative, we established a multidisciplinary team including physician representatives from pediatric surgery, emergency medicine, and radiology at Children's Mercy Hospital (CMH) in Kansas City, Missouri. The project was approved as a QI project intended to improve practice within
R. Sola Jr. et al. / Journal of Pediatric Surgery 53 (2018) 984–987
CMH and was deemed nonhuman subjects research by the CMH Office of Research Integrity. As part of the QI initiative the team created a diagnostic algorithm to formalize the evaluation of patients with suspected appendicitis that would allow for earlier surgical consultation and prioritize US imaging over CT scans. The results of our algorithm implementation have been previously published [20]. In addition to the diagnostic algorithm, the team created a standard template for the reporting of appendicitis-related ultrasound findings by our radiologists. 1.1.1. Retrospective review and data collection Our team initially set out to establish a baseline understanding of the current state of the diagnostic workup for appendicitis in our ED. This was conducted as a retrospective review of all patients diagnosed with appendicitis in the ED from January 1, 2011 to June 30, 2014. Inclusion criteria were all patients seen at CMH with acute appendicitis as identified by ICD-9 codes (540, 540.0, 540.1, 540.9, or 541) or CPT codes (44,950, 44,960, or 44,970). Exclusion criteria were any patient not seen in the CMH ED during the same visit or with outside imaging
985
already diagnostic of acute appendicitis. Data collected included age, gender, body mass index (BMI), information related to diagnostic imaging if performed, treatment of appendicitis, intraoperative findings, and pathology. US reports were also reviewed and categorized as: negative, positive, or indeterminate. 1.1.2. Creation of a standardized ultrasonography reporting template In addition to creation of the new diagnostic algorithm (Fig. 1), our team created a standardized template for the reporting of US findings when evaluating patients with suspected appendicitis. The diagnostic algorithm prioritized the use of US over CT, and included pathways where the ED and/or pediatric surgical providers could request a US for further workup. The standardized US report consisted of four categories: Category 1—negative for appendicitis, Category 2—appendix not visualized without any secondary signs of appendicitis, Category 3—appendix not visualized with secondary signs of appendicitis, Category 4— findings consistent with appendicitis. With this standardized US report, treating providers were instructed that Categories 1 & 2 should be
Fig. 1. Diagnostic algorithm utilized for suspected appendicitis. Alt indicates alternative.
986
R. Sola Jr. et al. / Journal of Pediatric Surgery 53 (2018) 984–987
interpreted as negative for appendicitis, while Categories 3 & 4 should be interpreted as positive for appendicitis. Before implementation of the standardized US report, it was presented at departmental/divisional meetings in pediatric surgery, emergency medicine, and radiology for consensus approval. 1.1.3. Prospective evaluation and data collection After approval, the diagnostic algorithm and standardized US report were implemented on August 1, 2014 and prospectively followed until July 31, 2015. During the 1-year prospective evaluation period, data were collected. The inclusion criteria for prospective follow-up were any patient evaluated in the CMH ED with a suspicion of appendicitis. Exclusion criteria included any patient that had outside imaging already diagnostic of appendicitis. Prospectively followed patients were captured by several methods: (i) The pediatric appendicitis score (PAS), which was utilized as part of the diagnostic algorithm for the initial evaluation, was built into the electronic medical record (EMR). Once the emergency medicine provider selected the individual components of the PAS, a cumulative score was calculated and stored in the patient's record. A daily and weekly report of PAS entries into the EMR was autoforwarded to a surgical research nurse (KMJ) and the principal investigator (PI) (SRS). (ii) A weekly report of all patients discharged home from the ED with a diagnosis of “abdominal pain” in the EMR was autoforwarded to the PI. (iii) A weekly report of all patients that underwent a US or CT scan for “appendicitis,” “appy,” “right lower quadrant,” or “RLQ” in the ED was also autoforwarded to the PI. (iv) Finally, a weekly report of all patients that had an appendectomy (identified by CPT codes 44,950, 44,960, and 44,970) at CMH was autoforwarded to the PI. For every patient on these reports, the EMR was reviewed. If the subject met inclusion criteria, then subject data were entered by the research nurse into a secured electronic spreadsheet for analysis. Each subject's medical record was independently reviewed by the PI for validation of data entry. Data collected during prospective follow-up included age, gender, BMI, information related to diagnostic imaging used, disposition from ED, treatment of appendicitis, intraoperative findings, pathology, and adherence to the standardized US report. In addition, any patient included that was discharged from the ED or admitted to the hospital with a diagnosis other than appendicitis received a follow-up telephone call by ED personnel or our research nurse between 24 h and 1 week from their ED visit to confirm resolution of symptoms. During the 1-year prospective evaluation period, we continued with ongoing education and feedback for emergency medicine, radiology, and pediatric surgery providers to increase awareness and adherence to the standardized US report. This was conducted as reminders at departmental/divisional meetings and as individualized feedback to providers that showed variance from utilization of the standard US report. 1.2. Statistics Descriptive statistics including counts and percentages were analyzed. Pearson's chi-square and Fisher's exact tests were used for categorical variables, and the frequencies were reported as a percentage of the group of origin. Statistical significance was set at p b 0.05, and all reported p values are two-tailed. 2. Results 2.1. Comparison of ultrasonography results before and after implementation of standardized ultrasonography reporting There were 627 patients diagnosed with appendicitis in the ED during the retrospective review period (January 1, 2011 to June 30, 2014), and 289 (46.1%) of them had a US as part of their workup. There were 840 patients seen in the ED for suspected appendicitis during the prospective evaluation period after implementation of the
diagnostic algorithm and standardized US reporting (August 1, 2014 to July 31, 2015). Of these prospectively followed patients, there were 267 (31.7%) diagnosed with appendicitis and 236 (88.4%) of them had a US as part of their workup. The utilization of US was significantly higher in patients diagnosed with appendicitis after implementation of the diagnostic algorithm and standardized US reporting (88.4% vs. 46.1%, p b 0.01). The preimplementation appendicitis cohort that had a US (n = 289) and the postimplementation appendicitis cohort that had a US (n = 236) had very similar demographics. There were no statistically significant differences in age, gender distribution, or BMI between the two cohorts (Table 1). However, the frequency of indeterminate US reports decreased significantly after implementation of standardized US reporting for appendicitis. After implementation, indeterminate US results decreased from 44.3% to 13.1% (p b 0.01). Additionally, the frequency of US reports read as positive increased from 46.4% to 66.1% (p b 0.01), and US reports read as negative increased from 9.3% to 20.8% (p b 0.01) (Table 2). These changes were realized after implementation of standardized US reporting without adversely affecting clinical outcomes. The negative appendectomy rate, based on pathology evaluation, was 9.0% in the preimplementation cohort compared to 9.3% in the postimplementation cohort (p = 1.0) (Table 2). 2.2. Effects of standardized ultrasonography reporting on all patients with suspected appendicitis After implementation of the diagnostic algorithm and standardized US reporting for appendicitis 766 (91.2%) of all 840 patients with suspected appendicitis in the ED were evaluated with US. The 74 patients that were not evaluated with a US were discharged based on history and physical alone, taken to the operating room without imaging, or underwent evaluation with CT without prior US. Of these 766 patients, the US was reported positive for appendicitis in 177 (23%), negative for appendicitis in 511 (67%), and indeterminate in 78 (10%). This resulted in a sensitivity of 69.2% and specificity of 94.0% for US in patients with suspected appendicitis after implementation of the standardized reporting (indeterminate results were counted as negative). This was a significant increase in sensitivity when compared to the US results in the retrospectively reviewed appendicitis cohort (69.2% vs. 50.6%, p b 0.01). The missed appendectomy rate was 0.8% after implementation. The overall compliance with utilization of standardized US reporting by radiology was 85%. Additionally, the telephone follow-up rate for patients discharged from the ED was 86% during the prospective evaluation period. 3. Discussion The reliance on CT scans for the diagnosis of appendicitis in children has significantly increased over the years [20–22]. Current recommendations state that US should be the initial imaging modality for the evaluation of appendicitis in children [8]. At CMH, we were concerned about our higher utilization of CT and lower reliance upon US for the diagnosis of appendicitis as compared to our peers. In an effort to decrease CT utilization and increase US utilization through a QI initiative, we established a multidisciplinary team to evaluate and address this concern. The retrospective review data confirmed our suspicions and revealed a CT utilization rate of 75.4% and only 24.4% utilization of US prior to CT for the diagnosis of appendicitis in our ED. Through our multidisciplinary team, we then created a diagnostic algorithm that used PAS for risk stratification of patients with suspected appendicitis, focused on early surgical consultation, and prioritized US utilization before CT for diagnostic imaging [20]. However, one additional concern was our prior experience with utilizing US for the evaluation of appendicitis and the large number of US reports with an indeterminate result. In an effort to address this concern, we created a standardized US reporting template and implemented it at the same time as the newly created diagnostic algorithm. We also established a detailed workflow that allowed us to
R. Sola Jr. et al. / Journal of Pediatric Surgery 53 (2018) 984–987 Table 1 Demographics of patients with appendicitis before and after implementation of standardized ultrasonography reporting.
Age (years) Gender (% female) BMI
Prior to Diagnostic Algorithm & Standardized US Report (N = 289)
After Diagnostic Algorithm & Standardized US Report (N = 236)
p value
10.4 45.2% 20.5
10.2 41.1% 19.9
NS NS NS
identify patients evaluated for suspected appendicitis in the ED and prospectively follow them. In addition, we formalized a plan for telephone follow-up for all patients that were not diagnosed with appendicitis during their ED visit so we could accurately capture any patient with missed appendicitis. The simultaneous implementation of a complete diagnostic algorithm and prospective evaluation with telephone follow-up was an important factor that allowed providers to feel comfortable during the evaluation period to comply with the standardized US reporting and interpretation. In addition, it allowed us the ability to track the missed appendicitis rate and accurately calculate the specificity of the standardized US reporting, which differentiates this project from most others. After aggressive education efforts within pediatric surgery, radiology, and emergency medicine, the standardized US reporting was implemented. During the 1-year prospective evaluation period, we continued with ongoing education and feedback to our radiologists to increase adherence to the standardized US report. This was conducted as reminders at departmental/divisional meetings and as individualized feedback to providers that showed variance from utilization of the standard US report. These ongoing efforts resulted in an increase in compliance from 76% in the first quarter to 86%–88% during the last three quarters of our evaluation period. Our comparative analysis of patients diagnosed with appendicitis in the ED before and after the implementation of the standardized US reporting demonstrated impressive results. The frequency of indeterminate US result reporting was cut by two-thirds (44.3% to 13.1%). In addition, we significantly increased the sensitivity of US for children with suspected appendicitis at our institution from 50.6% to 69.2%. The specificity of US with the standardized reporting was 94.0%. This was all accomplished without any significant change in the negative appendectomy rate. During the prospective follow-up of all patients with suspected appendicitis in the ED, we were able to reach 86% of patients that were not diagnosed with appendicitis for telephone followup. The missed appendectomy rate was 0.8%. While the overall sensitivity of US in our experience is still low at 69.2% compared to the literature, the diagnostic algorithm as a whole had a sensitivity of 98.6% and a specificity of 94.4% emphasizing the importance of the role of an algorithm as a whole compared to its individual parts. Additionally, these findings demonstrate the importance of making significant practice changes such as this as part of a larger
Table 2 Comparison of patients with appendicitis before and after implementation of standardized ultrasonography reporting.
US Results Positive Negative Indeterminate Negative appendectomy rate
Prior to Diagnostic Algorithm & Standardized US Report (N = 289)
After Diagnostic Algorithm & Standardized US Report (N = 236)
134 (46.4%) 27 (9.3%) 128 (44.3%) 26 (9.0%)
156 (66.1%) 49 (20.8%) 31 (13.1%) 22 (9.3%)
p value
b0.01
NS
987
multidisciplinary effort to ensure maintaining high quality of care for patients. We expect as the experience with US increases at our hospital over time, the sensitivity and specificity of US will also increase. In an effort to maintain continuous quality improvement, there are continued educational and training efforts for our US technologists, who perform the studies with both resident and attending radiology coverage 24 h a day. These efforts should result in a lower incidence of missed appendicitis, a lower negative appendectomy rate, and an even greater shift from utilization of CT to US for the diagnosis of appendicitis at our hospital. One limitation of this project is that during the prospective followup we were unable to reach 14% of patients discharged from the ED. 4. Conclusions Our data suggest that implementation of standardized ultrasonography reporting template for the evaluation of pediatric appendicitis significantly decreases the number of indeterminate results. We recommend that this type of change in reporting be implemented as part of a multidisciplinary quality improvement initiative that also relies upon a comprehensive diagnostic algorithm to account for the initial learning curve necessary to increase experience. References [1] Witt WP, Weiss AJ, Elixhauser A. Overview of hospital stays for children in the United States, 2012. https://www.hcup-us.ahrq.gov/reports/statbriefs/sb187Hospital-Stays-Children-2012.jsp; 2014, Accessed date: 7 December 2016. [2] Doria AS, Moineddin R, Kellenberger CJ, et al. US or CT for diagnosis of appendicitis in children and adults? A meta-analysis. Radiology 2006;241(1):83–94. [3] Sivit CJ. Imaging the child with right lower quadrant pain and suspected appendicitis: current concepts. Pediatr Radiol 2004;34(6):447–53. [4] Pearce MS, Salotti JA, Little MP, et al. Radiation exposure from CT scans in childhood and subsequent risk of leukaemia and brain tumours: a retrospective cohort study. Lancet 2012;380(9840):499–505. [5] Brenner D, Elliston C, Hall E, et al. Estimated risks of radiation-induced fatal cancer from pediatric CT. AJR Am J Roentgenol 2001;176(2):289–96. [6] Brenner DJ, Hall EJ. Computed tomography—an increasing source of radiation exposure. N Engl J Med 2007;357(22):2277–84. [7] http://www.choosingwisely.org/clinician-lists/american-college-radiology-ct-toevaluate-appendicitis-in-children/, Accessed date: 12 June 2017. [8] Smith MP, Katz DS, Lalani T, et al. ACR Appropriateness Criteria(R) right lower quadrant pain—suspected appendicitis. Ultrasound Q 2015;31(2):85–91. [9] Yu YR, Shah SR. Can the diagnosis of appendicitis be made without a computed tomography scan? Adv Surg 2017;51(1):11–28. [10] Fallon SC, Orth RC, Guillerman RP, et al. Development and validation of an ultrasound scoring system for children with suspected acute appendicitis. Pediatr Radiol 2015;45(13):1945–52. [11] Ramachandran P, Sivit CJ, Newman KD, et al. Ultrasonography as an adjunct in the diagnosis of acute appendicitis: a 4-year experience. J Pediatr Surg 1996;31(1): 164–7 [discussion 7-9]. [12] Otero HJ, Crowder L. Imaging utilization for the diagnosis of appendicitis in standalone children's hospitals in the United States: trends and costs. J Am Coll Radiol 2017;14(5):603–8. [13] Cohen B, Bowling J, Midulla P, et al. The non-diagnostic ultrasound in appendicitis: is a non-visualized appendix the same as a negative study? J Pediatr Surg 2015;50(6): 923–7. [14] Alter SM, Walsh B, Lenehan PJ, et al. Ultrasound for diagnosis of appendicitis in a community hospital emergency department has a high rate of nondiagnostic studies. J Emerg Med 2017;52(6):833–8. [15] Ross MJ, Liu H, Netherton SJ, et al. Outcomes of children with suspected appendicitis and incompletely visualized appendix on ultrasound. Acad Emerg Med 2014;21(5): 538–42. [16] Schuh S, Man C, Cheng A, et al. Predictors of non-diagnostic ultrasound scanning in children with suspected appendicitis. J Pediatr 2011;158(1):112–8. [17] Kotagal M, Richards MK, Chapman T, et al. Improving ultrasound quality to reduce computed tomography use in pediatric appendicitis: the Safe and Sound campaign. Am J Surg 2015;209(5):896–900 [discussion]. [18] Bachur RG, Hennelly K, Callahan MJ, et al. Advanced radiologic imaging for pediatric appendicitis, 2005-2009: trends and outcomes. J Pediatr 2012;160(6):1034–8. [19] Ramarajan N, Krishnamoorthi R, Gharahbaghian L, et al. Clinical correlation needed: what do emergency physicians do after an equivocal ultrasound for pediatric acute appendicitis? J Clin Ultrasound 2014;42(7):385–94. [20] Shah SR, Sinclair KA, Theut SB, et al. Computed tomography utilization for the diagnosis of acute appendicitis in children decreases with a diagnostic algorithm. Ann Surg 2016;264(3):474–81. [21] Sivit CJ. Controversies in emergency radiology: acute appendicitis in children—the case for CT. Emerg Radiol 2004;10(5):238–40. [22] Fahimi J, Herring A, Harries A, et al. Computed tomography use among children presenting to emergency departments with abdominal pain. Pediatrics 2012;130(5):e1069-5.