- Email: [email protected]
Improving the diagnostic accuracy of appendicitis using a multidisciplinary pathway
YJPSU-59584; No of Pages 4 Journal of Pediatric Surgery xxx (xxxx) xxx
Contents lists available at ScienceDirect
Journal of Pediatric Surgery journal homepage: www.elsevier.com/locate/jpedsurg
Improving the diagnostic accuracy of appendicitis using a multidisciplinary pathway☆ Saroop Dhatt a, Vikram Sabhaney b, Heather Bray c, Erik D. Skarsgard d,⁎ a
University of British Columbia, Faculty of Medicine, British Columbia Children's Hospital, Vancouver, Canada Department of Pediatrics, University of British Columbia, Vancouver, Canada Department of Radiology, University of British Columbia, Vancouver, Canada d Department of Surgery, University of British Columbia, Vancouver, Canada b c
a r t i c l e
i n f o
Article history: Received 8 January 2020 Accepted 25 January 2020 Available online xxxx Key words: Appendicitis Pediatric Diagnostic pathway Alvarado score Ultrasound
a b s t r a c t Background/Purpose: Improvement opportunities exist in the accuracy and timeliness of the diagnosis of childhood appendicitis. The purpose of our study was to conduct a post-implementation audit of a diagnostic pathway for children with suspected appendicitis presenting to our pediatric emergency department. Methods: We adopted a diagnostic pathway that utilized a validated risk of appendicitis stratification tool (Alvarado Score) with protocolized use of abdominal ultrasound for moderate risk patients. We conducted a 10% convenience sample audit of pathway patients treated over the subsequent 18-month period. Outcome measures included false negative and positive rates, sensitivity, specificity, and overall pathway accuracy. Results: One hundred thirty-four pathway patients, of which 22 (16.4%) had appendicitis confirmed pathologically, were evaluated. The risk group distribution of patients was: low risk (29%), moderate risk (60%), and high risk (11%). The negative appendectomy rate was 4.4% (reduced from 14% pre-pathway), and the false negative (missed appendicitis) rate was 3.0%. No patients received CT scans. Pathway sensitivity was 81.8%% (95% CI 59.7% to 94.8%), specificity-92.9%% (95% CI 86.4%–96.9%), and overall accuracy-91.0% (95% CI 84.9%–95.3%). Conclusion: Implementation of a diagnostic pathway achieved a high level of accuracy and reduced our institutional negative appendectomy rate by 67%. The audit identified additional pathway improvement opportunities. Levels of Evidence: Level IV. © 2020 Elsevier Inc. All rights reserved.
Children with abdominal pain that may be caused by acute appendicitis represent a significant patient burden which present for Emergency Department (ED) diagnosis. Delay in the accurate diagnosis and timely treatment of childhood appendicitis contributes potentially avoidable morbidity and cost in both children's and non-children's hospital populations [1, 2]. Targeted improvement in the diagnostic accuracy and expediency as well as resource utilization associated with the processes of proving or excluding a diagnosis of appendicitis have the potential to improve clinical outcomes, cost-effectiveness, and patient experience. An essential driver for quality improvement (QI) activities is an awareness of outlier performance relative to peer organizations. Prior to undertaking this study, the negative appendectomy rate at our hospital was twice the national average for children's hospitals. We hypothesized that adoption of a diagnostic pathway that leverage standardized
☆ Declarations of Interest: None ⁎ Corresponding author at: British Columbia Children's Hospital, Department of Surgery, K0-110 ACB, 4480 Oak Street, Vancouver, British Columbia, Canada V6H 3V4. Tel.: + 1 604 875 3744; fax: + 1 604 875 2721. E-mail address: [email protected] (E.D. Skarsgard).
and timely assessment, selective use of medical imaging and optimized patient flow would enable more accurate diagnosis and treatment, improved outcomes and cost-effectiveness, and enhanced patient experience. 1. Methods Beginning in September 2016, key stakeholders from Pediatric Surgery (PS), Emergency Medicine (EM) and Diagnostic Radiology (DR) in our free-standing children's hospital, worked collaboratively to develop and implement a QI initiative that included a standardized diagnostic pathway for patients with suspected appendicitis (Fig. 1), with the goal of improving performance in key benchmarks, most notably the negative appendectomy rate. A requirement for institutional review board (IRB) approval was waived based on the QI nature of the initiative. The pathway utilized the Alvarado Score (AS), one of several validated pediatric appendicitis scoring tools which combines historical symptoms, clinical signs, and laboratory findings, to stratify patients into low, moderate, and high risk groups for any child who was considered to have possible appendicitis based on initial EM physician
https://doi.org/10.1016/j.jpedsurg.2020.01.040 0022-3468/© 2020 Elsevier Inc. All rights reserved.
Please cite this article as: S. Dhatt, V. Sabhaney, H. Bray, et al., Improving the diagnostic accuracy of appendicitis using a multidisciplinary pathway, Journal of Pediatric Surgery, https://doi.org/10.1016/j.jpedsurg.2020.01.040
2
S. Dhatt et al. / Journal of Pediatric Surgery xxx (xxxx) xxx
Fig. 1. Alvarado Score-based diagnostic pathway.
assessment [1–14]. The choice of the AS tool was made by the EM physicians, with cut-offs for the low- and high- risk categories based on a published optimization study [4]. The teams agreed to the following AS group cutoffs: i) 0–4 = low risk: patient discharged home with selective follow-up; ii) 5–8 = moderate risk: abdominal ultrasound (US) examination performed; and iii) 9–10 = high risk: direct referral to PS. To facilitate assessment of the moderate risk group, the Department of Radiology committed US availability between 0800 and 2300 h. US examinations were performed selectively in high risk patients if requested by PS. If the US could not be completed by 2300 h, patients were observed in the ED overnight or admitted by the PS service. A standard data collection form which included the AS and patient management flow chart on one side, and consensus criteria for a sonographic diagnosis of appendicitis (non-compressible appendix N6 mm in diameter, echogenic mesenteric fat, free fluid, fluid collection/abscess or inflammatory mass) and a final impression from US examination (negative, equivocal or positive for appendicitis; or an alternate diagnosis) on the other side was created for compliance documentation. The intention was that any patient identified as having possible appendicitis after EM physician assessment, would have the form attached to the chart, undergo scoring, and if US was required, the form would accompany the patient to US, to be filled in manually by the interpreting radiologist or resident/fellow. Completed forms were collected and archived. For cases where the data collection form was not fully completed at the time of patient assessment, the medical records were reviewed to obtain missing data. The pathway was implemented in late 2016. Following pathway implementation, we conducted a retrospective audit to assess both compliance with and performance of this QI initiative. The cohort consisted of an estimated 10% convenience sample of patients between the ages of 3 and 18 that presented to the ED for possible appendicitis assessment between March 2017 and August 2018. This random sample was obtained from the archived forms, and the
10% estimate was based on the known number of appendectomies performed during the study period (n = 216), which is 10 times the number of appendectomies reported in this study (n = 22). Patient and visit characteristics were extracted from the data collection form and the medical chart. Patient characteristics included age and sex, visit characteristics included the AS, whether an US was performed and its result, whether PS was consulted, and if the patient went on to have a laparoscopic appendectomy (LA). Return visits to the ED within 72 h of the index visit were also recorded. Among patients undergoing LA, the final diagnosis (simple or perforated appendicitis, or normal appendix) was based on the pathology report. “Test positive” patients were those with high risk AS (9–10), and those moderate risk patients with an US that was positive for appendicitis. “Test negative” patients were those who had a low risk AS (0–4), and those with moderate risk AS (5–8), in whom US was negative for suspected appendicitis, equivocal, suggested an alternative diagnosis, or was not performed. Pathway performance was assessed by the negative appendectomy rate, false positive rate, false negative rate, sensitivity, specificity, positive and negative predictive values and overall accuracy. 1.1. Statistical analysis Confidence intervals (95%) for each of the sample proportions were calculated using the Clopper-Pearson exact method for binomial proportions. 2. Results One hundred thirty-four patients, of which 69 (51.5%) were male, were included in the analysis. The mean age of patients was 10 ± 3.4 years. The distribution of patients by AS risk grouping was as follows: low risk – 39 (29.1%); moderate risk – 81 (60.4%); and high risk
Please cite this article as: S. Dhatt, V. Sabhaney, H. Bray, et al., Improving the diagnostic accuracy of appendicitis using a multidisciplinary pathway, Journal of Pediatric Surgery, https://doi.org/10.1016/j.jpedsurg.2020.01.040
S. Dhatt et al. / Journal of Pediatric Surgery xxx (xxxx) xxx Table 1 Patient (N = 134) demographics, risk groups and rates of confirmed appendicitis. Age (N, %) b5 5–9 N10 Gender (N, %) Male Female Alvarado Risk Group (N, %) Low Moderate High
4 (3.0) 60 (44.8) 70 (52.2) 69 (51.5) 65 (48.5) Diagnosed with appendicitis (N, %) 39 (29.1) 81 (60.4) 14 (19.5)
1 (2.6) 14 (17.3) 7 (50.0)
– 14 (10.4%). The overall rate of appendicitis in the diagnostic study cohort was 22/134 = 16.4% (Table 1). The diagnostic flow of pathway patients is summarized in Fig. 2. Ultrasound was performed in 89 patients (66.4%). Distribution of requests within each risk group were as follows: low risk – 1 patient (2.6%); moderate risk – 77 (95.1%); and high risk −11 (78.6%). Four moderate risk patients did not have US, and none had appendicitis. Six of 11 requests for US in high risk patients were from EM physicians, with the rest requested by PS. Twenty-three patients underwent LA, with 22 confirmed to have either simple (16) or perforated (6) appendicitis, for a perforation rate of 27.3%. The distribution of patients with proven appendicitis by AS risk group was as follows: low risk – 4.5%, moderate risk – 63.6%, and high risk – 31.8%. The negative appendectomy rate was 1/23 (4.3%). The patient with a normal appendix was high risk by AS, who was taken to surgery after an equivocal US and found to have a ruptured corpus luteal cyst. There were eight return visits to the emergency department within 72 h, of which one proved to have appendicitis. Seven of eight patients were from the low or moderate risk category, and none turned out to have appendicitis. The only case of a patient discharged from the ED who had appendicitis was a patient with an initial Alvarado score of 9 who was sent home by PS without imaging, only to return the next day and be correctly diagnosed with appendicitis. Using the definitions of “test positive” and “test negative” described in the methods, the overall sensitivity and specificity of the pathway was 81.8%% (95% CI 59.7%–94.8%) and 92.9% (95% CI 86.4%–96.9%), respectively. The positive and negative predictive values were 69.2%
3
(95% CI 52.9%–81.9%) and 96.3% (95% CI 91.5%–98.4%) respectively, while the overall accuracy (true positives + true negatives / all patients) of the pathway was 91.0% (95% CI 84.9%–95.3%). Of the low risk group, 38 patients were correctly identified as not having appendicitis, and there was one false negative. For moderate risk pathway patients, 11 of the 12 patients with positive US examinations had confirmed appendicitis. In this group, there were three false negatives. Of the 14 high-risk pathway patients, 3 had no imaging and were taken to surgery following PS consultation, and all 3 had appendicitis. Eleven high-risk patients had ultrasound exams. PS requested five of these, and four were positive for appendicitis, which was confirmed at surgery, while the fifth, with an equivocal US was found at surgery to have a normal appendix and ruptured corpus luteal cyst as noted above. Of six high-risk patients who had an US requested by the ED, five had incorrect Alvarado scoring, most commonly from a failure to include the white blood count (WBC) results (three patients); none of these patients had evidence of appendicitis by US. No patients in this study underwent diagnostic computed tomography (CT) scanning. 3. Discussion The incentive for this QI project was an institutional negative appendectomy rate of nearly 14% for the 3 years prior to the start of this project, which was well above the published national average of 6.8% for Canadian children's hospitals [5]. This provided an engagement platform for discussion between PS, EM and DR on how to formalize and implement a diagnostic pathway that would improve diagnostic accuracy, without negatively affecting the perforation rate, which averaged 37% in the 3 years prior to pathway implementation. Prior to this pathway, US utilization in the diagnosis of appendicitis was highly variable, and US reporting was similarly non-standardized. In our patient cohort, an AS cut-off of less than 5 safely excluded patients from requiring further investigations with a very low false negative rate (2.6%) within this group. The one patient with a low-risk AS who was found to have appendicitis received an US to rule out ovarian pathology. Within the moderate risk patient group (which represented 60% of all patients with suspected appendicitis), the combination of AS and US resulted in the correct diagnosis of acute appendicitis in 11 of 14 patients. The three patients with negative or equivocal US findings and who were subsequently correctly diagnosed with appendicitis were not discharged from the ED and received a PS consult during
Fig. 2. Diagnostic flow of pathway patients.
Please cite this article as: S. Dhatt, V. Sabhaney, H. Bray, et al., Improving the diagnostic accuracy of appendicitis using a multidisciplinary pathway, Journal of Pediatric Surgery, https://doi.org/10.1016/j.jpedsurg.2020.01.040
4
S. Dhatt et al. / Journal of Pediatric Surgery xxx (xxxx) xxx
their visit. We did note apparent pathway compliance violations among six of 14 high-risk patients who had US requested by the ED, without PS consultation. In three of these patients, US requests were made before the WBC results were back. Because the AS is heavily weighted by both the total WBC (2 points) and a “left shift” in the proportion of polymorphs and immature forms (1 point), a failure to include the WBC meant that these patients were inappropriately assigned to be moderate risk. In our post-audit discussions with ED physicians, this seemed to be out of concern that the window for an evening US would close if the request was held until the laboratory results were back. Our outcomes were comparable to existing published reports. Saucier et al. published an observational study using the Pediatric Appendicitis Score to stratify patients into low-, moderate- and high-risk groups, with US reserved for moderate risk patients [6]. The rate of appendicitis in this population was 33%, which was twice that of our population. The overall diagnostic accuracy of the pathway was 94%. The rates of appendicitis by low-, moderate- and high-risk category were 0%, 57% and 43% respectively, and the false negative rate was 4.4%. Surgeons requested computed tomography (CT) scans in 7% of patients. Others have sought to compare the predictive diagnostic accuracy of AS alone for low and high risk patients, with discretionary use of US for moderate risk patients and have shown lower rates of CT scan and reduced admission to ED observational units with this protocol [7]. Another study compared diagnostic combinations of AS, C-reactive protein (CRP) and US and found that the highest positive predictive value for the diagnosis of acute appendicitis was achieved when AS was combined with the consulting surgeons' discretionary use of US [8]. These protocols have also been shown to have applicability in non-pediatric hospital settings, with demonstration of preserved or reduced negative appendectomy rates coincident with institutional commitments to reduce CT scan utilization [9–12]. The low rate (16%) of appendicitis in our cohort relative to other reports is notable. One potential explanation is our health region's established referral practice for children's surgery: nearly 1/3 of appendectomies performed in our hospital are in children who present in transfer from another hospital with a confirmed or strongly suspected diagnosis of appendicitis, for whom re-triaging with our institutional pathway is either unnecessary or may lead to additional diagnostic delay. If we accept that patients presenting initially to our ED with possible appendicitis represent an inherently lower risk group, it may be appropriate to attempt to diminish the burden of US requests per positive study, by increasing the AS threshold for low risk to 5, and/or reducing the threshold for high risk to an AS of 8. Increasing the cut-off for low risk to an AS of 5 increases the likelihood of false negatives, thus such a change would need to be accompanied by active surveillance of low risk patients discharged with a higher AS. Another change under consideration would be the adoption of the Pediatric Appendicitis Risk Calculator (pARC) which, in direct comparison to the Pediatric Appendicitis Score was more discriminating in its assignment of patients to either a low- or high-risk group, and may allow us to utilize US more efficiently without affecting overall diagnostic accuracy [13]. Our QI audit has several limitations. Despite an intent to track all cases, we were unable to determine the total denominator of eligible patients over the study period. This appears to have been due to a failure to complete and/or appropriately archive the pathway sheets. We did not track the impact of the pathway on clinical outcomes, including surgical site infection (SSI) rates or length of hospital stay, nor did we sur-
vey patient experience. Although we did note a reduction in the rate of perforated appendicitis to 27% (from 37% pre-implementation), we did not collect comparative diagnostic cycle time data to suggest whether times to diagnosis or treatment where affected by the pathway. Finally, another significant limitation is a lack of awareness of the impact of the pathway on costs, particularly in DR. 4. Conclusions Despite these limitations, this diagnostic pathway for appendicitis performed well in our institution, and importantly, was associated with a reduction in the negative appendectomy rate from 14% to 4%, with a slight reduction in the rate of perforated appendicitis. It has nurtured a culture of collaboration and continuous improvement across our departments, and has supported practice change among our radiologists, with the adoption of standardized reporting criteria for the US diagnosis of appendicitis that deals specifically with the interpretive uncertainty associated with non-visualization of the appendix [14]. It has also set the stage for collaboration in the development of diagnostic pathways for other specific patient groups. Funding sources This research did not receive any financial support from funding agencies in the public, commercial, or not-for-profit sectors. References [1] Cameron DB, Graham DA, Milliren CE, et al. Quantifying the burden of interhospital cost variation in pediatric surgery: implications for the prioritization of comparative effectiveness research. JAMA Pediatr 2017 Feb 6;171(2):e163926. [2] Tom CM, Won RP, Lee AD, et al. Outcomes and costs of common surgical procedures at children's and nonchildren's hospitals. J Surg Res 2018 Dec;232:63–71. [3] Schneider C, Kharbanda A, Bachur R. Evaluating appendicitis scoring systems using a prospective pediatric cohort. Ann Emerg Med 2007;49(6):778–84 784.e1. [4] Ebell MH, Shinholser J. What are the most clinically useful cutoffs for the Alvarado and Pediatric Appendicitis Scores? A systematic review. Ann Emerg Med 2014;64 (4):365–372.e2. [5] Thompson GC, Schuh S, Gravel J, et al. Variation in the diagnosis and management of appendicitis at Canadian pediatric hospitals. Acad Emerg Med 2015 Jul;22(7): 811–22. [6] Saucier A, Huang EY, Emeremni CA, et al. Prospective evaluation of a clinical pathway for suspected appendicitis. Pediatrics 2014 Jan;133(1):e88–95. [7] Fleischman RJ, Devine MK, Yagapen MA, et al. Evaluation of a novel pediatric appendicitis pathway using high- and low-risk scoring systems. Pediatr Emerg Care 2013 Oct;29(10):1060–5. [8] Zouari M, Jallouli M, Louati H, et al. Predictive value of C-reactive protein, ultrasound and Alvarado score in acute appendicitis: a prospective pediatric cohort. Am J Emerg Med 2016 Feb;34(2):189–92. [9] Kobayashi E, Johnson B, Goetz K, et al. Does the implementation of a pediatric appendicitis pathway promoting ultrasound work outside of a children's hospital? Am J Surg 2018 May;215(5):917–20. [10] Santillanes G, Simms S, Gausche-Hill M, et al. Prospective evaluation of a clinical practice guideline for diagnosis of appendicitis in children. Acad Emerg Med 2012 Aug;19(8):886–93. [11] Russell WS, Schuh AM, Hill JG, et al. Clinical practice guidelines for pediatric appendicitis evaluation can decrease computed tomography utilization while maintaining diagnostic accuracy. Pediatr Emerg Care 2013 May;29(5):568–73. [12] Wagenaar AE, Tashiro J, Wang B, et al. Protocol for suspected pediatric appendicitis limits computed tomography utilization. J Surg Res 2015 Nov;199(1):153–8. [13] Kharbanda AB, Vazquez-Benitez G, Ballard DW, et al. Development and validation of a novel Pediatric Appendicitis Risk Calculator (pARC). Pediatrics 2018 Apr;141(4). https://doi.org/10.1542/peds.2017-2699 Epub 2018 Mar 13. [14] Larson DB, Trout AT, Fierke SR, et al. Improvement in diagnostic accuracy of ultrasound of the pediatric appendix through the use of equivocal interpretive categories. AJR Am J Roentgenol 2015 Apr;204(4):849–56.
Please cite this article as: S. Dhatt, V. Sabhaney, H. Bray, et al., Improving the diagnostic accuracy of appendicitis using a multidisciplinary pathway, Journal of Pediatric Surgery, https://doi.org/10.1016/j.jpedsurg.2020.01.040
Contents lists available at ScienceDirect
Journal of Pediatric Surgery journal homepage: www.elsevier.com/locate/jpedsurg
Improving the diagnostic accuracy of appendicitis using a multidisciplinary pathway☆ Saroop Dhatt a, Vikram Sabhaney b, Heather Bray c, Erik D. Skarsgard d,⁎ a
University of British Columbia, Faculty of Medicine, British Columbia Children's Hospital, Vancouver, Canada Department of Pediatrics, University of British Columbia, Vancouver, Canada Department of Radiology, University of British Columbia, Vancouver, Canada d Department of Surgery, University of British Columbia, Vancouver, Canada b c
a r t i c l e
i n f o
Article history: Received 8 January 2020 Accepted 25 January 2020 Available online xxxx Key words: Appendicitis Pediatric Diagnostic pathway Alvarado score Ultrasound
a b s t r a c t Background/Purpose: Improvement opportunities exist in the accuracy and timeliness of the diagnosis of childhood appendicitis. The purpose of our study was to conduct a post-implementation audit of a diagnostic pathway for children with suspected appendicitis presenting to our pediatric emergency department. Methods: We adopted a diagnostic pathway that utilized a validated risk of appendicitis stratification tool (Alvarado Score) with protocolized use of abdominal ultrasound for moderate risk patients. We conducted a 10% convenience sample audit of pathway patients treated over the subsequent 18-month period. Outcome measures included false negative and positive rates, sensitivity, specificity, and overall pathway accuracy. Results: One hundred thirty-four pathway patients, of which 22 (16.4%) had appendicitis confirmed pathologically, were evaluated. The risk group distribution of patients was: low risk (29%), moderate risk (60%), and high risk (11%). The negative appendectomy rate was 4.4% (reduced from 14% pre-pathway), and the false negative (missed appendicitis) rate was 3.0%. No patients received CT scans. Pathway sensitivity was 81.8%% (95% CI 59.7% to 94.8%), specificity-92.9%% (95% CI 86.4%–96.9%), and overall accuracy-91.0% (95% CI 84.9%–95.3%). Conclusion: Implementation of a diagnostic pathway achieved a high level of accuracy and reduced our institutional negative appendectomy rate by 67%. The audit identified additional pathway improvement opportunities. Levels of Evidence: Level IV. © 2020 Elsevier Inc. All rights reserved.
Children with abdominal pain that may be caused by acute appendicitis represent a significant patient burden which present for Emergency Department (ED) diagnosis. Delay in the accurate diagnosis and timely treatment of childhood appendicitis contributes potentially avoidable morbidity and cost in both children's and non-children's hospital populations [1, 2]. Targeted improvement in the diagnostic accuracy and expediency as well as resource utilization associated with the processes of proving or excluding a diagnosis of appendicitis have the potential to improve clinical outcomes, cost-effectiveness, and patient experience. An essential driver for quality improvement (QI) activities is an awareness of outlier performance relative to peer organizations. Prior to undertaking this study, the negative appendectomy rate at our hospital was twice the national average for children's hospitals. We hypothesized that adoption of a diagnostic pathway that leverage standardized
☆ Declarations of Interest: None ⁎ Corresponding author at: British Columbia Children's Hospital, Department of Surgery, K0-110 ACB, 4480 Oak Street, Vancouver, British Columbia, Canada V6H 3V4. Tel.: + 1 604 875 3744; fax: + 1 604 875 2721. E-mail address: [email protected] (E.D. Skarsgard).
and timely assessment, selective use of medical imaging and optimized patient flow would enable more accurate diagnosis and treatment, improved outcomes and cost-effectiveness, and enhanced patient experience. 1. Methods Beginning in September 2016, key stakeholders from Pediatric Surgery (PS), Emergency Medicine (EM) and Diagnostic Radiology (DR) in our free-standing children's hospital, worked collaboratively to develop and implement a QI initiative that included a standardized diagnostic pathway for patients with suspected appendicitis (Fig. 1), with the goal of improving performance in key benchmarks, most notably the negative appendectomy rate. A requirement for institutional review board (IRB) approval was waived based on the QI nature of the initiative. The pathway utilized the Alvarado Score (AS), one of several validated pediatric appendicitis scoring tools which combines historical symptoms, clinical signs, and laboratory findings, to stratify patients into low, moderate, and high risk groups for any child who was considered to have possible appendicitis based on initial EM physician
https://doi.org/10.1016/j.jpedsurg.2020.01.040 0022-3468/© 2020 Elsevier Inc. All rights reserved.
Please cite this article as: S. Dhatt, V. Sabhaney, H. Bray, et al., Improving the diagnostic accuracy of appendicitis using a multidisciplinary pathway, Journal of Pediatric Surgery, https://doi.org/10.1016/j.jpedsurg.2020.01.040
2
S. Dhatt et al. / Journal of Pediatric Surgery xxx (xxxx) xxx
Fig. 1. Alvarado Score-based diagnostic pathway.
assessment [1–14]. The choice of the AS tool was made by the EM physicians, with cut-offs for the low- and high- risk categories based on a published optimization study [4]. The teams agreed to the following AS group cutoffs: i) 0–4 = low risk: patient discharged home with selective follow-up; ii) 5–8 = moderate risk: abdominal ultrasound (US) examination performed; and iii) 9–10 = high risk: direct referral to PS. To facilitate assessment of the moderate risk group, the Department of Radiology committed US availability between 0800 and 2300 h. US examinations were performed selectively in high risk patients if requested by PS. If the US could not be completed by 2300 h, patients were observed in the ED overnight or admitted by the PS service. A standard data collection form which included the AS and patient management flow chart on one side, and consensus criteria for a sonographic diagnosis of appendicitis (non-compressible appendix N6 mm in diameter, echogenic mesenteric fat, free fluid, fluid collection/abscess or inflammatory mass) and a final impression from US examination (negative, equivocal or positive for appendicitis; or an alternate diagnosis) on the other side was created for compliance documentation. The intention was that any patient identified as having possible appendicitis after EM physician assessment, would have the form attached to the chart, undergo scoring, and if US was required, the form would accompany the patient to US, to be filled in manually by the interpreting radiologist or resident/fellow. Completed forms were collected and archived. For cases where the data collection form was not fully completed at the time of patient assessment, the medical records were reviewed to obtain missing data. The pathway was implemented in late 2016. Following pathway implementation, we conducted a retrospective audit to assess both compliance with and performance of this QI initiative. The cohort consisted of an estimated 10% convenience sample of patients between the ages of 3 and 18 that presented to the ED for possible appendicitis assessment between March 2017 and August 2018. This random sample was obtained from the archived forms, and the
10% estimate was based on the known number of appendectomies performed during the study period (n = 216), which is 10 times the number of appendectomies reported in this study (n = 22). Patient and visit characteristics were extracted from the data collection form and the medical chart. Patient characteristics included age and sex, visit characteristics included the AS, whether an US was performed and its result, whether PS was consulted, and if the patient went on to have a laparoscopic appendectomy (LA). Return visits to the ED within 72 h of the index visit were also recorded. Among patients undergoing LA, the final diagnosis (simple or perforated appendicitis, or normal appendix) was based on the pathology report. “Test positive” patients were those with high risk AS (9–10), and those moderate risk patients with an US that was positive for appendicitis. “Test negative” patients were those who had a low risk AS (0–4), and those with moderate risk AS (5–8), in whom US was negative for suspected appendicitis, equivocal, suggested an alternative diagnosis, or was not performed. Pathway performance was assessed by the negative appendectomy rate, false positive rate, false negative rate, sensitivity, specificity, positive and negative predictive values and overall accuracy. 1.1. Statistical analysis Confidence intervals (95%) for each of the sample proportions were calculated using the Clopper-Pearson exact method for binomial proportions. 2. Results One hundred thirty-four patients, of which 69 (51.5%) were male, were included in the analysis. The mean age of patients was 10 ± 3.4 years. The distribution of patients by AS risk grouping was as follows: low risk – 39 (29.1%); moderate risk – 81 (60.4%); and high risk
Please cite this article as: S. Dhatt, V. Sabhaney, H. Bray, et al., Improving the diagnostic accuracy of appendicitis using a multidisciplinary pathway, Journal of Pediatric Surgery, https://doi.org/10.1016/j.jpedsurg.2020.01.040
S. Dhatt et al. / Journal of Pediatric Surgery xxx (xxxx) xxx Table 1 Patient (N = 134) demographics, risk groups and rates of confirmed appendicitis. Age (N, %) b5 5–9 N10 Gender (N, %) Male Female Alvarado Risk Group (N, %) Low Moderate High
4 (3.0) 60 (44.8) 70 (52.2) 69 (51.5) 65 (48.5) Diagnosed with appendicitis (N, %) 39 (29.1) 81 (60.4) 14 (19.5)
1 (2.6) 14 (17.3) 7 (50.0)
– 14 (10.4%). The overall rate of appendicitis in the diagnostic study cohort was 22/134 = 16.4% (Table 1). The diagnostic flow of pathway patients is summarized in Fig. 2. Ultrasound was performed in 89 patients (66.4%). Distribution of requests within each risk group were as follows: low risk – 1 patient (2.6%); moderate risk – 77 (95.1%); and high risk −11 (78.6%). Four moderate risk patients did not have US, and none had appendicitis. Six of 11 requests for US in high risk patients were from EM physicians, with the rest requested by PS. Twenty-three patients underwent LA, with 22 confirmed to have either simple (16) or perforated (6) appendicitis, for a perforation rate of 27.3%. The distribution of patients with proven appendicitis by AS risk group was as follows: low risk – 4.5%, moderate risk – 63.6%, and high risk – 31.8%. The negative appendectomy rate was 1/23 (4.3%). The patient with a normal appendix was high risk by AS, who was taken to surgery after an equivocal US and found to have a ruptured corpus luteal cyst. There were eight return visits to the emergency department within 72 h, of which one proved to have appendicitis. Seven of eight patients were from the low or moderate risk category, and none turned out to have appendicitis. The only case of a patient discharged from the ED who had appendicitis was a patient with an initial Alvarado score of 9 who was sent home by PS without imaging, only to return the next day and be correctly diagnosed with appendicitis. Using the definitions of “test positive” and “test negative” described in the methods, the overall sensitivity and specificity of the pathway was 81.8%% (95% CI 59.7%–94.8%) and 92.9% (95% CI 86.4%–96.9%), respectively. The positive and negative predictive values were 69.2%
3
(95% CI 52.9%–81.9%) and 96.3% (95% CI 91.5%–98.4%) respectively, while the overall accuracy (true positives + true negatives / all patients) of the pathway was 91.0% (95% CI 84.9%–95.3%). Of the low risk group, 38 patients were correctly identified as not having appendicitis, and there was one false negative. For moderate risk pathway patients, 11 of the 12 patients with positive US examinations had confirmed appendicitis. In this group, there were three false negatives. Of the 14 high-risk pathway patients, 3 had no imaging and were taken to surgery following PS consultation, and all 3 had appendicitis. Eleven high-risk patients had ultrasound exams. PS requested five of these, and four were positive for appendicitis, which was confirmed at surgery, while the fifth, with an equivocal US was found at surgery to have a normal appendix and ruptured corpus luteal cyst as noted above. Of six high-risk patients who had an US requested by the ED, five had incorrect Alvarado scoring, most commonly from a failure to include the white blood count (WBC) results (three patients); none of these patients had evidence of appendicitis by US. No patients in this study underwent diagnostic computed tomography (CT) scanning. 3. Discussion The incentive for this QI project was an institutional negative appendectomy rate of nearly 14% for the 3 years prior to the start of this project, which was well above the published national average of 6.8% for Canadian children's hospitals [5]. This provided an engagement platform for discussion between PS, EM and DR on how to formalize and implement a diagnostic pathway that would improve diagnostic accuracy, without negatively affecting the perforation rate, which averaged 37% in the 3 years prior to pathway implementation. Prior to this pathway, US utilization in the diagnosis of appendicitis was highly variable, and US reporting was similarly non-standardized. In our patient cohort, an AS cut-off of less than 5 safely excluded patients from requiring further investigations with a very low false negative rate (2.6%) within this group. The one patient with a low-risk AS who was found to have appendicitis received an US to rule out ovarian pathology. Within the moderate risk patient group (which represented 60% of all patients with suspected appendicitis), the combination of AS and US resulted in the correct diagnosis of acute appendicitis in 11 of 14 patients. The three patients with negative or equivocal US findings and who were subsequently correctly diagnosed with appendicitis were not discharged from the ED and received a PS consult during
Fig. 2. Diagnostic flow of pathway patients.
Please cite this article as: S. Dhatt, V. Sabhaney, H. Bray, et al., Improving the diagnostic accuracy of appendicitis using a multidisciplinary pathway, Journal of Pediatric Surgery, https://doi.org/10.1016/j.jpedsurg.2020.01.040
4
S. Dhatt et al. / Journal of Pediatric Surgery xxx (xxxx) xxx
their visit. We did note apparent pathway compliance violations among six of 14 high-risk patients who had US requested by the ED, without PS consultation. In three of these patients, US requests were made before the WBC results were back. Because the AS is heavily weighted by both the total WBC (2 points) and a “left shift” in the proportion of polymorphs and immature forms (1 point), a failure to include the WBC meant that these patients were inappropriately assigned to be moderate risk. In our post-audit discussions with ED physicians, this seemed to be out of concern that the window for an evening US would close if the request was held until the laboratory results were back. Our outcomes were comparable to existing published reports. Saucier et al. published an observational study using the Pediatric Appendicitis Score to stratify patients into low-, moderate- and high-risk groups, with US reserved for moderate risk patients [6]. The rate of appendicitis in this population was 33%, which was twice that of our population. The overall diagnostic accuracy of the pathway was 94%. The rates of appendicitis by low-, moderate- and high-risk category were 0%, 57% and 43% respectively, and the false negative rate was 4.4%. Surgeons requested computed tomography (CT) scans in 7% of patients. Others have sought to compare the predictive diagnostic accuracy of AS alone for low and high risk patients, with discretionary use of US for moderate risk patients and have shown lower rates of CT scan and reduced admission to ED observational units with this protocol [7]. Another study compared diagnostic combinations of AS, C-reactive protein (CRP) and US and found that the highest positive predictive value for the diagnosis of acute appendicitis was achieved when AS was combined with the consulting surgeons' discretionary use of US [8]. These protocols have also been shown to have applicability in non-pediatric hospital settings, with demonstration of preserved or reduced negative appendectomy rates coincident with institutional commitments to reduce CT scan utilization [9–12]. The low rate (16%) of appendicitis in our cohort relative to other reports is notable. One potential explanation is our health region's established referral practice for children's surgery: nearly 1/3 of appendectomies performed in our hospital are in children who present in transfer from another hospital with a confirmed or strongly suspected diagnosis of appendicitis, for whom re-triaging with our institutional pathway is either unnecessary or may lead to additional diagnostic delay. If we accept that patients presenting initially to our ED with possible appendicitis represent an inherently lower risk group, it may be appropriate to attempt to diminish the burden of US requests per positive study, by increasing the AS threshold for low risk to 5, and/or reducing the threshold for high risk to an AS of 8. Increasing the cut-off for low risk to an AS of 5 increases the likelihood of false negatives, thus such a change would need to be accompanied by active surveillance of low risk patients discharged with a higher AS. Another change under consideration would be the adoption of the Pediatric Appendicitis Risk Calculator (pARC) which, in direct comparison to the Pediatric Appendicitis Score was more discriminating in its assignment of patients to either a low- or high-risk group, and may allow us to utilize US more efficiently without affecting overall diagnostic accuracy [13]. Our QI audit has several limitations. Despite an intent to track all cases, we were unable to determine the total denominator of eligible patients over the study period. This appears to have been due to a failure to complete and/or appropriately archive the pathway sheets. We did not track the impact of the pathway on clinical outcomes, including surgical site infection (SSI) rates or length of hospital stay, nor did we sur-
vey patient experience. Although we did note a reduction in the rate of perforated appendicitis to 27% (from 37% pre-implementation), we did not collect comparative diagnostic cycle time data to suggest whether times to diagnosis or treatment where affected by the pathway. Finally, another significant limitation is a lack of awareness of the impact of the pathway on costs, particularly in DR. 4. Conclusions Despite these limitations, this diagnostic pathway for appendicitis performed well in our institution, and importantly, was associated with a reduction in the negative appendectomy rate from 14% to 4%, with a slight reduction in the rate of perforated appendicitis. It has nurtured a culture of collaboration and continuous improvement across our departments, and has supported practice change among our radiologists, with the adoption of standardized reporting criteria for the US diagnosis of appendicitis that deals specifically with the interpretive uncertainty associated with non-visualization of the appendix [14]. It has also set the stage for collaboration in the development of diagnostic pathways for other specific patient groups. Funding sources This research did not receive any financial support from funding agencies in the public, commercial, or not-for-profit sectors. References [1] Cameron DB, Graham DA, Milliren CE, et al. Quantifying the burden of interhospital cost variation in pediatric surgery: implications for the prioritization of comparative effectiveness research. JAMA Pediatr 2017 Feb 6;171(2):e163926. [2] Tom CM, Won RP, Lee AD, et al. Outcomes and costs of common surgical procedures at children's and nonchildren's hospitals. J Surg Res 2018 Dec;232:63–71. [3] Schneider C, Kharbanda A, Bachur R. Evaluating appendicitis scoring systems using a prospective pediatric cohort. Ann Emerg Med 2007;49(6):778–84 784.e1. [4] Ebell MH, Shinholser J. What are the most clinically useful cutoffs for the Alvarado and Pediatric Appendicitis Scores? A systematic review. Ann Emerg Med 2014;64 (4):365–372.e2. [5] Thompson GC, Schuh S, Gravel J, et al. Variation in the diagnosis and management of appendicitis at Canadian pediatric hospitals. Acad Emerg Med 2015 Jul;22(7): 811–22. [6] Saucier A, Huang EY, Emeremni CA, et al. Prospective evaluation of a clinical pathway for suspected appendicitis. Pediatrics 2014 Jan;133(1):e88–95. [7] Fleischman RJ, Devine MK, Yagapen MA, et al. Evaluation of a novel pediatric appendicitis pathway using high- and low-risk scoring systems. Pediatr Emerg Care 2013 Oct;29(10):1060–5. [8] Zouari M, Jallouli M, Louati H, et al. Predictive value of C-reactive protein, ultrasound and Alvarado score in acute appendicitis: a prospective pediatric cohort. Am J Emerg Med 2016 Feb;34(2):189–92. [9] Kobayashi E, Johnson B, Goetz K, et al. Does the implementation of a pediatric appendicitis pathway promoting ultrasound work outside of a children's hospital? Am J Surg 2018 May;215(5):917–20. [10] Santillanes G, Simms S, Gausche-Hill M, et al. Prospective evaluation of a clinical practice guideline for diagnosis of appendicitis in children. Acad Emerg Med 2012 Aug;19(8):886–93. [11] Russell WS, Schuh AM, Hill JG, et al. Clinical practice guidelines for pediatric appendicitis evaluation can decrease computed tomography utilization while maintaining diagnostic accuracy. Pediatr Emerg Care 2013 May;29(5):568–73. [12] Wagenaar AE, Tashiro J, Wang B, et al. Protocol for suspected pediatric appendicitis limits computed tomography utilization. J Surg Res 2015 Nov;199(1):153–8. [13] Kharbanda AB, Vazquez-Benitez G, Ballard DW, et al. Development and validation of a novel Pediatric Appendicitis Risk Calculator (pARC). Pediatrics 2018 Apr;141(4). https://doi.org/10.1542/peds.2017-2699 Epub 2018 Mar 13. [14] Larson DB, Trout AT, Fierke SR, et al. Improvement in diagnostic accuracy of ultrasound of the pediatric appendix through the use of equivocal interpretive categories. AJR Am J Roentgenol 2015 Apr;204(4):849–56.
Please cite this article as: S. Dhatt, V. Sabhaney, H. Bray, et al., Improving the diagnostic accuracy of appendicitis using a multidisciplinary pathway, Journal of Pediatric Surgery, https://doi.org/10.1016/j.jpedsurg.2020.01.040