- Email: [email protected]
Biomarkers to estimate the probability of complicated appendicitis
Journal of Pediatric Surgery 53 (2018) 437–440
Contents lists available at ScienceDirect
Journal of Pediatric Surgery journal homepage: www.elsevier.com/locate/jpedsurg
Biomarkers to estimate the probability of complicated appendicitis☆ Meghan C. Daly a,b,⁎, Daniel von Allmen c, Hector R. Wong a,d a
Division of Critical Care Medicine, Cincinnati Children's Hospital Medical Center and Cincinnati Children's Research Foundation, Cincinnati, OH, USA Department of Surgery, University of Cincinnati College of Medicine, Cincinnati, OH, USA c Department of Surgery, Cincinnati Children's Hospital Medical Center, Cincinnati, OH, USA d Department of Pediatrics, University of Cincinnati College of Medicine, Cincinnati, OH, USA b
a r t i c l e
i n f o
Article history: Received 10 January 2017 Received in revised form 2 August 2017 Accepted 2 September 2017 Key words: Appendicitis Biomarker Matrix metalloproteinases Tissue inhibitors of metalloproteinase Conservative
a b s t r a c t Background: The conventional paradigm that all children with appendicitis require an appendectomy is being challenged by the idea that some patients may be successfully managed non-operatively. The study aimed to determine if matrix metalloproteinases (MMPs) and tissue inhibitors of metalloproteinase (TIMPs) are candidate biomarkers for estimating the probability of complicated appendicitis in pediatric patients. Methods: The study was a single-institution, prospective cohort study. MMP and TIMP serum protein concentrations were measured in patients with suspected appendicitis. Three hundred and thirty-one patients were enrolled with appendicitis. Classification and Regression Tree (CART) analysis was used to determine the combination of candidate biomarkers that best predicted complicated appendicitis. Results: The CART-generated decision tree for the derivation cohort included WBC count, MMP-8, MMP-9, MMP12, TIMP-2, and TIMP-4 and had the following test characteristics for estimating the probability of complicated appendicitis (95% CI): AUC 0.86 (0.81–0.90); sensitivity 91% (83–96); specificity 61% (53–68); positive predictive value 58% (50–66); negative predictive value 92% (84–96); positive likelihood ratio (LR) 2.3 (1.9–2.8); and negative LR 0.15 (0.08–0.3). Conclusions: MMPs and TIMPs have the potential to serve as biomarkers to estimate the probability of complicated appendicitis in pediatric patients. The multi-biomarker-based decision tree has test characteristics suggesting clinical utility for decision making. Level of Evidence: Level II: Study of Diagnostic Test. Published by Elsevier Inc.
Acute appendicitis is the most common condition requiring urgent abdominal surgery in children [1]. The longstanding rationale for appendectomy is the prevention of complications secondary to disease progression and perforation [2]. However, the traditional paradigm that all children with appendicitis require appendectomy is now being challenged by the concept that a subgroup of children with low risk of disease progression can be successfully managed non-operatively with antibiotics and supportive care alone [3–10]. A pilot study was performed to investigate the feasibility of this non-operative management approach [11]. A recent meta-analysis reported success rates of nonoperative management to be 91% at 30-day follow-up and 73% at oneyear follow-up [12]. Another prospective observational study using rigorous inclusion criteria demonstrated success rates of nonoperative management to be 87% at 18-month follow-up [13].
☆ Conflicts of Interest and Source of Funding: The authors have no competing interests to report. The study was supported by NIH T32GM008478, R01GM099773, and R01GM108025. ⁎ Corresponding author at: University of Cincinnati Medical Center, 231 Albert Sabin Way, ML 0558, Cincinnati, OH 45267. Tel.: +1 315 569 2351; fax: +1 513 558 3474. E-mail address: [email protected] (M.C. Daly). https://doi.org/10.1016/j.jpedsurg.2017.09.004 0022-3468/Published by Elsevier Inc.
Given the increasing interest in the non-operative management of appendicitis, it is crucial to reliably identify patients appropriate for non-operative management. A companion diagnostic test to reliably estimate which patients can be safely managed in this manner would be highly valuable. Our preclinical studies have identified matrix metalloproteinase-8 (MMP8) as an important mediator of intestinal injury in animal models [14,15]. We hypothesized that MMP-8, other matrix metalloproteinase family members, and endogenous tissue inhibitors of metalloproteinases (TIMPs) can serve as companion diagnostic biomarkers to estimate the probability of complicated appendicitis in pediatric patients.
1. Methods 1.1. Patient cohort and data collection This is a single institution, prospective cohort study. The study conformed to good clinical practice guidelines and followed the recommendations of the Declaration of Helsinki. The study protocol was approved in September 2014 by the Cincinnati Children's Hospital
438
M.C. Daly et al. / Journal of Pediatric Surgery 53 (2018) 437–440
Medical Center (CCHMC) Institutional Review Board (ID 2014–5392). The CCHMC IRB granted a waiver of informed consent. Pediatric patients presenting to the emergency department with suspected appendicitis, and a serum sample obtained at the time of presentation were eligible for study enrollment. The primary outcome variable was complicated appendicitis as defined by an abscessed, perforated, or gangrenous appendix at the time of appendectomy. Both the operative report and pathology report were reviewed to determine a classification of complicated appendicitis. Study subjects were followed to determine their clinical course after evaluation for appendicitis. 1.2. Candidate biomarkers MMP and TIMP serum protein concentrations were measured using a multiplex magnetic bead-based immunoassay designed by BIORAD Corporation (Hercules, CA, USA) and a Luminex 100/200 System (Luminex Corporation, Austin, TX, USA), according to the manufactures' specifications. The multiplex panel included measurements for MMP-1, -2, -3, -7, -8, -9, -10, -12, -13 and TIMP-1, -2, -3, -4. 1.3. Statistical analysis The primary outcome variable for all modeling procedures was the presence of complicated appendicitis. Descriptive statistics and comparisons were conducted using SigmaStat Software (Systat Software, Inc., San Jose, CA, USA). Receiver operating characteristic curves were constructed to determine the ability of each candidate biomarker to estimate the probability of complicated appendicitis. Classification and Regression Tree (CART) methodology (Salford Predictive Modeler v8.0, Salford Systems, San Diego, CA) was used to determine the combination of candidate biomarkers that best predicted complicated appendicitis [16]. All 13 candidate biomarkers, as well as white blood cell (WBC) count were considered as predictor variables in the CART analysis. Weighting of cases or introducing a cost for incorrect predictions was not used in the modeling procedures. The derivation cohort was randomly selected to include 80% of the study cohort; the test cohort consisted of the remaining 20%. Performance of the derivation and test cohort decision trees are reported using diagnostic test statistics with 95% confidence intervals. Statistical significance was defined as p b 0.05. 2. Results 2.1. Derivation of the decision tree Five hundred and eighty-seven subjects who presented to the emergency department with suspected appendicitis were enrolled in the study. Among these, 331 had surgically confirmed appendicitis, and 125 (38%) of these had complicated appendicitis. The primary analysis is based on these 331 subjects with appendicitis. We randomly selected 80% of these subjects (n = 265) for the derivation cohort. The demographic and clinical characteristics of the study subjects in the derivation cohort are depicted in Table 1. Among the individual candidate biomarkers, MMP-8 had the greatest area under the curve (AUC = 0.68; p b 0.001) for estimating the probability of complicated appendicitis. MMP-9, MMP-10, MMP-12, MMP-13, TIMP-2 and TIMP-4 also had AUCs approaching 0.60 and p values ≤ 0.05 (Table 2). WBC count alone had an AUC of 0.77, p b 0.001. Fig. 1 shows the derived decision tree. Maximum accuracy for estimating the risk of complicated appendicitis was attained with 5 of the 13 candidate biomarkers: MMP-8, −9, −12, TIMP2, and TIMP4. There were three low-risk terminal nodes (≤ 20.8% risk of complicated appendicitis; nodes 1, 3, and 7), two moderate risk terminal nodes (27.5–47.6% risk of complicated appendicitis; nodes 2 and 4), and three high-risk terminal nodes (59.1–90.0% risk of complicated appendicitis; nodes 5, 6, and 8). Table 3 shows the diagnostic test characteristics of the decision tree for the derivation cohort, wherein subjects in the
Table 1 Demographic and clinical characteristics of the derivation and test cohorts.
N (% total) # Males (%) # Females (%) Median age years (range) [IQR] # for race (%) Caucasian African American Other # with temp N38 °C (%) # with vomiting (%)
Derivation cohort (n = 265)
Test cohort (n = 66)
Uncomplicated
Complicated
Uncomplicated
Complicated
165 (62) 104 (63) 61 (37) 11
100 (38) 65 (65) 35 (35) 10
41 (62) 21 (51) 20 (49) 10
25 (38) 13 (52) 12 (48) 10
[9–14]
[7–12.5]
[8–13]
[8–12]
129 (78) 17 (10) 19 (12) 8 (5)
82 (82) 12 (12) 6 (6) 37 (37)
35 (85) 2 (5) 4 (10) 6 (15)
19 (76) 2 (8) 4 (16) 9 (36)
85 (52)
69 (69)
20 (49)
15 (60)
low-risk terminal nodes are classified as predicted to not have complicated appendicitis, and subjects in the moderate and high risk terminal nodes are classified as predicted to have complicated appendicitis.
2.2. Testing the decision tree The demographic and clinical characteristics of the study subjects in the test cohort (n = 66) are depicted in Table 1. The test cohort subjects were classified based on the derived decision tree, without any modifications. Table 4 shows the diagnostic test characteristics of the decision tree in the test cohort.
2.3. Secondary considerations Fig. 2 compares the receiver operating characteristic (ROC) curves for the decision tree and WBC count alone for all subjects in the derivation and test cohorts. The AUC of the decision tree (0.86; 95% CI: 0.82 to 0.90) was superior to that of WBC count alone (0.77; 95% CI: 0.72 to 0.82; p = 0.0003) for estimating the risk of complicated appendicitis. Ten of the initial 587subjects enrolled were taken to the operating room for an appendectomy but were found to have a normal appendix. All of these patients fell into low-risk terminal nodes (nodes 1, 3, or 7). The decision tree would have identified these subjects as having a low probability of complicated appendicitis and they might have avoided an operation under a non-operative management protocol.
Table 2 Areas under the receiver operating curves for individual biomarkers. Biomarker
AUC
95% CI
p value
Higher value associated with complicated appendicitis MMP1 0.51 0.50–0.58 MMP2 0.52 0.46–0.59 MMP3 0.56 0.50–0.62 MMP7 0.51 0.50–0.58 MMP8 0.68 0.63–0.74 MMP9 0.62 0.56–0.69 MMP10 0.59 0.53–0.65 MMP13 0.57 0.51–0.63 TIMP4 0.58 0.52–0.64 WBC 0.77 0.71–0.84
0.667 0.502 0.071 0.658 b0.001 b0.001 0.006 0.031 0.012 b0.001
Lower value associated with complicated appendicitis MMP12 0.57 0.51–0.64 TIMP1 0.54 0.48–0.60 TIMP2 0.61 0.55–0.67 TIMP3 0.52 0.45–0.58
0.025 0.223 b0.001 0.618
M.C. Daly et al. / Journal of Pediatric Surgery 53 (2018) 437–440
439
Fig. 1. The classification tree. The classification tree includes matrix metalloproteinase-8 (MMP-8), MMP-9, MMP-12, WBC count, tissue inhibitor of metalloproteinase 2 (TIMP2), and TIMP4. The MMP and TIMP data are shown as ng/ml. The root node provides the total number of samples, and the number of samples associated with no disease activity and disease activity, with the respective rates. Each daughter node provides the respective decision rule criterion and the number of samples associated with no disease activity and disease activity, with the respective rates. Terminal nodes (TN) TN1, TN3, and TN7 are low-risk terminal nodes (≤20.8% risk of complicated appendicitis). TN2 and TN4 are moderate risk terminal nodes (27.5 to 47.6% risk of complicated appendicitis) and TN5, TN6, and TN8 are high-risk terminal nodes (59.1 to 90.0% risk of complicated appendicitis). To calculate the diagnostic test characteristics, all low-risk samples are predicted to be associated with uncomplicated appendicitis, while all intermediate and high-risk samples are predicted to be associated with complicated appendicitis.
3. Discussion We have derived a biomarker-based risk stratification tool to reliably estimate the risk of complicated appendicitis in pediatric patients. Reliably estimating this risk is a key decision point for determining which patients are potentially eligible for non-operative management of acute appendicitis, and which patients warrant traditional appendectomy. The ideal characteristics of a companion test that informs clinical decision making regarding non-operative management of appendicitis include a high sensitivity, a high negative predictive value, and a low negative likelihood ratio. The decision tree demonstrated these test characteristics in both the derivation and test cohorts. Such test characteristics would allow for reliable inclusion of low-risk patients in a non-operative management protocol, with simultaneous exclusion of higher risk patients. The potential advantages of successful non-operative management of appendicitis, without the need for surgery, are self-evident. Peri-operative complications occur in approximately eight to 11% of patients undergoing appendectomy and recovery may require absence of up to one week of
school [17,18]. Several recent trials of non-operative management of appendicitis have reported no increased risk of developing perforated appendicitis [8,11,19,20]. Initial cost-effective analysis supports the use of non-operative management in uncomplicated appendicitis [7,21]. However, non-operative management has been associated with more subsequent emergency room visits and hospitalizations compared to those patients managed operatively [22]. The impact of medical interventions on patient-centered outcomes is becoming increasingly important in the current era of demand-side strategic purchasing. In our cohort, individual biomarkers demonstrated significantly lower AUCs than the derived decision tree, with MMP-8 having the greatest AUC of 0.68. WBC has been traditionally utilized as a supportive laboratory finding in the diagnosis of appendicitis [23]. Sensitivity and specificity of WBC count in this setting have been reported to range from 70% to 80% and 60% to 68%, respectively [24–27]. The AUC of the WBC count alone in our cohort was 0.77 for estimating the risk of complicated appendicitis. This was inferior to that of the decision tree, indicating that the decision tree provides predictive information beyond that of
Table 3 Diagnostic test characteristics of the classification tree for estimating the risk of complicated appendicitis in the derivation cohort.
Table 4 Diagnostic test characteristics of the classification tree for estimating the risk of complicated appendicitis in the test cohort.
Category # of true positives # of false positives # of true negatives # of false negatives Sensitivity Specificity Positive predictive value Negative predictive value Positive likelihood ratio Negative likelihood ratio Area under the curve
Category 91 65 100 9 91% (83–96) 61% (53–68) 58% (50–66) 92% (84–96) 2.3 (1.9–2.8) 0.15 (0.08–0.3) 0.86 (0.81–0.90)
# of true positives # of false positives # of true negatives # of false negatives Sensitivity Specificity Positive predictive value Negative predictive value Positive likelihood ratio Negative likelihood ratio Area under the curve
24 14 27 1 96% (78–100) 66% (49–79) 63% (46–78) 96% (80–100) 2.8 (1.8–4.3) 0.06 (0.01–0.4) 0.85 (0.76–0.94)
440
M.C. Daly et al. / Journal of Pediatric Surgery 53 (2018) 437–440
Fig. 2. Comparison of the receiver operating characteristics (ROC) curves for the classification tree and WBC count in all subjects. The ROC for the classification tree is shown as a black line, and the ROC for WBC count is shown as a gray line. The area under the curve for the classification tree (0.86, 95% CI: 0.82 to 0.90) was greater than that of the WBC count alone (0.77, 95% CI: 0.72 to 0.82, p = 0.0003).
the WBC count alone. Given the shortcomings of a single biomarker to accurately detect the presence of complicated appendicitis in pediatric patients, a combination of biomarkers may be more advantageous. To our knowledge, we are not aware of a validated stratification tool assessing the probability of complicated appendicitis that performs in an equivalent manner to that of our derived classification tree. We note the limitations of our study. CART analysis has the potential to over-fit a dataset. We attempted to address this issue by dividing our cohort into separate derivation and test cohorts. Nonetheless, our model requires further validation in a prospectively enrolled cohort. In addition, the panel of biomarkers used to estimate the risk of complicated appendicitis is currently available only as research-based assays and they require batching. In order to use these biomarkers for clinical decision making, a rapid assay would need to be developed. The technology to develop such a test currently exists. To conclude, these data suggest that MMPs and TIMPs have the potential to serve as biomarkers to estimate the probability of complicated appendicitis in pediatric patients. The combination of MMP-8, MMP-9, MMP-12, TIMP-2, and TIMP-4 produced the most reliable test characteristics. The multi-biomarker-based decision tree has test characteristics suggesting clinical utility for decision making and may serve to identify pediatric patients with appendicitis that can be safely managed non-operatively. Acknowledgments The authors would like to thank Patrick Lahni for the technical support. References [1] Sivit CJ, Siegel MJ, Applegate KE, et al. When appendicitis is suspected in children. Radiographics 2001;21:247–62. [2] Richardson WS. The evolution of early appendectomy as standard treatment from appendicitis: what we can learn from the past in adopting new medical therapies. Am Surg 2015;81:161–5. [3] Abes M, Petik B, Kazil S. Nonoperative treatment of acute appendicitis in children. J Pediatr Surg 2007;42:1439–42. [4] Eriksson S, Granstrom L. Randomized controlled trial of appendicectomy versus antibiotic therapy for acute appendicitis. Br J Surg 1995;82:166–9. [5] Hansson J, Korner U, Ludwigs K, et al. Antibiotics as first-line therapy for acute appendicitis: evidence for a change in clinical practice. World J Surg 2012;36:2028–36. [6] Styrud J, Eriksson S, Nilsson I, et al. Appendectomy versus antibiotic treatment in acute appendicitis. A prospective multicenter randomized controlled trial. World J Surg 2006;30:1033–7.
[7] Di Saverio S, Sibilio A, Giorgini E, et al. The NOTA study (non operative treatment for acute appendicitis): prospective study on the efficacy and safety of antibiotics (amoxicillin and clavulanic acid) for treating patients with right lower quadrant abdominal pain and long-term follow-up of conservatively treated suspected appendicitis. Ann Surg 2014;260:109–17. [8] Salminen P, Paajanen H, Rautio T, et al. Antibiotic therapy vs appendectomy for treatment of uncomplicated acute appendicitis: the APPAC randomized clinical trial. JAMA 2015;313:2340–8. [9] Simillis C, Symeonides P, Shorthouse AJ, et al. A meta-analysis comparing conservative treatment versus acute appendectomy for complicated appendicitis (abscess or phlegmon). Surgery 2010;147:818–29. [10] Malik AA, Bari SU. Conservative management of acute appendicitis. J Gastrointest Surg 2009;13:966–70. [11] Minneci PC, Sulkowski JP, Nacion KM, et al. Feasibility of a nonoperative management strategy for uncomplicated acute appendicitis in children. J Am Coll Surg 2014;219:272–9. [12] Huang L, Yin Y, Yang L, et al. Comparison of antibiotic therapy and appendectomy for acute uncomplicated appendicitis in children: a meta-analysis. JAMA Pediatr 2017;171:426–34. [13] Steiner Z, Buklan G, Stackievicz R, et al. Conservative treatment in uncomplicated acute appendicitis: reassessment of practice safety. Eur J Pediatr 2017;176:521–7. [14] Solan PD, Dunsmore KE, Denenberg AG, et al. A novel role for matrix metalloproteinase-8 in sepsis. Crit Care Med 2012;40:379–87. [15] Atkinson SJ, Nolan M, Klingbeil L, et al. Intestine-derived matrix metalloproteinase-8 is a critical mediator of polymicrobial peritonitis. Crit Care Med 2016;44:e200–6. [16] Muller R, Mockel M. Logistic regression and CART in the analysis of multimarker studies. Clin Chim Acta 2008;394:1–6. [17] Wilms IM, de Hoog DE, de Visser DC, et al. Appendectomy versus antibiotic treatment for acute appendicitis. Cochrane Database Syst Rev 2011:CD008359. [18] Vahdad MR, Troebs RB, Nissen M, et al. Laparoscopic appendectomy for perforated appendicitis in children has complication rates comparable with those of open appendectomy. J Pediatr Surg 2013;48:555–61. [19] Varadhan KK, Neal KR, Lobo DN. Safety and efficacy of antibiotics compared with appendicectomy for treatment of uncomplicated acute appendicitis: meta-analysis of randomised controlled trials. BMJ 2012;344:e2156. [20] Rollins KE, Varadhan KK, Neal KR, et al. Antibiotics versus appendicectomy for the treatment of uncomplicated acute appendicitis: an updated meta-analysis of randomised controlled trials. World J Surg 2016;40(10):2305–18. [21] Wu JX, Dawes AJ, Sacks GD, et al. Cost effectiveness of nonoperative management versus laparoscopic appendectomy for acute uncomplicated appendicitis. Surgery 2015;158:712–21. [22] Bachur RG, Lipsett SC, Monuteaux MC. Outcomes of nonoperative management of uncomplicated appendicitis. Pediatrics 2017;140(1) [pii: e20170048]. [23] Bates MF, Khander A, Steigman SA, et al. Use of white blood cell count and negative appendectomy rate. Pediatrics 2014;133:E39–44. [24] Mekhail P, Naguib N, Yanny F, et al. Appendicitis in paediatric age group: correlation between preoperative inflammatory markers and postoperative histological diagnosis. Br J Surg 2010;97:123. [25] Beltran MA, Almonacid J, Vicencio A, et al. Predictive value of white blood cell count and Creactive protein in children with appendicitis. J Pediatr Surg 2007;42:1208–14. [26] Wang LT, Prentiss KA, Simon JZ, et al. The use of white blood cell count and left shift in the diagnosis of appendicitis in children. Pediatr Emerg Care 2007;23:69–76. [27] Bundy DG, Byerley JS, Liles EA, et al. Does this child have appendicitis? JAMA 2007; 298:438–51.
Contents lists available at ScienceDirect
Journal of Pediatric Surgery journal homepage: www.elsevier.com/locate/jpedsurg
Biomarkers to estimate the probability of complicated appendicitis☆ Meghan C. Daly a,b,⁎, Daniel von Allmen c, Hector R. Wong a,d a
Division of Critical Care Medicine, Cincinnati Children's Hospital Medical Center and Cincinnati Children's Research Foundation, Cincinnati, OH, USA Department of Surgery, University of Cincinnati College of Medicine, Cincinnati, OH, USA c Department of Surgery, Cincinnati Children's Hospital Medical Center, Cincinnati, OH, USA d Department of Pediatrics, University of Cincinnati College of Medicine, Cincinnati, OH, USA b
a r t i c l e
i n f o
Article history: Received 10 January 2017 Received in revised form 2 August 2017 Accepted 2 September 2017 Key words: Appendicitis Biomarker Matrix metalloproteinases Tissue inhibitors of metalloproteinase Conservative
a b s t r a c t Background: The conventional paradigm that all children with appendicitis require an appendectomy is being challenged by the idea that some patients may be successfully managed non-operatively. The study aimed to determine if matrix metalloproteinases (MMPs) and tissue inhibitors of metalloproteinase (TIMPs) are candidate biomarkers for estimating the probability of complicated appendicitis in pediatric patients. Methods: The study was a single-institution, prospective cohort study. MMP and TIMP serum protein concentrations were measured in patients with suspected appendicitis. Three hundred and thirty-one patients were enrolled with appendicitis. Classification and Regression Tree (CART) analysis was used to determine the combination of candidate biomarkers that best predicted complicated appendicitis. Results: The CART-generated decision tree for the derivation cohort included WBC count, MMP-8, MMP-9, MMP12, TIMP-2, and TIMP-4 and had the following test characteristics for estimating the probability of complicated appendicitis (95% CI): AUC 0.86 (0.81–0.90); sensitivity 91% (83–96); specificity 61% (53–68); positive predictive value 58% (50–66); negative predictive value 92% (84–96); positive likelihood ratio (LR) 2.3 (1.9–2.8); and negative LR 0.15 (0.08–0.3). Conclusions: MMPs and TIMPs have the potential to serve as biomarkers to estimate the probability of complicated appendicitis in pediatric patients. The multi-biomarker-based decision tree has test characteristics suggesting clinical utility for decision making. Level of Evidence: Level II: Study of Diagnostic Test. Published by Elsevier Inc.
Acute appendicitis is the most common condition requiring urgent abdominal surgery in children [1]. The longstanding rationale for appendectomy is the prevention of complications secondary to disease progression and perforation [2]. However, the traditional paradigm that all children with appendicitis require appendectomy is now being challenged by the concept that a subgroup of children with low risk of disease progression can be successfully managed non-operatively with antibiotics and supportive care alone [3–10]. A pilot study was performed to investigate the feasibility of this non-operative management approach [11]. A recent meta-analysis reported success rates of nonoperative management to be 91% at 30-day follow-up and 73% at oneyear follow-up [12]. Another prospective observational study using rigorous inclusion criteria demonstrated success rates of nonoperative management to be 87% at 18-month follow-up [13].
☆ Conflicts of Interest and Source of Funding: The authors have no competing interests to report. The study was supported by NIH T32GM008478, R01GM099773, and R01GM108025. ⁎ Corresponding author at: University of Cincinnati Medical Center, 231 Albert Sabin Way, ML 0558, Cincinnati, OH 45267. Tel.: +1 315 569 2351; fax: +1 513 558 3474. E-mail address: [email protected] (M.C. Daly). https://doi.org/10.1016/j.jpedsurg.2017.09.004 0022-3468/Published by Elsevier Inc.
Given the increasing interest in the non-operative management of appendicitis, it is crucial to reliably identify patients appropriate for non-operative management. A companion diagnostic test to reliably estimate which patients can be safely managed in this manner would be highly valuable. Our preclinical studies have identified matrix metalloproteinase-8 (MMP8) as an important mediator of intestinal injury in animal models [14,15]. We hypothesized that MMP-8, other matrix metalloproteinase family members, and endogenous tissue inhibitors of metalloproteinases (TIMPs) can serve as companion diagnostic biomarkers to estimate the probability of complicated appendicitis in pediatric patients.
1. Methods 1.1. Patient cohort and data collection This is a single institution, prospective cohort study. The study conformed to good clinical practice guidelines and followed the recommendations of the Declaration of Helsinki. The study protocol was approved in September 2014 by the Cincinnati Children's Hospital
438
M.C. Daly et al. / Journal of Pediatric Surgery 53 (2018) 437–440
Medical Center (CCHMC) Institutional Review Board (ID 2014–5392). The CCHMC IRB granted a waiver of informed consent. Pediatric patients presenting to the emergency department with suspected appendicitis, and a serum sample obtained at the time of presentation were eligible for study enrollment. The primary outcome variable was complicated appendicitis as defined by an abscessed, perforated, or gangrenous appendix at the time of appendectomy. Both the operative report and pathology report were reviewed to determine a classification of complicated appendicitis. Study subjects were followed to determine their clinical course after evaluation for appendicitis. 1.2. Candidate biomarkers MMP and TIMP serum protein concentrations were measured using a multiplex magnetic bead-based immunoassay designed by BIORAD Corporation (Hercules, CA, USA) and a Luminex 100/200 System (Luminex Corporation, Austin, TX, USA), according to the manufactures' specifications. The multiplex panel included measurements for MMP-1, -2, -3, -7, -8, -9, -10, -12, -13 and TIMP-1, -2, -3, -4. 1.3. Statistical analysis The primary outcome variable for all modeling procedures was the presence of complicated appendicitis. Descriptive statistics and comparisons were conducted using SigmaStat Software (Systat Software, Inc., San Jose, CA, USA). Receiver operating characteristic curves were constructed to determine the ability of each candidate biomarker to estimate the probability of complicated appendicitis. Classification and Regression Tree (CART) methodology (Salford Predictive Modeler v8.0, Salford Systems, San Diego, CA) was used to determine the combination of candidate biomarkers that best predicted complicated appendicitis [16]. All 13 candidate biomarkers, as well as white blood cell (WBC) count were considered as predictor variables in the CART analysis. Weighting of cases or introducing a cost for incorrect predictions was not used in the modeling procedures. The derivation cohort was randomly selected to include 80% of the study cohort; the test cohort consisted of the remaining 20%. Performance of the derivation and test cohort decision trees are reported using diagnostic test statistics with 95% confidence intervals. Statistical significance was defined as p b 0.05. 2. Results 2.1. Derivation of the decision tree Five hundred and eighty-seven subjects who presented to the emergency department with suspected appendicitis were enrolled in the study. Among these, 331 had surgically confirmed appendicitis, and 125 (38%) of these had complicated appendicitis. The primary analysis is based on these 331 subjects with appendicitis. We randomly selected 80% of these subjects (n = 265) for the derivation cohort. The demographic and clinical characteristics of the study subjects in the derivation cohort are depicted in Table 1. Among the individual candidate biomarkers, MMP-8 had the greatest area under the curve (AUC = 0.68; p b 0.001) for estimating the probability of complicated appendicitis. MMP-9, MMP-10, MMP-12, MMP-13, TIMP-2 and TIMP-4 also had AUCs approaching 0.60 and p values ≤ 0.05 (Table 2). WBC count alone had an AUC of 0.77, p b 0.001. Fig. 1 shows the derived decision tree. Maximum accuracy for estimating the risk of complicated appendicitis was attained with 5 of the 13 candidate biomarkers: MMP-8, −9, −12, TIMP2, and TIMP4. There were three low-risk terminal nodes (≤ 20.8% risk of complicated appendicitis; nodes 1, 3, and 7), two moderate risk terminal nodes (27.5–47.6% risk of complicated appendicitis; nodes 2 and 4), and three high-risk terminal nodes (59.1–90.0% risk of complicated appendicitis; nodes 5, 6, and 8). Table 3 shows the diagnostic test characteristics of the decision tree for the derivation cohort, wherein subjects in the
Table 1 Demographic and clinical characteristics of the derivation and test cohorts.
N (% total) # Males (%) # Females (%) Median age years (range) [IQR] # for race (%) Caucasian African American Other # with temp N38 °C (%) # with vomiting (%)
Derivation cohort (n = 265)
Test cohort (n = 66)
Uncomplicated
Complicated
Uncomplicated
Complicated
165 (62) 104 (63) 61 (37) 11
100 (38) 65 (65) 35 (35) 10
41 (62) 21 (51) 20 (49) 10
25 (38) 13 (52) 12 (48) 10
[9–14]
[7–12.5]
[8–13]
[8–12]
129 (78) 17 (10) 19 (12) 8 (5)
82 (82) 12 (12) 6 (6) 37 (37)
35 (85) 2 (5) 4 (10) 6 (15)
19 (76) 2 (8) 4 (16) 9 (36)
85 (52)
69 (69)
20 (49)
15 (60)
low-risk terminal nodes are classified as predicted to not have complicated appendicitis, and subjects in the moderate and high risk terminal nodes are classified as predicted to have complicated appendicitis.
2.2. Testing the decision tree The demographic and clinical characteristics of the study subjects in the test cohort (n = 66) are depicted in Table 1. The test cohort subjects were classified based on the derived decision tree, without any modifications. Table 4 shows the diagnostic test characteristics of the decision tree in the test cohort.
2.3. Secondary considerations Fig. 2 compares the receiver operating characteristic (ROC) curves for the decision tree and WBC count alone for all subjects in the derivation and test cohorts. The AUC of the decision tree (0.86; 95% CI: 0.82 to 0.90) was superior to that of WBC count alone (0.77; 95% CI: 0.72 to 0.82; p = 0.0003) for estimating the risk of complicated appendicitis. Ten of the initial 587subjects enrolled were taken to the operating room for an appendectomy but were found to have a normal appendix. All of these patients fell into low-risk terminal nodes (nodes 1, 3, or 7). The decision tree would have identified these subjects as having a low probability of complicated appendicitis and they might have avoided an operation under a non-operative management protocol.
Table 2 Areas under the receiver operating curves for individual biomarkers. Biomarker
AUC
95% CI
p value
Higher value associated with complicated appendicitis MMP1 0.51 0.50–0.58 MMP2 0.52 0.46–0.59 MMP3 0.56 0.50–0.62 MMP7 0.51 0.50–0.58 MMP8 0.68 0.63–0.74 MMP9 0.62 0.56–0.69 MMP10 0.59 0.53–0.65 MMP13 0.57 0.51–0.63 TIMP4 0.58 0.52–0.64 WBC 0.77 0.71–0.84
0.667 0.502 0.071 0.658 b0.001 b0.001 0.006 0.031 0.012 b0.001
Lower value associated with complicated appendicitis MMP12 0.57 0.51–0.64 TIMP1 0.54 0.48–0.60 TIMP2 0.61 0.55–0.67 TIMP3 0.52 0.45–0.58
0.025 0.223 b0.001 0.618
M.C. Daly et al. / Journal of Pediatric Surgery 53 (2018) 437–440
439
Fig. 1. The classification tree. The classification tree includes matrix metalloproteinase-8 (MMP-8), MMP-9, MMP-12, WBC count, tissue inhibitor of metalloproteinase 2 (TIMP2), and TIMP4. The MMP and TIMP data are shown as ng/ml. The root node provides the total number of samples, and the number of samples associated with no disease activity and disease activity, with the respective rates. Each daughter node provides the respective decision rule criterion and the number of samples associated with no disease activity and disease activity, with the respective rates. Terminal nodes (TN) TN1, TN3, and TN7 are low-risk terminal nodes (≤20.8% risk of complicated appendicitis). TN2 and TN4 are moderate risk terminal nodes (27.5 to 47.6% risk of complicated appendicitis) and TN5, TN6, and TN8 are high-risk terminal nodes (59.1 to 90.0% risk of complicated appendicitis). To calculate the diagnostic test characteristics, all low-risk samples are predicted to be associated with uncomplicated appendicitis, while all intermediate and high-risk samples are predicted to be associated with complicated appendicitis.
3. Discussion We have derived a biomarker-based risk stratification tool to reliably estimate the risk of complicated appendicitis in pediatric patients. Reliably estimating this risk is a key decision point for determining which patients are potentially eligible for non-operative management of acute appendicitis, and which patients warrant traditional appendectomy. The ideal characteristics of a companion test that informs clinical decision making regarding non-operative management of appendicitis include a high sensitivity, a high negative predictive value, and a low negative likelihood ratio. The decision tree demonstrated these test characteristics in both the derivation and test cohorts. Such test characteristics would allow for reliable inclusion of low-risk patients in a non-operative management protocol, with simultaneous exclusion of higher risk patients. The potential advantages of successful non-operative management of appendicitis, without the need for surgery, are self-evident. Peri-operative complications occur in approximately eight to 11% of patients undergoing appendectomy and recovery may require absence of up to one week of
school [17,18]. Several recent trials of non-operative management of appendicitis have reported no increased risk of developing perforated appendicitis [8,11,19,20]. Initial cost-effective analysis supports the use of non-operative management in uncomplicated appendicitis [7,21]. However, non-operative management has been associated with more subsequent emergency room visits and hospitalizations compared to those patients managed operatively [22]. The impact of medical interventions on patient-centered outcomes is becoming increasingly important in the current era of demand-side strategic purchasing. In our cohort, individual biomarkers demonstrated significantly lower AUCs than the derived decision tree, with MMP-8 having the greatest AUC of 0.68. WBC has been traditionally utilized as a supportive laboratory finding in the diagnosis of appendicitis [23]. Sensitivity and specificity of WBC count in this setting have been reported to range from 70% to 80% and 60% to 68%, respectively [24–27]. The AUC of the WBC count alone in our cohort was 0.77 for estimating the risk of complicated appendicitis. This was inferior to that of the decision tree, indicating that the decision tree provides predictive information beyond that of
Table 3 Diagnostic test characteristics of the classification tree for estimating the risk of complicated appendicitis in the derivation cohort.
Table 4 Diagnostic test characteristics of the classification tree for estimating the risk of complicated appendicitis in the test cohort.
Category # of true positives # of false positives # of true negatives # of false negatives Sensitivity Specificity Positive predictive value Negative predictive value Positive likelihood ratio Negative likelihood ratio Area under the curve
Category 91 65 100 9 91% (83–96) 61% (53–68) 58% (50–66) 92% (84–96) 2.3 (1.9–2.8) 0.15 (0.08–0.3) 0.86 (0.81–0.90)
# of true positives # of false positives # of true negatives # of false negatives Sensitivity Specificity Positive predictive value Negative predictive value Positive likelihood ratio Negative likelihood ratio Area under the curve
24 14 27 1 96% (78–100) 66% (49–79) 63% (46–78) 96% (80–100) 2.8 (1.8–4.3) 0.06 (0.01–0.4) 0.85 (0.76–0.94)
440
M.C. Daly et al. / Journal of Pediatric Surgery 53 (2018) 437–440
Fig. 2. Comparison of the receiver operating characteristics (ROC) curves for the classification tree and WBC count in all subjects. The ROC for the classification tree is shown as a black line, and the ROC for WBC count is shown as a gray line. The area under the curve for the classification tree (0.86, 95% CI: 0.82 to 0.90) was greater than that of the WBC count alone (0.77, 95% CI: 0.72 to 0.82, p = 0.0003).
the WBC count alone. Given the shortcomings of a single biomarker to accurately detect the presence of complicated appendicitis in pediatric patients, a combination of biomarkers may be more advantageous. To our knowledge, we are not aware of a validated stratification tool assessing the probability of complicated appendicitis that performs in an equivalent manner to that of our derived classification tree. We note the limitations of our study. CART analysis has the potential to over-fit a dataset. We attempted to address this issue by dividing our cohort into separate derivation and test cohorts. Nonetheless, our model requires further validation in a prospectively enrolled cohort. In addition, the panel of biomarkers used to estimate the risk of complicated appendicitis is currently available only as research-based assays and they require batching. In order to use these biomarkers for clinical decision making, a rapid assay would need to be developed. The technology to develop such a test currently exists. To conclude, these data suggest that MMPs and TIMPs have the potential to serve as biomarkers to estimate the probability of complicated appendicitis in pediatric patients. The combination of MMP-8, MMP-9, MMP-12, TIMP-2, and TIMP-4 produced the most reliable test characteristics. The multi-biomarker-based decision tree has test characteristics suggesting clinical utility for decision making and may serve to identify pediatric patients with appendicitis that can be safely managed non-operatively. Acknowledgments The authors would like to thank Patrick Lahni for the technical support. References [1] Sivit CJ, Siegel MJ, Applegate KE, et al. When appendicitis is suspected in children. Radiographics 2001;21:247–62. [2] Richardson WS. The evolution of early appendectomy as standard treatment from appendicitis: what we can learn from the past in adopting new medical therapies. Am Surg 2015;81:161–5. [3] Abes M, Petik B, Kazil S. Nonoperative treatment of acute appendicitis in children. J Pediatr Surg 2007;42:1439–42. [4] Eriksson S, Granstrom L. Randomized controlled trial of appendicectomy versus antibiotic therapy for acute appendicitis. Br J Surg 1995;82:166–9. [5] Hansson J, Korner U, Ludwigs K, et al. Antibiotics as first-line therapy for acute appendicitis: evidence for a change in clinical practice. World J Surg 2012;36:2028–36. [6] Styrud J, Eriksson S, Nilsson I, et al. Appendectomy versus antibiotic treatment in acute appendicitis. A prospective multicenter randomized controlled trial. World J Surg 2006;30:1033–7.
[7] Di Saverio S, Sibilio A, Giorgini E, et al. The NOTA study (non operative treatment for acute appendicitis): prospective study on the efficacy and safety of antibiotics (amoxicillin and clavulanic acid) for treating patients with right lower quadrant abdominal pain and long-term follow-up of conservatively treated suspected appendicitis. Ann Surg 2014;260:109–17. [8] Salminen P, Paajanen H, Rautio T, et al. Antibiotic therapy vs appendectomy for treatment of uncomplicated acute appendicitis: the APPAC randomized clinical trial. JAMA 2015;313:2340–8. [9] Simillis C, Symeonides P, Shorthouse AJ, et al. A meta-analysis comparing conservative treatment versus acute appendectomy for complicated appendicitis (abscess or phlegmon). Surgery 2010;147:818–29. [10] Malik AA, Bari SU. Conservative management of acute appendicitis. J Gastrointest Surg 2009;13:966–70. [11] Minneci PC, Sulkowski JP, Nacion KM, et al. Feasibility of a nonoperative management strategy for uncomplicated acute appendicitis in children. J Am Coll Surg 2014;219:272–9. [12] Huang L, Yin Y, Yang L, et al. Comparison of antibiotic therapy and appendectomy for acute uncomplicated appendicitis in children: a meta-analysis. JAMA Pediatr 2017;171:426–34. [13] Steiner Z, Buklan G, Stackievicz R, et al. Conservative treatment in uncomplicated acute appendicitis: reassessment of practice safety. Eur J Pediatr 2017;176:521–7. [14] Solan PD, Dunsmore KE, Denenberg AG, et al. A novel role for matrix metalloproteinase-8 in sepsis. Crit Care Med 2012;40:379–87. [15] Atkinson SJ, Nolan M, Klingbeil L, et al. Intestine-derived matrix metalloproteinase-8 is a critical mediator of polymicrobial peritonitis. Crit Care Med 2016;44:e200–6. [16] Muller R, Mockel M. Logistic regression and CART in the analysis of multimarker studies. Clin Chim Acta 2008;394:1–6. [17] Wilms IM, de Hoog DE, de Visser DC, et al. Appendectomy versus antibiotic treatment for acute appendicitis. Cochrane Database Syst Rev 2011:CD008359. [18] Vahdad MR, Troebs RB, Nissen M, et al. Laparoscopic appendectomy for perforated appendicitis in children has complication rates comparable with those of open appendectomy. J Pediatr Surg 2013;48:555–61. [19] Varadhan KK, Neal KR, Lobo DN. Safety and efficacy of antibiotics compared with appendicectomy for treatment of uncomplicated acute appendicitis: meta-analysis of randomised controlled trials. BMJ 2012;344:e2156. [20] Rollins KE, Varadhan KK, Neal KR, et al. Antibiotics versus appendicectomy for the treatment of uncomplicated acute appendicitis: an updated meta-analysis of randomised controlled trials. World J Surg 2016;40(10):2305–18. [21] Wu JX, Dawes AJ, Sacks GD, et al. Cost effectiveness of nonoperative management versus laparoscopic appendectomy for acute uncomplicated appendicitis. Surgery 2015;158:712–21. [22] Bachur RG, Lipsett SC, Monuteaux MC. Outcomes of nonoperative management of uncomplicated appendicitis. Pediatrics 2017;140(1) [pii: e20170048]. [23] Bates MF, Khander A, Steigman SA, et al. Use of white blood cell count and negative appendectomy rate. Pediatrics 2014;133:E39–44. [24] Mekhail P, Naguib N, Yanny F, et al. Appendicitis in paediatric age group: correlation between preoperative inflammatory markers and postoperative histological diagnosis. Br J Surg 2010;97:123. [25] Beltran MA, Almonacid J, Vicencio A, et al. Predictive value of white blood cell count and Creactive protein in children with appendicitis. J Pediatr Surg 2007;42:1208–14. [26] Wang LT, Prentiss KA, Simon JZ, et al. The use of white blood cell count and left shift in the diagnosis of appendicitis in children. Pediatr Emerg Care 2007;23:69–76. [27] Bundy DG, Byerley JS, Liles EA, et al. Does this child have appendicitis? JAMA 2007; 298:438–51.