European Journal of Radiology 98 (2018) 174–178
Contents lists available at ScienceDirect
European Journal of Radiology journal homepage: www.elsevier.com/locate/ejrad
Research article
Clinical-radiological scoring system for enhanced diagnosis of acute appendicitis Manoj Mannila,b, Christos Polysopoulosc, Dominik Weishauptb, Anika Hansmannb,
T
⁎
a
Institute of Diagnostic and Interventional Radiology, University Hospital Zurich, University of Zurich, Raemistrasse 100, CH-8091 Zurich, Switzerland Institute of Radiology and Nuclear Medicine, Triemli Hospital, Birmensdorferstrasse 497, CH-8063 Zurich, Switzerland c Department of Biostatistics, Epidemiology, Biostatistics and Prevention Institute (EBPI), University of Zurich, Hirschengraben 84, CH-8001 Zurich, Switzerland b
A R T I C L E I N F O
A B S T R A C T
Keywords: Ultrasound Appendicitis Modified alvarado score Clinical-radiological scoring system Appendix diameter Vermix
Introduction: Acute appendicitis is the most common surgical condition in industrialized countries. However, diagnosis in borderline cases is often cumbersome and requires follow-up examinations, and/or a Computed Tomography examination. Therefore, our aim was to develop a combined clinical and sonographic score to enhance prediction of acute appendicitis. Patients and methods: The modified Alvarado score and various established sonographic criteria were investigated in 132 patients with suspected acute appendicitis. Two models were computed accounting for missing values. After analysis of sensitivity and specificity for the modified Alvarado score, logistic regression analysis was performed to identify significantly contributing sonographic features. Results: The threshold of the logistic regression analysis of the two models resulted in the formula: modified Alvarado score + axial Appendix diameter [mm]. When the Appendix diameter is > 8 mm, 1 point/mm is added, while for a diameter < 8 mm, 1 point/mm is subtracted. The cut-off value is ≥ 13 for acute appendicitis with a sensitivity 91.4% and a specificity of 100%, compared to a sensitivity of 90% and a specificity of 84.4% for the modified Alvarado score in our cohort. Conclusion: The established modified Alvarado score for diagnosis of acute Appendicitis can be improved by adding the axial diameter of the Appendix in a sonographic examination.
1. Introduction Acute appendicitis is the most common surgical condition in industrialized countries. It is the second most common cause of right lower quadrant pain of patients presenting to the emergency department after non-specific abdominal pain and constipation [1,2]. The lifetime incidence is reported to be in the range of 7–14% [3]. According to the American College of Radiology (ACR) appropriateness guidelines, computed tomography (CT) with intravenous contrast material is recommended for evaluation of adults and adolescents suspected of suffering from acute Appendicitis, while abdominal ultrasonography (US) is recommended for use in pregnant patients and children younger than the age of 14 years [4,5]. Since the introduction of pre-surgical imaging, negative appendectomies, treatment-associated complications, as well as overall healthcare costs have been significantly reduced [6]. While the use of abdominal CT is proven to be of higher sensitivity compared to abdominal ultrasound (sensitivity and specificity for US diagnosis of Appendicitis in adults are 83% (95% CI: 78%, 87%) and 93% (95% CI: 90%, 96%), compared to CT sensitivity ⁎
and specificity of 94% (95% CI: 92%, 95%) and 94% (95% CI: 94%, 96%), respectively), the use of potentially harmful ionizing radiation needs to be taken into consideration [7,8]. Abdominal US utilizes no ionizing radiation and shows a similar specificity for the diagnosis of acute Appendicitis compared to CT [9,10]. First line abdominal US could reduce radiation exposure by 57% and imaging costs by 45% compared to initial contrast enhanced CT [11]. Furthermore, data from large retrospective trials show radiation of repeated CT scans may triple the risk of brain cancer and leukaemia in younger populations [9,12,13]. There is a plethora of scores available for cases of suspected acute Appendicitis, including but not limited to the Adult Appendicitis Score [14], Appendicitis Inflammatory Response Score [15], Pediatric Appendicitis Score [16], and RIPASA score for Asian individuals [17]. However, the most frequently used measure to clinically evaluate acute Appendicitis is the Alvarado Score in its modified version consisting of 7 parameters (Table 1) [18,19]. Scores range from 0 to 9 points and the absolute score positively correlates with the likelihood of acute Appendicitis.
Corresponding author. E-mail address:
[email protected] (A. Hansmann).
https://doi.org/10.1016/j.ejrad.2017.11.020 Received 20 July 2017; Received in revised form 23 October 2017; Accepted 27 November 2017 0720-048X/ © 2017 Elsevier B.V. All rights reserved.
European Journal of Radiology 98 (2018) 174–178
M. Mannil et al.
Table 1 Modified Alvarado Score [7;8].
Table 2 Descriptive Statistics.
Variables
Clinical features
Score
Parameter
Cohort
Mean
Median
Symptoms
Migratory right iliac fossa (RIF) pain Anorexia Nausea and vomiting Tenderness (RIF) Rebound tenderness Elevated temperature Leukocytosis
1 1 1 2 1 1 2 9
Age [years]
Appendicitis Group Control Group
31.3 26.4
22 (6;87) 21 (8;80)
Gender [%]
Gender
Appendicitis Group
female male
54 46
Control Group 78.1 21.9
Cohort
Mean
Median
Appendicitis Group Control Group
6.5 2.8
6.5 (4;9) 2 (0;7)
Cohort
Mean
Median
Appendicitis Group Control Group
10.2 6.2
10 (5;17) 7 (3;9)
Signs
Laboratory Total Score
Modified Alvarado score
In light of the artificial separation between clinical and imageguided diagnosis of acute Appendicitis with potentially inconclusive or contradicting outcomes, a clinical-pathological-radiological scoring system was recently proposed [10,20]. In this context, Álvarez et al. recently evaluated the Alvarado score in its full form and a range of sonographic features [21] with promising results. The purpose of this study was to develop a combined clinical-radiological scoring system using the modified Alvarado score and only significantly contributing sonographic findings in order to enhance the prediction of histo-pathologically proven acute Appendicitis and provide guidance to examiners in case the Appendex cannot be visualized in abdominal ultrasound. The hypothesis was that a clinical-radiological scoring system can enhance sensitivity and specificity compared to the commonly used Alvarado score alone.
Appendix diameter [mm]
the root of the mesentery with displacement of bowel loops and hyperemia. Mesenteric lymphadenopathy was defined as ≥ 3 oval/round lymph nodes with a diameter of ≥ 10 mm and diminished fatty hilum. The diagnosis of perforation had to be confirmed by CT imaging or surgery with histo-pathology results. 2.2. Ultrasound technique All examinations were performed using the latest generation of GE Logiq E9 ultrasound machines (General Electric Healthcare, Little Chalfont, UK). A complete abdominal Ultrasound examination using a 4 Hz (C1-5) convex transducer was followed by a focused examination of the bowels using a 9 Hz (9L) linear transducer. Color duplex sonography was regularly used to visualize parenchymal organ perfusion and hyperemia. The abdominal ultrasound was internally standardized and performed by board certified radiologists or supervised residents. The above mentioned ultrasound criteria were routinely examined and explicitly mentioned in the final report.
2. Subjects and methods 2.1. Patients and study design Ethical approval of the local Ethic’s commission (blinded for review) was obtained for the retrospective analysis of pre-surgical imaging studies in patients with suspected acute Appendicitis. Between 2012 and 2014 492 patients with clinically suspected diagnosis of acute Appendicitis presented to the local emergency department and received an initial abdominal ultrasound. Patients without following surgical intervention, without histo-pathologically confirmed diagnosis, or with a first-line CT/MRI examination were excluded (n = 298). 62 patients were excluded as the imaging studies and/or clinical reports were incomplete. 132 patients were consecutively enrolled into the study. 100 patients had a histo-pathologically confirmed case of acute Appendicitis. The remaining 32 cases with negative surgical-pathological evaluation were used as a control group. The mean age was 31.3 (6;87) years for the Appendicitis group and 26.4 (8;80) years for the control group. The Appendicitis group had 54% female and 46% male patients. The control group had 78.1% female patients and 21.9% male patients (Table 2). The clinical outcome measure, Alvarado Score in its modified version, was performed on all study participants [18,19]. The necessary clinical information was obtained via the local clinical information system. The clinical components of the modified Alvarado score had to be explicitly mentioned in the clinical history (positive or negative). The ultrasound examination of the abdomen was standardized to internal guidelines. All images and reports were retrospectively reanalyzed according following criteria: Appendix found, axial Appendix diameter, Appendix compressible, Appendix stone, abscess formation, perforation, primary gynecological/urological findings, large bowel inflammation, small bowel inflammation, inflamed mesenteric fatty tissue, free fluid. All endpoints were classified according to an internal consensus and evaluated by two radiologists with 4 and 10 years of experience in gastrointestinal ultrasound. The axial Appendix diameter was measured perpendicular to the longitudinal axis at its widest diameter from outer wall to outer wall. Inflammation of the mesenteric tissue was defined as a hyperechogenic, well-defined fatty structure in
2.3. Statistical analysis For statistical analysis R software by R Core Team (URL http:// www.R-project.org/) was used. Two models were computed depending on the visualization of the Appendix (Table 3). To impute for missing values of Appendix diameter, compressibility and Appendix stone, imputation was performed using the random forest method (missForest R package) [22]. Logistic regression analysis was performed to identify significantly contributing sonographic criteria for score generation. Threshold analysis of the new score evaluated cut-off values to determine the diagnosis of acute Appendicitis in accordance with the histo-pathological gold standard. Model selection was conducted using bestglm R package, with Morgan and Tatar’s method of complete enumeration [23]. Results were based on Bayesian Information Criterion. A ROC-analysis was performed computing sensitivity and specificity. After cross validation using the R package cvAUC, the two regression equations were combined to find the optimal threshold for the Appendicitis group and control group. 3. Results 100 patients had a histo-pathologically confirmed case of acute Appendicitis. The remaining 32 patients with a confirmed alternative diagnosis served as a control group. Descriptive statistical analysis is depicted in Table 2. The mean modified Alvarado score was 6.5 (4–9) points for the Appendicitis group and 2.8 (0–7) points for the control 175
European Journal of Radiology 98 (2018) 174–178
M. Mannil et al.
Table 3 Descriptive statistics of Ultrasound findings. Variable
Levels
N Appendicitis Group
% Appendicitis Group
N Control Group
% Control Group
N all
% all
Appendix found
No Yes No Not applicable Yes No Not applicable Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes No Yes
7 93 87 12 1 73 7 20 97 3 90 10 91 9 94 6 92 8 8 92 86 14 59 41
7 93 91.6 7.4 1 73 7 20 97 3 90 10 91 9 94 6 92 8 8 92 86 14 59 41
19 13 0 19 8 12 19 1 32 0 32 0 28 4 29 3 27 5 27 5 23 9 27 5
59.4 40.6 0 70.4 29.6 37.5 59.4 3.1 100 0 100 0 87.5 12.5 90.6 9.4 84.4 15.6 84.4 15.6 71.9 28.1 84.4 15.6
26 106 87 26 9 85 26 21 129 3 122 10 119 13 123 9 119 13 35 97 109 23 86 46
19.7 80.3 71.3 21.3 7.4 64.4 19.7 15.9 97.7 2.3 92.4 7.6 90.2 9.8 93.2 6.8 90.2 9.8 26.5 73.5 82.6 17.4 65.2 34.9
Appendix compressible
Appendix stone
Abscess Perforation Gynecological/Urological Pathologies Large bowel inflammation Small bowel inflammation Fatty tissue inflamed Mesenterial lymphadenopathy Free fluid
Table 4 Diagnosis of control group (n = 32). Diagnosis of control group
Number
Gastroenteritis Inconclusive results (spontaneous recovery) Intestinal motility disorders Carcinoma (Sigmoid, Conglomerate) Urinary tract infection Cholecystitis Colitis Iliocaecal invagination Ruptured Ovarian cyst
10 10 4 2 2 1 1 1 1
Fig. 2. ROC curve for modified Alvarado score in case the Appendix cannot be identified. The blue shaded area describes the uncertainty around the placement values with a depiction of confidence intervals for sensitivity and specificity. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)
examination results are depicted in Table 3. Overall 13 patients had a complicated Appendicitis with perforation and/or abscess formation. In case the Appendix could not be visualized despite of a modified Alvorado score ≥ 8 points, a perforation was ruled out by either CT or direct surgery. In the control group (n = 32) non-specific abdominal pain, gastroenteritis and constipation (62.5%) were the most common diagnoses (Table 4). In both groups there was a strong correlation between the histo-pathological outcome and the Alvarado score in its modified version. At the threshold of the ROC analysis, the modified Alvarado score alone showed a sensitivity of 90% and a specificity of 84.4% (Fig. 1). In case experienced sonographers could not visualize the Appendix, the modified Alvarado score still showed a sensitivity of 85.7% and specificity of 89.5% in comparison to the histo-pathological gold standard (Fig. 2). In the first model, cases of failed US Appendix visualization were included (n = 26). A logistic regression was performed for the remaining eight sonographic criteria, excluding Appendix identification, Appendix axial diameter, Appendicolith presence, and Appendix
Fig. 1. ROC curve for modified Alvarado score in all cases of clinical suspicion of acute appendicitis. The blue shaded area describes the uncertainty around the placement values with a depiction of confidence intervals for sensitivity and specificity. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)
group. Mean axial Appendix diameter was 10.2 (5–19) mm for the Appendicitis groups and 6.2 (3–9) mm for the control group. By performing abdominal ultrasound the Appendix could be visualized in 93% of the Appendicitis group and in 40.6% of the control group. Detailed 176
European Journal of Radiology 98 (2018) 174–178
M. Mannil et al.
clinical-radiological scoring system, we tested the established modified Alvarado score and various ultrasound criteria in patients with clinical suspicion of acute Appendicitis [2,20]. The modified Alvarado score alone showed an excellent overall sensitivity of 90% and a specificity of 84.4% respectively, in a cohort of 100 patients and 32 controls with a histo-pathological gold standard. Correlation analysis of a battery of 12 additional sonographic criteria, showed that no sonographic parameter adds significant amount of information to the modified Alvarado score other than the axial Appendix diameter (p = 0.003). These results are in accordance with the findings of Álvarez et al. [21] in an adult population. As they also observed the highest discrimatory power in the Appendix diameter [mm] among all other tested sonographic features. Their obtained test characteristics regarding sensitivity and specificity are in the same range as our findings with 91.4% and 100%, respectively. However, we provided additional analyses in case the Appendix cannot be visualized by an experienced examiner using abdominal ultrasound in a separate statistical model imputed for missing values of Appendix diameter, compressibility and Appendix stone. We observed that in this case secondary ultrasound findings such as free fluid or mesenteric lymphadenopathy do not significantly improve the modified Alvarado score in decision-making regarding the final diagnosis. Apart from the modified Alvarado score, follow-up US examinations or use of other imaging modalities may therefore need to be considered. In summary, the new clinical-radiological score shows an increased sensitivity of 91.4% (87%–100%) and a markedly increased specificity of 100% (77%–100%), compared to the modified Alvarado score alone with a sensitivity of 90% and a specificity of 84.4%. An analysis of the aforementioned secondary ultrasound criteria yielded a sensitivity of 77.8% and a specificity of 59.6%. Retrospective chart review methods were taken into account during study design. However, we must acknowledge several limitations. First, the retrospective nature of the study has inherent limitations. Clinical information was obtained by analyzing clinical reports. Information to each criterion of the modified Alvarado score had to be explicitly mentioned in the clinical documentation for patients to be included. As imputation of missing data was actively avoided, patients with incomplete clinical or imaging data sets were largely excluded, which imposes a possible selection bias. Similarly, we only included patients who underwent surgical treatment of acute Appendicitis, as this is the local standard treatment. Therefore, the presence of a histo-pathological gold standard itself might impose a selection bias criterion. Furthermore, there was only a small number of patients (n = 13) with confirmed complicated onset of acute Appendicitis. This is mainly due to direct surgical treatment or pre-surgical CT imaging in patients with markedly increased inflammation markers and/or symptoms of a hyperacute abdominal emergency. Therefore the clinical-radiological score cannot be applied in suspected complicated disease. While our clinical-radiological scoring system was applied to both, pediatric and adult populations (ages 6–87), further sub-grouping may be necessary to enhance test performance. All US examinations were performed or supervised by experienced board certified radiologist. However, US remains an examiner-dependent imaging modality. Not visualizing the Appendix due to lack of dexterity or experience will produce false negative score results when applying this clinical-radiological scoring system in daily practice. Finally, our findings of increased risk for acute Appendicitis according to the diameter holds true within a certain range of approximately 4–15 mm. The Appendix diameter will usually not exceed 15 mm, as perforation is more likely to occur than further dilatation. Important differential diagnosis of enlarged Appendix are tumors [24] and Appendiceal mucocele (prevalence 0.3%) with an intramural accumulation of mucus. In Appendiceal mucocele the axial diameter reflects the chronicity of the disease. The clinical-radiological scoring system may be of added value in assisting decision-making in borderline cases of suspected acute Appendicitis. It may help avoiding redundant imaging studies with
compressibility. Results showed that no other sonographic parameter adds further significant amount of information to the modified Alvarado score. In the second model, cases of successful US Appendix visualization were included (n = 106). The modified Alvarado score and all obtained US criteria were used as predictors. Model selection returned modified Alvarado score (p < 0.003), Appendix found (p = 0.001) and axial Appendix diameter (p = 0.003) as significantly contributing parameters (Supplemental Table 1). For score generation, we generated the regression outputs of the two models (Appendix not found (1) and Appendix found (2)). The regression equations for our cohorts were:
y1 = −9.7 + 4.2⋅x + 1.57⋅mod Alvarado Score
(1)
y2 = −16.2 + 4.95⋅x + 1.07⋅mod Alvarado Score + 1.1 ⋅Appendix diameter
(2)
Combining these two regression equations one can calculate the optimal threshold for classifying the Appendicitis group and control group according to the histo-pathological gold standard. In case the Appendix cannot be visualized by US (1), the modified Alvarado score alone remains the method of choice with a score ≥ 8 showing strong evidence for the diagnosis of acute Appendicitis. In case the Appendix can be visualized (2) the nominal value of Appendix diameter in [mm] is added to the modified Alvarado score, resulting in: modified Alvarado score + axial Appendix diameter [mm]
(3)
A combined score of ≥ 13 points is highly specific for a case of acute Appendicitis. For optimal model fit, one may add or subtract one point per [mm] axial Appendix diameter, according to the absolute deviation from 8 mm. For test statistics, a modified ROC-curve was plotted adding the axial diameter of the Appendix to the existing statistical model (Fig. 3). The newly calculated confidence intervals at the threshold show an increased sensitivity of 91.4% (87%–100%) and a markedly increased specificity of 100% (77%–100%) with an AUC of 93.8% (88.2–99.3%). An AUC cross validation confirmed the obtained results. Two clinical examples are provided in Fig. 4. For instance, in case of an axial diameter of 7 mm and a modified Alvarado score of 7 points, one may subtract 1 point to gain a combined score of 13 points, suggesting clinical-radiological evidence for the diagnosis of acute Appendicitis (Fig. 4A). 4. Discussion In light of recent evidence-based findings and the suggestion of a
Fig. 3. ROC curve when Appendix is found and Appendix diameter is added to the model.
177
European Journal of Radiology 98 (2018) 174–178
M. Mannil et al.
Fig. 4. (a) 31 year old female with histologically proven appendicitis and a modified Alvarado score of 7: The diameter of the Appendix in ultrasound measures 7 mm resulting in a clinic-radiological score of 13. (b) 26 year old female with histologically proven absence of appendicitis and a modified Alvarado score of 6: The diameter of the Appendix in ultrasound measures 6 mm resulting in a clinic-radiological score of 10.
ionizing radiation, negative appendectomies and/or treatment-associated complications. In general the combination of axial Appendix diameter [mm] and the modified Alvarado score is not US specific and could possibly be extended to other imaging modalities (e.g. MRI, CT) [25–27]. Due to the lifetime prevalence of acute Appendicitis being between 7–14%, our findings need to be validated in a larger, multicenter clinical prospective trial.
[6]
[7]
[8]
5. Conclusion [9]
Incorporation of initial Ultrasound examinations in the evaluation of acute Appendicitis is proven to be of diagnostic value, while reducing overall exposure of ionizing radiation and healthcare cost. The clinically used modified Alvarado score provides excellent diagnostic value. In combining the sonographically determined axial diameter of the Appendix and the modified Alvarado score to a clinical-radiological scoring system, markedly improved sensitivity and specificity can be achieved.
[10] [11]
[12]
[13]
Funding sources [14]
This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.
[15]
Conflict of interest
[16] [17]
There is no conflict of interest. This research did not receive any specific grant from funding agencies in the public, commercial, or notfor-profit sectors.
[18] [19]
Acknowledgements [20]
We would like to thank all participants of the study. [21]
Appendix A. Supplementary data Supplementary data associated with this article can be found, in the online version, at https://doi.org/10.1016/j.ejrad.2017.11.020.
[22]
References
[24]
[23]
[1] J.J. Brown, Acute appendicitis: the radiologist's role, Radiology 180 (1) (1991) 13–14. [2] N.M. Blitman, M. Anwar, K.B. Brady, B.H. Taragin, K. Freeman, Value of focused appendicitis ultrasound and alvarado score in predicting appendicitis in children: can we reduce the use of CT? AJR Am. J. Roentgenol. 204 (6) (2015) W707–12. [3] D.R. Flum, Clinical practice. Acute appendicitis–appendectomy or the antibiotics first strategy, N. Engl. J. Med. 372 (20) (2015) 1937–1943. [4] I. Petkovska, D.R. Martin, M.F. Covington, S. Urbina, E. Duke, Z.J. Daye, L.A. Stolz, S.M. Keim, J.R. Costello, S. Chundru, H. Arif-Tiwari, D. Gilbertson-Dahdal, L. Gries, B. Kalb, Accuracy of unenhanced MR imaging in the detection of acute appendicitis: single-institution clinical performance review, Radiology (2016) 150468. [5] M.P. Rosen, A. Ding, M.A. Blake, M.E. Baker, B.D. Cash, J.L. Fidler, T.H. Grant,
[25]
[26]
[27]
178
F.L. Greene, B. Jones, D.S. Katz, T. Lalani, F.H. Miller, W.C. Small, S. Spottswood, G.S. Sudakoff, M. Tulchinsky, D.M. Warshauer, J. Yee, B.D. Coley, ACR Appropriateness Criteria(R) right lower quadrant pain–suspected appendicitis, J. Am. Coll. Radiol. 8 (11) (2011) 749–755. M.J. Lahaye, D.M. Lambregts, E. Mutsaers, B.A. Essers, S. Breukink, V.C. Cappendijk, G.L. Beets, R.G. Beets-Tan, Mandatory imaging cuts costs and reduces the rate of unnecessary surgeries in the diagnostic work-up of patients suspected of having appendicitis, Eur. Radiol. 25 (5) (2015) 1464–1470. G. Mostbeck, E.J. Adam, M.B. Nielsen, M. Claudon, D. Clevert, C. Nicolau, C. Nyhsen, C.M. Owens, How to diagnose acute appendicitis: ultrasound first, Insights Imaging 7 (2) (2016) 255–263. A.S. Doria, R. Moineddin, C.J. Kellenberger, M. Epelman, J. Beyene, S. Schuh, P.S. Babyn, P.T. Dick, US or CT for diagnosis of appendicitis in children and adults? a meta-analysis, Radiology 241 (1) (2006) 83–94. M. Chien, A. Habis, L. Glynn, A. O'Connor, T.L. Smith, F. Prendergast, Staged Imaging Pathway for the Evaluation of Pediatric Appendicitis, Pediatr Surg Int, 2016. F. Tatli, U. Ekici, M. Kanlioz, O. Gozeneli, A. Uzunkoy, Y. Yucel, A. Dirican, Ultrasonography in diagnosis of acute appendicitis, Ann. Ital. Chir. 87 (2016) 152–154. J.R. Pare, B.K. Langlois, S.A. Scalera, L.F. Husain, C. Douriez, H. Chiu, K. Carmody, Revival of the use of ultrasound in screening for appendicitis in young adult men, J. Clin. Ultrasound 44 (1) (2016) 3–11. M.O. Sauvain, K. Slankamenac, M.K. Muller, S. Wildi, U. Metzger, W. Schmid, J. Wydler, P.A. Clavien, D. Hahnloser, Delaying Surgery to Perform CT Scans for Suspected Appendicitis Decreases the Rate of Negative Appendectomies Without Increasing the Rate of Perforation nor Postoperative Complications, Langenbecks Arch Surg, 2016. M.S. Pearce, J.A. Salotti, M.P. Little, K. McHugh, C. Lee, K.P. Kim, N.L. Howe, C.M. Ronckers, P. Rajaraman, A.W. Sir Craft, L. Parker, A. Berrington de Gonzalez, Radiation exposure from CT scans in childhood and subsequent risk of leukaemia and brain tumours: a retrospective cohort study, Lancet 380 (9840) (2012) 499–505. H.E. Sammalkorpi, P. Mentula, H. Savolainen, A. Leppaniemi, The introduction of adult appendicitis score reduced negative appendectomy rate, Scand. J. Surg. 106 (3) (2017) 196–201. B. Von-Muhlen, O. Franzon, M.G. Beduschi, N. Kruel, D. Lupselo, AIR score assessment for acute appendicitis, Arq Bras Cir Dig 28 (3) (2015) 171–173. M. Samuel, Pediatric appendicitis score, J. Pediatr. Surg. 37 (6) (2002) 877–881. C.F. Chong, M.I. Adi, A. Thien, A. Suyoi, A.J. Mackie, A.S. Tin, S. Tripathi, N.H. Jaman, K.K. Tan, K.Y. Kok, V.V. Mathew, O. Paw, H.B. Chua, S.K. Yapp, Development of the RIPASA score: a new appendicitis scoring system for the diagnosis of acute appendicitis, Singapore Med. J. 51 (3) (2010) 220–225. A. Alvarado, A practical score for the early diagnosis of acute appendicitis, Ann. Emerg. Med. 15 (5) (1986) 557–564. M. Kalan, D. Talbot, W.J. Cunliffe, A.J. Rich, Evaluation of the modified Alvarado score in the diagnosis of acute appendicitis: a prospective study, Ann. R. Coll. Surg. Engl. 76 (6) (1994) 418–419. J. Debnath, R. Ravikumar, C.G. Muralidharan, G. Singh, Alvarado score: is it time to develop a clinical-pathological-radiological scoring system for diagnosing acute appendicitis? Am. J. Emerg. Med. 33 (6) (2015) 839–840. J.F. Álvarez, J.M. Gómez López, A.M. González Chávez, B.V. Acosta, A.D. Álvarez Hénandez, S.R. Pascual, A.A. García Vázquez, The use of ultrasonographic and Alvarado scores in evaluating abdominal pain suggestive of acute appendicitis, Int. Surg. J. 4 (2) (2017) 757–763. D.J. Stekhoven, P. Buhlmann, MissForest–non-parametric missing value imputation for mixed-type data, Bioinformatics 28 (1) (2012) 112–118. A.I.X. McLeod, C. Xu, Package ‘bestglm', (2015) (https://cran.r-project.org/web/ packages/bestglm/bestglm.pdf .). K.S. Lee, L.H. Tang, J. Shia, P.B. Paty, M.R. Weiser, J.G. Guillem, L.K. Temple, G.M. Nash, D. Reidy, L. Saltz, M.J. Gollub, Goblet cell carcinoid neoplasm of the appendix: clinical and CT features, Eur. J. Radiol. 82 (1) (2013) 85–89. E. Inci, E. Hocaoglu, S. Aydin, F. Palabiyik, T. Cimilli, A.N. Turhan, E. Aygun, Efficiency of unenhanced MRI in the diagnosis of acute appendicitis: comparison with Alvarado scoring system and histopathological results, Eur. J. Radiol. 80 (2) (2011) 253–258. E. Chabanova, I. Balslev, M. Achiam, Y.W. Nielsen, S. Adamsen, P. Gocht-Jensen, S.K. Brisling, V.B. Logager, H.S. Thomsen, Unenhanced MR Imaging in adults with clinically suspected acute appendicitis, Eur. J. Radiol. 79 (2) (2011) 206–210. P. Poortman, P.N. Lohle, C.M. Schoemaker, M.A. Cuesta, H.J. Oostvogel, E.S. de Lange-de Klerk, J.F. Hamming, Improving the false-negative rate of CT in acute appendicitisReassessment of CT images by body imaging radiologists: a blinded prospective study, Eur. J. Radiol. 74 (1) (2010) 67–70.