- Email: [email protected]
External validation of the STONE score and derivation of the modified STONE score
American Journal of Emergency Medicine xxx (2016) xxx–xxx
Contents lists available at ScienceDirect
American Journal of Emergency Medicine journal homepage: www.elsevier.com/locate/ajem
Original Contribution
External validation of the STONE score and derivation of the modified STONE score☆,☆☆ Byunghyun Kim, MD 1, Kyuseok Kim, MD, PhD ⁎, Joonghee Kim, MD, You Hwan Jo, MD, PhD, Jae Hyuk Lee, MD, PhD, Ji Eun Hwang, MD, Jongdae Park, MD Department of Emergency Medicine, Seoul National University, Bundang Hospital, Sungnam-si, Gyeonggi-do 463-707, Republic of Korea
a r t i c l e
i n f o
Article history: Received 12 April 2016 Received in revised form 20 May 2016 Accepted 21 May 2016 Available online xxxx
a b s t r a c t Objective: The STONE score is a clinical prediction rule for the presence of uncomplicated ureter stones with a low probability of acutely important alternative findings. This study performed an external validation of the STONE score, focusing on the Korean population, and a derivation of the modified STONE score for better specificity and sensitivity. Methods: We retrospectively reviewed medical records of patients complaining of flank pain at a single emergency department from January 2013 to December 2014. Patients were categorized into 3 groups according to their STONE score. The prevalence of ureter stones and other alternative findings were calculated in each group. We derived a modified STONE score based on a multivariable analysis and performed an interval validation. Results: From the 700 patients included in the analysis, 555 patients (79%) had a ureter stone. The area under the receiver operating characteristic curve of the STONE score was 0.92. The sensitivity of the high stone score was 0.56. In the modified STONE score, nausea, vomiting, and racial predictors were substituted by C-reactive protein and previous stone history. The area under the receiver operating characteristic curve and sensitivity of the modified STONE score in the internal validation group significantly increased to 0.94 and 0.80, respectively. Conclusion: The STONE score can be used to predict a ureter stone with a low probability of other alternative findings. The modified STONE score might increase the diagnostic performance in suspicious urinary stone cases. Key points: We performed external validation of the STONE score and derivation of the modified STONE score. This scoring system could help the clinicians with radiation reducing decision making. © 2016 Elsevier Inc. All rights reserved.
1. Introduction Ureter stones featuring acute flank pain and hematuria are commonly encountered in the emergency department (ED) and account for more than 1 million annual ED visits in the United States [1,2]. The worldwide prevalence and incidence of ureter stones vary from 2% to 20% depending on the country or region, and the incidence rate is increasing [3,4]. The annual incidence rate of ureter stones in Korea is 457 per 100 000. This rate is higher than that previously reported in Japan but lower than in Western countries [5].
☆ Prior presentations: None. ☆☆ Funding sources: None. ⁎ Corresponding author at: Department of Emergency Medicine, Seoul National University, Bundang Hospital, 300 Gumi-dong, Bundang-gu, Sungnam-si, Gyeonggi-do 463-707, Republic of Korea. Tel.: +82 31 787 7572; fax: +82 31 787 4055. E-mail addresses: [email protected] (B. Kim), [email protected] (K. Kim). 1 Department of Emergency Medicine, Seoul National University, Bundang Hospital, 300 Gumi-dong, Bundang-gu, Sungnam-si, Gyeonggi-do, 463-707, Republic of Korea. Tel.: +82 31 415 5739; fax: +82 31,787 4055.
Recently, Moore et al [6] derived the STONE score. It is a clinical prediction rule to evaluate the risk of ureter stones and clinically important alternative diagnoses for patients with suspected ureter stones. In the STONE score, 5 factors were found to be the most predictive of ureter stones: male sex, short duration of pain, nonblack race, presence of nausea or vomiting, and microscopic hematuria. The score ranges from 0 to 13. Patients are classified into low (0-5), moderated (6-9), and high (10-13) score groups. From those results, the authors propose using the STONE score to limit exposure to radiation and overutilization of imaging. Computed tomography (CT) is the preferred imaging method when patients initially present with suspected ureter stones in the United States and Europe [7,8]. As the use of a CT scan increases, the risk of a radiation hazard problem comes into question [9]. In Korea, relatively cheaper prices of CT scans compared to Western countries make CT scans even more frequently used and may lead to overuse and to radiation overexposure [10]. Increased CT scan use leads to increased diagnoses of ureter stones and other alternative clinical findings but does not lead to improvement in patient outcomes [11,12]. This study was performed to validate the STONE score in a single medical center in Korea. We also aimed to derive a modified STONE score and internally validate it.
http://dx.doi.org/10.1016/j.ajem.2016.05.061 0735-6757/© 2016 Elsevier Inc. All rights reserved.
Please cite this article as: Kim B, et al, External validation of the STONE score and derivation of the modified STONE score, Am J Emerg Med (2016), http://dx.doi.org/10.1016/j.ajem.2016.05.061
2
B. Kim et al. / American Journal of Emergency Medicine xxx (2016) xxx–xxx
Fig. 1. Flowchart of the enrolled patients. UA, urine analysis.
2. Materials and methods
2.2. Statistical analysis
2.1. Study design and selection of patients
2.2.1. External validation of the original STONE score For external validation of the original STONE score, the same number of points was assigned to each predictor as in original study (male sex, 2 points; duration of pain b6 hours, 3 points; duration of pain 6 to 24 hours, 1 point; nonblack race, 3 points; nausea alone, 1 point; vomiting, 2 points; hematuria on urine dipstick testing, 3 points). The patients were categorized into 3 groups according to their STONE score: low(0-5), moderate- (6-9), and high-score (10-13) groups. The prevalence of ureteral stones and important alternative diagnoses were determined for each group. We analyzed a receiver operating characteristic (ROC) curve and the area under the ROC curve (AUROC) to compare with data from the original study.
We retrospectively reviewed the medical records of adult patients older than 17 years who visited our ED for flank pain between January 2013 and December 2014. Eligible patients had a final clinical diagnosis combined with a CT scan using a CT scan. Exclusion criteria were the same as those in the original article: objective fever (N 37.7°C), presence of leukocytes in urine microscopy, known renal disease (including creatinine N1.5 mg/dL), history of trauma, or known active malignancy. The study facility was a 950-bed urban tertiary academic hospital with an annual ED census of approximately 80 000. The institutional review board approved the analysis and issued a waiver of consent. Table 1 Baseline characteristics of the study population and original study Characteristic
This validation cohort (n = 700), frequency (%)
Original validation cohort (n = 491), frequency (%)
Mean (SD) age (y) Male sex Race Asian White Black Hispanic Mixed
47.0 (14.3)
46 (15)
559 (65.6)
273 (56)
697 (99.9) 3 – – –
– 411 (84) 57 (12) – 23 (5)
Radiology findings Symptomatic 555 (79.3) ureteral stone 67 (9.6) Important alternative findings Disposition Admission or transfer
73 (10.4)
274 (56) 18 (3.7)
51 (11)
2.2.2. Derivation and internal validation of the modified STONE score For the derivation and internal validation of the modified STONE score, we randomly divided the enrolled patients into a derivation group consisting of 66% of the patients and an internal validation group with the remaining 33% of patients. After dichotomizing the variables, we investigated epidemiological factors and laboratory results using a univariable analysis in the derivation group. Subsequently, a multivariable analysis was performed to find independent risk factors using candidate variables with a factor significance of Pb .05 in the univariable analysis. The prediction model for ureter stones was developed using a regression coefficient–based scoring method. We found predictive factors in a multivariable analysis and assigned a weighted value to each factor using β coefficients that reflect predictive power [13,14]. The β coefficients in the multivariable logistic regression model were multiplied by 2 and then approximated to the nearest natural number. Patients were categorized into 3 groups according to the modified STONE score: low- (0-4), moderate- (5-9), and high-score (10-16) groups. The prevalence of ureteral stones and important alternative diagnoses were determined for each group. Finally, we analyzed an ROC curve and AUROC to investigate the performance of the modified STONE score.
Please cite this article as: Kim B, et al, External validation of the STONE score and derivation of the modified STONE score, Am J Emerg Med (2016), http://dx.doi.org/10.1016/j.ajem.2016.05.061
B. Kim et al. / American Journal of Emergency Medicine xxx (2016) xxx–xxx
3
Fig. 2. Prevalence of ureteral stone by STONE score category in this external validation study and original validation study. Percentages at top of bars indicate prevalence of stone in each group. Values under bars indicate percentages of each group form overall enrolled patients.
The remaining cohort of 33% of patients was used to validate the prediction model internally. We computed the number of patients and proportion according to the risk score assigned to each predictive factor. Patients in the validation cohort were also assigned to 3 groups based on their scores. The prevalence of ureteral stones and important alternative diagnoses were determined for each group. We also analyzed an ROC curve and computed the AUROC to quantify the performance of the modified STONE score.
3. Results 3.1. External validation of the STONE score
2.2.3. Comparison of diagnostic performance between the STONE score and modified STONE score An AUROC was used to compare the diagnostic performance between the 2 score systems. We performed a McNemar test to compare the sensitivities of the 2 scoring systems. All analyses were conducted using STATA (version 13; StataCorp, College Station, TX).
During the study period, a total of 1791 patients visited our ED complaining of flank pain. Of these patients, 945 had a final clinical diagnosis combined with a CT scan on the same day. In total, 109 of the 945 enrolled patients were excluded because of fever, pyuria, urinalysis data loss, azotemia, known malignancy, and trauma history (Fig. 1). Finally, 700 patients were included in the final analysis, of which 555 patients were diagnosed with a ureter stone (79%) and 67 had other clinically significant findings (10%). Table 1 shows the baseline characteristics of the validation cohorts compared with that of the original study. The prevalence of ureter stones in the 3 score groups compared with that of the original validation study is represented in Fig. 2. In the high STONE score group, 309 patients (98.7%) were diagnosed with ureter stone, indicating that the high STONE score in this external validation
Table 2 Characteristics of derivation and internal validation cohorts
Table 3 Univariable analysis of clinical factors in patients in the derivation cohort
Characteristic
Derivation cohort (n = 469), Validation cohort (n = 231), frequency (%) frequency (%)
Factors
Ureter stone (n = 372)
Other diagnosis (n OR (95% CI) = 97)
Mean (SD) age (y) Male sex Race Asian White Black Hispanic Mixed
46.6 (14.1) 310 (66.0)
47.6 (14.4) 149 (64.5)
270 (72%) 249 (67%)
40 (41%) 46 (47%)
3.77 (2.37-6.00) b.01 2.24 (1.42-3.53) b.01
467 (99.9) 2 – – –
230 (99.9) 1 – – –
269 (72%) 99 (26%) 57 (15%) 311 (83%) 141 (38%)
22 (23%) 11 (11%) 7 (7%) 11 (11%) 13 (13%)
8.9 (5.26-15.1) 2.83 (1.45-5.53) 2.32 (1.03-5.28) 39.4 (19.9-78.2) 3.94 (2.12-7.35)
372 (79.2)
183 (79.2)
337 (91%)
65 (67%)
4.74 (2.74-8.20) b.01
43 (9.2)
24 (10.4)
Male sex Age b50 Duration of pain b6 h Nausea Vomiting Hematuria Previous stone history CRP b 0.5 (mg/dL)a BUN b26 (mg/dL) Cr b1.2 (mg/dL) WBC b12000 cells/mL
336 (90%)
95 (98%)
1.28 (0.25-6.46)
.30
347 (93%) 311 (83%)
95 (98%) 90 (92%)
0.29 (0.07-1.25) 0.40 (0.18-0.90)
.098 .026
Radiology findings Symptomatic ureteral stone Important alternative findings Disposition Admission or transfer
48 (10.2)
24 (10.4)
P
b.01 b.01 b.01 b.01 b.01
Abbreviations: Cr, creatinine; BUN, blood urea nitrogen. a Forty-five patients without CRP data were not included in analysis.
Please cite this article as: Kim B, et al, External validation of the STONE score and derivation of the modified STONE score, Am J Emerg Med (2016), http://dx.doi.org/10.1016/j.ajem.2016.05.061
4
B. Kim et al. / American Journal of Emergency Medicine xxx (2016) xxx–xxx
Table 4 Multivariate analysis of predictive factors in the derivation cohort Factors
Odds ratio (95% CI)
β-Coefficient
Score
Male sex Duration of pain b6 h Hematuria Previous stone history CRP b0.5a Total
3.71 (1.87-7.32)
1.31
3
5.39 (2.70-10.79) 25.2 (11.9-53.6) 2.82 (1.22-6.51) 3.29 (1.40-7.74)
1.68 3.23 1.04 1.19
3 6 2 2 0-16
a
Forty-five patients without CRP data were not included in analysis.
cohort mostly excludes other significant diagnoses. The AUROC of the STONE score in the external validation cohort was 0.92 (95% confidence interval [CI], 0.90-0.95). Assuming the patients in the high stone score group were positive for ureter stone and the other 2 groups were negative, the sensitivity of the STONE score was 0.56, and the specificity was 0.97. 3.2. Derivation and internal validation of the modified STONE score For the derivation and internal validation of the modified STONE score, the 700 enrolled patients were randomly divided into derivation (469 patients) and validation (231 patients) cohorts. Table 2 shows the baseline demographic findings and laboratory tests for each cohort. The
overall prevalence of ureter stones was not significantly different between 2 cohorts (79.2 vs 79.2). In the univariable analysis, the following 8 variables were significant factors: male sex, age younger than 50 years, duration of pain less than 6 hours, presence of nausea, presence of vomiting, presence of hematuria on microscopy, presence of previous history of ureter stone, C-reactive protein (CRP) less than 0.5 mg/dL; white blood cell (WBC) less than 12 000 cells/μL (Table 3). We subsequently used these 8 variables for the multivariable analysis. Finally, the following five variables were demonstrated as useful to predict ureter stones with the multivariable analysis: male sex, duration of pain less than 6 hours, presence of hematuria on microscopy, presence of previous history of ureter stone, and CRP less than 0.5 mg/dL. The β coefficients in this model were doubled and rounded to the nearest number. The computed numbers were assigned to the independent variables (Table 4). Seventy-one patients did not have CRP data, and 1 patient did not have WBC count. From the 71 patients, 45 patients were assigned to derivation cohorts, and 26 patients were assigned to validation cohorts. The score for CRP factor in the modified STONE score for those patients was counted as 0. The prevalence of ureter stones and other clinically important findings in the 3 modified STONE score groups of the derivation and validation cohorts are represented in Fig. 3. In both cohorts, more than 98% of the patients in the highscore group were diagnosed with ureter stones. In the moderate and low groups, the prevalence of ureter stones was decreased in both cohorts. The rates were 61.4% and 14.7% in the derivation cohort and
Fig. 3. A, Prevalence of ureteral stone by the modified STONE score category in derivation and validation cohort. B, Prevalence of clinically important alternative diagnoses by the modified STONE score category in derivation and validation cohort. Percentages at top of bars indicate prevalence of stone in each group. Values under bars indicate percentages of each group form overall enrolled patients.
Please cite this article as: Kim B, et al, External validation of the STONE score and derivation of the modified STONE score, Am J Emerg Med (2016), http://dx.doi.org/10.1016/j.ajem.2016.05.061
B. Kim et al. / American Journal of Emergency Medicine xxx (2016) xxx–xxx
5
Fig. 4. Comparison of ROC curve between STONE score and modified STONE score in the internal validation cohort.
58.5% and 19.3% in validation cohort, respectively. On the other hand, as the score decreased from the high group to the moderate and low groups, the prevalence of other clinically important findings increased. The rates were 1.9%, 16.0%, and 34.4% in derivation cohort and 0.7%, 24.5%, and 32.3% in the internal validation cohort, respectively. The AUROC of the modified STONE score was 0.94 (95% CI, 0.91-0.97) which was significantly higher than that of the STONE score in the internal validation cohort (Fig. 4). Assuming the high stone score group patients were positive for ureter stones and the other 2 groups were negative, the sensitivity of the modified STONE score was 0.80, and specificity was 0.99, which was statistically higher than that for STONE score in the same cohort (Pb .01 by McNemar test). 4. Discussion In this study, we evaluated the performance of the STONE score in a Korean population and confirmed that the high STONE score could mostly predict ureter stones with a low probability of other significant alternative findings. The overall prevalence of ureter stones in the study population was 79%, which is higher than that of the original Table 5 Types and frequency of acutely important alternative findings Important alternative findings
Case
Abscess, peritoneal Acute appendicitis Acute pyelonephritis Angiomyolipoma Acute cholecystitis Compression fracture of spine Diverticulitis Enterocolitis Fat necrosis GB stone Herpes zoster Hydrosalphynx Ileus Infective spondylitis Uterine myoma Newly detected malignancy Ovarian cyst Pelvic congestion disease Pelvic inflammatory disease Pneumonia Pulmonary thromboembolism Renal infarction Ulcerative colitis Ureteral stricture
1 2 9 1 1 3 4 3 1 3 3 1 2 1 1 8 2 1 3 7 1 3 1 5
STONE study (56%) and other previous studies, in which the overall prevalence of ureter stone in CT scan varied from 34% to 78% [15,16]. In this study, only 52.8% (945/1791) of the patients with flank pain had a CT compared to 42.5% to 77% in other countries [12,17,18]. During the study periods, our institution had no strict indication for a CT scan for flank pain patients. Instead, there was a campaign for reducing radiation hazards by reconsidering unnecessary imaging, including x-rays and CT scans. Many patients complaining of abdominal or flank pain but with relatively normal laboratory test and well-controlled pain were discharged without CT scan. This might explain the higher prevalence of urinary stones in this study. The purpose of the STONE scoring system was to reduce the radiation hazard. Thus, we validated the STONE score in an even more selected patient group, which further supports the usefulness of STONE score. Of the 4 patients with other significant alternative findings, 3 were diagnosed with acute pyelonephritis, and the other one was diagnosed with angiomyolipoma. In the original study, the rate of clinically important incidental findings in the high-score group was 1.6%. However, the exact diagnosis of those people was not known. The overall prevalence of the clinically significant findings in this study was higher than that of the original study (3.7%). The prevalence of incidental findings from CT scans for suspected ureter stone patients varies from 10.0% to 22.1% [16,19,20]. This difference might be a result of the different of study populations. In this study, the enrolled patients were from patients who visited the ED with flank pain regardless of a physician's suspicion of ureter stone. In the original study, only a “flank pain protocol” noncontrast CT was used, whereas in this external validation study, various CT protocols covering the abdomen and pelvis were used regardless of contrast use. This might make the incidental findings other than urinary stone detected more often. Alternative diagnoses were similar with the original study, including newly diagnosed malignancy; ovarian problems; and infectious diseases, such as diverticulitis, appendicitis, and pneumonia (Table 5) [15,16]. After development of the STONE score, Wang et al [21] published an external validation study of the STONE score. From the multiinstitutional study, the author proved that the STONE score was statistically associated with ureter stones and successfully predicted ureter stones. However, in the external validation study, the prevalence of ureter stones in the high group (72.7%) was lower in the original study (88.6%). The sensitivity of the high STONE score was 53%. The author concluded that a high STONE score was not likely sufficient to provide a clear course of action. Furthermore, development of the STONE score is needed. Wang et al performed a CT scan in all patients. This decreased the incidence of ureteral stones from the original and current studies (39% vs 56% vs 79%). We believe that this may have caused the different results.
Please cite this article as: Kim B, et al, External validation of the STONE score and derivation of the modified STONE score, Am J Emerg Med (2016), http://dx.doi.org/10.1016/j.ajem.2016.05.061
6
B. Kim et al. / American Journal of Emergency Medicine xxx (2016) xxx–xxx
After the external validation of the STONE score, we derived the modified STONE score without the nausea and vomiting variables. In the derivation of the modified STONE, we added the 2 variables, previous ureter stone history, and low CRP (b0.5 mg/dL). Half of ureter stones recur within 5 years. A low CRP might exclude other infectious conditions, such as acute pyelonephritis or renal abscess [22]. C-reactive protein could also be used as a predictor of spontaneous passage of ureter stones. Simple uncomplicated ureter stone patients might have low CRP levels [23,24]. The AUROC of the modified STONE and the sensitivity of the high score were significantly higher than those of the STONE score, which needs further multicenter external validation. 5. Limitations Our study has several limitations. First, this was a retrospective single center study. Therefore, biases intrinsic to such design could have influenced the results. Second, the study population had a much higher prevalence of ureter stone, which should be mentioned. Distinct from the original study, we did not frequently use so-called flank pain protocol CT scans. We selected a study population whose chief complaint was flank pain with a final clinical diagnosis combined with a CT scan that was obtained from CT scans of any protocols. External validation beyond Korean population and refining the STONE score from a more generous population would be needed. 6. Conclusion In this external validation study, we found that the STONE score was significantly associated with ureter stones. Adding CRP and the presence of previous ureter stone history, the sensitivity and specificity of the score improved for the diagnosis of ureter stones and other clinically significant findings. References [1] Johri N, Cooper B, Robertson W, Choong S, Rickards D, Unwin R. An update and practical guide to renal stone management. Nephron Clin Pract 2010;116:c159–71. [2] Andreeva IV, Kozlov SN, Korolev SV, Belikov AN, Grinev AV, Evstaf'ev VV, et al. Diagnostic and treatment patterns in management of male patients with nongonococcal urethritis: results of Russian multicentral cross-sectional study. Antibiot Khimioter 2012;57:32–40. [3] Turney BW, Reynard JM, Noble JG, Keoghane SR. Trends in urological stone disease. BJU Int 2012;109:1082–7. [4] Roudakova K, Monga M. The evolving epidemiology of stone disease. Indian J Urol 2014;30:44–8.
[5] Bae SR, Seong JM, Kim LY, Paick SH, Kim HG, Lho YS, et al. The epidemiology of renoureteral stone disease in Koreans: a nationwide population-based study. Urolithiasis 2014;42:109–14. [6] Moore CL, Bomann S, Daniels B, Luty S, Molinaro A, Singh D, et al. Derivation and validation of a clinical prediction rule for uncomplicated ureteral stone—the STONE score: retrospective and prospective observational cohort studies. BMJ 2014;348, g2191. [7] Fulgham PF, Assimos DG, Pearle MS, Preminger GM. Clinical effectiveness protocols for imaging in the management of ureteral calculous disease: AUA technology assessment. J Urol 2013;189:1203–13. [8] Ziemba JB, Matlaga BR. Guideline of guidelines: kidney stones. BJU Int 2015;116: 184–9. [9] Brenner DJ, Hall EJ. Computed tomography—an increasing source of radiation exposure. N Engl J Med 2007;357:2277–84. [10] Lee W. Current status of medical radiation exposure and regulation efforts. J Korean Med Assoc 2011;54:1248–52. [11] Smith-Bindman R. Ultrasonography vs. CT for suspected nephrolithiasis. N Engl J Med 2014;371:2531. [12] Westphalen AC, Hsia RY, Maselli JH, Wang R, Gonzales R. Radiological imaging of patients with suspected urinary tract stones: national trends, diagnoses, and predictors. Acad Emerg Med 2011;18:699–707. [13] Laupacis A, Sekar N, Stiell IG. Clinical prediction rules. A review and suggested modifications of methodological standards. JAMA 1997;277:488–94. [14] Moons KG, Harrell FE, Steyerberg EW. Should scoring rules be based on odds ratios or regression coefficients? J Clin Epidemiol 2002;55:1054–5. [15] Ather MH, Faizullah K, Achakzai I, Siwani R, Irani F. Alternate and incidental diagnoses on noncontrast-enhanced spiral computed tomography for acute flank pain. Urol J 2009;6:14–8. [16] Moore CL, Daniels B, Singh D, Luty S, Molinaro A. Prevalence and clinical importance of alternative causes of symptoms using a renal colic computed tomography protocol in patients with flank or back pain and absence of pyuria. Acad Emerg Med 2013; 20:470–8. [17] Ahmed F, Zafar AM, Khan N, Haider Z, Ather MH. A paradigm shift in imaging for renal colic—is it time to say good bye to an old trusted friend? Int J Surg 2010;8: 252–6. [18] Hyams ES, Korley FK, Pham JC, Matlaga BR. Trends in imaging use during the emergency department evaluation of flank pain. J Urol 2011;186:2270–4. [19] Samim M, Goss S, Luty S, Weinreb J, Moore C. Incidental findings on CT for suspected renal colic in emergency department patients: prevalence and types in 5,383 consecutive examinations. J Am Coll Radiol 2015;12:63–9. [20] Lokken RP, Sadow CA, Silverman SG. Diagnostic yield of CT urography in the evaluation of young adults with hematuria. AJR Am J Roentgenol 2012;198: 609–15. [21] Wang RC, Rodriguez RM, Moghadassi M, Noble V, Bailitz J, Mallin M, et al. External validation of the STONE score, a clinical prediction rule for ureteral stone: an observational multi-institutional study. Ann Emerg Med 2015. [22] van der Starre WE, Zunder SM, Vollaard AM, van Nieuwkoop C, Stalenhoef JE, Delfos NM, et al. Prognostic value of pro-adrenomedullin, procalcitonin and C-reactive protein in predicting outcome of febrile urinary tract infection. Clin Microbiol Infect 2014;20:1048–54. [23] Ozcan C, Aydogdu O, Senocak C, Damar E, Eraslan A, Oztuna D, et al. Predictive factors for spontaneous stone passage and the potential role of serum C-reactive protein in patients with 4 to 10 mm distal ureteral stones: a prospective clinical study. J Urol 2015;194:1009–13. [24] Park CH, Ha JY, Park CH, Kim CI, Kim KS, Kim BH. Relationship between spontaneous passage rates of ureteral stones less than 8 mm and serum C-reactive protein levels and neutrophil percentages. Korean J Urol 2013;54:615–8.
Please cite this article as: Kim B, et al, External validation of the STONE score and derivation of the modified STONE score, Am J Emerg Med (2016), http://dx.doi.org/10.1016/j.ajem.2016.05.061
Contents lists available at ScienceDirect
American Journal of Emergency Medicine journal homepage: www.elsevier.com/locate/ajem
Original Contribution
External validation of the STONE score and derivation of the modified STONE score☆,☆☆ Byunghyun Kim, MD 1, Kyuseok Kim, MD, PhD ⁎, Joonghee Kim, MD, You Hwan Jo, MD, PhD, Jae Hyuk Lee, MD, PhD, Ji Eun Hwang, MD, Jongdae Park, MD Department of Emergency Medicine, Seoul National University, Bundang Hospital, Sungnam-si, Gyeonggi-do 463-707, Republic of Korea
a r t i c l e
i n f o
Article history: Received 12 April 2016 Received in revised form 20 May 2016 Accepted 21 May 2016 Available online xxxx
a b s t r a c t Objective: The STONE score is a clinical prediction rule for the presence of uncomplicated ureter stones with a low probability of acutely important alternative findings. This study performed an external validation of the STONE score, focusing on the Korean population, and a derivation of the modified STONE score for better specificity and sensitivity. Methods: We retrospectively reviewed medical records of patients complaining of flank pain at a single emergency department from January 2013 to December 2014. Patients were categorized into 3 groups according to their STONE score. The prevalence of ureter stones and other alternative findings were calculated in each group. We derived a modified STONE score based on a multivariable analysis and performed an interval validation. Results: From the 700 patients included in the analysis, 555 patients (79%) had a ureter stone. The area under the receiver operating characteristic curve of the STONE score was 0.92. The sensitivity of the high stone score was 0.56. In the modified STONE score, nausea, vomiting, and racial predictors were substituted by C-reactive protein and previous stone history. The area under the receiver operating characteristic curve and sensitivity of the modified STONE score in the internal validation group significantly increased to 0.94 and 0.80, respectively. Conclusion: The STONE score can be used to predict a ureter stone with a low probability of other alternative findings. The modified STONE score might increase the diagnostic performance in suspicious urinary stone cases. Key points: We performed external validation of the STONE score and derivation of the modified STONE score. This scoring system could help the clinicians with radiation reducing decision making. © 2016 Elsevier Inc. All rights reserved.
1. Introduction Ureter stones featuring acute flank pain and hematuria are commonly encountered in the emergency department (ED) and account for more than 1 million annual ED visits in the United States [1,2]. The worldwide prevalence and incidence of ureter stones vary from 2% to 20% depending on the country or region, and the incidence rate is increasing [3,4]. The annual incidence rate of ureter stones in Korea is 457 per 100 000. This rate is higher than that previously reported in Japan but lower than in Western countries [5].
☆ Prior presentations: None. ☆☆ Funding sources: None. ⁎ Corresponding author at: Department of Emergency Medicine, Seoul National University, Bundang Hospital, 300 Gumi-dong, Bundang-gu, Sungnam-si, Gyeonggi-do 463-707, Republic of Korea. Tel.: +82 31 787 7572; fax: +82 31 787 4055. E-mail addresses: [email protected] (B. Kim), [email protected] (K. Kim). 1 Department of Emergency Medicine, Seoul National University, Bundang Hospital, 300 Gumi-dong, Bundang-gu, Sungnam-si, Gyeonggi-do, 463-707, Republic of Korea. Tel.: +82 31 415 5739; fax: +82 31,787 4055.
Recently, Moore et al [6] derived the STONE score. It is a clinical prediction rule to evaluate the risk of ureter stones and clinically important alternative diagnoses for patients with suspected ureter stones. In the STONE score, 5 factors were found to be the most predictive of ureter stones: male sex, short duration of pain, nonblack race, presence of nausea or vomiting, and microscopic hematuria. The score ranges from 0 to 13. Patients are classified into low (0-5), moderated (6-9), and high (10-13) score groups. From those results, the authors propose using the STONE score to limit exposure to radiation and overutilization of imaging. Computed tomography (CT) is the preferred imaging method when patients initially present with suspected ureter stones in the United States and Europe [7,8]. As the use of a CT scan increases, the risk of a radiation hazard problem comes into question [9]. In Korea, relatively cheaper prices of CT scans compared to Western countries make CT scans even more frequently used and may lead to overuse and to radiation overexposure [10]. Increased CT scan use leads to increased diagnoses of ureter stones and other alternative clinical findings but does not lead to improvement in patient outcomes [11,12]. This study was performed to validate the STONE score in a single medical center in Korea. We also aimed to derive a modified STONE score and internally validate it.
http://dx.doi.org/10.1016/j.ajem.2016.05.061 0735-6757/© 2016 Elsevier Inc. All rights reserved.
Please cite this article as: Kim B, et al, External validation of the STONE score and derivation of the modified STONE score, Am J Emerg Med (2016), http://dx.doi.org/10.1016/j.ajem.2016.05.061
2
B. Kim et al. / American Journal of Emergency Medicine xxx (2016) xxx–xxx
Fig. 1. Flowchart of the enrolled patients. UA, urine analysis.
2. Materials and methods
2.2. Statistical analysis
2.1. Study design and selection of patients
2.2.1. External validation of the original STONE score For external validation of the original STONE score, the same number of points was assigned to each predictor as in original study (male sex, 2 points; duration of pain b6 hours, 3 points; duration of pain 6 to 24 hours, 1 point; nonblack race, 3 points; nausea alone, 1 point; vomiting, 2 points; hematuria on urine dipstick testing, 3 points). The patients were categorized into 3 groups according to their STONE score: low(0-5), moderate- (6-9), and high-score (10-13) groups. The prevalence of ureteral stones and important alternative diagnoses were determined for each group. We analyzed a receiver operating characteristic (ROC) curve and the area under the ROC curve (AUROC) to compare with data from the original study.
We retrospectively reviewed the medical records of adult patients older than 17 years who visited our ED for flank pain between January 2013 and December 2014. Eligible patients had a final clinical diagnosis combined with a CT scan using a CT scan. Exclusion criteria were the same as those in the original article: objective fever (N 37.7°C), presence of leukocytes in urine microscopy, known renal disease (including creatinine N1.5 mg/dL), history of trauma, or known active malignancy. The study facility was a 950-bed urban tertiary academic hospital with an annual ED census of approximately 80 000. The institutional review board approved the analysis and issued a waiver of consent. Table 1 Baseline characteristics of the study population and original study Characteristic
This validation cohort (n = 700), frequency (%)
Original validation cohort (n = 491), frequency (%)
Mean (SD) age (y) Male sex Race Asian White Black Hispanic Mixed
47.0 (14.3)
46 (15)
559 (65.6)
273 (56)
697 (99.9) 3 – – –
– 411 (84) 57 (12) – 23 (5)
Radiology findings Symptomatic 555 (79.3) ureteral stone 67 (9.6) Important alternative findings Disposition Admission or transfer
73 (10.4)
274 (56) 18 (3.7)
51 (11)
2.2.2. Derivation and internal validation of the modified STONE score For the derivation and internal validation of the modified STONE score, we randomly divided the enrolled patients into a derivation group consisting of 66% of the patients and an internal validation group with the remaining 33% of patients. After dichotomizing the variables, we investigated epidemiological factors and laboratory results using a univariable analysis in the derivation group. Subsequently, a multivariable analysis was performed to find independent risk factors using candidate variables with a factor significance of Pb .05 in the univariable analysis. The prediction model for ureter stones was developed using a regression coefficient–based scoring method. We found predictive factors in a multivariable analysis and assigned a weighted value to each factor using β coefficients that reflect predictive power [13,14]. The β coefficients in the multivariable logistic regression model were multiplied by 2 and then approximated to the nearest natural number. Patients were categorized into 3 groups according to the modified STONE score: low- (0-4), moderate- (5-9), and high-score (10-16) groups. The prevalence of ureteral stones and important alternative diagnoses were determined for each group. Finally, we analyzed an ROC curve and AUROC to investigate the performance of the modified STONE score.
Please cite this article as: Kim B, et al, External validation of the STONE score and derivation of the modified STONE score, Am J Emerg Med (2016), http://dx.doi.org/10.1016/j.ajem.2016.05.061
B. Kim et al. / American Journal of Emergency Medicine xxx (2016) xxx–xxx
3
Fig. 2. Prevalence of ureteral stone by STONE score category in this external validation study and original validation study. Percentages at top of bars indicate prevalence of stone in each group. Values under bars indicate percentages of each group form overall enrolled patients.
The remaining cohort of 33% of patients was used to validate the prediction model internally. We computed the number of patients and proportion according to the risk score assigned to each predictive factor. Patients in the validation cohort were also assigned to 3 groups based on their scores. The prevalence of ureteral stones and important alternative diagnoses were determined for each group. We also analyzed an ROC curve and computed the AUROC to quantify the performance of the modified STONE score.
3. Results 3.1. External validation of the STONE score
2.2.3. Comparison of diagnostic performance between the STONE score and modified STONE score An AUROC was used to compare the diagnostic performance between the 2 score systems. We performed a McNemar test to compare the sensitivities of the 2 scoring systems. All analyses were conducted using STATA (version 13; StataCorp, College Station, TX).
During the study period, a total of 1791 patients visited our ED complaining of flank pain. Of these patients, 945 had a final clinical diagnosis combined with a CT scan on the same day. In total, 109 of the 945 enrolled patients were excluded because of fever, pyuria, urinalysis data loss, azotemia, known malignancy, and trauma history (Fig. 1). Finally, 700 patients were included in the final analysis, of which 555 patients were diagnosed with a ureter stone (79%) and 67 had other clinically significant findings (10%). Table 1 shows the baseline characteristics of the validation cohorts compared with that of the original study. The prevalence of ureter stones in the 3 score groups compared with that of the original validation study is represented in Fig. 2. In the high STONE score group, 309 patients (98.7%) were diagnosed with ureter stone, indicating that the high STONE score in this external validation
Table 2 Characteristics of derivation and internal validation cohorts
Table 3 Univariable analysis of clinical factors in patients in the derivation cohort
Characteristic
Derivation cohort (n = 469), Validation cohort (n = 231), frequency (%) frequency (%)
Factors
Ureter stone (n = 372)
Other diagnosis (n OR (95% CI) = 97)
Mean (SD) age (y) Male sex Race Asian White Black Hispanic Mixed
46.6 (14.1) 310 (66.0)
47.6 (14.4) 149 (64.5)
270 (72%) 249 (67%)
40 (41%) 46 (47%)
3.77 (2.37-6.00) b.01 2.24 (1.42-3.53) b.01
467 (99.9) 2 – – –
230 (99.9) 1 – – –
269 (72%) 99 (26%) 57 (15%) 311 (83%) 141 (38%)
22 (23%) 11 (11%) 7 (7%) 11 (11%) 13 (13%)
8.9 (5.26-15.1) 2.83 (1.45-5.53) 2.32 (1.03-5.28) 39.4 (19.9-78.2) 3.94 (2.12-7.35)
372 (79.2)
183 (79.2)
337 (91%)
65 (67%)
4.74 (2.74-8.20) b.01
43 (9.2)
24 (10.4)
Male sex Age b50 Duration of pain b6 h Nausea Vomiting Hematuria Previous stone history CRP b 0.5 (mg/dL)a BUN b26 (mg/dL) Cr b1.2 (mg/dL) WBC b12000 cells/mL
336 (90%)
95 (98%)
1.28 (0.25-6.46)
.30
347 (93%) 311 (83%)
95 (98%) 90 (92%)
0.29 (0.07-1.25) 0.40 (0.18-0.90)
.098 .026
Radiology findings Symptomatic ureteral stone Important alternative findings Disposition Admission or transfer
48 (10.2)
24 (10.4)
P
b.01 b.01 b.01 b.01 b.01
Abbreviations: Cr, creatinine; BUN, blood urea nitrogen. a Forty-five patients without CRP data were not included in analysis.
Please cite this article as: Kim B, et al, External validation of the STONE score and derivation of the modified STONE score, Am J Emerg Med (2016), http://dx.doi.org/10.1016/j.ajem.2016.05.061
4
B. Kim et al. / American Journal of Emergency Medicine xxx (2016) xxx–xxx
Table 4 Multivariate analysis of predictive factors in the derivation cohort Factors
Odds ratio (95% CI)
β-Coefficient
Score
Male sex Duration of pain b6 h Hematuria Previous stone history CRP b0.5a Total
3.71 (1.87-7.32)
1.31
3
5.39 (2.70-10.79) 25.2 (11.9-53.6) 2.82 (1.22-6.51) 3.29 (1.40-7.74)
1.68 3.23 1.04 1.19
3 6 2 2 0-16
a
Forty-five patients without CRP data were not included in analysis.
cohort mostly excludes other significant diagnoses. The AUROC of the STONE score in the external validation cohort was 0.92 (95% confidence interval [CI], 0.90-0.95). Assuming the patients in the high stone score group were positive for ureter stone and the other 2 groups were negative, the sensitivity of the STONE score was 0.56, and the specificity was 0.97. 3.2. Derivation and internal validation of the modified STONE score For the derivation and internal validation of the modified STONE score, the 700 enrolled patients were randomly divided into derivation (469 patients) and validation (231 patients) cohorts. Table 2 shows the baseline demographic findings and laboratory tests for each cohort. The
overall prevalence of ureter stones was not significantly different between 2 cohorts (79.2 vs 79.2). In the univariable analysis, the following 8 variables were significant factors: male sex, age younger than 50 years, duration of pain less than 6 hours, presence of nausea, presence of vomiting, presence of hematuria on microscopy, presence of previous history of ureter stone, C-reactive protein (CRP) less than 0.5 mg/dL; white blood cell (WBC) less than 12 000 cells/μL (Table 3). We subsequently used these 8 variables for the multivariable analysis. Finally, the following five variables were demonstrated as useful to predict ureter stones with the multivariable analysis: male sex, duration of pain less than 6 hours, presence of hematuria on microscopy, presence of previous history of ureter stone, and CRP less than 0.5 mg/dL. The β coefficients in this model were doubled and rounded to the nearest number. The computed numbers were assigned to the independent variables (Table 4). Seventy-one patients did not have CRP data, and 1 patient did not have WBC count. From the 71 patients, 45 patients were assigned to derivation cohorts, and 26 patients were assigned to validation cohorts. The score for CRP factor in the modified STONE score for those patients was counted as 0. The prevalence of ureter stones and other clinically important findings in the 3 modified STONE score groups of the derivation and validation cohorts are represented in Fig. 3. In both cohorts, more than 98% of the patients in the highscore group were diagnosed with ureter stones. In the moderate and low groups, the prevalence of ureter stones was decreased in both cohorts. The rates were 61.4% and 14.7% in the derivation cohort and
Fig. 3. A, Prevalence of ureteral stone by the modified STONE score category in derivation and validation cohort. B, Prevalence of clinically important alternative diagnoses by the modified STONE score category in derivation and validation cohort. Percentages at top of bars indicate prevalence of stone in each group. Values under bars indicate percentages of each group form overall enrolled patients.
Please cite this article as: Kim B, et al, External validation of the STONE score and derivation of the modified STONE score, Am J Emerg Med (2016), http://dx.doi.org/10.1016/j.ajem.2016.05.061
B. Kim et al. / American Journal of Emergency Medicine xxx (2016) xxx–xxx
5
Fig. 4. Comparison of ROC curve between STONE score and modified STONE score in the internal validation cohort.
58.5% and 19.3% in validation cohort, respectively. On the other hand, as the score decreased from the high group to the moderate and low groups, the prevalence of other clinically important findings increased. The rates were 1.9%, 16.0%, and 34.4% in derivation cohort and 0.7%, 24.5%, and 32.3% in the internal validation cohort, respectively. The AUROC of the modified STONE score was 0.94 (95% CI, 0.91-0.97) which was significantly higher than that of the STONE score in the internal validation cohort (Fig. 4). Assuming the high stone score group patients were positive for ureter stones and the other 2 groups were negative, the sensitivity of the modified STONE score was 0.80, and specificity was 0.99, which was statistically higher than that for STONE score in the same cohort (Pb .01 by McNemar test). 4. Discussion In this study, we evaluated the performance of the STONE score in a Korean population and confirmed that the high STONE score could mostly predict ureter stones with a low probability of other significant alternative findings. The overall prevalence of ureter stones in the study population was 79%, which is higher than that of the original Table 5 Types and frequency of acutely important alternative findings Important alternative findings
Case
Abscess, peritoneal Acute appendicitis Acute pyelonephritis Angiomyolipoma Acute cholecystitis Compression fracture of spine Diverticulitis Enterocolitis Fat necrosis GB stone Herpes zoster Hydrosalphynx Ileus Infective spondylitis Uterine myoma Newly detected malignancy Ovarian cyst Pelvic congestion disease Pelvic inflammatory disease Pneumonia Pulmonary thromboembolism Renal infarction Ulcerative colitis Ureteral stricture
1 2 9 1 1 3 4 3 1 3 3 1 2 1 1 8 2 1 3 7 1 3 1 5
STONE study (56%) and other previous studies, in which the overall prevalence of ureter stone in CT scan varied from 34% to 78% [15,16]. In this study, only 52.8% (945/1791) of the patients with flank pain had a CT compared to 42.5% to 77% in other countries [12,17,18]. During the study periods, our institution had no strict indication for a CT scan for flank pain patients. Instead, there was a campaign for reducing radiation hazards by reconsidering unnecessary imaging, including x-rays and CT scans. Many patients complaining of abdominal or flank pain but with relatively normal laboratory test and well-controlled pain were discharged without CT scan. This might explain the higher prevalence of urinary stones in this study. The purpose of the STONE scoring system was to reduce the radiation hazard. Thus, we validated the STONE score in an even more selected patient group, which further supports the usefulness of STONE score. Of the 4 patients with other significant alternative findings, 3 were diagnosed with acute pyelonephritis, and the other one was diagnosed with angiomyolipoma. In the original study, the rate of clinically important incidental findings in the high-score group was 1.6%. However, the exact diagnosis of those people was not known. The overall prevalence of the clinically significant findings in this study was higher than that of the original study (3.7%). The prevalence of incidental findings from CT scans for suspected ureter stone patients varies from 10.0% to 22.1% [16,19,20]. This difference might be a result of the different of study populations. In this study, the enrolled patients were from patients who visited the ED with flank pain regardless of a physician's suspicion of ureter stone. In the original study, only a “flank pain protocol” noncontrast CT was used, whereas in this external validation study, various CT protocols covering the abdomen and pelvis were used regardless of contrast use. This might make the incidental findings other than urinary stone detected more often. Alternative diagnoses were similar with the original study, including newly diagnosed malignancy; ovarian problems; and infectious diseases, such as diverticulitis, appendicitis, and pneumonia (Table 5) [15,16]. After development of the STONE score, Wang et al [21] published an external validation study of the STONE score. From the multiinstitutional study, the author proved that the STONE score was statistically associated with ureter stones and successfully predicted ureter stones. However, in the external validation study, the prevalence of ureter stones in the high group (72.7%) was lower in the original study (88.6%). The sensitivity of the high STONE score was 53%. The author concluded that a high STONE score was not likely sufficient to provide a clear course of action. Furthermore, development of the STONE score is needed. Wang et al performed a CT scan in all patients. This decreased the incidence of ureteral stones from the original and current studies (39% vs 56% vs 79%). We believe that this may have caused the different results.
Please cite this article as: Kim B, et al, External validation of the STONE score and derivation of the modified STONE score, Am J Emerg Med (2016), http://dx.doi.org/10.1016/j.ajem.2016.05.061
6
B. Kim et al. / American Journal of Emergency Medicine xxx (2016) xxx–xxx
After the external validation of the STONE score, we derived the modified STONE score without the nausea and vomiting variables. In the derivation of the modified STONE, we added the 2 variables, previous ureter stone history, and low CRP (b0.5 mg/dL). Half of ureter stones recur within 5 years. A low CRP might exclude other infectious conditions, such as acute pyelonephritis or renal abscess [22]. C-reactive protein could also be used as a predictor of spontaneous passage of ureter stones. Simple uncomplicated ureter stone patients might have low CRP levels [23,24]. The AUROC of the modified STONE and the sensitivity of the high score were significantly higher than those of the STONE score, which needs further multicenter external validation. 5. Limitations Our study has several limitations. First, this was a retrospective single center study. Therefore, biases intrinsic to such design could have influenced the results. Second, the study population had a much higher prevalence of ureter stone, which should be mentioned. Distinct from the original study, we did not frequently use so-called flank pain protocol CT scans. We selected a study population whose chief complaint was flank pain with a final clinical diagnosis combined with a CT scan that was obtained from CT scans of any protocols. External validation beyond Korean population and refining the STONE score from a more generous population would be needed. 6. Conclusion In this external validation study, we found that the STONE score was significantly associated with ureter stones. Adding CRP and the presence of previous ureter stone history, the sensitivity and specificity of the score improved for the diagnosis of ureter stones and other clinically significant findings. References [1] Johri N, Cooper B, Robertson W, Choong S, Rickards D, Unwin R. An update and practical guide to renal stone management. Nephron Clin Pract 2010;116:c159–71. [2] Andreeva IV, Kozlov SN, Korolev SV, Belikov AN, Grinev AV, Evstaf'ev VV, et al. Diagnostic and treatment patterns in management of male patients with nongonococcal urethritis: results of Russian multicentral cross-sectional study. Antibiot Khimioter 2012;57:32–40. [3] Turney BW, Reynard JM, Noble JG, Keoghane SR. Trends in urological stone disease. BJU Int 2012;109:1082–7. [4] Roudakova K, Monga M. The evolving epidemiology of stone disease. Indian J Urol 2014;30:44–8.
[5] Bae SR, Seong JM, Kim LY, Paick SH, Kim HG, Lho YS, et al. The epidemiology of renoureteral stone disease in Koreans: a nationwide population-based study. Urolithiasis 2014;42:109–14. [6] Moore CL, Bomann S, Daniels B, Luty S, Molinaro A, Singh D, et al. Derivation and validation of a clinical prediction rule for uncomplicated ureteral stone—the STONE score: retrospective and prospective observational cohort studies. BMJ 2014;348, g2191. [7] Fulgham PF, Assimos DG, Pearle MS, Preminger GM. Clinical effectiveness protocols for imaging in the management of ureteral calculous disease: AUA technology assessment. J Urol 2013;189:1203–13. [8] Ziemba JB, Matlaga BR. Guideline of guidelines: kidney stones. BJU Int 2015;116: 184–9. [9] Brenner DJ, Hall EJ. Computed tomography—an increasing source of radiation exposure. N Engl J Med 2007;357:2277–84. [10] Lee W. Current status of medical radiation exposure and regulation efforts. J Korean Med Assoc 2011;54:1248–52. [11] Smith-Bindman R. Ultrasonography vs. CT for suspected nephrolithiasis. N Engl J Med 2014;371:2531. [12] Westphalen AC, Hsia RY, Maselli JH, Wang R, Gonzales R. Radiological imaging of patients with suspected urinary tract stones: national trends, diagnoses, and predictors. Acad Emerg Med 2011;18:699–707. [13] Laupacis A, Sekar N, Stiell IG. Clinical prediction rules. A review and suggested modifications of methodological standards. JAMA 1997;277:488–94. [14] Moons KG, Harrell FE, Steyerberg EW. Should scoring rules be based on odds ratios or regression coefficients? J Clin Epidemiol 2002;55:1054–5. [15] Ather MH, Faizullah K, Achakzai I, Siwani R, Irani F. Alternate and incidental diagnoses on noncontrast-enhanced spiral computed tomography for acute flank pain. Urol J 2009;6:14–8. [16] Moore CL, Daniels B, Singh D, Luty S, Molinaro A. Prevalence and clinical importance of alternative causes of symptoms using a renal colic computed tomography protocol in patients with flank or back pain and absence of pyuria. Acad Emerg Med 2013; 20:470–8. [17] Ahmed F, Zafar AM, Khan N, Haider Z, Ather MH. A paradigm shift in imaging for renal colic—is it time to say good bye to an old trusted friend? Int J Surg 2010;8: 252–6. [18] Hyams ES, Korley FK, Pham JC, Matlaga BR. Trends in imaging use during the emergency department evaluation of flank pain. J Urol 2011;186:2270–4. [19] Samim M, Goss S, Luty S, Weinreb J, Moore C. Incidental findings on CT for suspected renal colic in emergency department patients: prevalence and types in 5,383 consecutive examinations. J Am Coll Radiol 2015;12:63–9. [20] Lokken RP, Sadow CA, Silverman SG. Diagnostic yield of CT urography in the evaluation of young adults with hematuria. AJR Am J Roentgenol 2012;198: 609–15. [21] Wang RC, Rodriguez RM, Moghadassi M, Noble V, Bailitz J, Mallin M, et al. External validation of the STONE score, a clinical prediction rule for ureteral stone: an observational multi-institutional study. Ann Emerg Med 2015. [22] van der Starre WE, Zunder SM, Vollaard AM, van Nieuwkoop C, Stalenhoef JE, Delfos NM, et al. Prognostic value of pro-adrenomedullin, procalcitonin and C-reactive protein in predicting outcome of febrile urinary tract infection. Clin Microbiol Infect 2014;20:1048–54. [23] Ozcan C, Aydogdu O, Senocak C, Damar E, Eraslan A, Oztuna D, et al. Predictive factors for spontaneous stone passage and the potential role of serum C-reactive protein in patients with 4 to 10 mm distal ureteral stones: a prospective clinical study. J Urol 2015;194:1009–13. [24] Park CH, Ha JY, Park CH, Kim CI, Kim KS, Kim BH. Relationship between spontaneous passage rates of ureteral stones less than 8 mm and serum C-reactive protein levels and neutrophil percentages. Korean J Urol 2013;54:615–8.
Please cite this article as: Kim B, et al, External validation of the STONE score and derivation of the modified STONE score, Am J Emerg Med (2016), http://dx.doi.org/10.1016/j.ajem.2016.05.061