- Email: [email protected]
Development of a nomogram to predict small bowel obstruction using point-of-care ultrasound in the emergency department
YAJEM-158638; No of Pages 5 American Journal of Emergency Medicine xxx (xxxx) xxx
Contents lists available at ScienceDirect
American Journal of Emergency Medicine journal homepage: www.elsevier.com/locate/ajem
Development of a nomogram to predict small bowel obstruction using point-of-care ultrasound in the emergency department☆ Hamid Shokoohi, MD, MPH a,b,⁎, Keith S. Boniface, MD b, Michael A. Loesche, MD, Ph.D. c, Nicole M. Duggan, MD c, Jordan B. King, PharmD, M.S. d a
Department of Emergency Medicine, Massachusetts General Hospital, Harvard Medical School, Boston, MA, United States of America Department of Emergency Medicine, The George Washington University Medical Center, Washington, DC, United States of America Department of Emergency Medicine, Massachusetts General Hospital, Harvard Affiliated Emergency Medicine Residency Program, United States of America d Department of Pharmacy, Kaiser Permanente Colorado Region, Aurora, CO & Department of Population Health Sciences, School of Medicine, University of Utah, Salt Lake City, UT, United States of America b c
a r t i c l e
i n f o
Article history: Received 10 August 2019 Received in revised form 3 December 2019 Accepted 4 December 2019 Available online xxxx
a b s t r a c t Objective: Early diagnostic prediction in patients with small bowel obstruction (SBO) can improve time to definitive management and disposition in the emergency department. We sought to develop a nomogram to leverage point-of-care ultrasound (POCUS) and maximize accuracy of prediction of SBO diagnosis. Methods: Using data from a prospective cohort of 125 patients with suspected SBO who were evaluated with POCUS in the ED, we developed a nomogram integrating age, gender, comorbidities, prior abdominal surgery, physician's pre-test probability, and POCUS findings to determine post-test risk of SBO. The primary outcome was to develop a nomogram to allow calculating output probabilities for predictive models using POCUS findings. The discriminative ability of the nomogram was tested using a C-statistics, calibration plots, and receiver operating characteristic curves. Results: The derivation cohort included 125 patients with a median age of 54 years who underwent POCUS for a suspected SBO. One-fourth of the patients (25.6% [32/125]) had SBO. Using a retrospective stepwise selection of clinically important variables with the POCUS results, the final nomogram incorporated four relevant factors for the prediction of SBO: small bowel diameter (odds ratio [OR] per 1 mm increase, 1.10; 95% CI, 1.03–1.17; P = 0.001), positive free intraperitoneal fluid between bowel loops (OR, 8.19; 95% CI, 2.62–25.62; P b 0.001), clinician's moderate (OR, 5.94; 95% CI, 0.83–42.57; P = 0.08) or high pretest probability (OR, 11.26; 95% CI, 1.44–88.25; P = 0.02), and patient age (OR per 1 year increase, 1.03; 95% CI, 1.00 to1.07; P = 0.08).The discriminative ability and calibration of the nomogram revealed good predictive ability as indicated by the C-statistic of 0.89 for the SBO diagnosis. Conclusion: A unique nomogram incorporating patient age, physician pretest probability of SBO, and POCUS measurements of small bowel diameter and the presence of free intraperitoneal fluid between bowel loops was developed to accurately predict the diagnosis of SBO in the emergency department. The nomogram should be externally validated in a novel cohort of patients at risk for SBO to better assess predictability and generalizability. © 2019 Published by Elsevier Inc.
1. Background Small bowel obstruction (SBO) is a common presentation to the Emergency Department (ED), representing 1/6 of all surgical admissions to the hospital [1]. In up to two-thirds of patients with uncomplicated SBO, medical management remains the most appropriate approach [1].
☆ Study Institution: The study was conducted at the George Washington University Hospital in Washington DC. ⁎ Corresponding author at: 326 Cambridge Street, Suite 410, Boston, MA, United States of America. E-mail address: [email protected] (H. Shokoohi).
However, in other cases, SBO is still a common indication for emergency laparotomy and laparoscopy [1,2]. Ideally, use of a validated decision tool for predicting SBO would improve patient care by establishing an expedited surgical consultation, and prioritizing further imaging. Diagnosing SBO using bedside tools such as point-of-care ultrasound (POCUS) and decision rules would expedite clinical decision-making by emergency physicians and minimize time to definitive management and disposition. SBO management decisions are often based on cross-sectional imaging such as computed tomography (CT) scans which can introduce a significant time delay in patients' ED course. While CT scan is currently is the gold standard imaging modality for diagnosing SBO, use of POCUS
https://doi.org/10.1016/j.ajem.2019.12.010 0735-6757/© 2019 Published by Elsevier Inc.
Please cite this article as: H. Shokoohi, K.S. Boniface, M.A. Loesche, et al., Development of a nomogram to predict small bowel obstruction using point-of-care ultrasound in the e..., American Journal of Emergency Medicine, https://doi.org/10.1016/j.ajem.2019.12.010
2
H. Shokoohi et al. / American Journal of Emergency Medicine xxx (xxxx) xxx
is gaining more popularity in the EDs. Historically, when SBO was suspected, abdominal X-rays were performed as an initial diagnostic test, followed by CT if the X-rays were indeterminate. However, abdominal X-rays have poor test characteristics, including a non-actionable positive likelihood ratio (+LR) of only 1.64, whereas the +LR of ultrasound (US) by radiologists is 14.1, and of bedside POCUS performed by clinicians of 9.55–27.5 [3,4]. The negative LR of US by radiologists is 0.13 and of bedside POCUS is 0.04–0.08 [3,4]. These demonstrated test characteristics of US significantly change pre- to post-test probability, and can lead to either ruling in or ruling out disease. Given the demonstrated lower diagnostic accuracy for POCUS however, a predictive tool incorporating both clinical characteristics and POCUS findings could ideally improve the diagnostic precision over POCUS alone. Currently, models that incorporate patient-centered factors to predict the diagnosis of SBO are unavailable. Developing a nomogram that quantifies the combined contribution of several easily assessable demographic, clinical, and POCUS findings may provide a predicted probability for SBO diagnosis and help to rapidly identify those patients in need of early surgical consultation. In this study, we sought to develop a nomogram incorporating both clinical features and POCUS findings to enhance the ability to predict the diagnosis of SBO in the emergency department. 2. Methods 2.1. Patient population and data collection Subjects were derived from a prospective, single-institution database consisting of 125 patients with suspected SBO who underwent POCUS evaluation in the ED, between October 2014 and January 2017, in an urban, academic medical center with an emergency medicine residency program, an ultrasound fellowship program, and 73,000 annual ED visits. The methods and main results of this study have been previously published [5]. Briefly, patients were eligible for inclusion if all three of the following criteria were met: 1.) they presented with a chief complaint of abdominal pain, vomiting, or constipation; 2.) an attending physician rated their pre-test probability for SBO as greater than zero or very low (on a scale of zero, very low, low, medium, high), and 3.) the patient received an abdominal CT scan as part of their diagnostic course. Patients who were medically unstable, pregnant, unable to provide consent, or for whom a CT was not completed were excluded. The subjects' demographic and clinical data including age, sex, history of SBO, comorbidities, prior abdominal surgery, and clinician's pre-test probability of SBO was surveyed prior to POCUS or CT scan results. Patient data was abstracted from the subjects' electronic medical records by trained research assistants (RA) who were blinded to the US results. Patients' history of SBO and comorbidities were abstracted from the patients' digital medical records. The primary outcome was to develop a model using POCUS and clinical features to calculate post-test probability of SBO.
a linear probe when able given patient body habitus. Bowel obstruction was defined sonographically as a maximal small bowel diameter of N2.5 cm (outer wall to outer wall) and abnormal peristalsis (defined as either increased to-and-fro, or decreased/sluggish peristalsis). In addition, the relative degree of peristalsis, the bowel wall thickness, and the presence of free intraperitoneal fluid between bowel loops were also noted [6]. CT scans were performed with intravenous contrast and without oral contrast. The final diagnosis of SBO was determined by the attending read of the CT scan, as well as intra-operative findings for patients who underwent laparoscopy or laparotomy. The institutional review board at our institution approved this study. Informed consent was obtained for each patient.
2.3. Statistical analysis We developed and internally validated a nomogram to predict the diagnosis of SBO as a function of demographics, clinician's pretest probability, and POCUS findings. Variables evaluated as potential predictors of SBO included age, gender, SBO risk factors at baseline (e.g., history of SBO and prior abdominal surgery), physician determined pre-test likelihood of SBO, and POCUS findings of bowel diameter, bowel wall thickness, the presence of free intraperitoneal fluid between bowel loops, and type of peristalsis observed. The model was derived from the complete patient cohort. The associations of relevant clinical and demographical variables and POCUS findings with SBO diagnosis were assessed using multivariable logistic regression models. To improve the parsimony of the multivariable regression models, we employed a backward stepwise variable selection procedure removing variables with p-values N 0.2. Selected variables were incorporated in the nomogram to predict the probability of SBO diagnosis. The performance of the nomogram was evaluated using the C-statistic by Harrell et al. [7] The C-statistic estimates the probability of concordance between predicted and observed outcomes in rank order and is equivalent to the area under the receiver operating characteristic (ROC) curve. A Cstatistic of 0.5 indicates the absence of discrimination, whereas a Cstatistic of 1.0 indicates perfect separation of patients with different outcomes [8]. Internal validation was performed with a bootstrapping procedure in 200 samples drawn with replacement from the original sample, to determine the expected bias-corrected C-statistic from the model which accounts for overfitting [9,10]. Calibration was assessed with the Hosmer-Lemeshow goodness-of-fit test by grouping patients into quintiles of nomogram-predicted probabilities and then comparing the observed versus predicted proportions of SBOs within each group. All analyses were performed using STATA version 15.0 (StataCorp LP, College Station, TX). The nomogram was calculated using the “nomolog” Stata module [11].
2.2. Image acquisition and interpretation
3. Results
The POCUS findings and a dichotomous SBO diagnosis were documented by the RAs in real time. Sonographers were residents, fellows, and attendings who had previous training in emergency ultrasonography, with a minimum of a 16-hour course and a 2-week rotation in emergency ultrasonography. In addition, sonographers enrolling in the study had focused training in scanning for SBO including a 20-minute didactic presentation, and performed 5 precepted scans with the lead investigators on patients without bowel obstruction before enrolling for the study. Enrolling sonographers were blinded to clinical data including pretest probability of SBO. Treating providers and patients were blinded to the US results. Measurements were made with the patient lying supine, using predominantly a phased-array probe for most measurements, and
3.1. Characteristics of study subjects Patient characteristics in the original cohort is shown in Table 1. Overall, 125 patients met eligibility criteria and were enrolled in the study. The average age was 54 years, 46% were male, and 19% had a prior history of SBO. Thirty-two patients (25.6%) were ultimately diagnosed with an SBO. Patients with SBO were over three times more likely to have had a prior SBO and twice as likely to have an abdominal tumor, than patients without SBO (Table 1). Physician prediction of the likelihood of SBO prior to any imaging correlated with the incidence of SBO: 8.3% of the low pretest probability group, 22% of the moderate probability group, and 52% of the high probability group were ultimately diagnosed with SBO.
Please cite this article as: H. Shokoohi, K.S. Boniface, M.A. Loesche, et al., Development of a nomogram to predict small bowel obstruction using point-of-care ultrasound in the e..., American Journal of Emergency Medicine, https://doi.org/10.1016/j.ajem.2019.12.010
H. Shokoohi et al. / American Journal of Emergency Medicine xxx (xxxx) xxx
3
Table 1 Patient characteristics.* Characteristic
SBO present (n = 32)
No SBO present (n = 93)
Age, years Male sex Presenting symptoms Abdominal pain Vomiting Constipation SBO risk factors Prior SBO Abdominal surgery Abdominal hernia Abdominal tumor Inflammatory bowel disease Physician experience Attending/Fellow Resident POCUS results Bowel thickness b 3 mm Bowel thickness ≥ 3 mm Abnormal peristalsis Intraperitoneal fluid - No Intraperitoneal fluid - Yes
*57.1 15
(14.2) (46.9)
*53.3 43
(15.6) (46.2)
25 15 5
(78.1) (46.9) (15.6)
80 37 25
(86.0) (39.8) (26.9)
13 15 3 5 2
(40.6) (46.9) (9.4) (15.6) (6.3)
11 33 11 7 10
(11.8) (35.5) (11.8) (7.5) (10.8)
25 7
(78.1) (21.9)
63 30
(67.7) (32.3)
8 13 30 12 19
(38.1) (61.9) (96.8) (38.7) (61.3)
23 19 70 79 10
(54.8) (45.2) (78.7) (88.8) (11.2)
*Values are mean (SD), otherwise values are reported as No. (%). SBO = Small bowel obstruction.
3.2. Nomogram for SBO prediction Using a retrospective stepwise selection of clinically important variables with the POCUS results, the final nomogram incorporated four relevant factors for the prediction of SBO: small bowel diameter (odds ratio [OR] per 1 mm increase, 1.10; 95% CI, 1.03–1.17; P = 0.001), positive free intraperitoneal fluid between bowel loops (OR, 8.19; 95% CI, 2.62–25.62; P b 0.001), clinician's moderate (OR, 5.94; 95% CI, 0.83–42.57; P = 0.08) or high pretest probability (OR, 11.26; 95% CI, 1.44–88.25; P = 0.02), and patient age (OR per 1 year increase, 1.03; 95% CI, 1.00 to1.07; P = 0.08) (Table 2). The discriminative ability and calibration of the nomogram revealed good predictive ability as indicated by the C-statistic of 0.89 for SBO diagnosis. When performance was assessed, the final model correctly classified patients with an SBO as being high-risk (predicted likelihood of N50%) with a sensitivity of 59% and specificity of 91%. The C-statistic of the model overall was 0.89. This is compared to C-statistics of 0.70 from physician pre-test likelihood alone and 0.84 for bowel diameter alone (Fig. 1). Following the internal validation via bootstrapping, the expected bias-corrected C-statistic decreased to 0.86, accounting for 3% overfitting. The calibration plot for SBO probability consistently matched the ideal 45-degree reference line for patients within each quintile of predicted risk (Fig. 2). The discriminative ability and calibration of the nomograms revealed good predictive ability as indicated (Fig. 3).
Fig. 1. ROC curve of multivariable nomogram model (green line) compared with either bowel diameter (red line) or physician pre-test probability (blue line) alone. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)
“Instructions for use: Step 1. in the upper box locate the patient's age on the “Age” axis. Draw a line straight down to the “Points” axis to determine how many points the patient receives for age. Repeat this process for “Free fluid,” “Physician determined pre-test likelihood,” and “Bowel diameter, mm.” Step 2. Sum the points received for each predictor and locate this sum on the “Total points” axis in the lower box. Draw a line straight down to determine the patient's probability that a SBO is present. Case: Consider a patient age 60 (~3.5 points), a moderate pre-test likelihood of SBO (~3 points), with free fluid present (~3.5 points), and a bowel diameter of 35 mm (~6 points). The sum of all predictors is ~16 points and corresponds to a 75% estimated probability that an SBO is present.”
4. Discussion In this study, we developed a nomogram to numerically predict the diagnosis of SBO based on patients' demographics, emergency physician's
Table 2 Multivariable logistic regression model outcomes for predicting SBO in the nomogram derivation.
Age, per 1 year Bowel diameter, per 1 mm Low pre-test SBO likelihood Moderate pre-test SBO likelihood High pre-test SBO likelihood Free fluid present Intercept SBO, Small bowel obstruction.
Beta
95% confidence interval
0.03 0.10 0 (ref) 1.78 2.42 2.10 −8.17
−0.00 to 0.07 0.04 to 0.16 −0.19 to 3.75 0.36 to 4.48 0.96 to 3.24 −11.92 to −4.12
p-Value 0.086 0.001 0.076 0.021 b0.001 b0.001
Fig. 2. Calibration plot of the final nomogram model. The 45-degree straight line corresponds to the line of perfect calibration on which model predicted probabilities coincide with the observed proportions.
Please cite this article as: H. Shokoohi, K.S. Boniface, M.A. Loesche, et al., Development of a nomogram to predict small bowel obstruction using point-of-care ultrasound in the e..., American Journal of Emergency Medicine, https://doi.org/10.1016/j.ajem.2019.12.010
4
H. Shokoohi et al. / American Journal of Emergency Medicine xxx (xxxx) xxx
Fig. 3. A. This nomogram can be used to predict post-test likelihood of small bowel obstruction given predictors of clinical history and POCUS results. Each of the four variables (pre-test probability, age, presence of free intraperitoneal fluid, and bowel diameter in mm) has points associated with the variable – the sum of the four-point values then correlates with the estimated probability of SBO as shown at the bottom of the nomogram. B. Sample case demonstrating use of the Nomogram which predicts approximately 75% probability that an SBO is present.
pretest probability of the diagnoses, and POCUS findings. Our proposed nomogram demonstrated good discriminative ability, with a C-statistic of 0.89 for predicting SBO. We included and tested several factors felt to be potentially relevant to predicting the outcomes of SBO in these patients, and in the end bowel diameter, the presence of free intraperitoneal fluid, patient age, and clinicians' pre-test probability were the factors with significant association which are included in the final nomogram. This nomogram can be used to aid in the integration of ultrasound results into clinical decision-making at the bedside for presentations which may be uncertain. For instance, if a patient has a low pre-test probability and has a negative ultrasound (e.g. 24 mm bowel diameter
with no intraperitoneal fluid), a young patient would have a b1% posttest probability of SBO. With the same diameter small bowel on ultrasound, a low-risk 80-year-old patient with positive intraperitoneal free fluid would still have a N20% post-test probability of SBO. Given that a significant number of SBOs can be managed conservatively without need for operative intervention, overall early use of this predictive tool in patients' clinical course could ideally expedite care and ultimate disposition from the emergency department regardless of need for surgery. This tool may be applied as a nomogram as presented here, or can be converted in to an electronic calculator to facilitate rapid and accurate use in an ED setting. This nomogram can also
Please cite this article as: H. Shokoohi, K.S. Boniface, M.A. Loesche, et al., Development of a nomogram to predict small bowel obstruction using point-of-care ultrasound in the e..., American Journal of Emergency Medicine, https://doi.org/10.1016/j.ajem.2019.12.010
H. Shokoohi et al. / American Journal of Emergency Medicine xxx (xxxx) xxx
be used for teaching residents and students about statistics and model development as well. Regression analyses are often difficult to understand and do not lend themselves to an intuitive interpretation. Most trainees are visual learners and providing a graphical representation of our model can help demystify this approach. From future studies applying this tool to larger external datasets, we hope to incorporate artificial intelligence and machine learning approaches to not only identify patients with SBO, but perhaps even identify patients for whom bypassing standard workup and proceeding directly to the operating room for surgical management would be beneficial. Additionally, we predict application of our nomogram will improve diagnostic efficiency and ultimately decreased overall ED length of stay for patients with SBO, thus we intend to investigate these metrics in future studies. The test performance characteristics for POCUS in identifying SBO in the single center study from which data for this study was derived were not as good as those found in the meta-analyses of prior studies [3-5]. This may be a result of any number of patient and sonographer characteristics, the limited sample size, or perhaps the convenience sampling methods used in the original study. Additionally, in terms of pretest probability, clinicians were not allowed to enter “indeterminate” as their value, and similarly clinician confidence in their pretest probability for SBO was not assessed which may have led to an increased error rate. As this nomogram is based upon this single center data set, it is clear that the accuracy of POCUS affects the accuracy of the nomogram. Our study is among the first to include POCUS findings along with easily accessible patient-related factors and clinicians' assessment of pretest probability of disease in a nomogram to assist in rapid prediction of SBO diagnosis in the ED. We were not able to identify similar studies for the purpose of comparison. Unlike many other studies which are based on the surveillance, and retrospective databases, the performance of this nomogram was developed from a prospective cohort and was internally validated via bootstrapping. It should next be externally validated in a new patient population at risk for SBO.
5
may be inconsistent with that of an experienced attending physician, or for physicians who may require additional training in POCUS. Finally, our nomogram was internally validated using bootstrapping validation, however, for small data sets, internal validation by bootstrapping may not be sufficient to be indicative of the model's performance in future patient populations [12]. Future studies are needed to prospectively and externally validate the proposed nomogram before implementing it in clinical practice. 5. Conclusion We developed a unique nomogram to incorporate patient age, POCUS measurement of small bowel diameter, intraperitoneal free fluid between bowel loops, and physicians' pretest probability to accurately predict the diagnosis of SBO. This nomogram can ideally be used in the ED for more rapid and accurate prediction of the diagnosis in cases presenting with possible SBO, adjusting for different demographic, clinical, and US findings. The nomogram should next be externally validated to further assess the generalizability and applicability of this novel tool. Financial support This is a non-funded study, with no compensation or honoraria for participation, consulting or conducting the study. Resources required for this project was provided by institutional departmental funds at the George Washington University, Department of Emergency Medicine, with NO particular budgeting allocated to this project. Declaration of competing interest The authors do NOT have a financial interest or relationship to disclose regarding this research project. References
4.1. Limitations The present study had several limitations. The nomogram is developed based on data from a single center study with a small sample size. As such, some analyses may have been limited. We have tested and included most significant variables but given the limited sample size, the data on certain factors such as prior abdominal surgery and other comorbidities may have had too small an impact due to the small numbers to be included in the analysis; therefore, their effect or potential incorporation in the nomogram could not be assessed. We did not include physical examination findings such as abdominal distension or abnormal bowel sounds as these findings may vary based on experience of the assessing clinicians. We also did not include laboratory values such as serum lactate measurements as waiting for laboratory values to result would have introduced a time delay in the ability to apply the nomogram to a patient, and one of our goals was to create a measure that facilitates rapid prediction than can be achieved without waiting on additional imaging or blood test results. Diagnostic accuracy of our nomogram relies in part on the initial clinical gestalt of the physician to assess pretest probability, as well as the physician's ability to correctly identify measures such as bowel diameter and presence of free fluid on POCUS. We maintain that the fairly minimal training provided to sonographers in this study is sufficient to develop expertise in identifying SBO on POCUS. However, it is possible our measure may demonstrate decreased accuracy in the hands of less experienced clinicians such as those for whom pre-test probability
[1] Maung AA, Johnson DC, Piper GL, et al. Evaluation and management of small-bowel obstruction: an eastern association for the surgery of trauma practice management guideline. J Trauma Acute Care Surg 2012;73(5):s362–9. [2] Paulson E, Thompson W. Review of small bowel obstruction: the diagnosis and when to worry. Radiol J 2015;275:332–42. [3] Taylor MR, Lalani N. Adult small bowel obstruction. Acad Emerg Med 2013;20: 528–44. [4] Gottlieb M, Peksa GD, Pandurangadu AV, Nakitende D, Takhar S, Seethala RR. Utilization of ultrasound for the evaluation of small bowel obstruction: a systematic review and meta-analysis. Am J Emerg Med 2018;36(2):234–42 Feb. [5] Boniface KS, King JB, LeSaux MA, Haciski SC, Shokoohi H. Diagnostic accuracy and time-saving effects of point-of-care ultrasonography in patients with small bowel obstruction: a prospective study. Ann Emerg Med 2019 Jul 23. [pii: S0196-0644 (19)30443-3]. [6] Pourmand A, Dimbil U, Drake A, Shokoohi H. The accuracy of point-of-care ultrasound in detecting small bowel obstruction in emergency department. Emergency Medicine International 2018;2018(1):1–5. [7] Harrell FE, Lee KL, Califf RM, Pryor DB, Rosati RA. Regression modelling strategies for improved prognostic prediction. Stat Med 1984;3:143–52. [8] Hosmer DW, Lemeshow S. Applied logistic regression. 2nd. New York (NY: John Wiley & Sons; 2000; 162. [9] Harrell Jr FE, Lee KL, Mark DB. Tutorial in biostatistics: multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med 1996;15:361–87. [10] Steyerberg EQ, Harrell FE, Borsboom GJ, Eijkemans MJ, Vergouwe Y, Habbema JD. Internal validation of predictive models: efficiency of some procedures for logistic regression analysis. J Clin Epidemiol 2001;54:774–81. [11] Zlotnik A, Abraira V. A general-purpose nomogram generator for predictive logistic regression models. The Stata Journal 2015;15(2):537–46. [12] Dwivedi AK, Mallawaarachchi I, Alvarado LA. Analysis of sample size studies using nonparametric bootstrap test with pooled resampling method. Stat Method 2017; 36(14):2187–205.
Please cite this article as: H. Shokoohi, K.S. Boniface, M.A. Loesche, et al., Development of a nomogram to predict small bowel obstruction using point-of-care ultrasound in the e..., American Journal of Emergency Medicine, https://doi.org/10.1016/j.ajem.2019.12.010
Contents lists available at ScienceDirect
American Journal of Emergency Medicine journal homepage: www.elsevier.com/locate/ajem
Development of a nomogram to predict small bowel obstruction using point-of-care ultrasound in the emergency department☆ Hamid Shokoohi, MD, MPH a,b,⁎, Keith S. Boniface, MD b, Michael A. Loesche, MD, Ph.D. c, Nicole M. Duggan, MD c, Jordan B. King, PharmD, M.S. d a
Department of Emergency Medicine, Massachusetts General Hospital, Harvard Medical School, Boston, MA, United States of America Department of Emergency Medicine, The George Washington University Medical Center, Washington, DC, United States of America Department of Emergency Medicine, Massachusetts General Hospital, Harvard Affiliated Emergency Medicine Residency Program, United States of America d Department of Pharmacy, Kaiser Permanente Colorado Region, Aurora, CO & Department of Population Health Sciences, School of Medicine, University of Utah, Salt Lake City, UT, United States of America b c
a r t i c l e
i n f o
Article history: Received 10 August 2019 Received in revised form 3 December 2019 Accepted 4 December 2019 Available online xxxx
a b s t r a c t Objective: Early diagnostic prediction in patients with small bowel obstruction (SBO) can improve time to definitive management and disposition in the emergency department. We sought to develop a nomogram to leverage point-of-care ultrasound (POCUS) and maximize accuracy of prediction of SBO diagnosis. Methods: Using data from a prospective cohort of 125 patients with suspected SBO who were evaluated with POCUS in the ED, we developed a nomogram integrating age, gender, comorbidities, prior abdominal surgery, physician's pre-test probability, and POCUS findings to determine post-test risk of SBO. The primary outcome was to develop a nomogram to allow calculating output probabilities for predictive models using POCUS findings. The discriminative ability of the nomogram was tested using a C-statistics, calibration plots, and receiver operating characteristic curves. Results: The derivation cohort included 125 patients with a median age of 54 years who underwent POCUS for a suspected SBO. One-fourth of the patients (25.6% [32/125]) had SBO. Using a retrospective stepwise selection of clinically important variables with the POCUS results, the final nomogram incorporated four relevant factors for the prediction of SBO: small bowel diameter (odds ratio [OR] per 1 mm increase, 1.10; 95% CI, 1.03–1.17; P = 0.001), positive free intraperitoneal fluid between bowel loops (OR, 8.19; 95% CI, 2.62–25.62; P b 0.001), clinician's moderate (OR, 5.94; 95% CI, 0.83–42.57; P = 0.08) or high pretest probability (OR, 11.26; 95% CI, 1.44–88.25; P = 0.02), and patient age (OR per 1 year increase, 1.03; 95% CI, 1.00 to1.07; P = 0.08).The discriminative ability and calibration of the nomogram revealed good predictive ability as indicated by the C-statistic of 0.89 for the SBO diagnosis. Conclusion: A unique nomogram incorporating patient age, physician pretest probability of SBO, and POCUS measurements of small bowel diameter and the presence of free intraperitoneal fluid between bowel loops was developed to accurately predict the diagnosis of SBO in the emergency department. The nomogram should be externally validated in a novel cohort of patients at risk for SBO to better assess predictability and generalizability. © 2019 Published by Elsevier Inc.
1. Background Small bowel obstruction (SBO) is a common presentation to the Emergency Department (ED), representing 1/6 of all surgical admissions to the hospital [1]. In up to two-thirds of patients with uncomplicated SBO, medical management remains the most appropriate approach [1].
☆ Study Institution: The study was conducted at the George Washington University Hospital in Washington DC. ⁎ Corresponding author at: 326 Cambridge Street, Suite 410, Boston, MA, United States of America. E-mail address: [email protected] (H. Shokoohi).
However, in other cases, SBO is still a common indication for emergency laparotomy and laparoscopy [1,2]. Ideally, use of a validated decision tool for predicting SBO would improve patient care by establishing an expedited surgical consultation, and prioritizing further imaging. Diagnosing SBO using bedside tools such as point-of-care ultrasound (POCUS) and decision rules would expedite clinical decision-making by emergency physicians and minimize time to definitive management and disposition. SBO management decisions are often based on cross-sectional imaging such as computed tomography (CT) scans which can introduce a significant time delay in patients' ED course. While CT scan is currently is the gold standard imaging modality for diagnosing SBO, use of POCUS
https://doi.org/10.1016/j.ajem.2019.12.010 0735-6757/© 2019 Published by Elsevier Inc.
Please cite this article as: H. Shokoohi, K.S. Boniface, M.A. Loesche, et al., Development of a nomogram to predict small bowel obstruction using point-of-care ultrasound in the e..., American Journal of Emergency Medicine, https://doi.org/10.1016/j.ajem.2019.12.010
2
H. Shokoohi et al. / American Journal of Emergency Medicine xxx (xxxx) xxx
is gaining more popularity in the EDs. Historically, when SBO was suspected, abdominal X-rays were performed as an initial diagnostic test, followed by CT if the X-rays were indeterminate. However, abdominal X-rays have poor test characteristics, including a non-actionable positive likelihood ratio (+LR) of only 1.64, whereas the +LR of ultrasound (US) by radiologists is 14.1, and of bedside POCUS performed by clinicians of 9.55–27.5 [3,4]. The negative LR of US by radiologists is 0.13 and of bedside POCUS is 0.04–0.08 [3,4]. These demonstrated test characteristics of US significantly change pre- to post-test probability, and can lead to either ruling in or ruling out disease. Given the demonstrated lower diagnostic accuracy for POCUS however, a predictive tool incorporating both clinical characteristics and POCUS findings could ideally improve the diagnostic precision over POCUS alone. Currently, models that incorporate patient-centered factors to predict the diagnosis of SBO are unavailable. Developing a nomogram that quantifies the combined contribution of several easily assessable demographic, clinical, and POCUS findings may provide a predicted probability for SBO diagnosis and help to rapidly identify those patients in need of early surgical consultation. In this study, we sought to develop a nomogram incorporating both clinical features and POCUS findings to enhance the ability to predict the diagnosis of SBO in the emergency department. 2. Methods 2.1. Patient population and data collection Subjects were derived from a prospective, single-institution database consisting of 125 patients with suspected SBO who underwent POCUS evaluation in the ED, between October 2014 and January 2017, in an urban, academic medical center with an emergency medicine residency program, an ultrasound fellowship program, and 73,000 annual ED visits. The methods and main results of this study have been previously published [5]. Briefly, patients were eligible for inclusion if all three of the following criteria were met: 1.) they presented with a chief complaint of abdominal pain, vomiting, or constipation; 2.) an attending physician rated their pre-test probability for SBO as greater than zero or very low (on a scale of zero, very low, low, medium, high), and 3.) the patient received an abdominal CT scan as part of their diagnostic course. Patients who were medically unstable, pregnant, unable to provide consent, or for whom a CT was not completed were excluded. The subjects' demographic and clinical data including age, sex, history of SBO, comorbidities, prior abdominal surgery, and clinician's pre-test probability of SBO was surveyed prior to POCUS or CT scan results. Patient data was abstracted from the subjects' electronic medical records by trained research assistants (RA) who were blinded to the US results. Patients' history of SBO and comorbidities were abstracted from the patients' digital medical records. The primary outcome was to develop a model using POCUS and clinical features to calculate post-test probability of SBO.
a linear probe when able given patient body habitus. Bowel obstruction was defined sonographically as a maximal small bowel diameter of N2.5 cm (outer wall to outer wall) and abnormal peristalsis (defined as either increased to-and-fro, or decreased/sluggish peristalsis). In addition, the relative degree of peristalsis, the bowel wall thickness, and the presence of free intraperitoneal fluid between bowel loops were also noted [6]. CT scans were performed with intravenous contrast and without oral contrast. The final diagnosis of SBO was determined by the attending read of the CT scan, as well as intra-operative findings for patients who underwent laparoscopy or laparotomy. The institutional review board at our institution approved this study. Informed consent was obtained for each patient.
2.3. Statistical analysis We developed and internally validated a nomogram to predict the diagnosis of SBO as a function of demographics, clinician's pretest probability, and POCUS findings. Variables evaluated as potential predictors of SBO included age, gender, SBO risk factors at baseline (e.g., history of SBO and prior abdominal surgery), physician determined pre-test likelihood of SBO, and POCUS findings of bowel diameter, bowel wall thickness, the presence of free intraperitoneal fluid between bowel loops, and type of peristalsis observed. The model was derived from the complete patient cohort. The associations of relevant clinical and demographical variables and POCUS findings with SBO diagnosis were assessed using multivariable logistic regression models. To improve the parsimony of the multivariable regression models, we employed a backward stepwise variable selection procedure removing variables with p-values N 0.2. Selected variables were incorporated in the nomogram to predict the probability of SBO diagnosis. The performance of the nomogram was evaluated using the C-statistic by Harrell et al. [7] The C-statistic estimates the probability of concordance between predicted and observed outcomes in rank order and is equivalent to the area under the receiver operating characteristic (ROC) curve. A Cstatistic of 0.5 indicates the absence of discrimination, whereas a Cstatistic of 1.0 indicates perfect separation of patients with different outcomes [8]. Internal validation was performed with a bootstrapping procedure in 200 samples drawn with replacement from the original sample, to determine the expected bias-corrected C-statistic from the model which accounts for overfitting [9,10]. Calibration was assessed with the Hosmer-Lemeshow goodness-of-fit test by grouping patients into quintiles of nomogram-predicted probabilities and then comparing the observed versus predicted proportions of SBOs within each group. All analyses were performed using STATA version 15.0 (StataCorp LP, College Station, TX). The nomogram was calculated using the “nomolog” Stata module [11].
2.2. Image acquisition and interpretation
3. Results
The POCUS findings and a dichotomous SBO diagnosis were documented by the RAs in real time. Sonographers were residents, fellows, and attendings who had previous training in emergency ultrasonography, with a minimum of a 16-hour course and a 2-week rotation in emergency ultrasonography. In addition, sonographers enrolling in the study had focused training in scanning for SBO including a 20-minute didactic presentation, and performed 5 precepted scans with the lead investigators on patients without bowel obstruction before enrolling for the study. Enrolling sonographers were blinded to clinical data including pretest probability of SBO. Treating providers and patients were blinded to the US results. Measurements were made with the patient lying supine, using predominantly a phased-array probe for most measurements, and
3.1. Characteristics of study subjects Patient characteristics in the original cohort is shown in Table 1. Overall, 125 patients met eligibility criteria and were enrolled in the study. The average age was 54 years, 46% were male, and 19% had a prior history of SBO. Thirty-two patients (25.6%) were ultimately diagnosed with an SBO. Patients with SBO were over three times more likely to have had a prior SBO and twice as likely to have an abdominal tumor, than patients without SBO (Table 1). Physician prediction of the likelihood of SBO prior to any imaging correlated with the incidence of SBO: 8.3% of the low pretest probability group, 22% of the moderate probability group, and 52% of the high probability group were ultimately diagnosed with SBO.
Please cite this article as: H. Shokoohi, K.S. Boniface, M.A. Loesche, et al., Development of a nomogram to predict small bowel obstruction using point-of-care ultrasound in the e..., American Journal of Emergency Medicine, https://doi.org/10.1016/j.ajem.2019.12.010
H. Shokoohi et al. / American Journal of Emergency Medicine xxx (xxxx) xxx
3
Table 1 Patient characteristics.* Characteristic
SBO present (n = 32)
No SBO present (n = 93)
Age, years Male sex Presenting symptoms Abdominal pain Vomiting Constipation SBO risk factors Prior SBO Abdominal surgery Abdominal hernia Abdominal tumor Inflammatory bowel disease Physician experience Attending/Fellow Resident POCUS results Bowel thickness b 3 mm Bowel thickness ≥ 3 mm Abnormal peristalsis Intraperitoneal fluid - No Intraperitoneal fluid - Yes
*57.1 15
(14.2) (46.9)
*53.3 43
(15.6) (46.2)
25 15 5
(78.1) (46.9) (15.6)
80 37 25
(86.0) (39.8) (26.9)
13 15 3 5 2
(40.6) (46.9) (9.4) (15.6) (6.3)
11 33 11 7 10
(11.8) (35.5) (11.8) (7.5) (10.8)
25 7
(78.1) (21.9)
63 30
(67.7) (32.3)
8 13 30 12 19
(38.1) (61.9) (96.8) (38.7) (61.3)
23 19 70 79 10
(54.8) (45.2) (78.7) (88.8) (11.2)
*Values are mean (SD), otherwise values are reported as No. (%). SBO = Small bowel obstruction.
3.2. Nomogram for SBO prediction Using a retrospective stepwise selection of clinically important variables with the POCUS results, the final nomogram incorporated four relevant factors for the prediction of SBO: small bowel diameter (odds ratio [OR] per 1 mm increase, 1.10; 95% CI, 1.03–1.17; P = 0.001), positive free intraperitoneal fluid between bowel loops (OR, 8.19; 95% CI, 2.62–25.62; P b 0.001), clinician's moderate (OR, 5.94; 95% CI, 0.83–42.57; P = 0.08) or high pretest probability (OR, 11.26; 95% CI, 1.44–88.25; P = 0.02), and patient age (OR per 1 year increase, 1.03; 95% CI, 1.00 to1.07; P = 0.08) (Table 2). The discriminative ability and calibration of the nomogram revealed good predictive ability as indicated by the C-statistic of 0.89 for SBO diagnosis. When performance was assessed, the final model correctly classified patients with an SBO as being high-risk (predicted likelihood of N50%) with a sensitivity of 59% and specificity of 91%. The C-statistic of the model overall was 0.89. This is compared to C-statistics of 0.70 from physician pre-test likelihood alone and 0.84 for bowel diameter alone (Fig. 1). Following the internal validation via bootstrapping, the expected bias-corrected C-statistic decreased to 0.86, accounting for 3% overfitting. The calibration plot for SBO probability consistently matched the ideal 45-degree reference line for patients within each quintile of predicted risk (Fig. 2). The discriminative ability and calibration of the nomograms revealed good predictive ability as indicated (Fig. 3).
Fig. 1. ROC curve of multivariable nomogram model (green line) compared with either bowel diameter (red line) or physician pre-test probability (blue line) alone. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)
“Instructions for use: Step 1. in the upper box locate the patient's age on the “Age” axis. Draw a line straight down to the “Points” axis to determine how many points the patient receives for age. Repeat this process for “Free fluid,” “Physician determined pre-test likelihood,” and “Bowel diameter, mm.” Step 2. Sum the points received for each predictor and locate this sum on the “Total points” axis in the lower box. Draw a line straight down to determine the patient's probability that a SBO is present. Case: Consider a patient age 60 (~3.5 points), a moderate pre-test likelihood of SBO (~3 points), with free fluid present (~3.5 points), and a bowel diameter of 35 mm (~6 points). The sum of all predictors is ~16 points and corresponds to a 75% estimated probability that an SBO is present.”
4. Discussion In this study, we developed a nomogram to numerically predict the diagnosis of SBO based on patients' demographics, emergency physician's
Table 2 Multivariable logistic regression model outcomes for predicting SBO in the nomogram derivation.
Age, per 1 year Bowel diameter, per 1 mm Low pre-test SBO likelihood Moderate pre-test SBO likelihood High pre-test SBO likelihood Free fluid present Intercept SBO, Small bowel obstruction.
Beta
95% confidence interval
0.03 0.10 0 (ref) 1.78 2.42 2.10 −8.17
−0.00 to 0.07 0.04 to 0.16 −0.19 to 3.75 0.36 to 4.48 0.96 to 3.24 −11.92 to −4.12
p-Value 0.086 0.001 0.076 0.021 b0.001 b0.001
Fig. 2. Calibration plot of the final nomogram model. The 45-degree straight line corresponds to the line of perfect calibration on which model predicted probabilities coincide with the observed proportions.
Please cite this article as: H. Shokoohi, K.S. Boniface, M.A. Loesche, et al., Development of a nomogram to predict small bowel obstruction using point-of-care ultrasound in the e..., American Journal of Emergency Medicine, https://doi.org/10.1016/j.ajem.2019.12.010
4
H. Shokoohi et al. / American Journal of Emergency Medicine xxx (xxxx) xxx
Fig. 3. A. This nomogram can be used to predict post-test likelihood of small bowel obstruction given predictors of clinical history and POCUS results. Each of the four variables (pre-test probability, age, presence of free intraperitoneal fluid, and bowel diameter in mm) has points associated with the variable – the sum of the four-point values then correlates with the estimated probability of SBO as shown at the bottom of the nomogram. B. Sample case demonstrating use of the Nomogram which predicts approximately 75% probability that an SBO is present.
pretest probability of the diagnoses, and POCUS findings. Our proposed nomogram demonstrated good discriminative ability, with a C-statistic of 0.89 for predicting SBO. We included and tested several factors felt to be potentially relevant to predicting the outcomes of SBO in these patients, and in the end bowel diameter, the presence of free intraperitoneal fluid, patient age, and clinicians' pre-test probability were the factors with significant association which are included in the final nomogram. This nomogram can be used to aid in the integration of ultrasound results into clinical decision-making at the bedside for presentations which may be uncertain. For instance, if a patient has a low pre-test probability and has a negative ultrasound (e.g. 24 mm bowel diameter
with no intraperitoneal fluid), a young patient would have a b1% posttest probability of SBO. With the same diameter small bowel on ultrasound, a low-risk 80-year-old patient with positive intraperitoneal free fluid would still have a N20% post-test probability of SBO. Given that a significant number of SBOs can be managed conservatively without need for operative intervention, overall early use of this predictive tool in patients' clinical course could ideally expedite care and ultimate disposition from the emergency department regardless of need for surgery. This tool may be applied as a nomogram as presented here, or can be converted in to an electronic calculator to facilitate rapid and accurate use in an ED setting. This nomogram can also
Please cite this article as: H. Shokoohi, K.S. Boniface, M.A. Loesche, et al., Development of a nomogram to predict small bowel obstruction using point-of-care ultrasound in the e..., American Journal of Emergency Medicine, https://doi.org/10.1016/j.ajem.2019.12.010
H. Shokoohi et al. / American Journal of Emergency Medicine xxx (xxxx) xxx
be used for teaching residents and students about statistics and model development as well. Regression analyses are often difficult to understand and do not lend themselves to an intuitive interpretation. Most trainees are visual learners and providing a graphical representation of our model can help demystify this approach. From future studies applying this tool to larger external datasets, we hope to incorporate artificial intelligence and machine learning approaches to not only identify patients with SBO, but perhaps even identify patients for whom bypassing standard workup and proceeding directly to the operating room for surgical management would be beneficial. Additionally, we predict application of our nomogram will improve diagnostic efficiency and ultimately decreased overall ED length of stay for patients with SBO, thus we intend to investigate these metrics in future studies. The test performance characteristics for POCUS in identifying SBO in the single center study from which data for this study was derived were not as good as those found in the meta-analyses of prior studies [3-5]. This may be a result of any number of patient and sonographer characteristics, the limited sample size, or perhaps the convenience sampling methods used in the original study. Additionally, in terms of pretest probability, clinicians were not allowed to enter “indeterminate” as their value, and similarly clinician confidence in their pretest probability for SBO was not assessed which may have led to an increased error rate. As this nomogram is based upon this single center data set, it is clear that the accuracy of POCUS affects the accuracy of the nomogram. Our study is among the first to include POCUS findings along with easily accessible patient-related factors and clinicians' assessment of pretest probability of disease in a nomogram to assist in rapid prediction of SBO diagnosis in the ED. We were not able to identify similar studies for the purpose of comparison. Unlike many other studies which are based on the surveillance, and retrospective databases, the performance of this nomogram was developed from a prospective cohort and was internally validated via bootstrapping. It should next be externally validated in a new patient population at risk for SBO.
5
may be inconsistent with that of an experienced attending physician, or for physicians who may require additional training in POCUS. Finally, our nomogram was internally validated using bootstrapping validation, however, for small data sets, internal validation by bootstrapping may not be sufficient to be indicative of the model's performance in future patient populations [12]. Future studies are needed to prospectively and externally validate the proposed nomogram before implementing it in clinical practice. 5. Conclusion We developed a unique nomogram to incorporate patient age, POCUS measurement of small bowel diameter, intraperitoneal free fluid between bowel loops, and physicians' pretest probability to accurately predict the diagnosis of SBO. This nomogram can ideally be used in the ED for more rapid and accurate prediction of the diagnosis in cases presenting with possible SBO, adjusting for different demographic, clinical, and US findings. The nomogram should next be externally validated to further assess the generalizability and applicability of this novel tool. Financial support This is a non-funded study, with no compensation or honoraria for participation, consulting or conducting the study. Resources required for this project was provided by institutional departmental funds at the George Washington University, Department of Emergency Medicine, with NO particular budgeting allocated to this project. Declaration of competing interest The authors do NOT have a financial interest or relationship to disclose regarding this research project. References
4.1. Limitations The present study had several limitations. The nomogram is developed based on data from a single center study with a small sample size. As such, some analyses may have been limited. We have tested and included most significant variables but given the limited sample size, the data on certain factors such as prior abdominal surgery and other comorbidities may have had too small an impact due to the small numbers to be included in the analysis; therefore, their effect or potential incorporation in the nomogram could not be assessed. We did not include physical examination findings such as abdominal distension or abnormal bowel sounds as these findings may vary based on experience of the assessing clinicians. We also did not include laboratory values such as serum lactate measurements as waiting for laboratory values to result would have introduced a time delay in the ability to apply the nomogram to a patient, and one of our goals was to create a measure that facilitates rapid prediction than can be achieved without waiting on additional imaging or blood test results. Diagnostic accuracy of our nomogram relies in part on the initial clinical gestalt of the physician to assess pretest probability, as well as the physician's ability to correctly identify measures such as bowel diameter and presence of free fluid on POCUS. We maintain that the fairly minimal training provided to sonographers in this study is sufficient to develop expertise in identifying SBO on POCUS. However, it is possible our measure may demonstrate decreased accuracy in the hands of less experienced clinicians such as those for whom pre-test probability
[1] Maung AA, Johnson DC, Piper GL, et al. Evaluation and management of small-bowel obstruction: an eastern association for the surgery of trauma practice management guideline. J Trauma Acute Care Surg 2012;73(5):s362–9. [2] Paulson E, Thompson W. Review of small bowel obstruction: the diagnosis and when to worry. Radiol J 2015;275:332–42. [3] Taylor MR, Lalani N. Adult small bowel obstruction. Acad Emerg Med 2013;20: 528–44. [4] Gottlieb M, Peksa GD, Pandurangadu AV, Nakitende D, Takhar S, Seethala RR. Utilization of ultrasound for the evaluation of small bowel obstruction: a systematic review and meta-analysis. Am J Emerg Med 2018;36(2):234–42 Feb. [5] Boniface KS, King JB, LeSaux MA, Haciski SC, Shokoohi H. Diagnostic accuracy and time-saving effects of point-of-care ultrasonography in patients with small bowel obstruction: a prospective study. Ann Emerg Med 2019 Jul 23. [pii: S0196-0644 (19)30443-3]. [6] Pourmand A, Dimbil U, Drake A, Shokoohi H. The accuracy of point-of-care ultrasound in detecting small bowel obstruction in emergency department. Emergency Medicine International 2018;2018(1):1–5. [7] Harrell FE, Lee KL, Califf RM, Pryor DB, Rosati RA. Regression modelling strategies for improved prognostic prediction. Stat Med 1984;3:143–52. [8] Hosmer DW, Lemeshow S. Applied logistic regression. 2nd. New York (NY: John Wiley & Sons; 2000; 162. [9] Harrell Jr FE, Lee KL, Mark DB. Tutorial in biostatistics: multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med 1996;15:361–87. [10] Steyerberg EQ, Harrell FE, Borsboom GJ, Eijkemans MJ, Vergouwe Y, Habbema JD. Internal validation of predictive models: efficiency of some procedures for logistic regression analysis. J Clin Epidemiol 2001;54:774–81. [11] Zlotnik A, Abraira V. A general-purpose nomogram generator for predictive logistic regression models. The Stata Journal 2015;15(2):537–46. [12] Dwivedi AK, Mallawaarachchi I, Alvarado LA. Analysis of sample size studies using nonparametric bootstrap test with pooled resampling method. Stat Method 2017; 36(14):2187–205.
Please cite this article as: H. Shokoohi, K.S. Boniface, M.A. Loesche, et al., Development of a nomogram to predict small bowel obstruction using point-of-care ultrasound in the e..., American Journal of Emergency Medicine, https://doi.org/10.1016/j.ajem.2019.12.010