- Email: [email protected]
An Electronic Emergency Triage System to Improve Patient Distribution by Critical Outcomes
The Journal of Emergency Medicine, Vol. -, No. -, pp. 1–9, 2016 Copyright Ó 2016 Elsevier Inc. Printed in the USA. All rights reserved 0736-4679/$ - see front matter
http://dx.doi.org/10.1016/j.jemermed.2016.02.026
Computers in Emergency Medicine
AN ELECTRONIC EMERGENCY TRIAGE SYSTEM TO IMPROVE PATIENT DISTRIBUTION BY CRITICAL OUTCOMES Andrea Freyer Dugas, MD, PHD,* Thomas D. Kirsch, MD, MPH,*†‡ Matthew Toerper,* Fred Korley, MD, PHD,* Gayane Yenokyan, PHD,§ Daniel France, MPH, PHD,k David Hager, MD, PHD,{ and Scott Levin, PHD*† *Department of Emergency Medicine, Johns Hopkins University, Baltimore, Maryland, †Whiting School of Engineering, Johns Hopkins University, Baltimore, Maryland, ‡Department of International Health, Johns Hopkins University, Baltimore, Maryland, §Department of Biostatistics, Johns Hopkins University, Baltimore, Maryland, kDepartment of Anesthesiology, Vanderbilt University, Nashville, Tennessee, and {Department of Pulmonary and Critical Care Medicine, Johns Hopkins University, Baltimore Maryland Reprint Address: Andrea Freyer Dugas, MD, PHD, Department of Emergency Medicine, Johns Hopkins University, 5801 Smith Avenue, Davis Building Suite 3220, Baltimore MD, 21209
, Abstract—Background: Patient triage is necessary to manage excessive patient volumes and identify those with critical conditions. The most common triage system used today, Emergency Severity Index (ESI), focuses on resources utilized and critical outcomes. Objective: This study derives and validates a computer-based electronic triage system (ETS) to improve patient acuity distribution based on serious patient outcomes. Methods: This cross-sectional study of 25,198 (97 million weighted) adult emergency department visits from the 2009 National Hospital Ambulatory Medical Care Survey. The ETS distributes patients by using a composite outcome based on the estimated probability of mortality, intensive care unit admission, or transfer to operating room or catheterization suite. We compared the ETS with the ESI based on the differentiation of patients, outcomes, inpatient hospitalization, and resource utilization. Results: Of the patients included, 3.3% had the composite outcome and 14% were admitted, and 2.52 resources/patient were used. Of the 90% triaged to lowacuity levels, ETS distributed patients evenly (Level 3: 30%; Level 4: 30%, and Level 5: 29%) compared to ESI (46%, 34%, and 7%, respectively). The ETS betteridentified patients with the composite outcome present in 40% of ETS Level 1 vs. 17% for ESI and the ETS area under the receiver operating characteristic curve (AUC) was 0.83 vs. ESI 0.73. Similar results were found for hospital admission (ETS AUC = 0.83 vs. ESI AUC = 0.72). The ETS
demonstrated slight improvements in discriminating patient resource utilization. Conclusions: The ETS is a triage system based on the frequency of critical outcomes that demonstrate improved differentiation of patients compared to the current standard ESI. Ó 2016 Elsevier Inc. , Keywords—triage; emergency medicine; allocation; patient acuity; utilization
resource
INTRODUCTION Increasing patient visits and decreasing capacity due to closing facilities has led a growing number of patients to experience significant wait times to receive medical evaluation and treatment (1). Despite the move to early time to a provider, patient triage is necessary to effectively manage excessive volumes of patients and identify patients with critical and time-sensitive conditions (e.g., myocardial ischemia or sepsis) from those with lessurgent needs (e.g., indigestion or minor infections). Although triage decisions are straightforward for very high- or low-severity cases; the projected clinical course for the majority of patients is not obvious. Inability to quickly distinguish significantly ill patients (i.e., undertriage) can delay time-sensitive treatment and lead to
RECEIVED: 4 March 2015; FINAL SUBMISSION RECEIVED: 2 November 2015; ACCEPTED: 17 February 2016 1
2
deterioration, morbidity, and mortality (2–5). Overtriaged patients consume limited resources that might be directed more appropriately to those with higheracuity illness (6,7). As emergency care demands higher efficiency to manage growing patient volumes, an accurate and evidence-based triage system is required to provide safe and optimal care. Triage has been a long-standing principle in emergency medicine, but standardized triage tools are relatively new. Canada, Australia, and the United Kingdom have created their own triage instruments and, in the United States, 72% of emergency department (ED) patient visits are assessed using the Emergency Severity Index (ESI) (6,8–11). ESI is composed of a series of 3 questions used to assign patients to one of five acuity levels. The ESI triage process relies on experienced nurses’ judgments to assess patients according to the following questions: 1) Is the patient dying? 2) Should the patient wait? and 3) How many resources will this patient require? (6) Patients dying are categorized to Level 1 (immediate treatment); patients who should not wait are categorized to Level 2 (emergent treatment); and patients deemed safe to wait are stratified to Levels 3 (urgent treatment) through 5 (nonurgent treatment) based on anticipated resource utilization. Level 3 patients are expected to use the most resources (more than two resources) followed by Level 4 (one resource) and Level 5 (no resources). The triage provider can also re-categorize Level 3 patients up to Level 2, based on abnormal vital signs. ESI has been validated by its developers to outcomes of hospital admission and ED resource use (12–15). Overall, the ESI tool stratifies patients based on nurses’ experience and ‘‘sixth sense’’ for immediacy of medical need and resource utilization (6). Including resource utilization to determine a triage level makes the system unique among modern triage systems. Although currently in widespread use within the United States, ESI has several shortcomings (14,16,17). Foremost, it does not sufficiently discriminate and distribute patients across its five triage levels, with almost half of all ED patients nationally assigned to acuity Level 3 (18,19). This results in patients with a wide range of illness severity clustered to one large group, potentially delaying care to those most severely ill. This seems to counter the true objective of triage by not differentiating a majority of patient visits and, therefore, creates challenges in efficient resource distribution. Next, ESI has not been adequately validated against time-sensitive or critical care outcomes, which are important for an effective triage system. Finally, ESI relies heavily on subjectivity at triage and might be limited by untoward variability that can adversely affect patients through nurse inexperience, human error, or even systemic flaws in triage assessment.
A. F. Dugas et al.
To address these deficiencies, we utilized a large nationally representative sample of ED patient visits to develop an automated, computer-based, electronic triage system (i.e., ETS) designed to improve patient differentiation objectively based on the risk of critical patient outcomes. The ETS uses simple standardized patient information routinely collected at triage to predict risk for critical outcomes and distributes patients among five triage levels based on estimated risk. The ETS and ESI are then evaluated using measures targeted by both the ETS (distribution of patients and risk of critical outcomes) and ESI (risk of inpatient hospitalization and ED resource utilization). MATERIALS AND METHODS Study Design and Setting This is a retrospective cohort study of patient visits included in the 2009 National Hospital Ambulatory Medical Care Survey (NHAMCS). NHAMCS is an annually collected, nationally representative probability sample survey of ED visits conducted by the Centers for Disease Control and Prevention’s (CDC) National Center for Health Statistics (20). This study utilizes a pre-existing, publicly available, de-identified database; therefore, no additional participant consent or Institutional Review Board approval was needed for this analysis. The 2009 sample included 26,556 adult (18 years and older) patient records from 356 of 389 EDs (91.5% unweighted response rate), resulting in an unbiased weighted national sample of 102 million ED patient visits (20). Patients dead on arrival (11 patient visits [< 0.1%]), transferred to a psychiatric hospital (594 [2.2%]), or with an unknown outcome (753 patient visits [2.9%]) were excluded. The final study cohort of 25,198 patient visits provided data for a weighted nationally representative sample of 97 million patient visits. Data Collection and Methods of Measurement This study compares the derived ETS triage level to the ESI triage measurement recorded in the NHAMCS database. NHAMCS encodes triage as Level 1: immediate; Level 2: emergent; Level 3: urgent; Level 4: semi-urgent; and Level 5: nonurgent (19). The majority of EDs in the United States use ESI; however, some of the 356 EDs sampled in the NHAMCS may have used alternative triage systems. EDs using other systems, specifically other 3- or 4-level triage systems, were systematically recoded by the CDC to a 5-level ESI system within NHAMCS (19). The primary goal of the ETS is to discriminate ED patients based on the risk of critical or time-sensitive outcomes. These were compositely defined as either: 1)
Electronic Emergency Triage System
mortality; 2) direct admission from the ED to any intensive care unit (ICU), including medical, cardiac, neurologic and surgical units; or 3) direct transfer to the operating room (OR) or cardiovascular catheterization suite and called the ‘‘composite outcome.’’ Mortality was defined as death during the ED stay or corresponding hospitalization. Independent from our composite outcome measure, an admission to the hospital and the amount of resources utilized were used to evaluate the ETS and ESI comparatively. Resource utilization was defined per ESI guidelines as counts of distinct classes of resources for a patient visit, including blood tests, electrocardiography, plain x-ray studies, computed tomography (CT), ultrasound, magnetic resonance imaging, other imaging, medication administered in the ED, specialty consultation, and intravenous fluids. For example, x-ray studies of the knee and of the chest count as one resource used (i.e., x-ray), whereas an x-ray study of the chest and a CT of the neck counts as two resources used (i.e., x-ray and CT). In addition, each minor procedure performed counts as one resource, including suturing, stapling, incision and drainage, foreign body removal, bladder catheterization, and other procedures. Major procedures count as two resources, including central venous line placement, cardiopulmonary resuscitation, and endotracheal intubation. Only standard patient information available at triage was used as predictor variables for the ETS. Specifically, these included basic demographics, vital signs, primary chief complaint, and mode of arrival (i.e., ambulance or walk-in). Vital signs, including temperature, heart rate, systolic blood pressure, respiratory rate, and oxygen saturation were categorized as normal or gradations of abnormal according to previously developed divisions (see Table 1) (20–24). Chief complaints were captured through standard reasons for visit classification for ambulatory care (19,25). Common complaints, such as chest pain, abdominal pain, headache, shortness of breath, back pain, cough, nausea/vomiting, fever/chills, syncope, or dizziness were isolated and considered individually because of their high frequency and significance in predicting outcomes. The remaining chief complaints were categorized by body system to limit overspecification: psychiatric, nervous, cardiovascular, eyes and ears, respiratory, gastrointestinal, genitourinary, skin/hair/nails, musculoskeletal, injury or poisoning, and other. This clustering was created using clinical classification software developed by the Agency for Healthcare Research in Quality Healthcare Cost and Utilization Project (26). ETS Derivation The ETS was developed using logistic regression to estimate the probability of the composite outcome based on
3
data routinely collected at ED triage: demographics, vital signs, primary chief complaint, and mode of arrival. The coefficients estimated (i.e., predictor weights; see Table 1) may then be applied to any new ED patient to estimate their probability of composite outcome from their individual triage information. This patient is then assigned to one of five ETS triage levels based on their probability of them having any component of the composite outcome (Level 1: highest risk; Level 5: lowest risk). The risk profiles (i.e., cutoff probabilities) were chosen to mirror the ESI distribution of high acuity Level 1 (2% of ED visits) and Level 2 (8%) patients so as not to overwhelm resources dedicated to the highest severity patients. Risk points for ETS levels 3 to 5 were intentionally derived for an even distribution (i.e., each 30% of the remaining population) to maximize the discrimination of the patients. Statistical Analysis A small portion of NHAMCS vital signs and mode of arrival data were missing (i.e., 3.2% to 5.7% of these data), except that 13.5% of patients did not have oxygen saturation recorded. No patterns or bias were found in missing values, thus multiple imputation based on predictor variables was used to treat missing data. A patient visit was the unit of analysis for this study. Thus statistical analyses were performed on weighted ED patient data (i.e., 97 million visits) with all estimates meeting reliability requirements for sample size (>30 patient visits) and relative standard error (<30%) per CDC recommendations (20). Logistic regression was used to predict the probability of the composite outcome (i.e., mortality, admission to the ICU, or direct transport to the OR or cardiovascular catheterization suite) to derive the ETS acuity levels. Pooled parameter estimates were reported for multiple imputations. The ETS was crossvalidated and only out-of-sample predictions were evaluated. This was done by randomly parsing the data into 10 equal subsets. Logistic regression models were trained from 90% of these data producing outcome probability estimates for the remaining 10% test set. This process was repeated 10 times (i.e., 10-fold cross validation) yielding out-of-sample predictions for all patient visits. This approach was designed to assess predictive ability and determine the stability of predictor variable parameter estimates. The area under the receiver operating characteristic curve (AUC) was used to examine the ability of the ETS to discern patients with the composite outcome. The AUC was measured on out-of-sample predictions using the 10-fold cross-validation scheme described (27). To fairly compare the predictive power of both
4
A. F. Dugas et al.
Table 1. Electronic Triage System Predictor Variables and Corresponding Odds Ratios Variable Patient demographics and mode of arrival Age (baseline 18–30 years) 31–40 years 41–50 years 51–60 years 61–70 years 71–80 years >80 years Sex, male (baseline female) Mode of arrival ambulance (baseline other) Vital signs Temperature (baseline 95.1–100.4 F) Hypothermia (#95 F) Hyperthermia ($100.5 F) Heart rate (baseline 50–109 bpm) Bradycardia (<50 bpm) Mild tachycardia (110–119 bpm) Moderate tachycardia (120–129 bpm) Severe tachycardia ($130 bpm) Systolic blood pressure (baseline 100–200 mm Hg) Hypotension (<100 mm Hg) Hypertension (>200 mm Hg) Respiratory rate (baseline 14–19 breaths/min) Bradypnea (<14 breaths/min) Moderate tachypnea (20–27 breaths/min) Severe tachypnea ($28 breaths/min) Oxygen saturation (baseline 95%–100%) Mild hypoxia (90%–94%) Severe hypoxia (<90%) Chief complaint Chest pain (baseline general symptoms) Abdominal pain Headache Shortness of breath Back pain Cough Nausea/vomiting Fever/chills Syncope Dizziness Psychiatric complaint Other nervous system complaint Other cardiovascular system complaint Ears or eyes complaint Other respiratory complaint Other gastrointestinal system complaint Other genitourinary complaint Skin/nails/hair complaint Other musculoskeletal complaint Other injury or poisoning Other
Unweighted n (Weighted %)
Odds Ratio (95% CI)
6,807 (27.3) 4,470 (17.6) 4,381 (16.9) 3,646 (14.7) 2,225 (9.0) 1,805 (7.2) 1,864 (7.2) 10,747 (42.3) 4,544 (17.9)
1.34 (1.32–1.36) 1.75 (1.71–1.78) 2.31 (2.27–2.35) 3.19 (3.11–3.26) 3.88 (3.74–4.01) 4.11 (3.90–4.32) 1.42 (1.38–1.46) 4.53 (4.30–4.77)
24,498 (97.1) 109 (0.4) 591 (2.5) 22,725 (89.8) 154 (0.6) 1,391 (5.6) 577 (2.4) 351 (1.5) 24,040 (95.2) 808 (3.3) 350 (1.5) 15,518 (59.6) 456 (1.4) 8,751 (32.9) 473 (1.9) 22,363 (88.1) 1979 (8.3) 856 (3.6)
1.98 (1.82–2.15) 2.07 (1.78–2.41) 1.86 (1.64–2.11) 1.39 (1.10–1.77) 3.39 (3.10–3.71) 4.66 (4.28–5.09) 2.94 (2.75–3.14) 1.41 (1.22–1.63) 2.89 (2.06–4.05) 1.28 (1.19–1.38) 2.55 (2.42–2.69) 1.16 (0.93–1.46) 1.73 (1.22–2.46)
1,593 (6.4) 1,517 (8.2) 2,025 (4.5) 1,091 (4.2) 1,037 (5.1) 1,266 (2.4) 565 (3.0) 724 (1.7) 205 (0.9) 395 (1.6) 686 (2.2) 553 (2.2) 450 (1.9) 602 (2.1) 1,220 (5.0) 1,056 (4.1) 1,396 (5.1) 828 (3.3) 2,855 (11.3) 3,396 (13.5) 1,327 (4.6)
2.05 (1.95–2.15) 2.62 (2.52–2.75) 0.42 (0.41–0.43) 2.47 (2.43–2.51) 0.24 (0.23–0.25) 0.77 (0.75–0.79) 1.14 (1.10–1.18) 1.39 (1.26–1.54) 0.73 (0.69–0.76) 0.87 (0.82–0.91) 0.98 (0.95–1.02) 1.10 (1.06–1.13) 1.58 (1.54–1.62) 0.96 (0.85–1.07) 0.70 (0.68–0.71) 1.68 (1.64–1.72) 1.49 (1.37–1.62) 0.43 (0.40–0.44) 0.42 (0.41–0.44) 1.37 (1.32–1.43) 2.25 (2.16–2.35)
CI = confidence interval.
triage systems (ETS vs. ESI), the AUC was determined for the ordinal levels of each system. The ETS was also compared to ESI based on outcomes targeted by the ETS (distribution of patients and risk of composite outcome) and the ESI (risk of inpatient hospitalization and ED resource utilization). Two tailed z-tests were used to test for differences in the proportion of patient visits with these outcomes between each triage system.
RESULTS The ETS predicts an ED patient’s probability of the composite outcome (mortality, admission to the ICU, or direct transport to the OR or cardiovascular catheterization suite) using routinely available information obtained at ED triage. Of the 97 million ED visits included, 3.3% had the composite outcome (0.5% in-hospital mortality, 2.2% admitted to an ICU, 0.5% transferred to an OR,
Electronic Emergency Triage System
5
and 0.1% transferred to a cardiovascular catheterization suite). Factors associated with increased likelihood of a patient having the composite outcome include advancing age, male sex, arrival by ambulance, abnormal vital signs, and specific chief complaints (chest pain, abdominal pain, shortness of breath, and fever/chills) seen in Table 1. Like ESI, the ETS classifies patients into 5 levels with Level 1 requiring the most immediate attention based on the risk of them having the composite outcome. The risk of the composite outcome for Level 1 was set at $ 20%, Level 2 (7% to 20%), Level 3 (1.5% to 7%), Level 4 (0.5% to 1.5%), and Level 5 (<0.5%). ETS Evaluation Measures Comparisons In both of these triage systems, almost 90% of patients are assigned to the lowest acuity levels (Levels 3 to 5). The ETS improves the discrimination of these patients because it is designed to distribute patients evenly amongst the lowest acuity levels (Level 3: 30%; Level 4: 30%, Level 5: 29%) compared to the ESI distribution (46%, 34%, and 7%, respectively) as depicted in Figure 1. The distribution amongst high acuity Level 1 (2.3% for ETS and 1.9% for ESI) and Level 2 (8.2% for ETS and 11% for ESI) categories were similar by design. Despite similar overall distributions, the highacuity patients recognized by both the ETS and ESI were different, with only 31% of the highest acuity (i.e., Level 1 and Level 2) ESI patients also classified as high acuity by the ETS. The ETS exceled at differentiating patients based on predictions of the composite outcome. The AUC for the ETS (0.83) demonstrated significant (p < 0.001) improvement compared to ESI (0.73). This translated to the ETS classifying a greater proportion of patients with the composite outcome to higher severity levels (Figure 2). Within Level 1, 40% of ETS patients have the composite outcome compared to 17% for ESI. Levels 2 through 5 demonstrate similar rates of composite outcome between both triage systems. However, ETS Level 2 and 3 did identify a higher proportion (p < 0.001) of patient visits with composite outcome, while ETS Level 4 and 5 identified a lower proportion (p < 0.001) compared to the corresponding ESI level.
Figure 1. Severity level classification: proportion of patients assigned to each severity level for both electronic triage system (ETS) and Emergency Severity Index (ESI).
admitted (p < 0.001), while 44% of ETS and 44% of ESI Level 2 patients were admitted to the hospital (Figure 3). At the lower severity levels (Levels 3 to 5), ETS and ESI performed similarly with respect to discriminating hospital admission. Resource utilization is used as a critical determination of the immediacy of need for treatment by ESI design. Overall, the mean number of resources used by all patients was 2.52 (95% CI 0–6) and 13% used no resources. The ETS and ESI demonstrated similar discrimination across all severity levels (Figure 4) and specifically for
ESI Evaluation Measures Comparisons Hospital admission was evaluated as a marker of disease severity by ESI (12–15). Overall, 14% of all ED patient visits were admitted to the hospital. The AUC for the ETS predicting admission was 0.83 compared to ESI 0.72 (p < 0.001). ETS was also better able to identify patients admitted compared to ESI, with 80% of ETS and 39% of ESI patients categorized as Level 1
Figure 2. Patient outcome by severity level: proportion of patients in each severity level with the outcome of transport to the operating room or cardiac catheterization suite (OR/ CATH), admission to an intensive care unit (ICU), or mortality for both the electronic triage system (ETS) (left) and Emergency Severity Index (ESI) (right).
6
A. F. Dugas et al.
Figure 3. Admission by severity level: proportion of patients in each severity level who were admitted to the hospital for both the electronic triage system (ETS) and Emergency Severity Index (ESI).
the lower acuity levels (Levels 3–5) most applicable per the ESI algorithm. DISCUSSION The ETS is a newly proposed electronic triage instrument that differentiates patients based on critical and timesensitive outcomes. It can be easily incorporated into any electronic medical record system and immediately calculate a score once common patient information has been put into the system. The ETS was developed using a nationally representative sample of 97 million ED visits
Figure 4. Resource utilization by severity level: median number of medical resources used by patients in each severity level (boxes with upper [75%] and lower quartiles [25%]); line displaying 95% range for both the electronic triage system (ETS) and Emergency Severity Index (ESI).
accounting for geographic and facility variability, maximizing its potential for generalized application across the United States. Advantages of the ETS compared to the reference standard ESI include evidence (i.e., outcomes)-based development, improved distribution of patients across the five severity levels, and decreased reliance on subjective human experience or judgment. By designing an automatic triage system that uses discrete variables, the ETS reduces dependence on subjective decisions, while optimizing acuity level classification based on these clinically important outcomes. Previous evaluations of the validity of triage systems such as ESI, including those performed by the creators of ESI, used the outcomes of hospital admission and resource utilization as markers of appropriate triage (12,14). These outcomes fit with recommendations from a joint task force of members from the American College of Emergency Physicians and the Emergency Nurses Association, who reported, ‘‘Emergency departments in the United States need an ED triage tool that would strongly predict outcomes including severity of illness, mortality rate, and resource requirements’’ (15). Thus, we evaluated the validity of the ETS based on these same outcomes: hospital admission (a marker of severity of illness) and resource utilization (as defined by ESI), which the ETS is impartial to. The ETS performed substantially better in discerning admitted patients (Figure 3) and showed similar results to ESI for predicting resource utilization (Figure 4). In addition, The ETS demonstrated a more even distribution of patients across five levels and more accurate at identifying patients with other markers of severe illness and mortality (i.e., the composite outcome). The ETS was designed to achieve these outcomes and thus was expected to outperform ESI, but we consider measures such measures of severity critical to an effective triage system and thus motivated the ETS design. However, these measures were also balanced by comparing the effectiveness of both systems to established ESI measures (hospital admission and resource utilization) endorsed by ESI developers (12–14). Even though EDs are moving to more rapid assessment by a physician or mid-level provider, the initial distribution of patients within the ED is often determined by a primary triage process. A major disadvantage of the current standard ESI is the lack of distribution among lower triage levels. In our sample, 46% of patients are classified as triage Level 3 (urgent treatment), similar to prior reports of ESI use (18). This likely results from very few patients using fewer than two resources, as defined by ESI. This lack of discrimination diminishes the advantages of a five-level system and counters the objective of triage (i.e., patient differentiation). It prohibits identification of low risk patients (traditionally considered Level 4 or
Electronic Emergency Triage System
5) who may receive less intense work-ups and may be sent to a ‘‘fast-track’’ or may not require examination by board-certified emergency physician (28). This large group of undifferentiated patients also simultaneously increases risk for the more-severe patients grouped with the less-acute patients. In addition to patient safety, lack of discrimination of the ESI Level 3 distribution challenges efficient resource distribution and overall ED efficiency. The ETS was designed to address this by offering improvements in distributing the majority of lower acuity patients (i.e., 30% of patients in each triage Level 3 to 5). This may facilitate substantial gains in patient prioritization and distribution to different service areas and providers that are critical to ED operations. Although we propose the specific even distribution, the ETS may be customized to individual ED objectives for risk stratification and distribution of patients to match existing resource allocation schemes. The ETS uses critical and time-sensitive outcomes (mortality, admission to the ICU, or direct transport to the OR or cardiovascular catheterization suite) as a proxy for the need for emergent treatment. The ETS was designed for clinical decision support and is calculated immediately based on the first data recorded on a patient’s arrival. It is not intended to replace triage nurse or physician judgment, but to support it by providing the score as they assess a patient. An electronic system cannot account for details of patient history or appearance, which experienced triage providers recognize to indicate that a patient requires emergent attention. Hence, the assigned ETS score can be over-ridden based on a provider’s clinical concerns, specific types of injuries/illnesses (e.g., lacerations) or related to the availability of specialty services or treatment areas/options (e.g., fast track). However, the ETS can provide more consistent, evidence-based information to assist with making safer triage decisions. Limitations There are several limitations of the proposed the ETS that may decrease its ability to be generalized to all EDs. First, the ETS requires an electronic medical record with a computerized triage system to avoid manual calculations. However, as of 2012, > 85% of EDs in the United States (US) have access to electronic medical records and computerized triage systems (29). Adapting the ETS to different clinical settings and computer systems will take training and software interfaces. The ETS has also only been evaluated for an adult ED population with no inferences towards its utility in pediatric populations. Although the ETS is based on the NHAMCS nationally representative database, and is broadly generalizable, it is subject to any potential errors in the NHAMCS data itself (30). In particular, there is likely variation in
7
application of ESI or potentially other 5 level triage systems within the NHAMCS database introducing the potential for misclassification, which could, in turn, improve the apparent performance of the ETS relative to the ESI. However, ESI is used to triage the majority of ED patient visits in the United States, and our distribution of patients aligns with previous reports of institutions using an ESI triage system, thus impact on these analyses is likely minimal (8,18). Resource utilization is also believed to be under-reported in NHAMCS, which might impact these comparisons in our paper (30). CONCLUSIONS The ETS is a novel electronic triage system that uses commonly obtained triage information to automatically distribute adult ED patients across five severity levels based on critical and time-sensitive outcomes (mortality, admission to the ICU, or direct transport to the OR or cardiovascular catheterization suite). The algorithm was derived and validated using a nationally representative sample of 25,198 adult patient visits (97 million visits weighted) collected by the CDC. Compared to ESI, the ETS may more accurately discern high-acuity patients according to proxies for severity of illness in a medically undifferentiated adult ED population. In addition, the ETS may reduce subjectivity in triage evaluation, while more evenly distributing patients among lower-acuity levels to allow more efficient distribution of patients to treatment spaces and providers. Overall, is a critical outcomes-based triage system demonstrating improved differentiation of patients compared to the current US standard triage system, ESI. The ETS will require prospective evaluation in diverse ED care settings to understand its clinical utility and generalizability. Acknowledgments—This work was supported by the Department of Homeland Security (PACER: National Center for Study of Preparedness and Response [2010-ST-061-PA0001]).
REFERENCES 1. Institute of Medicine. The Future of Emergency Care in the United States Health System. Ann Emerg Med 2006;13:1081–5. 2. Seiger N, Van Veen M, Steyerber E, Ruige M, Van Meurs A, Moll A. Undertriage in the Manchester triage system: an assessment of severity and options for improvement. Arch Dis Child 2011;96: 653–7. 3. Haas B, Gomez D, Zagorski B, Stukel T, Rubenfeld G, Nathens A. Survival of the fittest: the hidden cost of undertriage of major trauma. J Am Coll Surg 2010;211:804–11. 4. Yurkova I, Wolf L. Under-triage as a significant factor affecting transfer time between the emergency department and the intensive care unit. J Emerg Nurse 2011;37:491–6.
8 5. Hitchcock M, Gillespie B, Crilly J, Chabover W. Triage: an investigation of the process and potential vulenerabilities. J Adv Nurs 2013;70:1–10. 6. Emergency Severity Index, Version 4: implementation handbook. AHRQ Publication No. 12–0014. Agency for Healthcare Research and Quality, 2011. Available at: http://www.ahrq.gov/research/esi/. Accessed November 1, 2012. 7. Dong SL, Bullard MJ, Meurer DP, et al. Comparing a novel computer triage program with standard triage. Acad Emerg Med 2005;12:502–7. 8. McHugh M, Tanabe P, McClelland M, Khare RK. More patients are triaged using the Emergency Severity Index than any other triage acuity system in the United States. Acad Emerg Med 2012;19: 106–9. 9. Meek R, Phiri W. Australian Triage Scale: consumer perspective. Emerg Med Australas 2005;17:212–7. 10. Manchester Triage Group. Emergency triage: Manchester Triage Group. London: BMJ Publishing Group; 1997. 11. Beveridge R, Clarke B, Janes N. Canadian emergency department triage and acuity scale; implementation guidelines. Can J Emerg Med 1999;1:2–28. 12. Eitel DR, Travers DA, Rosenau AM, Gilboy N, Wuerz RC. The emergency severity index triage algorithm version 2 is reliable and valid. Acad Emerg Med 2003;10:1070–80. 13. Tanabe P, Gimbel R, Yarnold PR, Adams JG. The Emergency Severity Index (version 3) 5-level triage system scores predict ED resource consumption. J Emerg Nurs 2004;30:22–9. 14. Wuerz RC, Milne LW, Eitel DR, Travers D, Gilboy N. Reliability and validity of a new five-level triage instrument. Acad Emerg Med 2000;7:236–42. 15. Fernandes C, Tanabe P, Gilboy N, et al. Five-level triage: a report from the ACEP/ENA Five-Level Triage Task Force. J Emerg Nurs 2005;31:39–50. 16. Tanabe P, Gimbel R, Yarnold PR, Kyriacou DN, Adams JG. Reliability and validity of scores on the Emergency Severity Index version 3. Acad Emerg Med 2004;11:59–65. 17. Wuerz RC, Travers D, Gilboy N, Eitel DR, Rosenau A, Yazhari R. Implementation and refinement of the emergency severity index. Acad Emerg Med 2001;8:170–6.
A. F. Dugas et al. 18. Christ M, Grossmann F, Winter D, Bingisser R, Platz E. Modern triage in the emergency department. Dtsch Arztebl Int 2010;107: 892–8. 19. 2009 NHAMCS Micro-data File Documentation. 2009. Available at: ftp://ftp.cdc.gov/pub/health_statistics/NCHS/Dataset_Docume ntation/NHAMCS/doc09.pdf. Accessed January 15, 2013. 20. NHAMCS scope and sample design. Available at: http://www.cdc. gov/nchs/ahcd/ahcd_scope.htm#nhamcs_scope. Accessed December 25, 2012. 21. Barfod C, Lauritzen MM, Danker JK, et al. Abnormal vital signs are strong predictors for intensive care unit admission and in-hospital mortality in adults triaged in the emergency department—a prospective cohort study. Scand J Trauma Resusc Emerg Med 2012;20:28. 22. Tortora GA. Principles of Anatomy and Physiology. 6th edn. Boston, MA: Addison Wesley. 23. Chobanian AV, Bakris GL, Black HR, et al. The Seventh Report of the Joint National Committee on Prevention, Detection, Evaluation, and Treatment of High Blood Pressure: the JNC 7 report. JAMA 2003;289:2560–72. 24. McGillicuddy DC, O’Connell FJ, Shapiro NI, et al. Emergency department abnormal vital sign ‘‘triggers’’ program improves time to therapy. Acad Emerg Med 2011;18:483–7. 25. Schneider D, Appleton L, McLemore T. A reason for visit classification for ambulatory care. National Center for Health Statistics. Vital Health Stat 2 1979. i–vi, 1–63. 26. Clinical Classifications Software (CCS) for ICD-9-CM. 2012. Available at: http://www.hcup-us.ahrq.gov/toolssoftware/ccs/ccs. jsp. Accessed December 25, 2012. 27. Geisse S. Predictive Inference. New York: Chapman and Hall; 1993. 28. Oredsson S, Jonsson H, Rognes J, et al. A systematic review of triage-related interventions to improve patient flow in emergency departments. Scand J Trauma Resusc Emerg Med 2011;19:43. 29. Charles D, King J, Patel V, Furukawa MF. Adoption of Electronic Health Record Systems among U.S. Non-Federal Acute Care Hospitals: 2008-2012. Available at: https://www.healthit.gov/sites/ default/files/oncdatabrief9final.pdf. Accessed February 25, 2016. 30. McCaig LF, Burt CW. Understanding and Interpreting the National Hospital Ambulatory Medical Care Survey: Key Questions and Answers. Ann Emerg Med 2012;60:716–21.
Electronic Emergency Triage System
ARTICLE SUMMARY 1. Why is this an important topic? Despite administrative changes to reduce the time to see a provider, triage remains an important function to identify seriously ill patients and prevent poor outcomes. 2. What does this study attempt to show? The study attempts to develop a triage system to identify patients based on their probability of a poor outcome, not simply on resource utilization and triage nurse decisions as with ESI. 3. What are the key findings? Using standard screening information the ETS automatically classifies patients and better identifies those with risk of a poor outcome. As importantly, it better distributes patients to each triage level and reduces ‘‘overtriage’’ to a single level (Level 3) that occurs with ESI. 4. How is patient care impacted? As this system can run simultaneously as the patient checks in and preliminary vital signs are recorded, it can provide an immediate estimate of patient risk to any provider conducting a preliminary triage evaluation. Because it is better able to identify patients at risk for poor outcomes, it can also reduce their occurrence and improve patient safety.
9
http://dx.doi.org/10.1016/j.jemermed.2016.02.026
Computers in Emergency Medicine
AN ELECTRONIC EMERGENCY TRIAGE SYSTEM TO IMPROVE PATIENT DISTRIBUTION BY CRITICAL OUTCOMES Andrea Freyer Dugas, MD, PHD,* Thomas D. Kirsch, MD, MPH,*†‡ Matthew Toerper,* Fred Korley, MD, PHD,* Gayane Yenokyan, PHD,§ Daniel France, MPH, PHD,k David Hager, MD, PHD,{ and Scott Levin, PHD*† *Department of Emergency Medicine, Johns Hopkins University, Baltimore, Maryland, †Whiting School of Engineering, Johns Hopkins University, Baltimore, Maryland, ‡Department of International Health, Johns Hopkins University, Baltimore, Maryland, §Department of Biostatistics, Johns Hopkins University, Baltimore, Maryland, kDepartment of Anesthesiology, Vanderbilt University, Nashville, Tennessee, and {Department of Pulmonary and Critical Care Medicine, Johns Hopkins University, Baltimore Maryland Reprint Address: Andrea Freyer Dugas, MD, PHD, Department of Emergency Medicine, Johns Hopkins University, 5801 Smith Avenue, Davis Building Suite 3220, Baltimore MD, 21209
, Abstract—Background: Patient triage is necessary to manage excessive patient volumes and identify those with critical conditions. The most common triage system used today, Emergency Severity Index (ESI), focuses on resources utilized and critical outcomes. Objective: This study derives and validates a computer-based electronic triage system (ETS) to improve patient acuity distribution based on serious patient outcomes. Methods: This cross-sectional study of 25,198 (97 million weighted) adult emergency department visits from the 2009 National Hospital Ambulatory Medical Care Survey. The ETS distributes patients by using a composite outcome based on the estimated probability of mortality, intensive care unit admission, or transfer to operating room or catheterization suite. We compared the ETS with the ESI based on the differentiation of patients, outcomes, inpatient hospitalization, and resource utilization. Results: Of the patients included, 3.3% had the composite outcome and 14% were admitted, and 2.52 resources/patient were used. Of the 90% triaged to lowacuity levels, ETS distributed patients evenly (Level 3: 30%; Level 4: 30%, and Level 5: 29%) compared to ESI (46%, 34%, and 7%, respectively). The ETS betteridentified patients with the composite outcome present in 40% of ETS Level 1 vs. 17% for ESI and the ETS area under the receiver operating characteristic curve (AUC) was 0.83 vs. ESI 0.73. Similar results were found for hospital admission (ETS AUC = 0.83 vs. ESI AUC = 0.72). The ETS
demonstrated slight improvements in discriminating patient resource utilization. Conclusions: The ETS is a triage system based on the frequency of critical outcomes that demonstrate improved differentiation of patients compared to the current standard ESI. Ó 2016 Elsevier Inc. , Keywords—triage; emergency medicine; allocation; patient acuity; utilization
resource
INTRODUCTION Increasing patient visits and decreasing capacity due to closing facilities has led a growing number of patients to experience significant wait times to receive medical evaluation and treatment (1). Despite the move to early time to a provider, patient triage is necessary to effectively manage excessive volumes of patients and identify patients with critical and time-sensitive conditions (e.g., myocardial ischemia or sepsis) from those with lessurgent needs (e.g., indigestion or minor infections). Although triage decisions are straightforward for very high- or low-severity cases; the projected clinical course for the majority of patients is not obvious. Inability to quickly distinguish significantly ill patients (i.e., undertriage) can delay time-sensitive treatment and lead to
RECEIVED: 4 March 2015; FINAL SUBMISSION RECEIVED: 2 November 2015; ACCEPTED: 17 February 2016 1
2
deterioration, morbidity, and mortality (2–5). Overtriaged patients consume limited resources that might be directed more appropriately to those with higheracuity illness (6,7). As emergency care demands higher efficiency to manage growing patient volumes, an accurate and evidence-based triage system is required to provide safe and optimal care. Triage has been a long-standing principle in emergency medicine, but standardized triage tools are relatively new. Canada, Australia, and the United Kingdom have created their own triage instruments and, in the United States, 72% of emergency department (ED) patient visits are assessed using the Emergency Severity Index (ESI) (6,8–11). ESI is composed of a series of 3 questions used to assign patients to one of five acuity levels. The ESI triage process relies on experienced nurses’ judgments to assess patients according to the following questions: 1) Is the patient dying? 2) Should the patient wait? and 3) How many resources will this patient require? (6) Patients dying are categorized to Level 1 (immediate treatment); patients who should not wait are categorized to Level 2 (emergent treatment); and patients deemed safe to wait are stratified to Levels 3 (urgent treatment) through 5 (nonurgent treatment) based on anticipated resource utilization. Level 3 patients are expected to use the most resources (more than two resources) followed by Level 4 (one resource) and Level 5 (no resources). The triage provider can also re-categorize Level 3 patients up to Level 2, based on abnormal vital signs. ESI has been validated by its developers to outcomes of hospital admission and ED resource use (12–15). Overall, the ESI tool stratifies patients based on nurses’ experience and ‘‘sixth sense’’ for immediacy of medical need and resource utilization (6). Including resource utilization to determine a triage level makes the system unique among modern triage systems. Although currently in widespread use within the United States, ESI has several shortcomings (14,16,17). Foremost, it does not sufficiently discriminate and distribute patients across its five triage levels, with almost half of all ED patients nationally assigned to acuity Level 3 (18,19). This results in patients with a wide range of illness severity clustered to one large group, potentially delaying care to those most severely ill. This seems to counter the true objective of triage by not differentiating a majority of patient visits and, therefore, creates challenges in efficient resource distribution. Next, ESI has not been adequately validated against time-sensitive or critical care outcomes, which are important for an effective triage system. Finally, ESI relies heavily on subjectivity at triage and might be limited by untoward variability that can adversely affect patients through nurse inexperience, human error, or even systemic flaws in triage assessment.
A. F. Dugas et al.
To address these deficiencies, we utilized a large nationally representative sample of ED patient visits to develop an automated, computer-based, electronic triage system (i.e., ETS) designed to improve patient differentiation objectively based on the risk of critical patient outcomes. The ETS uses simple standardized patient information routinely collected at triage to predict risk for critical outcomes and distributes patients among five triage levels based on estimated risk. The ETS and ESI are then evaluated using measures targeted by both the ETS (distribution of patients and risk of critical outcomes) and ESI (risk of inpatient hospitalization and ED resource utilization). MATERIALS AND METHODS Study Design and Setting This is a retrospective cohort study of patient visits included in the 2009 National Hospital Ambulatory Medical Care Survey (NHAMCS). NHAMCS is an annually collected, nationally representative probability sample survey of ED visits conducted by the Centers for Disease Control and Prevention’s (CDC) National Center for Health Statistics (20). This study utilizes a pre-existing, publicly available, de-identified database; therefore, no additional participant consent or Institutional Review Board approval was needed for this analysis. The 2009 sample included 26,556 adult (18 years and older) patient records from 356 of 389 EDs (91.5% unweighted response rate), resulting in an unbiased weighted national sample of 102 million ED patient visits (20). Patients dead on arrival (11 patient visits [< 0.1%]), transferred to a psychiatric hospital (594 [2.2%]), or with an unknown outcome (753 patient visits [2.9%]) were excluded. The final study cohort of 25,198 patient visits provided data for a weighted nationally representative sample of 97 million patient visits. Data Collection and Methods of Measurement This study compares the derived ETS triage level to the ESI triage measurement recorded in the NHAMCS database. NHAMCS encodes triage as Level 1: immediate; Level 2: emergent; Level 3: urgent; Level 4: semi-urgent; and Level 5: nonurgent (19). The majority of EDs in the United States use ESI; however, some of the 356 EDs sampled in the NHAMCS may have used alternative triage systems. EDs using other systems, specifically other 3- or 4-level triage systems, were systematically recoded by the CDC to a 5-level ESI system within NHAMCS (19). The primary goal of the ETS is to discriminate ED patients based on the risk of critical or time-sensitive outcomes. These were compositely defined as either: 1)
Electronic Emergency Triage System
mortality; 2) direct admission from the ED to any intensive care unit (ICU), including medical, cardiac, neurologic and surgical units; or 3) direct transfer to the operating room (OR) or cardiovascular catheterization suite and called the ‘‘composite outcome.’’ Mortality was defined as death during the ED stay or corresponding hospitalization. Independent from our composite outcome measure, an admission to the hospital and the amount of resources utilized were used to evaluate the ETS and ESI comparatively. Resource utilization was defined per ESI guidelines as counts of distinct classes of resources for a patient visit, including blood tests, electrocardiography, plain x-ray studies, computed tomography (CT), ultrasound, magnetic resonance imaging, other imaging, medication administered in the ED, specialty consultation, and intravenous fluids. For example, x-ray studies of the knee and of the chest count as one resource used (i.e., x-ray), whereas an x-ray study of the chest and a CT of the neck counts as two resources used (i.e., x-ray and CT). In addition, each minor procedure performed counts as one resource, including suturing, stapling, incision and drainage, foreign body removal, bladder catheterization, and other procedures. Major procedures count as two resources, including central venous line placement, cardiopulmonary resuscitation, and endotracheal intubation. Only standard patient information available at triage was used as predictor variables for the ETS. Specifically, these included basic demographics, vital signs, primary chief complaint, and mode of arrival (i.e., ambulance or walk-in). Vital signs, including temperature, heart rate, systolic blood pressure, respiratory rate, and oxygen saturation were categorized as normal or gradations of abnormal according to previously developed divisions (see Table 1) (20–24). Chief complaints were captured through standard reasons for visit classification for ambulatory care (19,25). Common complaints, such as chest pain, abdominal pain, headache, shortness of breath, back pain, cough, nausea/vomiting, fever/chills, syncope, or dizziness were isolated and considered individually because of their high frequency and significance in predicting outcomes. The remaining chief complaints were categorized by body system to limit overspecification: psychiatric, nervous, cardiovascular, eyes and ears, respiratory, gastrointestinal, genitourinary, skin/hair/nails, musculoskeletal, injury or poisoning, and other. This clustering was created using clinical classification software developed by the Agency for Healthcare Research in Quality Healthcare Cost and Utilization Project (26). ETS Derivation The ETS was developed using logistic regression to estimate the probability of the composite outcome based on
3
data routinely collected at ED triage: demographics, vital signs, primary chief complaint, and mode of arrival. The coefficients estimated (i.e., predictor weights; see Table 1) may then be applied to any new ED patient to estimate their probability of composite outcome from their individual triage information. This patient is then assigned to one of five ETS triage levels based on their probability of them having any component of the composite outcome (Level 1: highest risk; Level 5: lowest risk). The risk profiles (i.e., cutoff probabilities) were chosen to mirror the ESI distribution of high acuity Level 1 (2% of ED visits) and Level 2 (8%) patients so as not to overwhelm resources dedicated to the highest severity patients. Risk points for ETS levels 3 to 5 were intentionally derived for an even distribution (i.e., each 30% of the remaining population) to maximize the discrimination of the patients. Statistical Analysis A small portion of NHAMCS vital signs and mode of arrival data were missing (i.e., 3.2% to 5.7% of these data), except that 13.5% of patients did not have oxygen saturation recorded. No patterns or bias were found in missing values, thus multiple imputation based on predictor variables was used to treat missing data. A patient visit was the unit of analysis for this study. Thus statistical analyses were performed on weighted ED patient data (i.e., 97 million visits) with all estimates meeting reliability requirements for sample size (>30 patient visits) and relative standard error (<30%) per CDC recommendations (20). Logistic regression was used to predict the probability of the composite outcome (i.e., mortality, admission to the ICU, or direct transport to the OR or cardiovascular catheterization suite) to derive the ETS acuity levels. Pooled parameter estimates were reported for multiple imputations. The ETS was crossvalidated and only out-of-sample predictions were evaluated. This was done by randomly parsing the data into 10 equal subsets. Logistic regression models were trained from 90% of these data producing outcome probability estimates for the remaining 10% test set. This process was repeated 10 times (i.e., 10-fold cross validation) yielding out-of-sample predictions for all patient visits. This approach was designed to assess predictive ability and determine the stability of predictor variable parameter estimates. The area under the receiver operating characteristic curve (AUC) was used to examine the ability of the ETS to discern patients with the composite outcome. The AUC was measured on out-of-sample predictions using the 10-fold cross-validation scheme described (27). To fairly compare the predictive power of both
4
A. F. Dugas et al.
Table 1. Electronic Triage System Predictor Variables and Corresponding Odds Ratios Variable Patient demographics and mode of arrival Age (baseline 18–30 years) 31–40 years 41–50 years 51–60 years 61–70 years 71–80 years >80 years Sex, male (baseline female) Mode of arrival ambulance (baseline other) Vital signs Temperature (baseline 95.1–100.4 F) Hypothermia (#95 F) Hyperthermia ($100.5 F) Heart rate (baseline 50–109 bpm) Bradycardia (<50 bpm) Mild tachycardia (110–119 bpm) Moderate tachycardia (120–129 bpm) Severe tachycardia ($130 bpm) Systolic blood pressure (baseline 100–200 mm Hg) Hypotension (<100 mm Hg) Hypertension (>200 mm Hg) Respiratory rate (baseline 14–19 breaths/min) Bradypnea (<14 breaths/min) Moderate tachypnea (20–27 breaths/min) Severe tachypnea ($28 breaths/min) Oxygen saturation (baseline 95%–100%) Mild hypoxia (90%–94%) Severe hypoxia (<90%) Chief complaint Chest pain (baseline general symptoms) Abdominal pain Headache Shortness of breath Back pain Cough Nausea/vomiting Fever/chills Syncope Dizziness Psychiatric complaint Other nervous system complaint Other cardiovascular system complaint Ears or eyes complaint Other respiratory complaint Other gastrointestinal system complaint Other genitourinary complaint Skin/nails/hair complaint Other musculoskeletal complaint Other injury or poisoning Other
Unweighted n (Weighted %)
Odds Ratio (95% CI)
6,807 (27.3) 4,470 (17.6) 4,381 (16.9) 3,646 (14.7) 2,225 (9.0) 1,805 (7.2) 1,864 (7.2) 10,747 (42.3) 4,544 (17.9)
1.34 (1.32–1.36) 1.75 (1.71–1.78) 2.31 (2.27–2.35) 3.19 (3.11–3.26) 3.88 (3.74–4.01) 4.11 (3.90–4.32) 1.42 (1.38–1.46) 4.53 (4.30–4.77)
24,498 (97.1) 109 (0.4) 591 (2.5) 22,725 (89.8) 154 (0.6) 1,391 (5.6) 577 (2.4) 351 (1.5) 24,040 (95.2) 808 (3.3) 350 (1.5) 15,518 (59.6) 456 (1.4) 8,751 (32.9) 473 (1.9) 22,363 (88.1) 1979 (8.3) 856 (3.6)
1.98 (1.82–2.15) 2.07 (1.78–2.41) 1.86 (1.64–2.11) 1.39 (1.10–1.77) 3.39 (3.10–3.71) 4.66 (4.28–5.09) 2.94 (2.75–3.14) 1.41 (1.22–1.63) 2.89 (2.06–4.05) 1.28 (1.19–1.38) 2.55 (2.42–2.69) 1.16 (0.93–1.46) 1.73 (1.22–2.46)
1,593 (6.4) 1,517 (8.2) 2,025 (4.5) 1,091 (4.2) 1,037 (5.1) 1,266 (2.4) 565 (3.0) 724 (1.7) 205 (0.9) 395 (1.6) 686 (2.2) 553 (2.2) 450 (1.9) 602 (2.1) 1,220 (5.0) 1,056 (4.1) 1,396 (5.1) 828 (3.3) 2,855 (11.3) 3,396 (13.5) 1,327 (4.6)
2.05 (1.95–2.15) 2.62 (2.52–2.75) 0.42 (0.41–0.43) 2.47 (2.43–2.51) 0.24 (0.23–0.25) 0.77 (0.75–0.79) 1.14 (1.10–1.18) 1.39 (1.26–1.54) 0.73 (0.69–0.76) 0.87 (0.82–0.91) 0.98 (0.95–1.02) 1.10 (1.06–1.13) 1.58 (1.54–1.62) 0.96 (0.85–1.07) 0.70 (0.68–0.71) 1.68 (1.64–1.72) 1.49 (1.37–1.62) 0.43 (0.40–0.44) 0.42 (0.41–0.44) 1.37 (1.32–1.43) 2.25 (2.16–2.35)
CI = confidence interval.
triage systems (ETS vs. ESI), the AUC was determined for the ordinal levels of each system. The ETS was also compared to ESI based on outcomes targeted by the ETS (distribution of patients and risk of composite outcome) and the ESI (risk of inpatient hospitalization and ED resource utilization). Two tailed z-tests were used to test for differences in the proportion of patient visits with these outcomes between each triage system.
RESULTS The ETS predicts an ED patient’s probability of the composite outcome (mortality, admission to the ICU, or direct transport to the OR or cardiovascular catheterization suite) using routinely available information obtained at ED triage. Of the 97 million ED visits included, 3.3% had the composite outcome (0.5% in-hospital mortality, 2.2% admitted to an ICU, 0.5% transferred to an OR,
Electronic Emergency Triage System
5
and 0.1% transferred to a cardiovascular catheterization suite). Factors associated with increased likelihood of a patient having the composite outcome include advancing age, male sex, arrival by ambulance, abnormal vital signs, and specific chief complaints (chest pain, abdominal pain, shortness of breath, and fever/chills) seen in Table 1. Like ESI, the ETS classifies patients into 5 levels with Level 1 requiring the most immediate attention based on the risk of them having the composite outcome. The risk of the composite outcome for Level 1 was set at $ 20%, Level 2 (7% to 20%), Level 3 (1.5% to 7%), Level 4 (0.5% to 1.5%), and Level 5 (<0.5%). ETS Evaluation Measures Comparisons In both of these triage systems, almost 90% of patients are assigned to the lowest acuity levels (Levels 3 to 5). The ETS improves the discrimination of these patients because it is designed to distribute patients evenly amongst the lowest acuity levels (Level 3: 30%; Level 4: 30%, Level 5: 29%) compared to the ESI distribution (46%, 34%, and 7%, respectively) as depicted in Figure 1. The distribution amongst high acuity Level 1 (2.3% for ETS and 1.9% for ESI) and Level 2 (8.2% for ETS and 11% for ESI) categories were similar by design. Despite similar overall distributions, the highacuity patients recognized by both the ETS and ESI were different, with only 31% of the highest acuity (i.e., Level 1 and Level 2) ESI patients also classified as high acuity by the ETS. The ETS exceled at differentiating patients based on predictions of the composite outcome. The AUC for the ETS (0.83) demonstrated significant (p < 0.001) improvement compared to ESI (0.73). This translated to the ETS classifying a greater proportion of patients with the composite outcome to higher severity levels (Figure 2). Within Level 1, 40% of ETS patients have the composite outcome compared to 17% for ESI. Levels 2 through 5 demonstrate similar rates of composite outcome between both triage systems. However, ETS Level 2 and 3 did identify a higher proportion (p < 0.001) of patient visits with composite outcome, while ETS Level 4 and 5 identified a lower proportion (p < 0.001) compared to the corresponding ESI level.
Figure 1. Severity level classification: proportion of patients assigned to each severity level for both electronic triage system (ETS) and Emergency Severity Index (ESI).
admitted (p < 0.001), while 44% of ETS and 44% of ESI Level 2 patients were admitted to the hospital (Figure 3). At the lower severity levels (Levels 3 to 5), ETS and ESI performed similarly with respect to discriminating hospital admission. Resource utilization is used as a critical determination of the immediacy of need for treatment by ESI design. Overall, the mean number of resources used by all patients was 2.52 (95% CI 0–6) and 13% used no resources. The ETS and ESI demonstrated similar discrimination across all severity levels (Figure 4) and specifically for
ESI Evaluation Measures Comparisons Hospital admission was evaluated as a marker of disease severity by ESI (12–15). Overall, 14% of all ED patient visits were admitted to the hospital. The AUC for the ETS predicting admission was 0.83 compared to ESI 0.72 (p < 0.001). ETS was also better able to identify patients admitted compared to ESI, with 80% of ETS and 39% of ESI patients categorized as Level 1
Figure 2. Patient outcome by severity level: proportion of patients in each severity level with the outcome of transport to the operating room or cardiac catheterization suite (OR/ CATH), admission to an intensive care unit (ICU), or mortality for both the electronic triage system (ETS) (left) and Emergency Severity Index (ESI) (right).
6
A. F. Dugas et al.
Figure 3. Admission by severity level: proportion of patients in each severity level who were admitted to the hospital for both the electronic triage system (ETS) and Emergency Severity Index (ESI).
the lower acuity levels (Levels 3–5) most applicable per the ESI algorithm. DISCUSSION The ETS is a newly proposed electronic triage instrument that differentiates patients based on critical and timesensitive outcomes. It can be easily incorporated into any electronic medical record system and immediately calculate a score once common patient information has been put into the system. The ETS was developed using a nationally representative sample of 97 million ED visits
Figure 4. Resource utilization by severity level: median number of medical resources used by patients in each severity level (boxes with upper [75%] and lower quartiles [25%]); line displaying 95% range for both the electronic triage system (ETS) and Emergency Severity Index (ESI).
accounting for geographic and facility variability, maximizing its potential for generalized application across the United States. Advantages of the ETS compared to the reference standard ESI include evidence (i.e., outcomes)-based development, improved distribution of patients across the five severity levels, and decreased reliance on subjective human experience or judgment. By designing an automatic triage system that uses discrete variables, the ETS reduces dependence on subjective decisions, while optimizing acuity level classification based on these clinically important outcomes. Previous evaluations of the validity of triage systems such as ESI, including those performed by the creators of ESI, used the outcomes of hospital admission and resource utilization as markers of appropriate triage (12,14). These outcomes fit with recommendations from a joint task force of members from the American College of Emergency Physicians and the Emergency Nurses Association, who reported, ‘‘Emergency departments in the United States need an ED triage tool that would strongly predict outcomes including severity of illness, mortality rate, and resource requirements’’ (15). Thus, we evaluated the validity of the ETS based on these same outcomes: hospital admission (a marker of severity of illness) and resource utilization (as defined by ESI), which the ETS is impartial to. The ETS performed substantially better in discerning admitted patients (Figure 3) and showed similar results to ESI for predicting resource utilization (Figure 4). In addition, The ETS demonstrated a more even distribution of patients across five levels and more accurate at identifying patients with other markers of severe illness and mortality (i.e., the composite outcome). The ETS was designed to achieve these outcomes and thus was expected to outperform ESI, but we consider measures such measures of severity critical to an effective triage system and thus motivated the ETS design. However, these measures were also balanced by comparing the effectiveness of both systems to established ESI measures (hospital admission and resource utilization) endorsed by ESI developers (12–14). Even though EDs are moving to more rapid assessment by a physician or mid-level provider, the initial distribution of patients within the ED is often determined by a primary triage process. A major disadvantage of the current standard ESI is the lack of distribution among lower triage levels. In our sample, 46% of patients are classified as triage Level 3 (urgent treatment), similar to prior reports of ESI use (18). This likely results from very few patients using fewer than two resources, as defined by ESI. This lack of discrimination diminishes the advantages of a five-level system and counters the objective of triage (i.e., patient differentiation). It prohibits identification of low risk patients (traditionally considered Level 4 or
Electronic Emergency Triage System
5) who may receive less intense work-ups and may be sent to a ‘‘fast-track’’ or may not require examination by board-certified emergency physician (28). This large group of undifferentiated patients also simultaneously increases risk for the more-severe patients grouped with the less-acute patients. In addition to patient safety, lack of discrimination of the ESI Level 3 distribution challenges efficient resource distribution and overall ED efficiency. The ETS was designed to address this by offering improvements in distributing the majority of lower acuity patients (i.e., 30% of patients in each triage Level 3 to 5). This may facilitate substantial gains in patient prioritization and distribution to different service areas and providers that are critical to ED operations. Although we propose the specific even distribution, the ETS may be customized to individual ED objectives for risk stratification and distribution of patients to match existing resource allocation schemes. The ETS uses critical and time-sensitive outcomes (mortality, admission to the ICU, or direct transport to the OR or cardiovascular catheterization suite) as a proxy for the need for emergent treatment. The ETS was designed for clinical decision support and is calculated immediately based on the first data recorded on a patient’s arrival. It is not intended to replace triage nurse or physician judgment, but to support it by providing the score as they assess a patient. An electronic system cannot account for details of patient history or appearance, which experienced triage providers recognize to indicate that a patient requires emergent attention. Hence, the assigned ETS score can be over-ridden based on a provider’s clinical concerns, specific types of injuries/illnesses (e.g., lacerations) or related to the availability of specialty services or treatment areas/options (e.g., fast track). However, the ETS can provide more consistent, evidence-based information to assist with making safer triage decisions. Limitations There are several limitations of the proposed the ETS that may decrease its ability to be generalized to all EDs. First, the ETS requires an electronic medical record with a computerized triage system to avoid manual calculations. However, as of 2012, > 85% of EDs in the United States (US) have access to electronic medical records and computerized triage systems (29). Adapting the ETS to different clinical settings and computer systems will take training and software interfaces. The ETS has also only been evaluated for an adult ED population with no inferences towards its utility in pediatric populations. Although the ETS is based on the NHAMCS nationally representative database, and is broadly generalizable, it is subject to any potential errors in the NHAMCS data itself (30). In particular, there is likely variation in
7
application of ESI or potentially other 5 level triage systems within the NHAMCS database introducing the potential for misclassification, which could, in turn, improve the apparent performance of the ETS relative to the ESI. However, ESI is used to triage the majority of ED patient visits in the United States, and our distribution of patients aligns with previous reports of institutions using an ESI triage system, thus impact on these analyses is likely minimal (8,18). Resource utilization is also believed to be under-reported in NHAMCS, which might impact these comparisons in our paper (30). CONCLUSIONS The ETS is a novel electronic triage system that uses commonly obtained triage information to automatically distribute adult ED patients across five severity levels based on critical and time-sensitive outcomes (mortality, admission to the ICU, or direct transport to the OR or cardiovascular catheterization suite). The algorithm was derived and validated using a nationally representative sample of 25,198 adult patient visits (97 million visits weighted) collected by the CDC. Compared to ESI, the ETS may more accurately discern high-acuity patients according to proxies for severity of illness in a medically undifferentiated adult ED population. In addition, the ETS may reduce subjectivity in triage evaluation, while more evenly distributing patients among lower-acuity levels to allow more efficient distribution of patients to treatment spaces and providers. Overall, is a critical outcomes-based triage system demonstrating improved differentiation of patients compared to the current US standard triage system, ESI. The ETS will require prospective evaluation in diverse ED care settings to understand its clinical utility and generalizability. Acknowledgments—This work was supported by the Department of Homeland Security (PACER: National Center for Study of Preparedness and Response [2010-ST-061-PA0001]).
REFERENCES 1. Institute of Medicine. The Future of Emergency Care in the United States Health System. Ann Emerg Med 2006;13:1081–5. 2. Seiger N, Van Veen M, Steyerber E, Ruige M, Van Meurs A, Moll A. Undertriage in the Manchester triage system: an assessment of severity and options for improvement. Arch Dis Child 2011;96: 653–7. 3. Haas B, Gomez D, Zagorski B, Stukel T, Rubenfeld G, Nathens A. Survival of the fittest: the hidden cost of undertriage of major trauma. J Am Coll Surg 2010;211:804–11. 4. Yurkova I, Wolf L. Under-triage as a significant factor affecting transfer time between the emergency department and the intensive care unit. J Emerg Nurse 2011;37:491–6.
8 5. Hitchcock M, Gillespie B, Crilly J, Chabover W. Triage: an investigation of the process and potential vulenerabilities. J Adv Nurs 2013;70:1–10. 6. Emergency Severity Index, Version 4: implementation handbook. AHRQ Publication No. 12–0014. Agency for Healthcare Research and Quality, 2011. Available at: http://www.ahrq.gov/research/esi/. Accessed November 1, 2012. 7. Dong SL, Bullard MJ, Meurer DP, et al. Comparing a novel computer triage program with standard triage. Acad Emerg Med 2005;12:502–7. 8. McHugh M, Tanabe P, McClelland M, Khare RK. More patients are triaged using the Emergency Severity Index than any other triage acuity system in the United States. Acad Emerg Med 2012;19: 106–9. 9. Meek R, Phiri W. Australian Triage Scale: consumer perspective. Emerg Med Australas 2005;17:212–7. 10. Manchester Triage Group. Emergency triage: Manchester Triage Group. London: BMJ Publishing Group; 1997. 11. Beveridge R, Clarke B, Janes N. Canadian emergency department triage and acuity scale; implementation guidelines. Can J Emerg Med 1999;1:2–28. 12. Eitel DR, Travers DA, Rosenau AM, Gilboy N, Wuerz RC. The emergency severity index triage algorithm version 2 is reliable and valid. Acad Emerg Med 2003;10:1070–80. 13. Tanabe P, Gimbel R, Yarnold PR, Adams JG. The Emergency Severity Index (version 3) 5-level triage system scores predict ED resource consumption. J Emerg Nurs 2004;30:22–9. 14. Wuerz RC, Milne LW, Eitel DR, Travers D, Gilboy N. Reliability and validity of a new five-level triage instrument. Acad Emerg Med 2000;7:236–42. 15. Fernandes C, Tanabe P, Gilboy N, et al. Five-level triage: a report from the ACEP/ENA Five-Level Triage Task Force. J Emerg Nurs 2005;31:39–50. 16. Tanabe P, Gimbel R, Yarnold PR, Kyriacou DN, Adams JG. Reliability and validity of scores on the Emergency Severity Index version 3. Acad Emerg Med 2004;11:59–65. 17. Wuerz RC, Travers D, Gilboy N, Eitel DR, Rosenau A, Yazhari R. Implementation and refinement of the emergency severity index. Acad Emerg Med 2001;8:170–6.
A. F. Dugas et al. 18. Christ M, Grossmann F, Winter D, Bingisser R, Platz E. Modern triage in the emergency department. Dtsch Arztebl Int 2010;107: 892–8. 19. 2009 NHAMCS Micro-data File Documentation. 2009. Available at: ftp://ftp.cdc.gov/pub/health_statistics/NCHS/Dataset_Docume ntation/NHAMCS/doc09.pdf. Accessed January 15, 2013. 20. NHAMCS scope and sample design. Available at: http://www.cdc. gov/nchs/ahcd/ahcd_scope.htm#nhamcs_scope. Accessed December 25, 2012. 21. Barfod C, Lauritzen MM, Danker JK, et al. Abnormal vital signs are strong predictors for intensive care unit admission and in-hospital mortality in adults triaged in the emergency department—a prospective cohort study. Scand J Trauma Resusc Emerg Med 2012;20:28. 22. Tortora GA. Principles of Anatomy and Physiology. 6th edn. Boston, MA: Addison Wesley. 23. Chobanian AV, Bakris GL, Black HR, et al. The Seventh Report of the Joint National Committee on Prevention, Detection, Evaluation, and Treatment of High Blood Pressure: the JNC 7 report. JAMA 2003;289:2560–72. 24. McGillicuddy DC, O’Connell FJ, Shapiro NI, et al. Emergency department abnormal vital sign ‘‘triggers’’ program improves time to therapy. Acad Emerg Med 2011;18:483–7. 25. Schneider D, Appleton L, McLemore T. A reason for visit classification for ambulatory care. National Center for Health Statistics. Vital Health Stat 2 1979. i–vi, 1–63. 26. Clinical Classifications Software (CCS) for ICD-9-CM. 2012. Available at: http://www.hcup-us.ahrq.gov/toolssoftware/ccs/ccs. jsp. Accessed December 25, 2012. 27. Geisse S. Predictive Inference. New York: Chapman and Hall; 1993. 28. Oredsson S, Jonsson H, Rognes J, et al. A systematic review of triage-related interventions to improve patient flow in emergency departments. Scand J Trauma Resusc Emerg Med 2011;19:43. 29. Charles D, King J, Patel V, Furukawa MF. Adoption of Electronic Health Record Systems among U.S. Non-Federal Acute Care Hospitals: 2008-2012. Available at: https://www.healthit.gov/sites/ default/files/oncdatabrief9final.pdf. Accessed February 25, 2016. 30. McCaig LF, Burt CW. Understanding and Interpreting the National Hospital Ambulatory Medical Care Survey: Key Questions and Answers. Ann Emerg Med 2012;60:716–21.
Electronic Emergency Triage System
ARTICLE SUMMARY 1. Why is this an important topic? Despite administrative changes to reduce the time to see a provider, triage remains an important function to identify seriously ill patients and prevent poor outcomes. 2. What does this study attempt to show? The study attempts to develop a triage system to identify patients based on their probability of a poor outcome, not simply on resource utilization and triage nurse decisions as with ESI. 3. What are the key findings? Using standard screening information the ETS automatically classifies patients and better identifies those with risk of a poor outcome. As importantly, it better distributes patients to each triage level and reduces ‘‘overtriage’’ to a single level (Level 3) that occurs with ESI. 4. How is patient care impacted? As this system can run simultaneously as the patient checks in and preliminary vital signs are recorded, it can provide an immediate estimate of patient risk to any provider conducting a preliminary triage evaluation. Because it is better able to identify patients at risk for poor outcomes, it can also reduce their occurrence and improve patient safety.
9