- Email: [email protected]
Drivers of ED efficiency: a statistical and cluster analysis of volume, staffing, and operations
American Journal of Emergency Medicine 34 (2016) 155–161
Contents lists available at ScienceDirect
American Journal of Emergency Medicine journal homepage: www.elsevier.com/locate/ajem
Original Contribution
Drivers of ED efficiency: a statistical and cluster analysis of volume, staffing, and operations☆,☆☆ David Anderson, PhD a, Laura Pimentel, MD b,⁎, Bruce Golden, PhD c, Edward Wasil, PhD d, Jon Mark Hirshon, MD e a
Zicklin School of Business, Baruch College, City University of New York, New York, NY Department of Emergency Medicine, University of Maryland School of Medicine, Baltimore, MD Robert H. Smith School of Business, University of Maryland, College Park, MD d Kogod School of Business, American University, Washington, DC e Department of Emergency Medicine, University of Maryland School of Medicine, Baltimore, MD b c
a r t i c l e
i n f o
Article history: Received 13 June 2015 Received in revised form 18 September 2015 Accepted 30 September 2015
a b s t r a c t Study objective: The percentage of patients leaving before treatment is completed (LBTC) is an important indicator of emergency department performance. The objective of this study is to identify characteristics of hospital operations that correlate with LBTC rates. Methods: The Emergency Department Benchmarking Alliance 2012 and 2013 cross-sectional national data sets were analyzed using multiple regression and k-means clustering. Significant operational variables affecting LBTC including annual patient volume, percentage of high-acuity patients, percentage of patients admitted to the hospital, number of beds, academic status, waiting times to see a physician, length of stay (LOS), registered nurse (RN) staffing, and physician staffing were identified. LBTC was regressed onto these variables. Because of the strong correlation between waiting times measured as door to first provider (DTFP), we regressed DTFP onto the remaining predictors. Cluster analysis was applied to the data sets to further analyze the impact of individual predictors on LBTC and DTFP. Results: LOS and the time from DTFP were both strongly associated with LBTC rate (P b .001). Patient volume is not significantly associated with LBTC rate (P = .16). Cluster analysis demonstrates that physician and RN staffing ratios correlate with shorter DTFP and lower LBTC. Conclusion: Volume is not the main driver of LBTC. DTFP and LOS are much more strongly associated. We show that operational factors including LOS and physician and RN staffing decisions, factors under the control of hospital and physician executives, correlate with waiting time and, thus, in determining the LBTC rate. © 2015 Elsevier Inc. All rights reserved.
1. Introduction 1.1. Background Emergency departments (EDs) play a critical role within the American health care system, serving both as the key location for the delivery of acute clinical care as well as the primary access pathway for unscheduled hospital admissions [1]. The clinical operations of EDs are complex and affected by many factors [2]. Some factors are intrinsic to the nature of the hospital and catchment community, such as annual hospital volume, patient acuity, and patient socioeconomic status. Other factors relate to hospital management and administrative decisions such as physician and employee staffing, input and throughput
☆ Sources of support: None. ☆☆ Prior presentations: None. ⁎ Corresponding author at: 110 S. Paca St., 6th Fl. Suite 200, Baltimore, MD 21201. E-mail address: [email protected] (L. Pimentel). http://dx.doi.org/10.1016/j.ajem.2015.09.034 0735-6757/© 2015 Elsevier Inc. All rights reserved.
processes, and hospital patient flow [3]. ED crowding is an increasing national problem that adversely impacts patient care and negatively affects ED operations [4]. In response to crowding, The Joint Commission [5] has made ED patient throughput a hospital priority through the adoption of ED-specific core measures. (See Table 1). 1.2. Importance Because of the unique features of individual EDs and the interaction of many disparate characteristics, it is difficult for hospital administrators, ED directors, and ED nurse managers to accurately predict the impact of staffing decisions and process changes on patient flow. One operational parameter important to every ED leader is the percentage of patients who leave before treatment is completed (LBTC). This metric is defined as all patients who leave before being discharged by a physician. This includes patients who leave before or after a medical screening examination, those who elope from the ED, and those who leave the ED against medical advice. Not only is LBTC a risk for both the
156
D. Anderson et al. / American Journal of Emergency Medicine 34 (2016) 155–161
Table 1 The Joint Commission definitions of ED flow measures ED-specific Joint Commission core measures ED 1 ED 2
Median time from ED arrival to ED departure for admitted ED patients Admit decision time to ED departure for admitted patients
patients and the hospital, it can be considered the ultimate expression of a poor patient experience. The recent literature suggests that ED volume is directly correlated with LBTC. Welch et al used 2009 data from the Emergency Department Benchmarking Alliance (EDBA) [6] to demonstrate a direct association between ED volume and acuity with LBTC, door-to-physician times, and length of stay in the ED [7]. Handel et al noted a similar relationship [8]. Implementation of the Affordable Care Act appears to have increased ED volumes [9]; more patients now have insurance coverage, but they do not have adequate access to primary care [10]. Data from Oregon demonstrated a 40% increase in ED usage among patients during the first year of Medicaid coverage subsequent to being without health insurance [11]. Although ED volume appears to play a role in the LBTC rate, we hypothesize that operational factors, such as professional staffing ratios and patient wait time, play an underlying and more important role than volume alone. It is critically important to understand the underlying factors related to ED operations and LBTC to improve the efficiency and effectiveness of high-quality acute care delivery. 1.3. Goals of this investigation Through the use of data from the EDBA, our study examines the impact of ED operational metrics, specifically physician and registered nurse (RN) staffing ratios, on LBTC. Physician staffing ratios are defined
as the number of patients per staffed physician hours per day; RN staffing ratios are the number patients per RN staffed hours per day. Because of the well-known strong statistical correlation between LBTC and the time it takes a patient to see a physician or other provider [12], we also conducted a secondary analysis of operational metrics focusing on the time from door to first provider (DTFP). This interval is defined as the time from the first recorded entry of the patient’s arrival in the ED until the first notation of evaluation by a provider (attending physician, resident physician, or mid-level provider). Using 2012 and 2013 data from the EDBA, we specifically looked at the relative impact of physician and nursing staffing on DTFP in the context of other hospital and ED characteristics. 2. Methods 2.1. Study design This is a retrospective study of 2012 and 2013 EDBA data. EDBA is a nonprofit membership organization that annually collects self-reported ED-level demographic information and performance metrics from EDs across the United States. In 2012, 976 member EDs reported data. This information is collated in a spreadsheet organized into cohorts of departments of 20,000 volume bands for statistical and comparison purposes. The identities of individual EDs are blinded although clustered by state. Data were solicited from participating members at the departmental level in calendar years 2013 and 2014 for calendar years 2012 and 2013, respectively. A spreadsheet is distributed to all members on an annual basis. Because the information is completely free from commercial influence and utilized solely for benchmarking and performance improvement, it is ideal for analysis using statistical techniques. Pediatric, free standing, and urgent care EDs were omitted because of
Fig. 1. A: Histogram of number of ED beds in the data set before and after logarithmic transformation. After transformation, the histogram approaches a normal distribution. B: Histogram of ED volume bands in the data set before and after logarithmic transformation. After transformation, the histogram approximates a normal distribution.
D. Anderson et al. / American Journal of Emergency Medicine 34 (2016) 155–161
the unique features that distinguish them from traditional EDs housed in hospitals. Free standing EDs were excluded because their operational dynamics are fundamentally different from a hospital-based ED in that they do not admit patients to their facilities. This decreases the complexity and crowding created by boarding. They also have the advantage of complete dedication of ancillary services such as laboratory and radiology to ED patients. Of the 976 hospitals in the 2012 data set, 35 were free standing, 30 were pediatric, and 8 were urgent care facilities and, therefore, did not meet inclusion criteria. Of the remaining 903 hospitals, 407 had complete data entries in the fields of interest. The other 496 hospitals were excluded for missing data. Because such a high proportion of observations had missing data, we also analyzed the 2013 EDBA data set to compare the 2012 and 2013 results. Furthermore, we used multiple imputation [13] to replace the missing data in the 2012 data set, as a robustness check, to ensure that there was no bias introduced by the missing data that would change the results. The additional analyses are shown in the Appendix. Finally, we conducted an analysis of the missing data and compared it to the complete data sets. The institutional review board of our institution exempted the study. Permission was obtained from the Research Committee of the EDBA to use the EDBA blinded survey data for the study.
2.2. Data management and analysis We used the 2012 EDBA report to study the demographic, staffing, and operational factors most correlated with LBTC. The data are selfreported and represent the average annual performance on each metric for calendar year 2012. Our primary outcome variable of interest was LBTC. The predictor variables selected were those most commonly associated with ED operational performance based upon the current literature: annual ED patient volume, percentage of high-acuity patients [14] (defined as those with current procedural terminology [CPT] codes of 99291, 99284, and 99285) [15], percentage of patients admitted to the hospital, number of ED beds, average DTFP, length of stay (LOS) in the ED, RN hours staffed, RN staffing ratio, physician hours staffed, and physician staffing ratio. The LOS is the median LOS for all patients from the first recorded time of arrival until physically leaving the ED whether discharged home, admitted to the hospital, or transferred to another facility. For this study, we used a logarithmic transformation and regressed the logarithm of the LBTC rate onto the other ED parameters, as we hypothesized that the DTFP time would be the single most powerful predictor of LBTC. This transformation reduces the skewness of a distribution, makes it look (nearly) normally distributed, and reduces the impact of extreme values. The underlying assumption of any regression model is that you have independent, normally distributed variables. When we have distributions with long tails (like volume or bed size) it is standard to take the logarithm so that it becomes more normal. The only change is that instead of measuring the increase by 1 unit (1 bed, 1 patient, etc), we measure the impact of a 1% increase. Histograms of beds and volume before and after logarithmic transformations are shown in Fig. 1A and B. As a second step, we applied multiple regression to regress the DTFP time on the remaining ED characteristics to identify the significant predictors of DTFP time. Finally, we analyzed the data set using cluster analysis. Using k-means clustering, we grouped the EDs into 5 distinct clusters to better aggregate the EDs based upon their characteristics. Variables measuring the same hospital characteristics (staffing levels, volume, size, acuity of the patient population, etc.) were used as inputs for the clustering technique. We set k = 5 because the percentage of variation explained leveled off after 5 clusters. K-means is a standard clustering technique [16] that iteratively selects cluster centers, and assigns data to the nearest cluster. K-means is a widely used clustering method that has been successfully applied in a variety of settings. It
157
Table 2 Emergency department operational factors associated with left before treatment completed rate, EDBA data 2012 Variable
% Increase in LBTC rate
95% CI
P
Log of DTFP (min) LOS (min) % High CPT acuity Log of physician hours Log of RN hours Academic Log of volume Log of beds Percent admitted Patients per bed
41.907 59.999 −0.387 −13.929 17.351 8.329 49.182 −32.968 22.14 −0.399
[36.451, 47.579] [34.124, 90.865] [−0.577, −0.110] [−24.96, −1.271] [−.462, 38.353] [0.160, 17.163] [−15.49, 163.37] [−62.03, 18.340] [−34.76, 128.69] [−2.433, 1.677]
.00 .00 .01 .05 .06 .07 .16 .17 .52 .71
has been shown to consistently partition data sets into meaningful groupings [17]. 3. Results 3.1. Descriptive statistics We used a data set of 407 EDs for our analysis. ED volumes ranged from 1,575 to 153,600 patients per year, with a median volume of 38,700 patients. An average of 2.25% of patients left before treatment was completed. LBTC rates at individual hospitals ranged from 0% to 14.6%, with an inter-quartile range of 1% to 3%. In our data set, LBTC rate was moderately correlated with the logarithm of volume (r = 0.31). However, the correlation between the logarithm of volume and the logarithm of nurse staffing, and the logarithm of physician staffing was quite high (r = 0.92 and r = 0.85, respectively). We used regression modeling to further investigate these relationships. 4. Main results The time from DTFP, overall LOS, percentage of high-acuity patients (high CPT acuity), and physician hours are the most significant variables in explaining the LBTC rate, as given in Table 2. In the average case, the model estimates that an increase of 1% in physician hours leads to a 0.14% reduction in LBTC rate. These results control for waiting time, hospital size, volume, and academic status, RN staffing, patient mix, and mean treatment time. Our analysis reveals LBTC rate is strongly and significantly associated with DTFP time (P b .0001). To determine the variables that affect waiting time, we regressed the logarithm of waiting time on the factors used in the previous model (patient mix, patient volume, number of beds, RN staffing, physician staffing, academic status, length of treatment). The results are given in Table 3. When we regressed waiting time on the operational factors shown in Table 3, we see that the strongest correlate with how long patients spend in the waiting room is the LOS. Every 1-minute increase in the average LOS correlates with a 0.49% increase in the waiting time. We do not see any significant evidence of staffing levels affecting treatment times. HowTable 3 Emergency department operational factors associated with door-to-first-provider times, EDBA data 2012 Variable
% Increase in DTFP time
95% CI
P
% High CPT acuity LOS (min) Log of volume Log of physician hours Log of RN hours Percent admitted Log of beds Academic
−0.863 0.488 46.228 19.722 −8.607 39.097 1.005 1.005
[−0.929, −0.733] [0.291, 0.685] [4.791, 104.0] [−36.51, 9.899] [−34.50, 27.53] [−57.08, 350.8] [−23.23, 32.89] [−52.97, 116.9]
.00 .00 .02 .20 .57 .58 .93 .99
158
D. Anderson et al. / American Journal of Emergency Medicine 34 (2016) 155–161
Fig. 2. Relationship between hospital volume and nurse staffing level, EDBA data 2012.
ever, RN and physician staffing levels are so strongly correlated with volume that it is impossible to separate the impact of the 2 (r = 0.91 and 0.83, respectively); theoretically, they have opposite effects on waiting times, so it is hard to calculate any impact from staffing decisions. Fig. 2 shows the relationship between hospital volume and RN staffing level. 4.1. Cluster analysis To further clarify the relationship between staffing decisions and ED operations, we used cluster analysis to glean insights from groups in the data set that may not be apparent through regression analysis. Comparing similarities and differences between clusters gives us another way to examine the causes of differences in the LBTC rates in these hospitals. For the cluster analysis, all hospitals in a single cluster are similar with respect to the 11 measures shown in Table 4, whereas hospitals from different clusters are dissimilar. At each iteration, the cluster centers are moved to the point that minimizes the distance to all points assigned to that cluster, and then reassigns points to the nearest center. In Fig. 3, the relationships between the cluster averages are plotted in 2-dimensional space, using multi-dimensional scaling to preserve the relationships between the cluster centers. This plot is helpful in understanding how different the 5 clusters are from each other; the farther apart they are in Fig. 2, the more disparate the clusters are. There were 75 EDs in cluster 1, 103 in cluster 2, 65 in cluster 3, 58 in cluster 4, and 106 in cluster 5. In Table 4, we show the average value and rank order of each of the 11 metrics for each cluster. EDs in cluster 4 have, on average, 57.2%
Fig. 3. Emergency department cluster center locations in two-dimensional space. Coordinate 1 roughly translates to hospital characteristics (volume, acuity, academic status, size), whereas coordinate 2 roughly translates to operational characteristics (staffing decisions, outcomes, admission).
high-CPT-acuity patients, which is the lowest of any of the clusters. At the low end, EDs in cluster 4 have the fewest beds (average logarithm of number of beds = 2.433) and the lowest percentage of admitted patients (average of 12.7%). At the high end, EDs in cluster 2 average 67.9% high-CPT-acuity patients, which was the highest percentage of the 5 clusters. EDs in cluster 2 admitted the second highest percentage of patients (averaging 20.0%). 5. Discussion 5.1. Importance of results Our analysis of factors affecting ED patient flow and efficiency demonstrates the complexity of forces impacting clinical operations. Our initial regression of operational predictors of LBTC (Table 2) found that 4 were statistically significant with DTFP the strongest. RN staffing did not reach statistical significance but trended in the direction of correlation with lower LBTC. Of these 4 significant factors, the only demographic level variable is the percentage of high-acuity patients. This finding is consistent with an Australian study documenting that less urgent patients are more likely to leave without treatment than those triaged as acutely ill [12]. Higher acuity and even critically ill patients leave without treatment, too, however [18]. The other predictors, staffing and LOS, are operational variables strongly affected by management decisions. The importance of
Table 4 Emergency department cluster centers (averages for each cluster) and ranks (lowest to highest value) Cluster no.
LOG BED
LOG VOL
% ADMIT
RN RATIO
DOC RATIO
LOG DTFP
LOG RNH
LOG DOCH
LOS
LBTC
Cluster averages 1 0.646 2 0.679 3 0.616 4 0.572 5 0.653
HICPT
3.802 3.797 2.905 2.433 3.118
11.163 11.167 10.308 9.625 10.536
0.202 0.200 0.165 0.127 0.168
0.621 0.599 0.630 0.558 0.661
3.203 2.350 2.523 1.714 3.106
3.862 2.789 3.732 2.751 2.641
5.766 5.802 4.889 4.368 5.080
4.134 4.450 3.529 3.277 3.547
5.137 5.014 4.870 4.653 4.878
0.044 0.024 0.037 0.018 0.022
Cluster ranks 1 2 3 4 5
5 4 2 1 3
4 5 2 1 3
5 4 2 1 3
3 2 4 1 5
5 2 3 1 4
5 3 4 2 1
4 5 2 1 3
4 5 2 1 3
5 4 2 1 3
5 3 4 1 2
3 5 2 1 4
HICPT, % high current procedural terminology acuity; LOG BED, log of number of beds; LOG VOL, log of annual patient volume; % ADMIT, % of patients admitted; RN RATIO, volume/ registered nurse hours; DOC RATIO, volume/physician hours; LOG DTFP, log of door to first provider time; LOG RNH, log of registered nurse hours; LOG DOCH, log of physician hours; LBTC, % of patients leaving before treatment complete.
D. Anderson et al. / American Journal of Emergency Medicine 34 (2016) 155–161
159
Table 5 Missing data analysis Variable
Log door-to-doctor time
Log of beds
Log of volume
Log of RN hours
Log of MD hours
Log of treatment time
Academic
Percent admitted
Missing Nonmissing P of difference
3.31 3.22 .08
2.92 3.27 0
10.17 10.63 0
5.14 5.24 .26
3.57 3.79 0
4.83 4.95 .00003
0.00098 0.27 0
0.16 0.18 .00007
MD, medical doctor.
physician staffing and operational efficiency in ensuring that patients complete treatment is intuitive and confirmed by our data. Although there is a surface correlation between volume and higher elopement rates as noted by Welch[7] and Handel [8], our analysis demonstrates that this association is negated when analyzed in the context of other operational and staffing predictors.
mortality [21]. This is particularly important because the ED serves as the safety net for a system in which patients may encounter access block when seeking care in outpatient practices. Hospital and physician executives should seriously consider these results when budgeting for emergency services.
5.2. Multicollinearity
5.4. Limitations
Because of the very strong correlation between LBTC and DTFP, we explored the factors driving DTFP. This analysis confirmed the significant relationship between high acuity, LOS, and efficiency as measured by DTFP. In this regression, volume was a statistically important predictor but physician and nursing staffing were not. Analyzing the relationship between volume and staffing, however, we found very high correlations (Fig. 2). In multiple regressions, this relationship among continuous predictors, known as multicollinearity, distorts the measurement of the relationship between the highly correlated predictors with the outcome variable. [19,20] We then turned to cluster analysis to clarify the importance of each individual predictor variable.
Our analyses have several limitations. First, there are missing data, which might lead to bias in the estimates. Our analysis of the 2012 EDBA data set used 407 hospitals with complete data and excluded 496 hospitals with missing data. To investigate the effects of missing data, we analyzed both the 2013 EDBA data set, and used multiple imputation [13] to replace the missing data in the 2012 data set (Appendix). We find that the results are broadly consistent both across years and with the missing data imputed. The source of the missing data is nonresponse in one or more fields of a given ED’s data set. EDs with missing data are smaller, have less doctor staffing, less likely to be academic, and admit a lower fraction of patients. However, there are no differences in door-to-doctor time, or nurse-staffing. All the means are within the range of the other data set. (See Table 5.) These data are cross-sectional and correlational, so there is no ability to see how operational changes impact performance. In addition, because staffing and volume are so heavily correlated, it is hard to fully separate the effects of either one. The EDBA data are self-reported by member organizations. Membership is voluntary and open to hospitals or ED physician groups willing and able to pay a nominal annual membership fee. There is no mechanism in place neither to ensure the validity of the reported data nor to ensure that members are representative of all US EDs. Our staffing analysis considered physician and RN staffing only. We did not consider the impact of physician assistants, nurse practitioners, or ancillary nursing personnel. Because we were unable to identify validated metrics for the impact of these providers on staffing, we did not consider them in our model.
5.3. Cluster analysis significance The cluster analysis results summarized in Table 4 are very instructive in elucidating the importance of staffing to operational measures of efficiency. Cluster 4 had the best performance (shortest treatment times and lowest LBTC rate). This cluster has the smallest size hospitals with the lowest volumes. Characteristic of low volume EDs, Cluster 4 has the lowest patient-to-staff ratios. It also has the lowest percentage of high-acuity patients and the shortest LOS. Because the EDs in this cluster have markedly different operational characteristics than the EDs in the other 4 clusters, it is not surprising that they perform better. The most interesting contrast is between clusters 1 and 2. Both clusters contain large hospitals with approximately 50% academic hospitals (compared to 17% in the entire data set). However, cluster 2 performs much better than cluster 1. Cluster 2 has just more than half of the LBTC rate of cluster 1 (2.39% vs 4.37%, P = .03), and much shorter door-to-doctor times (P = .04). Both clusters are nearly identical in terms of volume, number of beds, acuity, and percent admitted (P N .1 for all), and only differ in terms of staffing (P = .00 for both physician staffing and RN staffing). Cluster 2 has higher RN staffing and much higher physician staffing, which correlates with shorter treatment times, shorter DTFP, and lower LBTC. When we look at Fig. 3, we see that clusters 1 and 2 are very similar with respect to coordinate 1, the demographics of volume and acuity, but very different with respect to coordinate 2, operational processes, staffing, and outcomes. Also of interest is the comparison between the operational performances of cluster 2 with cluster 4. Although very different with respect to size and complexity, staffing and operational outcomes are similar. These results confirm the findings of our regression analysis that volume alone does not correlate with operational inefficiency but is an artifact of multicollinearity with staffing. From a strategic and planning perspective, our results support the hypothesis that better resourced EDs with efficient patient flow processes perform better regardless of volume and acuity. From the perspective of quality, recent literature suggests that patients treated in better resourced EDs and hospitals have better outcomes with lower
6. Conclusions Our analysis demonstrates that the complexity of ED operations and the interaction of multiple factors can lead to incorrect conclusions about variables driving patient flow. By the use of regression and cluster analysis, we have demonstrated that the most important correlate with LBTC is DTFP. We have further identified that physician and RN staffing and ED operations, not volume, are the most important drivers of DTFP times. When high volume EDs are resourced similarly to low volume EDs, operational outcomes are similar. Further research directed toward understanding the complex interaction of demographic and operational variables is encouraged to assist hospital and physician executives with crucial decisions regarding optimal ED resource utilization.
Acknowledgment The authors thank the EDBA for providing access to the data utilized in this study.
160
D. Anderson et al. / American Journal of Emergency Medicine 34 (2016) 155–161
Appendix. Additional analyses: multiple imputation with the 2012 data set and logistic regression with the 2013 data set To evaluate whether the missing data biased our results, we used multiple imputation to replace the missing values in the 2012 data set. Multiple imputation is a method of replacing missing data by predicting its value based on the other observed variables, and accounts for the additional noise added by imputing the missing variables [13]. The mi library was used in the R programming language to perform the multiple imputation analysis. As an additional analysis, we replicated the analysis on the 2013 EDBA data set. In the 2013 EDBA data set, we removed pediatric, free standing, and urgent care EDs, and hospitals with missing data were excluded, leaving 507 EDs in the 2013 sample. The results from both analyses were broadly consistent with the findings presented in the Results section. Table A1 shows that the main drivers of the LBTC rate in the 2012 data set (using multiple imputation) are still length of treatment and time to first provider. The only change is that instead of seeing percent high acuity patients are positively associated with the LBTC rate, we see the percentage of patients who are admitted having the effect. This is not a major change, as both are measures of the severity, urgency, and resource intensiveness of patients, and the 2 measures are correlated (r = 0.5).
Table A1 Emergency department operational factors associated with left before treatment completed using multiple imputation, EDBA data 2012 Variable
% Increase in LBTC rate
95% CI
P
LOS (min) Log of DTFP (min) Percent admitted Log of RN hours Log of beds Academic Log of volume Log of physician hours % High CPT acuity Patients per bed
0.015 0.803 −9.335 0.300 −1.292 0.100 0.300 −0.300 −0.014 0.050
[0.0132, 0.017] [0.645, 0.961] [−0.141, −0.042] [−0.091, 0.694] [−2.827, 0.268] [−0.056, 0.257] [−0.091, 0.694] [−0.689, 0.092] [−0.048, 0.021] [−0.145, 0.246]
.000 .000 .001 .137 .146 .196 .288 .330 .486 .618
Table A2 shows the results of the LBTC regression using the 2013 data on the 507 hospitals with complete data. Again, first provider time and length of treatment are significantly associated with the LBTC rate. We also see some evidence that increased physician staffing is associated with decreased LBTC rate, which is consistent with our earlier findings in the Results section.
Table A2 Emergency department operational factors associated with left before treatment completed, EDBA data 2013 Variable
Percentage increase
95% CI
P
Log of DTFP (min) LOS (min) Log of physician hours Patients per bed Academic Percent admitted Log of beds % High CPT acuity Log of RN hours Log of volume
77.004 149.927 −34.884 0.100 −6.106 −0.349 45.208 −0.118 2.327 −2.566
[63.33, 91.81] [94.09, 221.8] [−49.92, −15.32] [0.003, 0.196] [−17.17, 6.443] [−0.755, 0.731] [−39.89, 250.7] [−0.486, 0.515] [−19.75, 30.47] [−57.80, 124.9]
.000 .000 .002 .048 .326 .390 .407 .651 .856 .951
Tables A3 and A4 also show results consistent with our main findings when examining door-to-first-provider time in both the 2012 data set with missing observations replaced using multiple imputation
and the 2013 data set. Again, in both regression models, the main driver of long door-to-first-provider times was the length of treatment. In addition, the percentage of high-acuity patients was associated with decreased first provider time in both models. The results of both models are consistent with the regression results presented earlier in the paper. Table A3 Emergency department operational factors associated with door-to-first provider times using multiple imputation, EDBA data 2012 Variable
Percent increase
95% CI
P
% High CPT acuity Log of volume LOS (minutes) Log of physician hours Log of beds Log of RN hours Academic Percent admitted
−0.993 36.615 0.401 −18.698 −10.774 −1.193 3.769 1.284
[−0.998, −0.959] [18.86, 57.01] [0.204, 0.597] [−31.03, 0.864] [−23.72, 4.372] [−16.02, 16.26] [−7.924, 16.94] [−0.866, 37.94]
.000 .000 .000 .071 .216 .318 .558 .620
Table A4 Emergency department operational factors associated with door-to-first provider times using multiple imputation, EDBA data 2013 Variable
Percentage Increase
95% C.I.
P value
% High CPT acuity LOS (minutes) Percent admitted Log of volume Log of physician hours Academic Log of beds Patients per bed Log of RN hours
−0.789 47.703 0.547 −8.332 −22.392 10.619 44.023 0.022 9.856
[−0.883, −0.617] [11.10, 96.35] [−0.376, 2.841] [−64.50, 136.7] [−42.38, 4.542] [−4.075, 27.56] [−46.99, 291.3] [−0.041, 0.085] [−16.50, 44.54]
0.000 0.008 0.044 0.086 0.096 0.166 0.475 0.495 0.503
The additional analyses on a separate data set (2013 EDBA) and the analyses using multiple imputation to replace missing values in the 2012 data set support that the conclusions discussed earlier in the Results section are both internally and externally consistent. The variables associated with the LBTC rate and the DTFP time are consistent from year to year when data sets with complete observations or those with missing observations replaced by multiple imputation methods are used. The results and interpretation of the data are consistent across 2 years, and the missing data do not significantly alter the results of the analyses.
References [1] Gonzalez M, Bauhoff S, Blanchard M, Abir M, Iyer N, Smith A, et al. The Evolving Role of Emergency Departments in the United States Rand Corporation; 2013. [2] Crane J, Noon C. The definitive guide to emergency department operational improvement : employing lean principles with current ED best practices to create the "no wait" department. Boca Raton: CRC Press; 2011. [3] Zun LS. Analysis of the literature on emergency department throughput. West J Emerg Med 2009;10(2):104–9. [4] Pines JM, Decker SL, Hu T. Exogenous predictors of national performance measures for emergency department crowding. Ann Emerg Med 2012;60(3):293–8. [5] The Joint Commission; 2014[Oakbrook Terrace, IL]. [6] Emergency Department Benchmarking Alliance; 2015[Newark, DE]. [7] Welch SJ, Augustine JJ, Dong L, Savitz LA, Snow G, James BC. Volume-related differences in emergency department performance. Jt Comm J Qual Patient Saf 2012; 38(9):395–402. [8] Handel DA, Fu R, Vu E, Augustine JJ, Hsia RY, Shufflebarger CM, et al. Association of emergency department and hospital characteristics with elopements and length of stay. J Emerg Med 2014;46(6):839–46. [9] Baldyga M. ER Visits Continue to Rise Since Implementation of Affordable Care Act. ACEP Newsroom. Dallas, TX: American College of Emergency Physicians; 2015. [10] Block L, Cook BG, Hanyok LA, de la Torre D, Rogers MM, Noronha G, et al. Coverage isn't enough: building primary care capacity in the setting of health reform. J Health Care Poor Underserved 2014;25(1):25–8.
D. Anderson et al. / American Journal of Emergency Medicine 34 (2016) 155–161 [11] Taubman SL, Allen HL, Wright BJ, Baicker K, Finkelstein AN. Medicaid increases emergency-department use: evidence from Oregon's Health Insurance Experiment. Science 2014;343(6168):263–8. [12] Mohsin M, Forero R, Ieraci S, Bauman AE, Young L, Santiano N. A population followup study of patients who left an emergency department without being seen by a medical officer. Emerg Med J 2007;24(3):175–9. [13] Rubin D. Multiple Imputation after 18+ years. J Am Stat Assoc 1996;91(434):473–89. [14] McMullan JT, Veser FH. Emergency department volume and acuity as factors in patients leaving without treatment. South Med J 2004;97(8):729–33. [15] American Medical Association: Current Procedural Terminology; 2014[Chicago, IL]. [16] Hartigan J, Mancheck A, Wong. Algorithm AS 136: A k-means clustering algorithm. Appl Stat 1979;28(1):100–8.
161
[17] Jain A. Data clustering: 50 years beyond K-means. Pattern Recognit Lett 2010;31(8): 651–66. [18] Rowe BH, Channan P, Bullard M, Blitz S, Saunders LD, Rosychuk RJ, et al. Characteristics of patients who leave emergency departments without being seen. Acad Emerg Med 2006;13(8):848–52. [19] Belsley DA. Regression diagnostics : identifying influential data and sources of collinearity. Hoboken, N.J.: Wiley-Interscience; 2004 [20] Yoo W, Mayberry R, Bae S, Singh K, Peter He Q, Lillard Jr JW. A Study of Effects of MultiCollinearity in the Multivariable Analysis. Int J Appl Sci Technol 2014;4(5): 9–19. [21] Doyle J, Graves J, Gruber J, Kleiner S. Measuring returns to hospital care: Evidence from ambulance referral patterns. J Pol Econ 2015;123(1):170–214.
Contents lists available at ScienceDirect
American Journal of Emergency Medicine journal homepage: www.elsevier.com/locate/ajem
Original Contribution
Drivers of ED efficiency: a statistical and cluster analysis of volume, staffing, and operations☆,☆☆ David Anderson, PhD a, Laura Pimentel, MD b,⁎, Bruce Golden, PhD c, Edward Wasil, PhD d, Jon Mark Hirshon, MD e a
Zicklin School of Business, Baruch College, City University of New York, New York, NY Department of Emergency Medicine, University of Maryland School of Medicine, Baltimore, MD Robert H. Smith School of Business, University of Maryland, College Park, MD d Kogod School of Business, American University, Washington, DC e Department of Emergency Medicine, University of Maryland School of Medicine, Baltimore, MD b c
a r t i c l e
i n f o
Article history: Received 13 June 2015 Received in revised form 18 September 2015 Accepted 30 September 2015
a b s t r a c t Study objective: The percentage of patients leaving before treatment is completed (LBTC) is an important indicator of emergency department performance. The objective of this study is to identify characteristics of hospital operations that correlate with LBTC rates. Methods: The Emergency Department Benchmarking Alliance 2012 and 2013 cross-sectional national data sets were analyzed using multiple regression and k-means clustering. Significant operational variables affecting LBTC including annual patient volume, percentage of high-acuity patients, percentage of patients admitted to the hospital, number of beds, academic status, waiting times to see a physician, length of stay (LOS), registered nurse (RN) staffing, and physician staffing were identified. LBTC was regressed onto these variables. Because of the strong correlation between waiting times measured as door to first provider (DTFP), we regressed DTFP onto the remaining predictors. Cluster analysis was applied to the data sets to further analyze the impact of individual predictors on LBTC and DTFP. Results: LOS and the time from DTFP were both strongly associated with LBTC rate (P b .001). Patient volume is not significantly associated with LBTC rate (P = .16). Cluster analysis demonstrates that physician and RN staffing ratios correlate with shorter DTFP and lower LBTC. Conclusion: Volume is not the main driver of LBTC. DTFP and LOS are much more strongly associated. We show that operational factors including LOS and physician and RN staffing decisions, factors under the control of hospital and physician executives, correlate with waiting time and, thus, in determining the LBTC rate. © 2015 Elsevier Inc. All rights reserved.
1. Introduction 1.1. Background Emergency departments (EDs) play a critical role within the American health care system, serving both as the key location for the delivery of acute clinical care as well as the primary access pathway for unscheduled hospital admissions [1]. The clinical operations of EDs are complex and affected by many factors [2]. Some factors are intrinsic to the nature of the hospital and catchment community, such as annual hospital volume, patient acuity, and patient socioeconomic status. Other factors relate to hospital management and administrative decisions such as physician and employee staffing, input and throughput
☆ Sources of support: None. ☆☆ Prior presentations: None. ⁎ Corresponding author at: 110 S. Paca St., 6th Fl. Suite 200, Baltimore, MD 21201. E-mail address: [email protected] (L. Pimentel). http://dx.doi.org/10.1016/j.ajem.2015.09.034 0735-6757/© 2015 Elsevier Inc. All rights reserved.
processes, and hospital patient flow [3]. ED crowding is an increasing national problem that adversely impacts patient care and negatively affects ED operations [4]. In response to crowding, The Joint Commission [5] has made ED patient throughput a hospital priority through the adoption of ED-specific core measures. (See Table 1). 1.2. Importance Because of the unique features of individual EDs and the interaction of many disparate characteristics, it is difficult for hospital administrators, ED directors, and ED nurse managers to accurately predict the impact of staffing decisions and process changes on patient flow. One operational parameter important to every ED leader is the percentage of patients who leave before treatment is completed (LBTC). This metric is defined as all patients who leave before being discharged by a physician. This includes patients who leave before or after a medical screening examination, those who elope from the ED, and those who leave the ED against medical advice. Not only is LBTC a risk for both the
156
D. Anderson et al. / American Journal of Emergency Medicine 34 (2016) 155–161
Table 1 The Joint Commission definitions of ED flow measures ED-specific Joint Commission core measures ED 1 ED 2
Median time from ED arrival to ED departure for admitted ED patients Admit decision time to ED departure for admitted patients
patients and the hospital, it can be considered the ultimate expression of a poor patient experience. The recent literature suggests that ED volume is directly correlated with LBTC. Welch et al used 2009 data from the Emergency Department Benchmarking Alliance (EDBA) [6] to demonstrate a direct association between ED volume and acuity with LBTC, door-to-physician times, and length of stay in the ED [7]. Handel et al noted a similar relationship [8]. Implementation of the Affordable Care Act appears to have increased ED volumes [9]; more patients now have insurance coverage, but they do not have adequate access to primary care [10]. Data from Oregon demonstrated a 40% increase in ED usage among patients during the first year of Medicaid coverage subsequent to being without health insurance [11]. Although ED volume appears to play a role in the LBTC rate, we hypothesize that operational factors, such as professional staffing ratios and patient wait time, play an underlying and more important role than volume alone. It is critically important to understand the underlying factors related to ED operations and LBTC to improve the efficiency and effectiveness of high-quality acute care delivery. 1.3. Goals of this investigation Through the use of data from the EDBA, our study examines the impact of ED operational metrics, specifically physician and registered nurse (RN) staffing ratios, on LBTC. Physician staffing ratios are defined
as the number of patients per staffed physician hours per day; RN staffing ratios are the number patients per RN staffed hours per day. Because of the well-known strong statistical correlation between LBTC and the time it takes a patient to see a physician or other provider [12], we also conducted a secondary analysis of operational metrics focusing on the time from door to first provider (DTFP). This interval is defined as the time from the first recorded entry of the patient’s arrival in the ED until the first notation of evaluation by a provider (attending physician, resident physician, or mid-level provider). Using 2012 and 2013 data from the EDBA, we specifically looked at the relative impact of physician and nursing staffing on DTFP in the context of other hospital and ED characteristics. 2. Methods 2.1. Study design This is a retrospective study of 2012 and 2013 EDBA data. EDBA is a nonprofit membership organization that annually collects self-reported ED-level demographic information and performance metrics from EDs across the United States. In 2012, 976 member EDs reported data. This information is collated in a spreadsheet organized into cohorts of departments of 20,000 volume bands for statistical and comparison purposes. The identities of individual EDs are blinded although clustered by state. Data were solicited from participating members at the departmental level in calendar years 2013 and 2014 for calendar years 2012 and 2013, respectively. A spreadsheet is distributed to all members on an annual basis. Because the information is completely free from commercial influence and utilized solely for benchmarking and performance improvement, it is ideal for analysis using statistical techniques. Pediatric, free standing, and urgent care EDs were omitted because of
Fig. 1. A: Histogram of number of ED beds in the data set before and after logarithmic transformation. After transformation, the histogram approaches a normal distribution. B: Histogram of ED volume bands in the data set before and after logarithmic transformation. After transformation, the histogram approximates a normal distribution.
D. Anderson et al. / American Journal of Emergency Medicine 34 (2016) 155–161
the unique features that distinguish them from traditional EDs housed in hospitals. Free standing EDs were excluded because their operational dynamics are fundamentally different from a hospital-based ED in that they do not admit patients to their facilities. This decreases the complexity and crowding created by boarding. They also have the advantage of complete dedication of ancillary services such as laboratory and radiology to ED patients. Of the 976 hospitals in the 2012 data set, 35 were free standing, 30 were pediatric, and 8 were urgent care facilities and, therefore, did not meet inclusion criteria. Of the remaining 903 hospitals, 407 had complete data entries in the fields of interest. The other 496 hospitals were excluded for missing data. Because such a high proportion of observations had missing data, we also analyzed the 2013 EDBA data set to compare the 2012 and 2013 results. Furthermore, we used multiple imputation [13] to replace the missing data in the 2012 data set, as a robustness check, to ensure that there was no bias introduced by the missing data that would change the results. The additional analyses are shown in the Appendix. Finally, we conducted an analysis of the missing data and compared it to the complete data sets. The institutional review board of our institution exempted the study. Permission was obtained from the Research Committee of the EDBA to use the EDBA blinded survey data for the study.
2.2. Data management and analysis We used the 2012 EDBA report to study the demographic, staffing, and operational factors most correlated with LBTC. The data are selfreported and represent the average annual performance on each metric for calendar year 2012. Our primary outcome variable of interest was LBTC. The predictor variables selected were those most commonly associated with ED operational performance based upon the current literature: annual ED patient volume, percentage of high-acuity patients [14] (defined as those with current procedural terminology [CPT] codes of 99291, 99284, and 99285) [15], percentage of patients admitted to the hospital, number of ED beds, average DTFP, length of stay (LOS) in the ED, RN hours staffed, RN staffing ratio, physician hours staffed, and physician staffing ratio. The LOS is the median LOS for all patients from the first recorded time of arrival until physically leaving the ED whether discharged home, admitted to the hospital, or transferred to another facility. For this study, we used a logarithmic transformation and regressed the logarithm of the LBTC rate onto the other ED parameters, as we hypothesized that the DTFP time would be the single most powerful predictor of LBTC. This transformation reduces the skewness of a distribution, makes it look (nearly) normally distributed, and reduces the impact of extreme values. The underlying assumption of any regression model is that you have independent, normally distributed variables. When we have distributions with long tails (like volume or bed size) it is standard to take the logarithm so that it becomes more normal. The only change is that instead of measuring the increase by 1 unit (1 bed, 1 patient, etc), we measure the impact of a 1% increase. Histograms of beds and volume before and after logarithmic transformations are shown in Fig. 1A and B. As a second step, we applied multiple regression to regress the DTFP time on the remaining ED characteristics to identify the significant predictors of DTFP time. Finally, we analyzed the data set using cluster analysis. Using k-means clustering, we grouped the EDs into 5 distinct clusters to better aggregate the EDs based upon their characteristics. Variables measuring the same hospital characteristics (staffing levels, volume, size, acuity of the patient population, etc.) were used as inputs for the clustering technique. We set k = 5 because the percentage of variation explained leveled off after 5 clusters. K-means is a standard clustering technique [16] that iteratively selects cluster centers, and assigns data to the nearest cluster. K-means is a widely used clustering method that has been successfully applied in a variety of settings. It
157
Table 2 Emergency department operational factors associated with left before treatment completed rate, EDBA data 2012 Variable
% Increase in LBTC rate
95% CI
P
Log of DTFP (min) LOS (min) % High CPT acuity Log of physician hours Log of RN hours Academic Log of volume Log of beds Percent admitted Patients per bed
41.907 59.999 −0.387 −13.929 17.351 8.329 49.182 −32.968 22.14 −0.399
[36.451, 47.579] [34.124, 90.865] [−0.577, −0.110] [−24.96, −1.271] [−.462, 38.353] [0.160, 17.163] [−15.49, 163.37] [−62.03, 18.340] [−34.76, 128.69] [−2.433, 1.677]
.00 .00 .01 .05 .06 .07 .16 .17 .52 .71
has been shown to consistently partition data sets into meaningful groupings [17]. 3. Results 3.1. Descriptive statistics We used a data set of 407 EDs for our analysis. ED volumes ranged from 1,575 to 153,600 patients per year, with a median volume of 38,700 patients. An average of 2.25% of patients left before treatment was completed. LBTC rates at individual hospitals ranged from 0% to 14.6%, with an inter-quartile range of 1% to 3%. In our data set, LBTC rate was moderately correlated with the logarithm of volume (r = 0.31). However, the correlation between the logarithm of volume and the logarithm of nurse staffing, and the logarithm of physician staffing was quite high (r = 0.92 and r = 0.85, respectively). We used regression modeling to further investigate these relationships. 4. Main results The time from DTFP, overall LOS, percentage of high-acuity patients (high CPT acuity), and physician hours are the most significant variables in explaining the LBTC rate, as given in Table 2. In the average case, the model estimates that an increase of 1% in physician hours leads to a 0.14% reduction in LBTC rate. These results control for waiting time, hospital size, volume, and academic status, RN staffing, patient mix, and mean treatment time. Our analysis reveals LBTC rate is strongly and significantly associated with DTFP time (P b .0001). To determine the variables that affect waiting time, we regressed the logarithm of waiting time on the factors used in the previous model (patient mix, patient volume, number of beds, RN staffing, physician staffing, academic status, length of treatment). The results are given in Table 3. When we regressed waiting time on the operational factors shown in Table 3, we see that the strongest correlate with how long patients spend in the waiting room is the LOS. Every 1-minute increase in the average LOS correlates with a 0.49% increase in the waiting time. We do not see any significant evidence of staffing levels affecting treatment times. HowTable 3 Emergency department operational factors associated with door-to-first-provider times, EDBA data 2012 Variable
% Increase in DTFP time
95% CI
P
% High CPT acuity LOS (min) Log of volume Log of physician hours Log of RN hours Percent admitted Log of beds Academic
−0.863 0.488 46.228 19.722 −8.607 39.097 1.005 1.005
[−0.929, −0.733] [0.291, 0.685] [4.791, 104.0] [−36.51, 9.899] [−34.50, 27.53] [−57.08, 350.8] [−23.23, 32.89] [−52.97, 116.9]
.00 .00 .02 .20 .57 .58 .93 .99
158
D. Anderson et al. / American Journal of Emergency Medicine 34 (2016) 155–161
Fig. 2. Relationship between hospital volume and nurse staffing level, EDBA data 2012.
ever, RN and physician staffing levels are so strongly correlated with volume that it is impossible to separate the impact of the 2 (r = 0.91 and 0.83, respectively); theoretically, they have opposite effects on waiting times, so it is hard to calculate any impact from staffing decisions. Fig. 2 shows the relationship between hospital volume and RN staffing level. 4.1. Cluster analysis To further clarify the relationship between staffing decisions and ED operations, we used cluster analysis to glean insights from groups in the data set that may not be apparent through regression analysis. Comparing similarities and differences between clusters gives us another way to examine the causes of differences in the LBTC rates in these hospitals. For the cluster analysis, all hospitals in a single cluster are similar with respect to the 11 measures shown in Table 4, whereas hospitals from different clusters are dissimilar. At each iteration, the cluster centers are moved to the point that minimizes the distance to all points assigned to that cluster, and then reassigns points to the nearest center. In Fig. 3, the relationships between the cluster averages are plotted in 2-dimensional space, using multi-dimensional scaling to preserve the relationships between the cluster centers. This plot is helpful in understanding how different the 5 clusters are from each other; the farther apart they are in Fig. 2, the more disparate the clusters are. There were 75 EDs in cluster 1, 103 in cluster 2, 65 in cluster 3, 58 in cluster 4, and 106 in cluster 5. In Table 4, we show the average value and rank order of each of the 11 metrics for each cluster. EDs in cluster 4 have, on average, 57.2%
Fig. 3. Emergency department cluster center locations in two-dimensional space. Coordinate 1 roughly translates to hospital characteristics (volume, acuity, academic status, size), whereas coordinate 2 roughly translates to operational characteristics (staffing decisions, outcomes, admission).
high-CPT-acuity patients, which is the lowest of any of the clusters. At the low end, EDs in cluster 4 have the fewest beds (average logarithm of number of beds = 2.433) and the lowest percentage of admitted patients (average of 12.7%). At the high end, EDs in cluster 2 average 67.9% high-CPT-acuity patients, which was the highest percentage of the 5 clusters. EDs in cluster 2 admitted the second highest percentage of patients (averaging 20.0%). 5. Discussion 5.1. Importance of results Our analysis of factors affecting ED patient flow and efficiency demonstrates the complexity of forces impacting clinical operations. Our initial regression of operational predictors of LBTC (Table 2) found that 4 were statistically significant with DTFP the strongest. RN staffing did not reach statistical significance but trended in the direction of correlation with lower LBTC. Of these 4 significant factors, the only demographic level variable is the percentage of high-acuity patients. This finding is consistent with an Australian study documenting that less urgent patients are more likely to leave without treatment than those triaged as acutely ill [12]. Higher acuity and even critically ill patients leave without treatment, too, however [18]. The other predictors, staffing and LOS, are operational variables strongly affected by management decisions. The importance of
Table 4 Emergency department cluster centers (averages for each cluster) and ranks (lowest to highest value) Cluster no.
LOG BED
LOG VOL
% ADMIT
RN RATIO
DOC RATIO
LOG DTFP
LOG RNH
LOG DOCH
LOS
LBTC
Cluster averages 1 0.646 2 0.679 3 0.616 4 0.572 5 0.653
HICPT
3.802 3.797 2.905 2.433 3.118
11.163 11.167 10.308 9.625 10.536
0.202 0.200 0.165 0.127 0.168
0.621 0.599 0.630 0.558 0.661
3.203 2.350 2.523 1.714 3.106
3.862 2.789 3.732 2.751 2.641
5.766 5.802 4.889 4.368 5.080
4.134 4.450 3.529 3.277 3.547
5.137 5.014 4.870 4.653 4.878
0.044 0.024 0.037 0.018 0.022
Cluster ranks 1 2 3 4 5
5 4 2 1 3
4 5 2 1 3
5 4 2 1 3
3 2 4 1 5
5 2 3 1 4
5 3 4 2 1
4 5 2 1 3
4 5 2 1 3
5 4 2 1 3
5 3 4 1 2
3 5 2 1 4
HICPT, % high current procedural terminology acuity; LOG BED, log of number of beds; LOG VOL, log of annual patient volume; % ADMIT, % of patients admitted; RN RATIO, volume/ registered nurse hours; DOC RATIO, volume/physician hours; LOG DTFP, log of door to first provider time; LOG RNH, log of registered nurse hours; LOG DOCH, log of physician hours; LBTC, % of patients leaving before treatment complete.
D. Anderson et al. / American Journal of Emergency Medicine 34 (2016) 155–161
159
Table 5 Missing data analysis Variable
Log door-to-doctor time
Log of beds
Log of volume
Log of RN hours
Log of MD hours
Log of treatment time
Academic
Percent admitted
Missing Nonmissing P of difference
3.31 3.22 .08
2.92 3.27 0
10.17 10.63 0
5.14 5.24 .26
3.57 3.79 0
4.83 4.95 .00003
0.00098 0.27 0
0.16 0.18 .00007
MD, medical doctor.
physician staffing and operational efficiency in ensuring that patients complete treatment is intuitive and confirmed by our data. Although there is a surface correlation between volume and higher elopement rates as noted by Welch[7] and Handel [8], our analysis demonstrates that this association is negated when analyzed in the context of other operational and staffing predictors.
mortality [21]. This is particularly important because the ED serves as the safety net for a system in which patients may encounter access block when seeking care in outpatient practices. Hospital and physician executives should seriously consider these results when budgeting for emergency services.
5.2. Multicollinearity
5.4. Limitations
Because of the very strong correlation between LBTC and DTFP, we explored the factors driving DTFP. This analysis confirmed the significant relationship between high acuity, LOS, and efficiency as measured by DTFP. In this regression, volume was a statistically important predictor but physician and nursing staffing were not. Analyzing the relationship between volume and staffing, however, we found very high correlations (Fig. 2). In multiple regressions, this relationship among continuous predictors, known as multicollinearity, distorts the measurement of the relationship between the highly correlated predictors with the outcome variable. [19,20] We then turned to cluster analysis to clarify the importance of each individual predictor variable.
Our analyses have several limitations. First, there are missing data, which might lead to bias in the estimates. Our analysis of the 2012 EDBA data set used 407 hospitals with complete data and excluded 496 hospitals with missing data. To investigate the effects of missing data, we analyzed both the 2013 EDBA data set, and used multiple imputation [13] to replace the missing data in the 2012 data set (Appendix). We find that the results are broadly consistent both across years and with the missing data imputed. The source of the missing data is nonresponse in one or more fields of a given ED’s data set. EDs with missing data are smaller, have less doctor staffing, less likely to be academic, and admit a lower fraction of patients. However, there are no differences in door-to-doctor time, or nurse-staffing. All the means are within the range of the other data set. (See Table 5.) These data are cross-sectional and correlational, so there is no ability to see how operational changes impact performance. In addition, because staffing and volume are so heavily correlated, it is hard to fully separate the effects of either one. The EDBA data are self-reported by member organizations. Membership is voluntary and open to hospitals or ED physician groups willing and able to pay a nominal annual membership fee. There is no mechanism in place neither to ensure the validity of the reported data nor to ensure that members are representative of all US EDs. Our staffing analysis considered physician and RN staffing only. We did not consider the impact of physician assistants, nurse practitioners, or ancillary nursing personnel. Because we were unable to identify validated metrics for the impact of these providers on staffing, we did not consider them in our model.
5.3. Cluster analysis significance The cluster analysis results summarized in Table 4 are very instructive in elucidating the importance of staffing to operational measures of efficiency. Cluster 4 had the best performance (shortest treatment times and lowest LBTC rate). This cluster has the smallest size hospitals with the lowest volumes. Characteristic of low volume EDs, Cluster 4 has the lowest patient-to-staff ratios. It also has the lowest percentage of high-acuity patients and the shortest LOS. Because the EDs in this cluster have markedly different operational characteristics than the EDs in the other 4 clusters, it is not surprising that they perform better. The most interesting contrast is between clusters 1 and 2. Both clusters contain large hospitals with approximately 50% academic hospitals (compared to 17% in the entire data set). However, cluster 2 performs much better than cluster 1. Cluster 2 has just more than half of the LBTC rate of cluster 1 (2.39% vs 4.37%, P = .03), and much shorter door-to-doctor times (P = .04). Both clusters are nearly identical in terms of volume, number of beds, acuity, and percent admitted (P N .1 for all), and only differ in terms of staffing (P = .00 for both physician staffing and RN staffing). Cluster 2 has higher RN staffing and much higher physician staffing, which correlates with shorter treatment times, shorter DTFP, and lower LBTC. When we look at Fig. 3, we see that clusters 1 and 2 are very similar with respect to coordinate 1, the demographics of volume and acuity, but very different with respect to coordinate 2, operational processes, staffing, and outcomes. Also of interest is the comparison between the operational performances of cluster 2 with cluster 4. Although very different with respect to size and complexity, staffing and operational outcomes are similar. These results confirm the findings of our regression analysis that volume alone does not correlate with operational inefficiency but is an artifact of multicollinearity with staffing. From a strategic and planning perspective, our results support the hypothesis that better resourced EDs with efficient patient flow processes perform better regardless of volume and acuity. From the perspective of quality, recent literature suggests that patients treated in better resourced EDs and hospitals have better outcomes with lower
6. Conclusions Our analysis demonstrates that the complexity of ED operations and the interaction of multiple factors can lead to incorrect conclusions about variables driving patient flow. By the use of regression and cluster analysis, we have demonstrated that the most important correlate with LBTC is DTFP. We have further identified that physician and RN staffing and ED operations, not volume, are the most important drivers of DTFP times. When high volume EDs are resourced similarly to low volume EDs, operational outcomes are similar. Further research directed toward understanding the complex interaction of demographic and operational variables is encouraged to assist hospital and physician executives with crucial decisions regarding optimal ED resource utilization.
Acknowledgment The authors thank the EDBA for providing access to the data utilized in this study.
160
D. Anderson et al. / American Journal of Emergency Medicine 34 (2016) 155–161
Appendix. Additional analyses: multiple imputation with the 2012 data set and logistic regression with the 2013 data set To evaluate whether the missing data biased our results, we used multiple imputation to replace the missing values in the 2012 data set. Multiple imputation is a method of replacing missing data by predicting its value based on the other observed variables, and accounts for the additional noise added by imputing the missing variables [13]. The mi library was used in the R programming language to perform the multiple imputation analysis. As an additional analysis, we replicated the analysis on the 2013 EDBA data set. In the 2013 EDBA data set, we removed pediatric, free standing, and urgent care EDs, and hospitals with missing data were excluded, leaving 507 EDs in the 2013 sample. The results from both analyses were broadly consistent with the findings presented in the Results section. Table A1 shows that the main drivers of the LBTC rate in the 2012 data set (using multiple imputation) are still length of treatment and time to first provider. The only change is that instead of seeing percent high acuity patients are positively associated with the LBTC rate, we see the percentage of patients who are admitted having the effect. This is not a major change, as both are measures of the severity, urgency, and resource intensiveness of patients, and the 2 measures are correlated (r = 0.5).
Table A1 Emergency department operational factors associated with left before treatment completed using multiple imputation, EDBA data 2012 Variable
% Increase in LBTC rate
95% CI
P
LOS (min) Log of DTFP (min) Percent admitted Log of RN hours Log of beds Academic Log of volume Log of physician hours % High CPT acuity Patients per bed
0.015 0.803 −9.335 0.300 −1.292 0.100 0.300 −0.300 −0.014 0.050
[0.0132, 0.017] [0.645, 0.961] [−0.141, −0.042] [−0.091, 0.694] [−2.827, 0.268] [−0.056, 0.257] [−0.091, 0.694] [−0.689, 0.092] [−0.048, 0.021] [−0.145, 0.246]
.000 .000 .001 .137 .146 .196 .288 .330 .486 .618
Table A2 shows the results of the LBTC regression using the 2013 data on the 507 hospitals with complete data. Again, first provider time and length of treatment are significantly associated with the LBTC rate. We also see some evidence that increased physician staffing is associated with decreased LBTC rate, which is consistent with our earlier findings in the Results section.
Table A2 Emergency department operational factors associated with left before treatment completed, EDBA data 2013 Variable
Percentage increase
95% CI
P
Log of DTFP (min) LOS (min) Log of physician hours Patients per bed Academic Percent admitted Log of beds % High CPT acuity Log of RN hours Log of volume
77.004 149.927 −34.884 0.100 −6.106 −0.349 45.208 −0.118 2.327 −2.566
[63.33, 91.81] [94.09, 221.8] [−49.92, −15.32] [0.003, 0.196] [−17.17, 6.443] [−0.755, 0.731] [−39.89, 250.7] [−0.486, 0.515] [−19.75, 30.47] [−57.80, 124.9]
.000 .000 .002 .048 .326 .390 .407 .651 .856 .951
Tables A3 and A4 also show results consistent with our main findings when examining door-to-first-provider time in both the 2012 data set with missing observations replaced using multiple imputation
and the 2013 data set. Again, in both regression models, the main driver of long door-to-first-provider times was the length of treatment. In addition, the percentage of high-acuity patients was associated with decreased first provider time in both models. The results of both models are consistent with the regression results presented earlier in the paper. Table A3 Emergency department operational factors associated with door-to-first provider times using multiple imputation, EDBA data 2012 Variable
Percent increase
95% CI
P
% High CPT acuity Log of volume LOS (minutes) Log of physician hours Log of beds Log of RN hours Academic Percent admitted
−0.993 36.615 0.401 −18.698 −10.774 −1.193 3.769 1.284
[−0.998, −0.959] [18.86, 57.01] [0.204, 0.597] [−31.03, 0.864] [−23.72, 4.372] [−16.02, 16.26] [−7.924, 16.94] [−0.866, 37.94]
.000 .000 .000 .071 .216 .318 .558 .620
Table A4 Emergency department operational factors associated with door-to-first provider times using multiple imputation, EDBA data 2013 Variable
Percentage Increase
95% C.I.
P value
% High CPT acuity LOS (minutes) Percent admitted Log of volume Log of physician hours Academic Log of beds Patients per bed Log of RN hours
−0.789 47.703 0.547 −8.332 −22.392 10.619 44.023 0.022 9.856
[−0.883, −0.617] [11.10, 96.35] [−0.376, 2.841] [−64.50, 136.7] [−42.38, 4.542] [−4.075, 27.56] [−46.99, 291.3] [−0.041, 0.085] [−16.50, 44.54]
0.000 0.008 0.044 0.086 0.096 0.166 0.475 0.495 0.503
The additional analyses on a separate data set (2013 EDBA) and the analyses using multiple imputation to replace missing values in the 2012 data set support that the conclusions discussed earlier in the Results section are both internally and externally consistent. The variables associated with the LBTC rate and the DTFP time are consistent from year to year when data sets with complete observations or those with missing observations replaced by multiple imputation methods are used. The results and interpretation of the data are consistent across 2 years, and the missing data do not significantly alter the results of the analyses.
References [1] Gonzalez M, Bauhoff S, Blanchard M, Abir M, Iyer N, Smith A, et al. The Evolving Role of Emergency Departments in the United States Rand Corporation; 2013. [2] Crane J, Noon C. The definitive guide to emergency department operational improvement : employing lean principles with current ED best practices to create the "no wait" department. Boca Raton: CRC Press; 2011. [3] Zun LS. Analysis of the literature on emergency department throughput. West J Emerg Med 2009;10(2):104–9. [4] Pines JM, Decker SL, Hu T. Exogenous predictors of national performance measures for emergency department crowding. Ann Emerg Med 2012;60(3):293–8. [5] The Joint Commission; 2014[Oakbrook Terrace, IL]. [6] Emergency Department Benchmarking Alliance; 2015[Newark, DE]. [7] Welch SJ, Augustine JJ, Dong L, Savitz LA, Snow G, James BC. Volume-related differences in emergency department performance. Jt Comm J Qual Patient Saf 2012; 38(9):395–402. [8] Handel DA, Fu R, Vu E, Augustine JJ, Hsia RY, Shufflebarger CM, et al. Association of emergency department and hospital characteristics with elopements and length of stay. J Emerg Med 2014;46(6):839–46. [9] Baldyga M. ER Visits Continue to Rise Since Implementation of Affordable Care Act. ACEP Newsroom. Dallas, TX: American College of Emergency Physicians; 2015. [10] Block L, Cook BG, Hanyok LA, de la Torre D, Rogers MM, Noronha G, et al. Coverage isn't enough: building primary care capacity in the setting of health reform. J Health Care Poor Underserved 2014;25(1):25–8.
D. Anderson et al. / American Journal of Emergency Medicine 34 (2016) 155–161 [11] Taubman SL, Allen HL, Wright BJ, Baicker K, Finkelstein AN. Medicaid increases emergency-department use: evidence from Oregon's Health Insurance Experiment. Science 2014;343(6168):263–8. [12] Mohsin M, Forero R, Ieraci S, Bauman AE, Young L, Santiano N. A population followup study of patients who left an emergency department without being seen by a medical officer. Emerg Med J 2007;24(3):175–9. [13] Rubin D. Multiple Imputation after 18+ years. J Am Stat Assoc 1996;91(434):473–89. [14] McMullan JT, Veser FH. Emergency department volume and acuity as factors in patients leaving without treatment. South Med J 2004;97(8):729–33. [15] American Medical Association: Current Procedural Terminology; 2014[Chicago, IL]. [16] Hartigan J, Mancheck A, Wong. Algorithm AS 136: A k-means clustering algorithm. Appl Stat 1979;28(1):100–8.
161
[17] Jain A. Data clustering: 50 years beyond K-means. Pattern Recognit Lett 2010;31(8): 651–66. [18] Rowe BH, Channan P, Bullard M, Blitz S, Saunders LD, Rosychuk RJ, et al. Characteristics of patients who leave emergency departments without being seen. Acad Emerg Med 2006;13(8):848–52. [19] Belsley DA. Regression diagnostics : identifying influential data and sources of collinearity. Hoboken, N.J.: Wiley-Interscience; 2004 [20] Yoo W, Mayberry R, Bae S, Singh K, Peter He Q, Lillard Jr JW. A Study of Effects of MultiCollinearity in the Multivariable Analysis. Int J Appl Sci Technol 2014;4(5): 9–19. [21] Doyle J, Graves J, Gruber J, Kleiner S. Measuring returns to hospital care: Evidence from ambulance referral patterns. J Pol Econ 2015;123(1):170–214.