- Email: [email protected]
Modeling emergency department nursing workload in real time: An exploratory study
International Emergency Nursing xxx (xxxx) xxxx
Contents lists available at ScienceDirect
International Emergency Nursing journal homepage: www.elsevier.com/locate/aaen
Modeling emergency department nursing workload in real time: An exploratory study Edwin L. Cloptona, , Eira Kristiina Hyrkäsb ⁎
a b
Emergency Department, Southern Maine HealthCare, One Medical Center Drive, Biddeford, ME 04005 USA Center for Nursing Research & Quality Outcomes, Maine Medical Center, 22 Bramhall Street, Portland, ME 04102 USA
ARTICLE INFO
ABSTRACT
Keywords: Workload Emergency nursing Emergency department ESI Triage Clinical staffing Scales
Study of emergency department (ED) nursing workload has been largely subsumed under the related but separate phenomenon of ED crowding. Nursing workload is difficult to quantify directly. This observational study explored modeling ED nursing workload indirectly, in real time, from quantitative data available from the patient tracking computer system (PTCS). Methods: Data on 2793 patient visits plus departmental statistics were collected during 167 60-minute survey periods (SP) in a 25-bed hospital ED in the United States. The charge nurse assessed a perceived workload score (WLS) according to pre-determined criteria following each SP as a validation measure. Data analysis: Correlations were calculated between the data and WLS, and strongly correlating variables were incorporated into linear regression models that sought to approximate WLS. Results: A measure of aggregate patient acuity derived from the Emergency Severity Index (ESI) was the strongest predictor of WLS (r = 0.7991). The best-performing model agreed with WLS in 64% of SPs. Conclusions: Good agreement between model output and WLS suggests that ED nursing workload can be estimated indirectly in real time using data from a PTCS. Strong correlation between the ESI derivative and WLS further validates ESI and suggests a new application for the ESI score.
1. Introduction Emergency nurses bear a substantial workload. Each year in the United States approximately 175 000 registered nurses [1,2] devote more than 250 million person-hours to patient care in emergency departments, providing direct patient care during some 140 million emergency department (ED) patient visits [3]. Nursing workload is known to be related to patient care and outcomes [4–6] and to staff morale [7,8]. Nursing workload is challenging to quantify. The ephemeral, processintensive nature of the work of nursing contributes to the puzzle: much of the work that nurses do leaves behind no tangible product for objective measurement. Researchers agree that quantifying nursing workload facilitates effective staffing, but despite decades of research and discussion in the literature, the complex and diverse nature of nursing has thus far defied attempts to formulate a broadly accepted quantitative measure [9–11]. 2. Background and literature review Nursing workload originally was measured to facilitate financial analysis and personnel planning. Measures developed for those purposes
⁎
summarize work done over a period of time or anticipate work that will be required in the future. Time-and-motion studies of nursing date to the industrial efficiency era of the 1910s–1920s [12]. (In ergonomics and industrial engineering the time-and-motion study is a standard observational tool used to analyze work by measuring the time and describing the physical movements required to complete a given task [13]). Studies that explicitly address “nursing workload” date from the 1970s onward. The Therapeutic Intervention Scoring System (TISS) [14] of 1974 and subsequent measures derived from it [15] were developed using a timed task/activity approach (Table 1, discussed below) to quantify overall nursing workload in the intensive care unit (ICU). Those systems and others like them report nursing workload with a resolution of entire work shifts or days rather than describing the current situation with a resolution of hours or minutes. Such systems often require manual retrospective reporting of nursing activities by the nurses themselves, which adds yet another task to their overall workload [16]. Although staff nurses receive initial training on applying the particular workload measurement system adopted by their institution, ongoing quality assurance tends to be lacking, and as a
Corresponding author. E-mail addresses: [email protected] (E.L. Clopton), [email protected] (E.K. Hyrkäs).
https://doi.org/10.1016/j.ienj.2019.100793 Received 27 January 2019; Received in revised form 22 August 2019; Accepted 5 September 2019 1755-599X/ © 2019 Elsevier Ltd. All rights reserved.
Please cite this article as: Edwin L. Clopton and Eira Kristiina Hyrkäs, International Emergency Nursing, https://doi.org/10.1016/j.ienj.2019.100793
International Emergency Nursing xxx (xxxx) xxxx
E.L. Clopton and E.K. Hyrkäs
Table 1 Characteristics of nursing workforce planning systems as presented by Hurst (2003). Nursing workforce planning system
Characteristics of conceptualized variables and analysis methods
Professional judgment
Depends on experience and professional knowledge to determine staffing needs.
Top-down formula-driven methods
Utilize metrics such as nurses per occupied bed and nurse:patient ratios that are set forth by regulatory bodies and professional associations.
Acuity-quality methods
Assign patients to categories of “dependency” from which nursing needs are inferred as determined by nursing activity analysis and facilityspecific work sampling.
Timed task/activity methods
Standardized time requirements (established by work sampling) for individual tasks included in nursing care plans are summed to estimate the time that should be (or should have been) required to care for a patient under a given care plan.
Mathematical regression analysis
Identifies relationships between variables (e.g., percentage of bed occupancy and nursing hours worked) and extrapolates from those relationships to predict future staffing needs.
consequence, classification and reporting of tasks—and thus the quality of the reports produced—can be inconsistent [16]. In 1998, Maxwell reported on an automated computer implementation of the GRASP (GRAphical System for Presentation; now a trademark of Infor, New York, New York) system of accounting for ED patient care hours based on discharge diagnoses [17]. The implementation’s automated estimates correlated well with manual tabulations when reporting monthly, but the author noted that the system performed less satisfactorily at shorter time scales due to loss of the regression effect whereby random variations among individual patients’ cases cancel out over time. Although not adapted to real-time application, Maxwell’s system is notable as apparently the earliest publication in the literature of an automated nursing workload monitoring system (i.e., one not driven by manual input) and for focusing specifically on the ED. Reflecting growing interest in the study of nursing workload, in 2003 Hurst systematically reviewed 500 publications related to nursing workforce planning and workload. He identified five “nursing workforce planning systems” reported in the literature to quantify nursing workload for the purpose of determining appropriate staffing levels [18]. Hurst’s categories, listed in Table 1 and discussed in detail by Twigg & Duffield [11], summarize past and present approaches to characterizing nursing workload ranging from intuitive to rigorously analytical. Research on ED performance challenges has been dominated by studies of the related but separate phenomenon of ED crowding, resulting in a substantial body of work [19,20]. Reeder & Garrison’s READI (Realtime Emergency Analysis of Demand Indicators) model of 2001 [21] was the first of several in-depth studies of ED crowding to be published [22–27]. These studies tend to describe the overall crowding outcome without addressing nursing workload per se. Models of crowding are of limited usefulness in studying nursing workload across the spectrum of working conditions, as indicated by Jones et al’s finding that the four ED crowding scales that they studied “lack scalability and do not perform as designed in EDs where crowding is not the norm” [28]. Though related, the concepts of crowding and nursing workload differ significantly. Characteristics of the construct nursing workload include bed occupancy, patient acuity, and available staff resources, whereas physical bed occupancy approaching or exceeding 100% is the dominant characteristic of crowding even though other factors may be included in studies or models [22,24,29]. Wretborn et al observe that “most of the negative effects of crowding … are mediated via a high staff workload” and present further insightful discussions of the relationship between the concepts of workload and crowding in the ED and the merits of using separate validation measures for studies of the two phenomena [24]. These researchers and others [26,27,30] also agree on the value of continuous data collection and analysis in managing a complex system such as an ED. As with nursing workload, no clear definition or standard measure of ED crowding has emerged from extensive research into the phenomenon [19,20,22,24,26,29]. The purpose of this paper is to report on an exploratory study that extends existing work on characterizing nursing workload by developing a model to quantify nursing workload in real time based on data
drawn continuously from a patient tracking computer system (PTCS) and without manual reporting by nurses. The goal was to apply linear regression to a set of variables identified among PTCS data to create a model that generates a numerical score corresponding to ED nursing workload intensity as perceived by ED charge nurses. To our knowledge, this study represents one of the first attempts at automatic, realtime quantitative analysis of nursing workload. 3. Methods 3.1. Setting The setting for this observational study was a regional hospital with 120 inpatient beds whose 25-bed emergency department received approximately 40 000 patient visits annually during the study. The PTCS in use throughout the study was Meditech Client Server™ P5.66 (Meditech Corporation, Westwood, Massachusetts). Staff size and composition, ED and hospital organization, and computer system remained essentially unchanged during the study. 3.2. Measures The Emergency Severity Index (ESI) triage tool is used by many EDs in the United States to estimate the acuity of patients presenting for treatment [31,32]. The two highest acuity levels assigned by ESI reflect the immediacy with which intervention is indicated (1 = emergent, 2 = urgent). The remaining three levels reflect the triage nurse’s estimate of the number of predefined diagnostic and treatment resources the patient will require in the ED (3 = several resources, 4 = 1 resource, 5 = no resources). EDs typically use ESI scores to assign patients to appropriate treatment areas, such as to a low-acuity “fast track,” and to help prioritize care when multiple patients are waiting. We defined workload broadly as the portion of the available finite capacity for work that is required to meet the present need, following O’Donnell & Eggemeier: “The term workload refers to that portion of the operator’s limited capacity actually required to perform a particular task.” [33] This definition is consistent with Swiger et al.’s more recent concept analysis-based definition of nursing workload [29]. We assumed that the number of tasks pending at a particular time would reflect the portion of the nurses’ collective capacity for work that is required to meet the needs of the ED at that time. The workload score instrument developed for the present study (Fig. 1) corresponds closely to the instrument developed and validated by Bernstein et al [22] in their study of ED crowding. Staffing level was the number of nurses present and available for direct patient care during the data collection survey period (SP, discussed below) and was not adjusted for personal activities such as meal breaks. Staffing data were collected manually because the PTCS did not track this variable. Direct, automated measurement of nursing workload from the electronic record is challenging. While designing the present study, an 2
International Emergency Nursing xxx (xxxx) xxxx
E.L. Clopton and E.K. Hyrkäs
Workload: The portion of the available capacity to perform work that is required to satisfy the present demand for work. Choose one number to characterize workload in the ED for the past 60 minutes:
1
None or Very Light
2
Light
ED resources greatly exceeded demand. Required patient care and other duties (if any) performed promptly with ample time to spare. ED resources somewhat exceeded demand. Required patient care and other tasks performed promptly with some time to spare. ED resources approximately equal to demand. Required patient
3
Moderate
care and other tasks performed promptly with little or no time to spare. Few or no tasks deferred. Few or no patients in waiting room. Demand somewhat exceeded ED resources. Some tasks deferred
4
Heavy
in order to perform to perform more urgent tasks. Tasks are pending part of the time. Some patients in waiting room. Demand greatly exceeded ED resources. Many tasks deferred to
5
Overwhelming
perform more urgent tasks; tasks are pending most or all of the time. Many patients or long waits in waiting room. Fig. 1. Workload Score (WLS) Instrument.
experienced data collector (the first author, E.C.) found it difficult to extract and categorize nursing activities accurately from the PTCS, and we determined that the task was not feasible for a computer algorithm of reasonable complexity. We observed that whether and how a given nursing activity was reflected in the electronic record varied according to the type of activity, from nurse to nurse, and from time to time for a given nurse. Some nursing activities were charted in response to explicit orders, and some were reflected in narrative notes but not as discrete, readily countable events. Also, importantly for a system operating in real time, some activities that contributed substantially to nursing workload were charted long after they occurred or not at all. A further example of intangible, undocumented factors impacting nursing workload is the frequency of interruptions as reported by Forsyth et al [34]. Therefore we opted to model nursing workload indirectly by seeking objective indicators among the quantitative data available from the PTCS that correlated strongly with an independent validation measure. Our methodological assumption was that a model based on indirect but relevant objective measures would be more stable and more reliable than a model based on direct measurement of variables such as chart entries of nursing activities that are characterized by nurses’ individual documentation styles and subjective observations. In the absence of a broadly recognized objective measure of nursing workload, we adopted provider perception (specifically that of the ED charge nurse) as the validation measure. Perception is acknowledged to be subjective and susceptible to bias, but Crane et al. emphasize the unique ability of provider perception to capture the cumulative effect on workload of complex interacting factors that may elude detection in studies of isolated objective measures [10]. We assumed that the ED charge nurses’ global view of the department would enable them to render an accurate assessment of the overall nursing workload at a given time and that they would give professional consideration to the task of
assessing the workload for purposes of the study. By surveying only the charge nurse we minimized intrusion upon the work of staff nurses. 3.3. Data collection Data were collected by the first author (E.C.) during a sample of one-hour survey periods (SPs) stratified to represent times of day and days of the week approximately equally. The goal was to obtain a balanced, representative data set from which to construct a model. We prepared a stratification worksheet (Fig. 2) dividing the seven weekdays into three categories of days and the 24-hour day into eight 3-hour blocks. Based on first-hand experience in the studied ED, we assumed that the days included in each category would have similar characteristics and thus would be approximately equivalent within categories for data collection purposes. SPs were a convenience sample selected to cover each time block within each day category. Additional SPs (shaded cells) were selected to sample under-represented workload levels. During each 60-minute SP we collected data on 11 variables for each patient and four department-wide measures (Table 2). Within 15 min following the end of the SP the data collector (E.C.) asked the charge nurse to assess the workload for the past hour using the workload instrument (Fig. 1) posted at the charge desk and recorded the workload score (WLS). 3.4. Data analysis The ESI algorithm assigns its smallest numerical value (1) to the most seriously ill/injured patients and its largest value (5) to the least seriously ill/injured. For consistency with the other variables studied whose values increase with increasing volume or severity, we calculated an “inverted ESI” for use within the models, IESI ≡ 6 – ESI, to 3
International Emergency Nursing xxx (xxxx) xxxx
E.L. Clopton and E.K. Hyrkäs
Fig. 2. Stratification worksheet for data collection. This worksheet guided selection of 60-minute survey periods (SPs) for the study as explained in the text. For clarity, years have been omitted from dates, and not all additional (shaded) SPs are shown.
return values ranging from 1 for the lowest-acuity patient to 5 for the highest. Bernstein et al. [22] and Epstein & Tian [26] employed the same technique in their models. We analyzed Pearson’s correlations between WLS and 34 variables as described below (Table 3). The number of laboratory orders for each patient was limited to 12 (the largest number of laboratory orders for a non-admitted patient in the sample) to reduce the influence of inpatients boarded in the ED who had aberrantly large numbers of inpatient laboratory orders (81 patients, 2.9%) that did not directly affect their care in the ED. However, inpatient orders were not excluded entirely because some admitted patients were boarded in the ED and thus contributed to the ED nursing workload. From the collected variables (Table 2) we derived “sum variables” for each SP: the sum and mean IESI scores, sums of the respective categories of orders entered (both cumulative and 60-minute), and the sum of all orders entered (both cumulative and 60-minute), as well as sums of patients in the ED, patient arrivals and departures, crisis patients (defined in footnote, Table 2), and nurses available for direct patient care. In addition, we hypothesized that an individual nurse’s share of the work represented by each sum variable would correlate more strongly with WLS than would the overall sum variables that did
not take staffing level into account. To test this hypothesis we created a set of 15 “per-RN” variables by dividing each sum variable by the number of nurses available for direct patient care during the SP. Preliminary results prompted us also to test the sensitivity of WLS to the volume of diagnostic orders entered during each 60-minute SP (“60minute variables”) as opposed to the volume of orders cumulative since registration in the ED (“cumulative variables”). To that end we retrieved 60-minute order entry volumes retrospectively from a stratified post hoc sample of SPs included in the study. An online calculator [35] was used to determine a sample size providing a 97% confidence level. Sixty-minute order volume data were obtained from 36 SPs (21.6%) comprising 570 patients (20.4%). All variables were screened for collinearity prior to inclusion in regression models to avoid overfitting. In addition to studying the linear regression statistics, we evaluated the percent of model outputs, rounded to the nearest integer value, that coincided with the perceived WLS stated by the charge nurses for the respective SPs. This percentage served as the primary outcome metric for comparison among models. Data were analyzed using R version 3.4.2 (The R Foundation for Statistical Computing, Vienna, Austria.)
Table 2 Variables collected during each survey period (SP). For Each Patient (measure)
For the Department (measure)
Age (years) Sex (female/male) Arrival during SP (yes/no) Arrival by EMS or law enforcement (yes/no) Crisis patient1 (yes/no) Departure during SP (yes/no) ESI score (1–5) ECG tests (number of orders) Laboratory tests (number of orders) Radiology tests (number of orders) Respiratory therapy interventions (number of orders)
Patient census (number of patients occupying an ED bed at any time during SP) RN staffing (number of RNs available for direct patient care) Waiting room time (longest waiting time in minutes at the end of SP) Workload score (WLS—see text)
1 Crisis patient: a patient presenting with chief complaint of mental health or behavioral disorder, acute alcohol or drug intoxication, or seeking detoxification for chemical dependency.
4
International Emergency Nursing xxx (xxxx) xxxx
E.L. Clopton and E.K. Hyrkäs
Table 3 Independent variables and correlations with perceived workload (WLS). WLS vs. Aggregate IESI Patient census Sum of all diagnostic orders (cumulative)1 Radiology orders (cumulative) Laboratory orders (cumulative) Aggregate IESI per RN2 ECG orders (cumulative) Patient census per RN Radiology orders (cumulative) per RN Sum of all orders (cumulative) per RN Laboratory orders (cumulative) per RN Maximum time in waiting room ECG orders (cumulative) per RN Number of RNs on duty Total patient arrivals by any means Mean IESI Laboratory orders (60 min)3
r
WLS vs.
0.7991 0.7591 0.7495 0.7291 0.7196 0.7016 0.6610 0.6447 0.6321 0.6290 0.5876 0.5618 0.5132 0.4896 0.4177 0.4110 0.4073
Sum of all orders (60 min) Respiratory Therapy orders (cumulative) Sum of all orders (60 min) per RN Laboratory orders (60 min) per RN Total patient departures from ED Respiratory Therapy orders (cumulative) per RN Radiology orders (60 min) per RN Total patient arrivals by any means per RN Total patient arrivals by EMS and law enforcement Radiology orders (60 min) Number of crisis patients in ED ECG orders (60 min) per RN Total patient departures per RN ECG orders (60 min) Respiratory Therapy orders (60 min) Number of crisis patients in ED per RN Respiratory Therapy orders (60 min) per RN
r 0.3798 0.3747 0.3721 0.3662 0.2833 0.2767 0.2521 0.2425 0.2231 0.2216 0.2163 0.1962 0.1599 0.1525 0.0513 0.0386 −0.0190
Notes: 1 Cumulative: sum of orders entered for all current emergency department patients, cumulative from start of each patient’s present ED visit. 2 Per RN: the stated variable divided by the number of nurses available for patient care during the survey period. Per-RN variables are italicized in the table. 3 60 min: sum of orders entered for all emergency department patients during the 60-minute survey period; based on a subset of 36/167 study periods (see text).
entered during the 60-minute SP) (p ≈ 0.004). Several models, some derived from linear regression analysis and some constructed manually, achieved correlations with WLS greater than 0.75 and agreement with WLS of 60% to 64% (Table 5). The following formula produced the best-performing model:
Table 4 Description of the data set. Variables Study Periods (SP) Patient Visits Sex: Female Male Age (years) Patients in ED per SP Arrivals in ED per SP Departures from ED per SP RNs available for direct patient care per SP
Numerical counts/ranges 167 2793 1527 (55%) 1266 (45%) Range: 0–100 Range: 3–36 Range: 0–11 Range: 0–11 Range: 3–8
Median
y= 0.02425 m+ 0.004301 n+ 0.003172 p+ 1.0256 where m = sum of IESI scores; n = sum of all orders, cumulative since registration; p = longest time (minutes) since registration among patients in waiting room at end of SP. Fig. 5 compares the output of the best-performing model with WLS.
53 17 3 5
5. Discussion
4. Results
The workload measure proposed here is consistent with other models of nursing workload and of ED crowding that seek to distill useful measures of nursing workload from the complex tangle of variables that comprise a functioning ED [15,20]. The model presented here can be considered an “acuity-quality” method as described by Hurst [18] (Table 1) because it extrapolates primarily from nursing assessments of the acuities of individual patients at triage to estimate the amount of work required by the current patient population at any given time. The present model differs from most other nursing workload models in being focused specifically on the ED rather than on the ICU or other inpatient units, and in being adaptable to automated real-time application rather than being updated once per shift or once per day based on manual reporting of nursing activities. The model also differs from ED crowding research in concentrating on ED nursing workload. Recent Australian [36,37] and Brazilian [38,39] formula-driven analyses that address workload directly share some similarities with our study but concentrate on nurse:patient ratios and timed tasks without acuity weighting, and they report on a scale of work shifts or full days rather than being adaptable to real-time application. The methodology of our study did not interfere with the work of ED staff nurses, in contrast with studies that administered questionnaires or other assessments to nurses during their work shifts [34]. The most interesting and significant finding of our study relates to the ESI triage algorithm. The aggregate acuity measure derived from ESI scores accounted statistically for almost two-thirds of the variance in WLS (r2 = 0.64); a model based solely on that measure agreed with
The data (Table 4) consisted of 2793 patient visits during 167 SPs on 108 different dates, from 7 April 2015 to 1 January 2017. The data set was complete except for two variables, number of patient departures and maximum waiting room wait, that were not recorded for several consecutive SPs early in the study. Distributions of ESI triage scores and of WLS scores reported by charge nurses both were heavily centrally weighted (Figs. 3 and 4). We attributed the small number of ESI Level 4 and Level 5 patients seen in the ED to lower-acuity patients seeking treatment at urgent care walk-in clinics operated by the hospital throughout its service area. Table 3 reports correlations between the 34 studied variables and WLS in descending order of strength of correlation. The correlations varied between positive and moderately strong (Aggregate IESI, r = 0.7991) and weakly negative (Respiratory Therapy orders [60 min] per RN, r = −0.0190). Our assumption for further analysis was that a greater value of r, indicating strong correlation between that variable and the perceived workload stated by the charge nurses (WLS), suggested that the variable might be a good surrogate indicator of nursing workload. Most per-RN variables (e.g., number of ECG orders per RN) correlated less strongly with WLS than did the corresponding sum variables (e.g., sum of all ECG orders), but the difference was not statistically significant (p ≈ 0.095). WLS correlated significantly more strongly with cumulative order volumes (e.g. number of ECG orders cumulative since registration) than with 60-minute order volumes (e.g., ECG orders 5
International Emergency Nursing xxx (xxxx) xxxx
E.L. Clopton and E.K. Hyrkäs
Fig. 3. Distribution of ESI triage scores (N = 2793).
Level 5 (Most Intense)
n = 10 (6.0%)
Level 4
n = 43 (25.7%)
Level 3
n = 51 (30.5%)
Level 2
n = 48 (28.7%)
Level 1 (Least Intense)
n = 15 (9.0%) 0
10
20
30
40
50
Fig. 4. Distribution of WLS intensities (n = 167).
perceived nursing workload (WLS) in 63% of SPs. This finding both further validates the ESI triage algorithm and suggests a novel application for the ESI triage score in predicting overall ED nursing workload. The relationship between correlations of WLS with the per-RN vs. sum variables was not expected. This finding implies that the workload measure was not sensitive to staffing level. As noted above, we hypothesized that the per-RN variables would correlate more strongly with WLS than would the sum variables. However, in 13 of the 15 pernurse/sum-variable pairings, the sum variables correlated more strongly with WLS. The difference in correlations was not statistically significant, but even where all outcomes are equally likely, the simple probability of the correlation of one category of variable being greater by chance in 13 of 15 cases is approximately 0.3%. This finding may be related to the subjectivity of the validation measure. At times of low patient census and low staffing, the relatively quiet environment might lead charge nurses to underestimate the workloads on individual nurses, and thus also to underestimate the WLS for those SPs. The decision not to count nursing orders or activities may have contributed to the statistically significant difference between correlations of WLS with 60-minute order volumes vs. cumulative order volumes. Because the majority of the work involved in diagnostic tests is performed by personnel other than nurses, the order volumes we studied did not directly involve ED nurses to a substantial degree. The stronger correlation between WLS and cumulative order volume
suggests that the work required to care for a given patient depended more on the overall complexity of patients’ cases, as implied by the total volume of diagnostic tests ordered, than on the volume of diagnostic orders being entered during any one-hour period during the visit. Performance of the models was encouraging: their output agreed with perceived workload (WLS) for up to 64% of SPs. Confidence in the findings would have been enhanced if the model had been validated using a second sample of randomly selected time periods. In the case of the present study, however, impending hospital-wide implementation of a new PTCS and the subsequent disruption of work flow in the ED would have meant a significant delay before validation could have begun, and direct comparability of data derived from the two systems could not be assured. 6. Limitations The sample size for analysis was reasonable, but all data were collected at a single site. We do not know the extent to which workflow, staffing patterns, and PTCS characteristics unique to this site might have affected the results. The heavy central weighting of the WLS scale left the distribution rather sparse at the extremes of its range (Fig. 4). That situation limited the power of the regression models to estimate values in those ranges accurately as shown in Fig. 5. Sampling over a longer period of time would have been required to strengthen representation of WLS Level 1
Table 5 Comparison of selected models. Formula Designation Formula Formula Formula Formula
1 2 3 4
Correlation (r) with WLS
r2
Agreement
0.7818 0.7991 0.7527 0.8064
0.6112 0.6385 0.5665 0.6504
60% 63% 60% 64%
Remarks Linear regression on all variables except per-RN and 60-minute variables Single regression on aggregate triage acuity alone Regression on the second- and third-strongest correlates, patient census and total diagnostic orders Best-performing linear regression model overall (see text for formula)
6
International Emergency Nursing xxx (xxxx) xxxx
E.L. Clopton and E.K. Hyrkäs
Fig. 5. Graphic comparison of computer model output with perceived WLS. The grey dots show the range and distribution of the computer model’s estimated workload scores for survey periods assessed at the indicated levels of perceived workload intensity (WLS) by charge nurses. Dot size indicates the number of survey periods for which the model produced that output value. Boxes represent the second and third quartile range of model outputs (bar marks the median) for each survey period. Median model outputs for levels 2 through 5 were close to the WLS values they sought to reflect, and the range of model outputs for level 3 was closest to the value sought. The narrow interquartile range for level 1 indicates the greatest precision, although the accuracy at that level was suboptimal.
and Level 5 SPs in the database. The PTCS does not reflect disruptive and workload-intensive events such as an emergent intubation in a timely or readily countable way even though such events are immediately apparent to a human observer. Therefore the model proposed here could not capture the realtime impact of those patient care events on nursing workload. The ESI acuity level assigned at triage, as applied at the study facility, does not dynamically reflect a patient’s condition. Only occasionally is a triage level revised upward if the patient’s condition changes sufficiently to warrant a higher priority for treatment. A patient who arrives at a high acuity level, is stabilized, and subsequently requires fewer resources for the remainder of his/her ED visit retains the initial high acuity level until disposition. Therefore the aggregate ESI measure in this study slightly over-estimated the current acuity in the ED. The subjective nature of the WLS makes the performance of the models challenging to assess. A study design that obtained WLS ratings from multiple raters instead of from the charge nurse alone would have permitted assessment of inter-rater reliability, although identifying a second rater with a perspective of the unit comparable to that of the charge nurse would have been a challenge. The WLS criteria in the workload instrument (Fig. 1) were clear, but assessment of WLS by the charge nurse was still subject to interpretation which may have affected the reliability of the validation measure. On a few occasions charge nurses acknowledged stating WLS scores at variance with the stated criteria but that they felt were warranted by other factors.
7. Implications for emergency nursing and recommendations for future research An understanding of nursing workload is widely acknowledged to be fundamental to managing delivery of high-quality health care. Equally widely acknowledged is the need for further work on the underlying problem of defining and characterizing nursing workload so that it can be effectively measured and reported [9,10,11,29]. To our knowledge, this exploratory study is one of the first attempts to develop an automated, real-time measure of ED nursing workload. Although preliminary, our results suggest that it is feasible to characterize ED nursing workload in real time from data extracted continuously from a PTCS, and without manual data reporting to compound the already substantial workload of nurses. The reliability and validity of our approach should be tested with larger samples, at additional sites, and in hospitals in other cultures and health care systems, and should be compared with existing methodologies. Real-time analysis of operations helps organizations adapt quickly to changing needs and conditions [27]. The availability of real-time workload data could facilitate decision-making by ED staff and management, for example by supporting activation of a “full-capacity protocol”, a set of pre-determined actions to be implemented throughout the hospital to help alleviate overload conditions in the ED [40]. The measure described here may also be a useful metric for administrators and researchers. Further research may identify similar measures applicable to other nursing units. The Emergency Severity Index (ESI) is well established in the United States as an ED triage tool. In addition to further validating the ESI 7
International Emergency Nursing xxx (xxxx) xxxx
E.L. Clopton and E.K. Hyrkäs
algorithm, our observations suggest a new application for the score in predicting ED nursing workload. Potential use of this information beyond triage imparts added importance to continued conscientious application of the algorithm by triage nurses. The inherent standardization of automated workload measures can enhance the transparency and consistency of information shared within departments and across organizations. While this was an exploratory study, our vision for its eventual application in emergency nursing includes a more fluid, efficient work environment with potential to enhance perceptions of the overall ED experience on the part of patients and of job success on the part of staff. Even the simple advantage of enabling staff to help patients understand why they are waiting can reflect positively in an organization’s patient satisfaction scores [41].
[8] [9]
[10]
[11] [12]
8. Conclusions
[13]
The present study indicates that a measure derived from the ESI triage algorithm, the aggregate inverted ESI score, can form the basis of an indirect quantitative measure of ED nursing workload. The results presented here further validate the ESI triage algorithm and suggest that it is feasible to model ED nursing workload indirectly in real time based on data readily available from a PTCS.
[14] [15] [16]
Ethical statement
[17]
The present study conformed to the principles of the Declaration of Helsinki and constituted non-human subject research. The study was strictly observational, using existing, de-identified available data. No interventions or clinical investigations were involved.
[18] [19] [20]
Funding source
[21]
The study received no grant or other funding from entities in the public, commercial, or not-for-profit sectors.
[22]
Declaration of Competing Interest
[23]
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
[24]
Acknowledgments [25]
The authors express their appreciation to John Dziodzio, Data Analyst, Pulmonary and Critical Care Medicine, Maine Medical Center, Portland, Maine, for his valuable contributions to data analysis for this project.
[26] [27]
References
[28]
[1] Budden JS, Moulton P, Harper KJ, Brunell ML, Smiley R. The 2015 nursing workforce survey, registered nurse results. J Nurs Reg 2016;7(1). https://doi.org/10. 1016/S2155-8256(16)31058-4. Supplement. [2] American College of Emergency Physicians. Emergency medicine statistical profile. http://www.acep.org/content.aspx?id=25234#sm. 000144xisrm294e7mt6529kgx5nzk7, accessed April 9, 2018. [3] Rui P, Kang K. National Hospital Ambulatory Medical Care Survey: 2014 Emergency Department Summary Tables. http://www.cdc.gov/nchs/data/nhamcs/web_tables/ 2014_ed_web_tables.pdf, accessed 3/2/2018. [4] Aiken LH, Clarke SP, Sloane DM, Sochalski JA, Silber JH. Hospital nurse staffing and patient mortality, nurse burnout, and job dissatisfaction. J Amer Med Assoc 2002;288(16):1987–93. https://doi.org/10.1001/jama.288.16.1987. [5] Korn R, Mansfield M. ED overcrowding: an assessment tool to monitor ED registered nurse workload that accounts for admitted patients residing in the emergency department. J Emerg Nurs 2008;34:441–6. https://doi.org/10.1016/j.jen.2007.06. 025. [6] Duffield C, Diers D, O-Brien-Pallas L, Aisbett C, Roche M, King M, Aisbett K. Nurse staffing, nursing workload, the work environment and patient outcomes. Appl Nurs Res 2011;24:244–55. https://doi.org/10.1016/aprn.2009.12.004. [7] Crilly J, Greenslade J, Lincoln C, Timms J, Fisher A. Measurement of the impact of
[29] [30]
[31] [32] [33] [34]
8
the working environment on emergency department nurses: a cross-sectional study. Int Emerg Nurs 2017;31:9–14. https://doi.org/10.1016/j.ienj.2016.04.005. Vahey DC, Aiken LH, Sloane DM, Clarke SP, Vargas D. Nurse burnout and patient satisfaction. Med Care 2004;42(2 Suppl):II56–66. https://doi.org/10.1097/01.mlr. 0000109126.50398.54. Myny D, Van Hecke A, De Bacquer D, Verhaeghe S, Gobert M, Defloor T, et al. Determining a set of relevant and measurable factors affecting nursing workload in the acute care hospital setting: a cross-sectional study. Int J Nurs Studies 2012;49(4):427–36. https://doi.org/10.1111/j.1365-2648.2011.05689.x. Crane PW, Zhou Y, Sun Y, Lin L, Schneider S. Entropy: A conceptual approach to measuring situation-level workload within emergency care and its relationship to emergency department crowding. J Emerg Med 2014;46(4):551–9. https://doi.org/ 10.1016/j.jemermed.2013.08.113. Twigg D, Duffield C. A review of workload measures: A context for a new staffing methodology in Western Australia. Int J Nurs Studies 2009;46:132–40. https://doi. org/10.1016/j.ijnurstu.2008.08.005. Reverby S. The search for the hospital yardstick. In: Leavitt JW, Numbers RL, editors. Sickness and Health In America. second ed.Madison: University of Wisconsin Press; 1985. p. 206–16. Lopetegui M, Yen Po-Yin, Lai A, Jeffries J, Embi P, Payne P. Time and motion studies in health care: what are we talking about? J Biomed Informatics 2014;49:292–9. https://doi.org/10.1016/j.jbi.2014.02.017. Cullen DJ, Civetta JM, Briggs BA, et al. Therapeutic Intervention Scoring System: A method for quantitative comparison of patient care. Crit Care Med 1974;2:57–60. Guccione A, Morena A, Pezzi A, Iapichino G. I carichi di lavoro infermieristico. Minerva Anestesiol 2004;70:411–6. Registered Nurses Association of Ontario. Reporting on nursing workload measurement systems—a discussion of the issues. Dated January 2005. https://rnao.ca/ sites/rnao-ca/files/storage/related/1554_RNAO_Workload_Measurement.pdf, accessed 10/8/2018. Maxwell DM. A computer-generated measure of nursing workload for the emergency department based on discharge diagnosis. Can J Nurs Admin 1998;11(1):52–64. Hurst K. Selecting and applying methods for estimating the size and mix of nursing teams. Leeds, UK: Nuffield Institute for Health; 2003. Solberg LI, Asplin BR, Weinick RM, Magid DJ. Emergency department crowding: consensus development of potential measures. Ann Emerg Med 2003;42(6):824–34. https://doi.org/10.1016/mem.2003.423. Hwang U, McCarthy ML, Aronsky D, Asplin B, Crane PW, Craven CK, et al. Measures of crowding in the emergency department: a systematic review. Acad Emerg Med 2011;18(5):527–38. https://doi.org/10.1111/j1553-2712.2011.01054. Reeder TJ, Garrison HG. When the safety net is unsafe: real-time assessment of the overcrowded emergency department. Acad Emerg Med 2001;8:1070–4. https://doi. org/10.1111/j.1553-2712.2001.tb01117.x. Bernstein SL, Verghese V, Leung W, Lunney A, Perez I. Development and validation of a new index to measure emergency department crowding. Acad Emerg Med 2003;10:938–42. https://doi.org/10.1197/S1069-6563(03)00311-7. Weiss SJ, Derlet R, Arndahl J, Ernst AA, Richards J, Fernández-Frankelton M, Schwab R, Stair TO, Vicellio P, Levy D, Brautigan M, Johnson A, Nick TG. Estimating the degree of emergency department overcrowding in academic medical centers: results of the National ED Overcrowding Study (NEDOCS). Acad Emerg Med 2004;11(1):38–50. https://doi.org/10.1197/j.aem.2003.07.017. Wretborn J, Khoshnood A, Wieloch M, Ekelund U. Skåne Emergency Department assessment of patient load (SEAL)—A model to estimate crowding based on workload in Swedish emergency departments. PLoS One 2015;10(6):e010020https://doi.org/10.1371/journal.pone.0130020. Asplin BR, Magid DJ, Rhodes KV, Solberg LI, Lurie N, Camargo Jr. CA. A conceptual model of emergency department crowding. Ann Emerg Med 2003;42:173–80. https://doi.org/10.1067/mem.2003.302. Epstein SK, Tian L. Development of an emergency department work score to predict ambulance diversion. Acad Emerg Med 2006;13:421–3. https://doi.org/10.1197/j. aem.2005.11.081. Martin N, Bergs J, Eerdekens D, Depaire B, Verelst S. Developing an emergency department crowding dashboard: a design science approach. Int Emerg Nurs 2018;39:68–76. https://doi.org/10.1016/j.ienj.2017.08.001. Jones SS, Allen TL, Flottemesch TJ, Welch SJ. An independent evaluation of four quantitative emergency department crowding scales. Acad Emerg Med 2006;13(11):1204–11. https://doi.org/10.1197/j.aem.2006.05.021. Swiger PA, Vance DE, Patrician PA. Nursing workload in the acute-care setting: a concept analysis of nursing workload. Nurs Outlook 2016;64:244–54. https://doi. org/10.1016/j.outlook.2016.01.003. Bergs J, Vandyjck D, Hoogmartens O, Heerinckx P, Van Sassenbroeck D, Depaire B, et al. Emergency department crowding: time to shift the paradigm from predicting and controlling to analyzing and managing. Int Emerg Nurs 2016;24:74–7. https:// doi.org/10.1016/j.ienj.2015.05.004. Gilboy N, Tanabe P, Travers DA. The emergency severity index version 4: changes to ESI level 1 and pediatric fever criteria. J Emerg Nurs 2005;31(4):357–62. https:// doi.org/10.1016/j.jen.2005.05.011. ESI, Emergency severity index. A valid and reliable five-level emergency department triage system. http://www.esitriage.org. Accessed November 15, 2017. O’Donnell D, Eggemeier T. Workload assessment methodology. Cognitive process and performance. In: Boff K, Thomas J, editors. Handbook of Perception and Human Performance. New York: Wiley; 1986. p. 42.1–9. Forsyth KL, Hawthorne HJ, El-Sherif N, Varghese RS, Ernste VK, Koenig J, et al. Interruptions experienced by emergency nurses: implications for subjective and objective measures of workload. J Emerg Nurs 2017;44:614–23. https://doi.org/10.
International Emergency Nursing xxx (xxxx) xxxx
E.L. Clopton and E.K. Hyrkäs 1016/j.jen.2018.02.001. [35] http://www.raosoft.com/samplesize. Accessed 9/13/2018. [36] Lyneham J, Cloughessy L, Martin V. Workloads in Australian emergency departments a descriptive study. Int Emerg Nurs 2008;16:200–6. https://doi.org/10. 1016/j.ienj.2008.05.006. [37] Varndell W, Ryan E, Jeffers A, Marquez-Hunt N. Emergency nursing workload and patient dependency in the ambulance bay: a prospective study. Australasian Emerg Nurs J 2016;19:210–6. https://doi.org/10.1016/j.aenj.2016.09.002. [38] Rossetti AC, Gaidzinski RR, Fugulin FM. Nursing workload in the emergency department: a methodological proposal. Rev Latino-Am Enfermagem
2013(Spec):225–32. https://doi.org/10.1590/S0104-11692013000700028. [39] Rossetti AC, Gaidzinski RR, Bracco MM. Determining workload and size of nursing team in the pediatric emergency department. Einstein 2014;12(2):217–22. https:// doi.org/10.1590/S1679-45082014AO2945. [40] Willard E, Carlton EF, Moffat L, Barth B. A full-capacity protocol allows for increased emergency patient volume and hospital admissions. J Emerg Nurs 2017;43(5):413–8. https://doi.org/10.1016/j.jen.2017.01.007. [41] Sonis JD, Aaronson EL, Lee RY, Philpotts LL, White BA. Emergency department patient experience: a systematic review of the literature. J Patient Experience 2018;5(2):101–6. https://doi.org/10.1177/2374373517731359.
9
Contents lists available at ScienceDirect
International Emergency Nursing journal homepage: www.elsevier.com/locate/aaen
Modeling emergency department nursing workload in real time: An exploratory study Edwin L. Cloptona, , Eira Kristiina Hyrkäsb ⁎
a b
Emergency Department, Southern Maine HealthCare, One Medical Center Drive, Biddeford, ME 04005 USA Center for Nursing Research & Quality Outcomes, Maine Medical Center, 22 Bramhall Street, Portland, ME 04102 USA
ARTICLE INFO
ABSTRACT
Keywords: Workload Emergency nursing Emergency department ESI Triage Clinical staffing Scales
Study of emergency department (ED) nursing workload has been largely subsumed under the related but separate phenomenon of ED crowding. Nursing workload is difficult to quantify directly. This observational study explored modeling ED nursing workload indirectly, in real time, from quantitative data available from the patient tracking computer system (PTCS). Methods: Data on 2793 patient visits plus departmental statistics were collected during 167 60-minute survey periods (SP) in a 25-bed hospital ED in the United States. The charge nurse assessed a perceived workload score (WLS) according to pre-determined criteria following each SP as a validation measure. Data analysis: Correlations were calculated between the data and WLS, and strongly correlating variables were incorporated into linear regression models that sought to approximate WLS. Results: A measure of aggregate patient acuity derived from the Emergency Severity Index (ESI) was the strongest predictor of WLS (r = 0.7991). The best-performing model agreed with WLS in 64% of SPs. Conclusions: Good agreement between model output and WLS suggests that ED nursing workload can be estimated indirectly in real time using data from a PTCS. Strong correlation between the ESI derivative and WLS further validates ESI and suggests a new application for the ESI score.
1. Introduction Emergency nurses bear a substantial workload. Each year in the United States approximately 175 000 registered nurses [1,2] devote more than 250 million person-hours to patient care in emergency departments, providing direct patient care during some 140 million emergency department (ED) patient visits [3]. Nursing workload is known to be related to patient care and outcomes [4–6] and to staff morale [7,8]. Nursing workload is challenging to quantify. The ephemeral, processintensive nature of the work of nursing contributes to the puzzle: much of the work that nurses do leaves behind no tangible product for objective measurement. Researchers agree that quantifying nursing workload facilitates effective staffing, but despite decades of research and discussion in the literature, the complex and diverse nature of nursing has thus far defied attempts to formulate a broadly accepted quantitative measure [9–11]. 2. Background and literature review Nursing workload originally was measured to facilitate financial analysis and personnel planning. Measures developed for those purposes
⁎
summarize work done over a period of time or anticipate work that will be required in the future. Time-and-motion studies of nursing date to the industrial efficiency era of the 1910s–1920s [12]. (In ergonomics and industrial engineering the time-and-motion study is a standard observational tool used to analyze work by measuring the time and describing the physical movements required to complete a given task [13]). Studies that explicitly address “nursing workload” date from the 1970s onward. The Therapeutic Intervention Scoring System (TISS) [14] of 1974 and subsequent measures derived from it [15] were developed using a timed task/activity approach (Table 1, discussed below) to quantify overall nursing workload in the intensive care unit (ICU). Those systems and others like them report nursing workload with a resolution of entire work shifts or days rather than describing the current situation with a resolution of hours or minutes. Such systems often require manual retrospective reporting of nursing activities by the nurses themselves, which adds yet another task to their overall workload [16]. Although staff nurses receive initial training on applying the particular workload measurement system adopted by their institution, ongoing quality assurance tends to be lacking, and as a
Corresponding author. E-mail addresses: [email protected] (E.L. Clopton), [email protected] (E.K. Hyrkäs).
https://doi.org/10.1016/j.ienj.2019.100793 Received 27 January 2019; Received in revised form 22 August 2019; Accepted 5 September 2019 1755-599X/ © 2019 Elsevier Ltd. All rights reserved.
Please cite this article as: Edwin L. Clopton and Eira Kristiina Hyrkäs, International Emergency Nursing, https://doi.org/10.1016/j.ienj.2019.100793
International Emergency Nursing xxx (xxxx) xxxx
E.L. Clopton and E.K. Hyrkäs
Table 1 Characteristics of nursing workforce planning systems as presented by Hurst (2003). Nursing workforce planning system
Characteristics of conceptualized variables and analysis methods
Professional judgment
Depends on experience and professional knowledge to determine staffing needs.
Top-down formula-driven methods
Utilize metrics such as nurses per occupied bed and nurse:patient ratios that are set forth by regulatory bodies and professional associations.
Acuity-quality methods
Assign patients to categories of “dependency” from which nursing needs are inferred as determined by nursing activity analysis and facilityspecific work sampling.
Timed task/activity methods
Standardized time requirements (established by work sampling) for individual tasks included in nursing care plans are summed to estimate the time that should be (or should have been) required to care for a patient under a given care plan.
Mathematical regression analysis
Identifies relationships between variables (e.g., percentage of bed occupancy and nursing hours worked) and extrapolates from those relationships to predict future staffing needs.
consequence, classification and reporting of tasks—and thus the quality of the reports produced—can be inconsistent [16]. In 1998, Maxwell reported on an automated computer implementation of the GRASP (GRAphical System for Presentation; now a trademark of Infor, New York, New York) system of accounting for ED patient care hours based on discharge diagnoses [17]. The implementation’s automated estimates correlated well with manual tabulations when reporting monthly, but the author noted that the system performed less satisfactorily at shorter time scales due to loss of the regression effect whereby random variations among individual patients’ cases cancel out over time. Although not adapted to real-time application, Maxwell’s system is notable as apparently the earliest publication in the literature of an automated nursing workload monitoring system (i.e., one not driven by manual input) and for focusing specifically on the ED. Reflecting growing interest in the study of nursing workload, in 2003 Hurst systematically reviewed 500 publications related to nursing workforce planning and workload. He identified five “nursing workforce planning systems” reported in the literature to quantify nursing workload for the purpose of determining appropriate staffing levels [18]. Hurst’s categories, listed in Table 1 and discussed in detail by Twigg & Duffield [11], summarize past and present approaches to characterizing nursing workload ranging from intuitive to rigorously analytical. Research on ED performance challenges has been dominated by studies of the related but separate phenomenon of ED crowding, resulting in a substantial body of work [19,20]. Reeder & Garrison’s READI (Realtime Emergency Analysis of Demand Indicators) model of 2001 [21] was the first of several in-depth studies of ED crowding to be published [22–27]. These studies tend to describe the overall crowding outcome without addressing nursing workload per se. Models of crowding are of limited usefulness in studying nursing workload across the spectrum of working conditions, as indicated by Jones et al’s finding that the four ED crowding scales that they studied “lack scalability and do not perform as designed in EDs where crowding is not the norm” [28]. Though related, the concepts of crowding and nursing workload differ significantly. Characteristics of the construct nursing workload include bed occupancy, patient acuity, and available staff resources, whereas physical bed occupancy approaching or exceeding 100% is the dominant characteristic of crowding even though other factors may be included in studies or models [22,24,29]. Wretborn et al observe that “most of the negative effects of crowding … are mediated via a high staff workload” and present further insightful discussions of the relationship between the concepts of workload and crowding in the ED and the merits of using separate validation measures for studies of the two phenomena [24]. These researchers and others [26,27,30] also agree on the value of continuous data collection and analysis in managing a complex system such as an ED. As with nursing workload, no clear definition or standard measure of ED crowding has emerged from extensive research into the phenomenon [19,20,22,24,26,29]. The purpose of this paper is to report on an exploratory study that extends existing work on characterizing nursing workload by developing a model to quantify nursing workload in real time based on data
drawn continuously from a patient tracking computer system (PTCS) and without manual reporting by nurses. The goal was to apply linear regression to a set of variables identified among PTCS data to create a model that generates a numerical score corresponding to ED nursing workload intensity as perceived by ED charge nurses. To our knowledge, this study represents one of the first attempts at automatic, realtime quantitative analysis of nursing workload. 3. Methods 3.1. Setting The setting for this observational study was a regional hospital with 120 inpatient beds whose 25-bed emergency department received approximately 40 000 patient visits annually during the study. The PTCS in use throughout the study was Meditech Client Server™ P5.66 (Meditech Corporation, Westwood, Massachusetts). Staff size and composition, ED and hospital organization, and computer system remained essentially unchanged during the study. 3.2. Measures The Emergency Severity Index (ESI) triage tool is used by many EDs in the United States to estimate the acuity of patients presenting for treatment [31,32]. The two highest acuity levels assigned by ESI reflect the immediacy with which intervention is indicated (1 = emergent, 2 = urgent). The remaining three levels reflect the triage nurse’s estimate of the number of predefined diagnostic and treatment resources the patient will require in the ED (3 = several resources, 4 = 1 resource, 5 = no resources). EDs typically use ESI scores to assign patients to appropriate treatment areas, such as to a low-acuity “fast track,” and to help prioritize care when multiple patients are waiting. We defined workload broadly as the portion of the available finite capacity for work that is required to meet the present need, following O’Donnell & Eggemeier: “The term workload refers to that portion of the operator’s limited capacity actually required to perform a particular task.” [33] This definition is consistent with Swiger et al.’s more recent concept analysis-based definition of nursing workload [29]. We assumed that the number of tasks pending at a particular time would reflect the portion of the nurses’ collective capacity for work that is required to meet the needs of the ED at that time. The workload score instrument developed for the present study (Fig. 1) corresponds closely to the instrument developed and validated by Bernstein et al [22] in their study of ED crowding. Staffing level was the number of nurses present and available for direct patient care during the data collection survey period (SP, discussed below) and was not adjusted for personal activities such as meal breaks. Staffing data were collected manually because the PTCS did not track this variable. Direct, automated measurement of nursing workload from the electronic record is challenging. While designing the present study, an 2
International Emergency Nursing xxx (xxxx) xxxx
E.L. Clopton and E.K. Hyrkäs
Workload: The portion of the available capacity to perform work that is required to satisfy the present demand for work. Choose one number to characterize workload in the ED for the past 60 minutes:
1
None or Very Light
2
Light
ED resources greatly exceeded demand. Required patient care and other duties (if any) performed promptly with ample time to spare. ED resources somewhat exceeded demand. Required patient care and other tasks performed promptly with some time to spare. ED resources approximately equal to demand. Required patient
3
Moderate
care and other tasks performed promptly with little or no time to spare. Few or no tasks deferred. Few or no patients in waiting room. Demand somewhat exceeded ED resources. Some tasks deferred
4
Heavy
in order to perform to perform more urgent tasks. Tasks are pending part of the time. Some patients in waiting room. Demand greatly exceeded ED resources. Many tasks deferred to
5
Overwhelming
perform more urgent tasks; tasks are pending most or all of the time. Many patients or long waits in waiting room. Fig. 1. Workload Score (WLS) Instrument.
experienced data collector (the first author, E.C.) found it difficult to extract and categorize nursing activities accurately from the PTCS, and we determined that the task was not feasible for a computer algorithm of reasonable complexity. We observed that whether and how a given nursing activity was reflected in the electronic record varied according to the type of activity, from nurse to nurse, and from time to time for a given nurse. Some nursing activities were charted in response to explicit orders, and some were reflected in narrative notes but not as discrete, readily countable events. Also, importantly for a system operating in real time, some activities that contributed substantially to nursing workload were charted long after they occurred or not at all. A further example of intangible, undocumented factors impacting nursing workload is the frequency of interruptions as reported by Forsyth et al [34]. Therefore we opted to model nursing workload indirectly by seeking objective indicators among the quantitative data available from the PTCS that correlated strongly with an independent validation measure. Our methodological assumption was that a model based on indirect but relevant objective measures would be more stable and more reliable than a model based on direct measurement of variables such as chart entries of nursing activities that are characterized by nurses’ individual documentation styles and subjective observations. In the absence of a broadly recognized objective measure of nursing workload, we adopted provider perception (specifically that of the ED charge nurse) as the validation measure. Perception is acknowledged to be subjective and susceptible to bias, but Crane et al. emphasize the unique ability of provider perception to capture the cumulative effect on workload of complex interacting factors that may elude detection in studies of isolated objective measures [10]. We assumed that the ED charge nurses’ global view of the department would enable them to render an accurate assessment of the overall nursing workload at a given time and that they would give professional consideration to the task of
assessing the workload for purposes of the study. By surveying only the charge nurse we minimized intrusion upon the work of staff nurses. 3.3. Data collection Data were collected by the first author (E.C.) during a sample of one-hour survey periods (SPs) stratified to represent times of day and days of the week approximately equally. The goal was to obtain a balanced, representative data set from which to construct a model. We prepared a stratification worksheet (Fig. 2) dividing the seven weekdays into three categories of days and the 24-hour day into eight 3-hour blocks. Based on first-hand experience in the studied ED, we assumed that the days included in each category would have similar characteristics and thus would be approximately equivalent within categories for data collection purposes. SPs were a convenience sample selected to cover each time block within each day category. Additional SPs (shaded cells) were selected to sample under-represented workload levels. During each 60-minute SP we collected data on 11 variables for each patient and four department-wide measures (Table 2). Within 15 min following the end of the SP the data collector (E.C.) asked the charge nurse to assess the workload for the past hour using the workload instrument (Fig. 1) posted at the charge desk and recorded the workload score (WLS). 3.4. Data analysis The ESI algorithm assigns its smallest numerical value (1) to the most seriously ill/injured patients and its largest value (5) to the least seriously ill/injured. For consistency with the other variables studied whose values increase with increasing volume or severity, we calculated an “inverted ESI” for use within the models, IESI ≡ 6 – ESI, to 3
International Emergency Nursing xxx (xxxx) xxxx
E.L. Clopton and E.K. Hyrkäs
Fig. 2. Stratification worksheet for data collection. This worksheet guided selection of 60-minute survey periods (SPs) for the study as explained in the text. For clarity, years have been omitted from dates, and not all additional (shaded) SPs are shown.
return values ranging from 1 for the lowest-acuity patient to 5 for the highest. Bernstein et al. [22] and Epstein & Tian [26] employed the same technique in their models. We analyzed Pearson’s correlations between WLS and 34 variables as described below (Table 3). The number of laboratory orders for each patient was limited to 12 (the largest number of laboratory orders for a non-admitted patient in the sample) to reduce the influence of inpatients boarded in the ED who had aberrantly large numbers of inpatient laboratory orders (81 patients, 2.9%) that did not directly affect their care in the ED. However, inpatient orders were not excluded entirely because some admitted patients were boarded in the ED and thus contributed to the ED nursing workload. From the collected variables (Table 2) we derived “sum variables” for each SP: the sum and mean IESI scores, sums of the respective categories of orders entered (both cumulative and 60-minute), and the sum of all orders entered (both cumulative and 60-minute), as well as sums of patients in the ED, patient arrivals and departures, crisis patients (defined in footnote, Table 2), and nurses available for direct patient care. In addition, we hypothesized that an individual nurse’s share of the work represented by each sum variable would correlate more strongly with WLS than would the overall sum variables that did
not take staffing level into account. To test this hypothesis we created a set of 15 “per-RN” variables by dividing each sum variable by the number of nurses available for direct patient care during the SP. Preliminary results prompted us also to test the sensitivity of WLS to the volume of diagnostic orders entered during each 60-minute SP (“60minute variables”) as opposed to the volume of orders cumulative since registration in the ED (“cumulative variables”). To that end we retrieved 60-minute order entry volumes retrospectively from a stratified post hoc sample of SPs included in the study. An online calculator [35] was used to determine a sample size providing a 97% confidence level. Sixty-minute order volume data were obtained from 36 SPs (21.6%) comprising 570 patients (20.4%). All variables were screened for collinearity prior to inclusion in regression models to avoid overfitting. In addition to studying the linear regression statistics, we evaluated the percent of model outputs, rounded to the nearest integer value, that coincided with the perceived WLS stated by the charge nurses for the respective SPs. This percentage served as the primary outcome metric for comparison among models. Data were analyzed using R version 3.4.2 (The R Foundation for Statistical Computing, Vienna, Austria.)
Table 2 Variables collected during each survey period (SP). For Each Patient (measure)
For the Department (measure)
Age (years) Sex (female/male) Arrival during SP (yes/no) Arrival by EMS or law enforcement (yes/no) Crisis patient1 (yes/no) Departure during SP (yes/no) ESI score (1–5) ECG tests (number of orders) Laboratory tests (number of orders) Radiology tests (number of orders) Respiratory therapy interventions (number of orders)
Patient census (number of patients occupying an ED bed at any time during SP) RN staffing (number of RNs available for direct patient care) Waiting room time (longest waiting time in minutes at the end of SP) Workload score (WLS—see text)
1 Crisis patient: a patient presenting with chief complaint of mental health or behavioral disorder, acute alcohol or drug intoxication, or seeking detoxification for chemical dependency.
4
International Emergency Nursing xxx (xxxx) xxxx
E.L. Clopton and E.K. Hyrkäs
Table 3 Independent variables and correlations with perceived workload (WLS). WLS vs. Aggregate IESI Patient census Sum of all diagnostic orders (cumulative)1 Radiology orders (cumulative) Laboratory orders (cumulative) Aggregate IESI per RN2 ECG orders (cumulative) Patient census per RN Radiology orders (cumulative) per RN Sum of all orders (cumulative) per RN Laboratory orders (cumulative) per RN Maximum time in waiting room ECG orders (cumulative) per RN Number of RNs on duty Total patient arrivals by any means Mean IESI Laboratory orders (60 min)3
r
WLS vs.
0.7991 0.7591 0.7495 0.7291 0.7196 0.7016 0.6610 0.6447 0.6321 0.6290 0.5876 0.5618 0.5132 0.4896 0.4177 0.4110 0.4073
Sum of all orders (60 min) Respiratory Therapy orders (cumulative) Sum of all orders (60 min) per RN Laboratory orders (60 min) per RN Total patient departures from ED Respiratory Therapy orders (cumulative) per RN Radiology orders (60 min) per RN Total patient arrivals by any means per RN Total patient arrivals by EMS and law enforcement Radiology orders (60 min) Number of crisis patients in ED ECG orders (60 min) per RN Total patient departures per RN ECG orders (60 min) Respiratory Therapy orders (60 min) Number of crisis patients in ED per RN Respiratory Therapy orders (60 min) per RN
r 0.3798 0.3747 0.3721 0.3662 0.2833 0.2767 0.2521 0.2425 0.2231 0.2216 0.2163 0.1962 0.1599 0.1525 0.0513 0.0386 −0.0190
Notes: 1 Cumulative: sum of orders entered for all current emergency department patients, cumulative from start of each patient’s present ED visit. 2 Per RN: the stated variable divided by the number of nurses available for patient care during the survey period. Per-RN variables are italicized in the table. 3 60 min: sum of orders entered for all emergency department patients during the 60-minute survey period; based on a subset of 36/167 study periods (see text).
entered during the 60-minute SP) (p ≈ 0.004). Several models, some derived from linear regression analysis and some constructed manually, achieved correlations with WLS greater than 0.75 and agreement with WLS of 60% to 64% (Table 5). The following formula produced the best-performing model:
Table 4 Description of the data set. Variables Study Periods (SP) Patient Visits Sex: Female Male Age (years) Patients in ED per SP Arrivals in ED per SP Departures from ED per SP RNs available for direct patient care per SP
Numerical counts/ranges 167 2793 1527 (55%) 1266 (45%) Range: 0–100 Range: 3–36 Range: 0–11 Range: 0–11 Range: 3–8
Median
y= 0.02425 m+ 0.004301 n+ 0.003172 p+ 1.0256 where m = sum of IESI scores; n = sum of all orders, cumulative since registration; p = longest time (minutes) since registration among patients in waiting room at end of SP. Fig. 5 compares the output of the best-performing model with WLS.
53 17 3 5
5. Discussion
4. Results
The workload measure proposed here is consistent with other models of nursing workload and of ED crowding that seek to distill useful measures of nursing workload from the complex tangle of variables that comprise a functioning ED [15,20]. The model presented here can be considered an “acuity-quality” method as described by Hurst [18] (Table 1) because it extrapolates primarily from nursing assessments of the acuities of individual patients at triage to estimate the amount of work required by the current patient population at any given time. The present model differs from most other nursing workload models in being focused specifically on the ED rather than on the ICU or other inpatient units, and in being adaptable to automated real-time application rather than being updated once per shift or once per day based on manual reporting of nursing activities. The model also differs from ED crowding research in concentrating on ED nursing workload. Recent Australian [36,37] and Brazilian [38,39] formula-driven analyses that address workload directly share some similarities with our study but concentrate on nurse:patient ratios and timed tasks without acuity weighting, and they report on a scale of work shifts or full days rather than being adaptable to real-time application. The methodology of our study did not interfere with the work of ED staff nurses, in contrast with studies that administered questionnaires or other assessments to nurses during their work shifts [34]. The most interesting and significant finding of our study relates to the ESI triage algorithm. The aggregate acuity measure derived from ESI scores accounted statistically for almost two-thirds of the variance in WLS (r2 = 0.64); a model based solely on that measure agreed with
The data (Table 4) consisted of 2793 patient visits during 167 SPs on 108 different dates, from 7 April 2015 to 1 January 2017. The data set was complete except for two variables, number of patient departures and maximum waiting room wait, that were not recorded for several consecutive SPs early in the study. Distributions of ESI triage scores and of WLS scores reported by charge nurses both were heavily centrally weighted (Figs. 3 and 4). We attributed the small number of ESI Level 4 and Level 5 patients seen in the ED to lower-acuity patients seeking treatment at urgent care walk-in clinics operated by the hospital throughout its service area. Table 3 reports correlations between the 34 studied variables and WLS in descending order of strength of correlation. The correlations varied between positive and moderately strong (Aggregate IESI, r = 0.7991) and weakly negative (Respiratory Therapy orders [60 min] per RN, r = −0.0190). Our assumption for further analysis was that a greater value of r, indicating strong correlation between that variable and the perceived workload stated by the charge nurses (WLS), suggested that the variable might be a good surrogate indicator of nursing workload. Most per-RN variables (e.g., number of ECG orders per RN) correlated less strongly with WLS than did the corresponding sum variables (e.g., sum of all ECG orders), but the difference was not statistically significant (p ≈ 0.095). WLS correlated significantly more strongly with cumulative order volumes (e.g. number of ECG orders cumulative since registration) than with 60-minute order volumes (e.g., ECG orders 5
International Emergency Nursing xxx (xxxx) xxxx
E.L. Clopton and E.K. Hyrkäs
Fig. 3. Distribution of ESI triage scores (N = 2793).
Level 5 (Most Intense)
n = 10 (6.0%)
Level 4
n = 43 (25.7%)
Level 3
n = 51 (30.5%)
Level 2
n = 48 (28.7%)
Level 1 (Least Intense)
n = 15 (9.0%) 0
10
20
30
40
50
Fig. 4. Distribution of WLS intensities (n = 167).
perceived nursing workload (WLS) in 63% of SPs. This finding both further validates the ESI triage algorithm and suggests a novel application for the ESI triage score in predicting overall ED nursing workload. The relationship between correlations of WLS with the per-RN vs. sum variables was not expected. This finding implies that the workload measure was not sensitive to staffing level. As noted above, we hypothesized that the per-RN variables would correlate more strongly with WLS than would the sum variables. However, in 13 of the 15 pernurse/sum-variable pairings, the sum variables correlated more strongly with WLS. The difference in correlations was not statistically significant, but even where all outcomes are equally likely, the simple probability of the correlation of one category of variable being greater by chance in 13 of 15 cases is approximately 0.3%. This finding may be related to the subjectivity of the validation measure. At times of low patient census and low staffing, the relatively quiet environment might lead charge nurses to underestimate the workloads on individual nurses, and thus also to underestimate the WLS for those SPs. The decision not to count nursing orders or activities may have contributed to the statistically significant difference between correlations of WLS with 60-minute order volumes vs. cumulative order volumes. Because the majority of the work involved in diagnostic tests is performed by personnel other than nurses, the order volumes we studied did not directly involve ED nurses to a substantial degree. The stronger correlation between WLS and cumulative order volume
suggests that the work required to care for a given patient depended more on the overall complexity of patients’ cases, as implied by the total volume of diagnostic tests ordered, than on the volume of diagnostic orders being entered during any one-hour period during the visit. Performance of the models was encouraging: their output agreed with perceived workload (WLS) for up to 64% of SPs. Confidence in the findings would have been enhanced if the model had been validated using a second sample of randomly selected time periods. In the case of the present study, however, impending hospital-wide implementation of a new PTCS and the subsequent disruption of work flow in the ED would have meant a significant delay before validation could have begun, and direct comparability of data derived from the two systems could not be assured. 6. Limitations The sample size for analysis was reasonable, but all data were collected at a single site. We do not know the extent to which workflow, staffing patterns, and PTCS characteristics unique to this site might have affected the results. The heavy central weighting of the WLS scale left the distribution rather sparse at the extremes of its range (Fig. 4). That situation limited the power of the regression models to estimate values in those ranges accurately as shown in Fig. 5. Sampling over a longer period of time would have been required to strengthen representation of WLS Level 1
Table 5 Comparison of selected models. Formula Designation Formula Formula Formula Formula
1 2 3 4
Correlation (r) with WLS
r2
Agreement
0.7818 0.7991 0.7527 0.8064
0.6112 0.6385 0.5665 0.6504
60% 63% 60% 64%
Remarks Linear regression on all variables except per-RN and 60-minute variables Single regression on aggregate triage acuity alone Regression on the second- and third-strongest correlates, patient census and total diagnostic orders Best-performing linear regression model overall (see text for formula)
6
International Emergency Nursing xxx (xxxx) xxxx
E.L. Clopton and E.K. Hyrkäs
Fig. 5. Graphic comparison of computer model output with perceived WLS. The grey dots show the range and distribution of the computer model’s estimated workload scores for survey periods assessed at the indicated levels of perceived workload intensity (WLS) by charge nurses. Dot size indicates the number of survey periods for which the model produced that output value. Boxes represent the second and third quartile range of model outputs (bar marks the median) for each survey period. Median model outputs for levels 2 through 5 were close to the WLS values they sought to reflect, and the range of model outputs for level 3 was closest to the value sought. The narrow interquartile range for level 1 indicates the greatest precision, although the accuracy at that level was suboptimal.
and Level 5 SPs in the database. The PTCS does not reflect disruptive and workload-intensive events such as an emergent intubation in a timely or readily countable way even though such events are immediately apparent to a human observer. Therefore the model proposed here could not capture the realtime impact of those patient care events on nursing workload. The ESI acuity level assigned at triage, as applied at the study facility, does not dynamically reflect a patient’s condition. Only occasionally is a triage level revised upward if the patient’s condition changes sufficiently to warrant a higher priority for treatment. A patient who arrives at a high acuity level, is stabilized, and subsequently requires fewer resources for the remainder of his/her ED visit retains the initial high acuity level until disposition. Therefore the aggregate ESI measure in this study slightly over-estimated the current acuity in the ED. The subjective nature of the WLS makes the performance of the models challenging to assess. A study design that obtained WLS ratings from multiple raters instead of from the charge nurse alone would have permitted assessment of inter-rater reliability, although identifying a second rater with a perspective of the unit comparable to that of the charge nurse would have been a challenge. The WLS criteria in the workload instrument (Fig. 1) were clear, but assessment of WLS by the charge nurse was still subject to interpretation which may have affected the reliability of the validation measure. On a few occasions charge nurses acknowledged stating WLS scores at variance with the stated criteria but that they felt were warranted by other factors.
7. Implications for emergency nursing and recommendations for future research An understanding of nursing workload is widely acknowledged to be fundamental to managing delivery of high-quality health care. Equally widely acknowledged is the need for further work on the underlying problem of defining and characterizing nursing workload so that it can be effectively measured and reported [9,10,11,29]. To our knowledge, this exploratory study is one of the first attempts to develop an automated, real-time measure of ED nursing workload. Although preliminary, our results suggest that it is feasible to characterize ED nursing workload in real time from data extracted continuously from a PTCS, and without manual data reporting to compound the already substantial workload of nurses. The reliability and validity of our approach should be tested with larger samples, at additional sites, and in hospitals in other cultures and health care systems, and should be compared with existing methodologies. Real-time analysis of operations helps organizations adapt quickly to changing needs and conditions [27]. The availability of real-time workload data could facilitate decision-making by ED staff and management, for example by supporting activation of a “full-capacity protocol”, a set of pre-determined actions to be implemented throughout the hospital to help alleviate overload conditions in the ED [40]. The measure described here may also be a useful metric for administrators and researchers. Further research may identify similar measures applicable to other nursing units. The Emergency Severity Index (ESI) is well established in the United States as an ED triage tool. In addition to further validating the ESI 7
International Emergency Nursing xxx (xxxx) xxxx
E.L. Clopton and E.K. Hyrkäs
algorithm, our observations suggest a new application for the score in predicting ED nursing workload. Potential use of this information beyond triage imparts added importance to continued conscientious application of the algorithm by triage nurses. The inherent standardization of automated workload measures can enhance the transparency and consistency of information shared within departments and across organizations. While this was an exploratory study, our vision for its eventual application in emergency nursing includes a more fluid, efficient work environment with potential to enhance perceptions of the overall ED experience on the part of patients and of job success on the part of staff. Even the simple advantage of enabling staff to help patients understand why they are waiting can reflect positively in an organization’s patient satisfaction scores [41].
[8] [9]
[10]
[11] [12]
8. Conclusions
[13]
The present study indicates that a measure derived from the ESI triage algorithm, the aggregate inverted ESI score, can form the basis of an indirect quantitative measure of ED nursing workload. The results presented here further validate the ESI triage algorithm and suggest that it is feasible to model ED nursing workload indirectly in real time based on data readily available from a PTCS.
[14] [15] [16]
Ethical statement
[17]
The present study conformed to the principles of the Declaration of Helsinki and constituted non-human subject research. The study was strictly observational, using existing, de-identified available data. No interventions or clinical investigations were involved.
[18] [19] [20]
Funding source
[21]
The study received no grant or other funding from entities in the public, commercial, or not-for-profit sectors.
[22]
Declaration of Competing Interest
[23]
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
[24]
Acknowledgments [25]
The authors express their appreciation to John Dziodzio, Data Analyst, Pulmonary and Critical Care Medicine, Maine Medical Center, Portland, Maine, for his valuable contributions to data analysis for this project.
[26] [27]
References
[28]
[1] Budden JS, Moulton P, Harper KJ, Brunell ML, Smiley R. The 2015 nursing workforce survey, registered nurse results. J Nurs Reg 2016;7(1). https://doi.org/10. 1016/S2155-8256(16)31058-4. Supplement. [2] American College of Emergency Physicians. Emergency medicine statistical profile. http://www.acep.org/content.aspx?id=25234#sm. 000144xisrm294e7mt6529kgx5nzk7, accessed April 9, 2018. [3] Rui P, Kang K. National Hospital Ambulatory Medical Care Survey: 2014 Emergency Department Summary Tables. http://www.cdc.gov/nchs/data/nhamcs/web_tables/ 2014_ed_web_tables.pdf, accessed 3/2/2018. [4] Aiken LH, Clarke SP, Sloane DM, Sochalski JA, Silber JH. Hospital nurse staffing and patient mortality, nurse burnout, and job dissatisfaction. J Amer Med Assoc 2002;288(16):1987–93. https://doi.org/10.1001/jama.288.16.1987. [5] Korn R, Mansfield M. ED overcrowding: an assessment tool to monitor ED registered nurse workload that accounts for admitted patients residing in the emergency department. J Emerg Nurs 2008;34:441–6. https://doi.org/10.1016/j.jen.2007.06. 025. [6] Duffield C, Diers D, O-Brien-Pallas L, Aisbett C, Roche M, King M, Aisbett K. Nurse staffing, nursing workload, the work environment and patient outcomes. Appl Nurs Res 2011;24:244–55. https://doi.org/10.1016/aprn.2009.12.004. [7] Crilly J, Greenslade J, Lincoln C, Timms J, Fisher A. Measurement of the impact of
[29] [30]
[31] [32] [33] [34]
8
the working environment on emergency department nurses: a cross-sectional study. Int Emerg Nurs 2017;31:9–14. https://doi.org/10.1016/j.ienj.2016.04.005. Vahey DC, Aiken LH, Sloane DM, Clarke SP, Vargas D. Nurse burnout and patient satisfaction. Med Care 2004;42(2 Suppl):II56–66. https://doi.org/10.1097/01.mlr. 0000109126.50398.54. Myny D, Van Hecke A, De Bacquer D, Verhaeghe S, Gobert M, Defloor T, et al. Determining a set of relevant and measurable factors affecting nursing workload in the acute care hospital setting: a cross-sectional study. Int J Nurs Studies 2012;49(4):427–36. https://doi.org/10.1111/j.1365-2648.2011.05689.x. Crane PW, Zhou Y, Sun Y, Lin L, Schneider S. Entropy: A conceptual approach to measuring situation-level workload within emergency care and its relationship to emergency department crowding. J Emerg Med 2014;46(4):551–9. https://doi.org/ 10.1016/j.jemermed.2013.08.113. Twigg D, Duffield C. A review of workload measures: A context for a new staffing methodology in Western Australia. Int J Nurs Studies 2009;46:132–40. https://doi. org/10.1016/j.ijnurstu.2008.08.005. Reverby S. The search for the hospital yardstick. In: Leavitt JW, Numbers RL, editors. Sickness and Health In America. second ed.Madison: University of Wisconsin Press; 1985. p. 206–16. Lopetegui M, Yen Po-Yin, Lai A, Jeffries J, Embi P, Payne P. Time and motion studies in health care: what are we talking about? J Biomed Informatics 2014;49:292–9. https://doi.org/10.1016/j.jbi.2014.02.017. Cullen DJ, Civetta JM, Briggs BA, et al. Therapeutic Intervention Scoring System: A method for quantitative comparison of patient care. Crit Care Med 1974;2:57–60. Guccione A, Morena A, Pezzi A, Iapichino G. I carichi di lavoro infermieristico. Minerva Anestesiol 2004;70:411–6. Registered Nurses Association of Ontario. Reporting on nursing workload measurement systems—a discussion of the issues. Dated January 2005. https://rnao.ca/ sites/rnao-ca/files/storage/related/1554_RNAO_Workload_Measurement.pdf, accessed 10/8/2018. Maxwell DM. A computer-generated measure of nursing workload for the emergency department based on discharge diagnosis. Can J Nurs Admin 1998;11(1):52–64. Hurst K. Selecting and applying methods for estimating the size and mix of nursing teams. Leeds, UK: Nuffield Institute for Health; 2003. Solberg LI, Asplin BR, Weinick RM, Magid DJ. Emergency department crowding: consensus development of potential measures. Ann Emerg Med 2003;42(6):824–34. https://doi.org/10.1016/mem.2003.423. Hwang U, McCarthy ML, Aronsky D, Asplin B, Crane PW, Craven CK, et al. Measures of crowding in the emergency department: a systematic review. Acad Emerg Med 2011;18(5):527–38. https://doi.org/10.1111/j1553-2712.2011.01054. Reeder TJ, Garrison HG. When the safety net is unsafe: real-time assessment of the overcrowded emergency department. Acad Emerg Med 2001;8:1070–4. https://doi. org/10.1111/j.1553-2712.2001.tb01117.x. Bernstein SL, Verghese V, Leung W, Lunney A, Perez I. Development and validation of a new index to measure emergency department crowding. Acad Emerg Med 2003;10:938–42. https://doi.org/10.1197/S1069-6563(03)00311-7. Weiss SJ, Derlet R, Arndahl J, Ernst AA, Richards J, Fernández-Frankelton M, Schwab R, Stair TO, Vicellio P, Levy D, Brautigan M, Johnson A, Nick TG. Estimating the degree of emergency department overcrowding in academic medical centers: results of the National ED Overcrowding Study (NEDOCS). Acad Emerg Med 2004;11(1):38–50. https://doi.org/10.1197/j.aem.2003.07.017. Wretborn J, Khoshnood A, Wieloch M, Ekelund U. Skåne Emergency Department assessment of patient load (SEAL)—A model to estimate crowding based on workload in Swedish emergency departments. PLoS One 2015;10(6):e010020https://doi.org/10.1371/journal.pone.0130020. Asplin BR, Magid DJ, Rhodes KV, Solberg LI, Lurie N, Camargo Jr. CA. A conceptual model of emergency department crowding. Ann Emerg Med 2003;42:173–80. https://doi.org/10.1067/mem.2003.302. Epstein SK, Tian L. Development of an emergency department work score to predict ambulance diversion. Acad Emerg Med 2006;13:421–3. https://doi.org/10.1197/j. aem.2005.11.081. Martin N, Bergs J, Eerdekens D, Depaire B, Verelst S. Developing an emergency department crowding dashboard: a design science approach. Int Emerg Nurs 2018;39:68–76. https://doi.org/10.1016/j.ienj.2017.08.001. Jones SS, Allen TL, Flottemesch TJ, Welch SJ. An independent evaluation of four quantitative emergency department crowding scales. Acad Emerg Med 2006;13(11):1204–11. https://doi.org/10.1197/j.aem.2006.05.021. Swiger PA, Vance DE, Patrician PA. Nursing workload in the acute-care setting: a concept analysis of nursing workload. Nurs Outlook 2016;64:244–54. https://doi. org/10.1016/j.outlook.2016.01.003. Bergs J, Vandyjck D, Hoogmartens O, Heerinckx P, Van Sassenbroeck D, Depaire B, et al. Emergency department crowding: time to shift the paradigm from predicting and controlling to analyzing and managing. Int Emerg Nurs 2016;24:74–7. https:// doi.org/10.1016/j.ienj.2015.05.004. Gilboy N, Tanabe P, Travers DA. The emergency severity index version 4: changes to ESI level 1 and pediatric fever criteria. J Emerg Nurs 2005;31(4):357–62. https:// doi.org/10.1016/j.jen.2005.05.011. ESI, Emergency severity index. A valid and reliable five-level emergency department triage system. http://www.esitriage.org. Accessed November 15, 2017. O’Donnell D, Eggemeier T. Workload assessment methodology. Cognitive process and performance. In: Boff K, Thomas J, editors. Handbook of Perception and Human Performance. New York: Wiley; 1986. p. 42.1–9. Forsyth KL, Hawthorne HJ, El-Sherif N, Varghese RS, Ernste VK, Koenig J, et al. Interruptions experienced by emergency nurses: implications for subjective and objective measures of workload. J Emerg Nurs 2017;44:614–23. https://doi.org/10.
International Emergency Nursing xxx (xxxx) xxxx
E.L. Clopton and E.K. Hyrkäs 1016/j.jen.2018.02.001. [35] http://www.raosoft.com/samplesize. Accessed 9/13/2018. [36] Lyneham J, Cloughessy L, Martin V. Workloads in Australian emergency departments a descriptive study. Int Emerg Nurs 2008;16:200–6. https://doi.org/10. 1016/j.ienj.2008.05.006. [37] Varndell W, Ryan E, Jeffers A, Marquez-Hunt N. Emergency nursing workload and patient dependency in the ambulance bay: a prospective study. Australasian Emerg Nurs J 2016;19:210–6. https://doi.org/10.1016/j.aenj.2016.09.002. [38] Rossetti AC, Gaidzinski RR, Fugulin FM. Nursing workload in the emergency department: a methodological proposal. Rev Latino-Am Enfermagem
2013(Spec):225–32. https://doi.org/10.1590/S0104-11692013000700028. [39] Rossetti AC, Gaidzinski RR, Bracco MM. Determining workload and size of nursing team in the pediatric emergency department. Einstein 2014;12(2):217–22. https:// doi.org/10.1590/S1679-45082014AO2945. [40] Willard E, Carlton EF, Moffat L, Barth B. A full-capacity protocol allows for increased emergency patient volume and hospital admissions. J Emerg Nurs 2017;43(5):413–8. https://doi.org/10.1016/j.jen.2017.01.007. [41] Sonis JD, Aaronson EL, Lee RY, Philpotts LL, White BA. Emergency department patient experience: a systematic review of the literature. J Patient Experience 2018;5(2):101–6. https://doi.org/10.1177/2374373517731359.
9