Number of intensivists per bed is associated with efficiency of Dutch intensive care units

Journal Pre-proof Number of intensivists per bed is associated with efficiency of Dutch intensive care units

Safira A. Wortel, Nicolette F. de Keizer, Ameen Abu-Hanna, Dave A. Dongelmans, Ferishta Bakhshi-Raiez PII:

S0883-9441(20)30802-9

DOI:

https://doi.org/10.1016/j.jcrc.2020.12.008

Reference:

YJCRC 53733

To appear in:

Journal of Critical Care

Please cite this article as: S.A. Wortel, N.F. de Keizer, A. Abu-Hanna, et al., Number of intensivists per bed is associated with efficiency of Dutch intensive care units, Journal of Critical Care (2020), https://doi.org/10.1016/j.jcrc.2020.12.008

This is a PDF file of an article that has undergone enhancements after acceptance, such as the addition of a cover page and metadata, and formatting for readability, but it is not yet the definitive version of record. This version will undergo additional copyediting, typesetting and review before it is published in its final form, but we are providing this version to give early visibility of the article. Please note that, during the production process, errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

© 2020 Published by Elsevier.

Journal Pre-proof

Number of intensivists per bed is associated with efficiency of Dutch Intensive Care Units Safira A. Wortel MSc.1,2,* [email protected], Nicolette F. de Keizer, PhD1,2, Ameen AbuHanna, PhD1, Dave A. Dongelmans MD, PhD2,3, Ferishta Bakhshi-Raiez, PhD1,2 1

Department of Medical Informatics, Amsterdam UMC, Location AMC, Amsterdam Public Health research institute, Amsterdam, The Netherlands 2 National Intensive Care Evaluation (NICE) Foundation, Department of Medical Informatics, Amsterdam UMC, Amsterdam, The Netherlands 3 Department of Intensive Care, Amsterdam UMC, location AMC, Amsterdam, The Netherlands *

Corresponding author at: Amsterdam UMC, University of Amsterdam, Department of Medical

Informatics, Meibergdreef 9, 1105 AZ Amsterdam, the Netherlands.

of

Abstract

ro

Purpose: To measure efficiency in Intensive Care Units (ICUs) and to determine which organizational

-p

factors are associated with ICU efficiency, taking confounding factors into account.

re

Materials and Methods: We used data of all consecutive admissions to Dutch ICUs between January 1, 2016 and January 1, 2019 and recorded ICU organizational factors. We calculated efficiency for

lP

each ICU by averaging the Standardized Mortality Ratio (SMR) and Standardized Resource Use (SRU)

na

and examined the relationship between various organizational factors and ICU efficiency. We thereby compared the results of linear regression models before and after covariate adjustment

Jo ur

using propensity scores.

Results: We included 164,399 admissions from 83 ICUs. ICU efficiency ranged from 0.51-1.42 (average 0.99, 0.15 SD). The unadjusted model as well as the propensity score adjusted model showed a significant association between the ratio of employed intensivists per ICU bed and ICU efficiency. Other organizational factors had no statistically significant association with ICU efficiency after adjustment. Conclusions: We found marked variability in efficiency in Dutch ICUs. After applying covariate adjustment using propensity scores, we identified one organizational factor, ratio intensivists per bed, having an association with ICU efficiency. Keywords: Intensive Care Unit, resource use, ICU mortality, organizational characteristics, quality indicators 1

Journal Pre-proof Introduction Hospitals and intensive care units (ICUs) in particular, continuously strive for improving their quality of care while facing pressure to reduce costs. To indicate room for improvement, benchmarking is often applied by comparing quality measures of each ICU with its peers, the national average scores or a group of top-performing ICUs. ICU performance is often measured and benchmarked with the Standardized Mortality Ratio (SMR) [1–4]. This indicator, however, measures effectiveness and does not take into account the resources needed to provide care. Therefore, benchmark information

of

containing only SMRs offers no insight into the efficiency of ICUs or how it can be improved.

ro

Many studies focusing on ICU efficiency use ICU length of stay as a proxy to determine the

-p

costs or resources used. They use this proxy as a complete overview of actual costs of care is complex

re

and difficult to compare between different settings [5–11]. A measure created to benchmark ICU efficiency, based on the length of stay of ICU survivors as well as non-survivors in ICUs, is the

lP

Standardized Resource Use (SRU) [7,9,10,12]. The SRU is the ratio between the observed total

na

number of ICU length of stay of all patients and the expected length of stay for each ICU [7]. Efficiency and effectiveness of care in ICUs are affected by the way ICUs are organized [13–

Jo ur

17]. Using outliers from both median SMR and SRU to identify most efficient ICUs, Rothen et al. found that only the presence of interprofessional clinical rounds and presence of an emergency department separated the most efficient from least efficient ICUs while other organizational factors regarding nurse and physician staffing did not [7]. Soares et al. used a similar approach, but found that the number of implemented clinical protocols was associated with both improved patient survival and more efficient resource use [10]. The methods applied in previous studies focused on identifying organizational factors associated with ICU efficiency by considering them as prognostic factors, and hence consider (noncausal) associations. However, investigating the causal effect of an organizational factor on an outcome necessitates correcting for potential confounding factors [18,19]. Methods based on reducing or eliminating the effects of confounding are increasingly being used [20]. Even though 2

Journal Pre-proof these methods are mostly applied in clinical observational studies focusing on measuring treatment effects on health outcomes [20–22], they could also help provide a better understanding of the relationship between organizational factors and ICU efficiency. We therefore aim to measure efficiency in ICUs and to determine which organizational factors are associated with ICU efficiency, taking confounding factors into account.

Materials and Methods

of

Data Source

ro

We used data from the Dutch National Intensive Care Evaluation (NICE) registry, a non-profit quality

-p

registry covering all Dutch ICUs [23]. All ICUs collect at least the minimal dataset (MDS) for each admitted patient. The MDS describes the severity of illness based on, among others, the Acute

re

Physiology And Chronic Health Evaluation (APACHE) IV model as well as outcome measures such as

lP

ICU and in-hospital mortality, length of stay and ICU readmission. Additionally, a mandatory part of the MDS includes the collection of basic organizational data, such as the volume of beds, and the

na

number of full-time equivalent (FTE) qualified ICU nurses and ICU physicians. These organizational

Jo ur

data are collected annually. Apart from the mandatory MDS module, ICUs can also participate in: the quality indicators module, the complications module, the Sequential Organ Failure Assessment (SOFA) module, the Sepsis module and the Nursing capacity module [35].

In-and exclusion criteria Patient data

We included data from all adult patients admitted to a Dutch ICU between January 1, 2016 and January 1, 2019. We excluded patient admissions according to the APACHE IV exclusion criteria: patients younger than 16 years; length of stay shorter than four hours or longer than one year; readmissions; patients admitted from another Coronary Care Unit (CCU) or ICU; patients with missing reason for admission or admission type; patients with burns; transplant patients or CCU or recovery

3

Journal Pre-proof patients [24]. ICUs that did not provide MDS data for at least 12 consecutive months between 2016 and 2019 and ICUs that did not provide organizational data were excluded.

Organizational data We used organizational data collected for the year 2017 and in case of missing values we used data from 2018 or 2016. The organizational data consisted of the following variables: number of stepdown beds with intensivist supervision, number of stepdown beds without intensivist supervision, presence of a separate CCU, number of CCU beds in the ICU, number of Post Anesthesia Care Unit

of

(PACU) beds with mechanical ventilation, number of PACU beds without mechanical ventilation,

ro

number of FTE intensivists in training, number of FTE qualified ICU nurses, number of FTE ICU nurses

-p

in training, number of FTE physicians in the ICU (excluding intensivists), number of FTE board

re

certified Intensivists, number of ICU beds and number of hospital beds. The different types of beds (stepdown with or without intensivist supervision, CCU, PACU with or without mechanical

lP

ventilation) and FTE intensivists in training were dichotomized due to reduced variance, the number

na

of hospital beds was included as a continuous variable, and factors regarding staffing were related to the number of ICU beds, e.g. the number of FTE qualified ICU nurses per bed.

Jo ur

In addition to these organizational factors, we also calculated the number of ICU admissions per ICU bed and the number of ICU admissions per month for each ICU. Lastly, we also counted how many registration modules each ICU participates in, as this is a proxy for the ICUs’ interest in quality of care. More details on all organizational factors can be found in the Electronic Supplementary File. In previous studies, the status of the hospital, "private" or "public", was added in the analyses. As all hospitals in the Netherlands are public hospitals, including the status of the hospital in our analyses was not necessary.

Outcome measures For each ICU, we calculated the SMR and SRU. The SMR is the ratio between observed mortality and expected mortality, where the latter is calculated by taking the sum of all patients’ recalibrated

4

Journal Pre-proof APACHE IV mortality risks. For calculating the expected mortality risk, the APACHE IV model was recalibrated by refitting a new logistic regression model using the in-hospital mortality as the dependent variable and the logit transformed original probability as the sole independent variable [25,26]. For calculating the SRU we used the approach described in Rothen et al.[7] and adapted it for the Dutch situation. Instead of the Simplified Acute Physiology Score, which is not available in the NICE registry, the expected length of stay was calculated based on the recalibrated APACHE IV probabilities. ICU efficiency score was represented by the average of the SRU and SMR. The lower

of

the ICU efficiency score the more efficient the ICU.

ro

Statistical analyses

-p

For all included ICUs, we used descriptive statistics and presented categorical variables using

re

absolute and relative frequencies while continuous variables are presented using mean and Standard

lP

Deviation (SD) or median and interquartile range (IQR) depending on their distribution. To determine which organizational factors are associated with ICU efficiency we applied three steps:

Step 1 Confounder selection

na

1)confounder selection, 2)propensity score calculation and 3)model construction.

Jo ur

To determine which organizational factors are associated with ICU efficiency, we selected potential confounders for each organizational factor. A confounder is a variable that influences both the exposure and the outcome but is not an effect of the exposure [19]. We used the 10% change-inestimate criterion as a heuristic for an initial selection of potential confounders [27,28]. Thereafter we validated the initial list of identified confounders with a domain expert (DD) and adjusted the list accordingly.

Step 2 Propensity score calculation We used propensity score modeling to account for confounding [29,30]. To this end, we predicted each exposure variable by using the identified confounders as covariates. Logistic regression was used to predict the dichotomous exposure variables. This resulted in the propensity scores: the

5

Journal Pre-proof probability of an ICU receiving or having the exposure variable given the values for its covariates. For continuous exposure variables, we calculated the generalized propensity scores according to Hirano & Imbens [31]. The generalized propensity score indicates the conditional density of the treatment given the covariates and has similar properties as the propensity scores [31]. Every organizational factor was considered the so-called exposure variable at one time during the analyses, meaning the process of confounder selection and propensity score calculation was repeated for each organizational factor.

of

Step 3 Model Construction

ro

For each exposure variable, we created two linear regression models: one model in which the

-p

outcome variable, ICU efficiency, is regressed solely on the exposure variable, and another model

re

which in addition to the exposure variable also contains the estimated propensity score, i.e.

significant.

lP

propensity score adjusted model [29,30]. A p-value of less than 0.05 was considered statistically

Jo ur

Results

na

All statistical analyses were performed using R version 3.6.0.

In total 86 ICUs collected patient data between January 1, 2016 and January 1, 2019. Three ICUs were excluded because they did not collect data for at least 12 consecutive months (N=1), or appeared to have unreliable or incompletely organizational data (N=2) (Figure 1). After excluding the patients due to the APACHE IV exclusion criteria, the remaining 164,399 records were used to calculate the SMR and SRU for each ICU. Characteristics of the included patients and ICUs are shown in the Electronic Supplementary Material (ESM) eTable 1 and table 1 respectively. The 83 included ICUs had an SMR ranging from 0.53 to 1.33 with an average of 1.0 (SD 0.18). The SRU varied between 0.49 to 1.81 with an average SRU of 0.98 (SD 0.21), and the ICU efficiency score varied between 0.51 to 1.42 with an average ICU efficiency score of 0.99 (SD 0.15). The SMR

6

Journal Pre-proof and SRU for each ICU are shown in figure 2 while the distributions of these indicators are shown in efigure 1 of the ESM. The number of identified confounders used to calculate the propensity scores differs between the exposure variables. Table 2 gives an overview of the identified confounders used as covariates in measuring the propensity score. Presence of stepdown beds with intensivist supervision and qualified ICU nurses to ICU bed ratio have the most confounders (N=8), while the number of hospital beds, the number of ICU admissions per ICU bed and the number of ICU admissions per

ro

of

month have the least (N=0).

In table 3, we present the coefficients of each organizational factor as the exposure variable

-p

in the unadjusted- and propensity score adjusted model. Of all exposure variables, only the number

re

of intensivists to ICU bed ratio was significantly associated with ICU efficiency. According to the

lP

propensity score adjusted model for the number of intensivists per bed ratio, each unit increase in this organizational factor (i.e. one more intensivist per bed) leads to a 0.25-point decrease in the ICU

Jo ur

Discussion

na

efficiency score, and thus an increase in ICU efficiency.

In this study, we measured the efficiency in Dutch ICUs and examined the relationship between various organizational factors and ICU efficiency. We thereby compared the results of linear regression models before and after covariate adjustment using propensity scores. We found that there is marked variability in efficiency between ICUs. The ratio of employed intensivists per ICU bed showed a significant independent association with ICU efficiency, before and after adjustment. Other organizational factors had no statistically significant association with ICU efficiency. Our efficiency measure combined case-mix adjusted mortality and case-mix adjusted length of stay. The presence of qualified ICU nurses and intensivists has been associated with in-hospital mortality before [32–34]. Intensivists play an important role in patient discharge, and hence the length of stay of a patient. Therefore, the association of one more intensivist per bed leading to a 7

Journal Pre-proof 0.25-point decrease in efficiency score, or in more practical terms, one more intensivist per ten beds leading to a 0.025-point decrease in efficiency score, was expectable. Our results do not correspond to findings from other studies. In both Rothen et al. and Soares et al., organizational factors related to staffing were not associated with efficiency in ICUs [7,10]. There are, however, key differences between these studies and ours that may have led to these differing results. Primarily these are methodological differences as well as differences in the inclusion or measurement of the organizational factors. To start with the latter, these studies

of

included more process related variables such as the presence of inter-professional clinical rounds,

ro

multidisciplinary meetings, implemented clinical protocols and the use of checklists, about which we

-p

have no data available in our dataset. The closest variable we had to process related factors is the number of registration modules that an ICU participates in. To participate in one of the optional

re

registration modules, e.g. the Quality Indicators module, an ICU must collect items according to the

lP

definitions of the Dutch Society of Intensive Care. Although they are not equivalent, the number of registration modules might imply that an ICU operates according to implemented protocols.

na

Furthermore, in previous studies, multiple countries have been included in the analyses, while we

Jo ur

studied only one. The ICUs included in our study might be more homogeneous compared to those included in Rothen et. al [7] which could explain why the range in SRU in Rothen et. al. (median SRU 1.07 (IQR 0.76–1.58)) is larger than the range in SRU found in our study (median SRU 0.98 (IQR 0.831.1). It should also be noted that compared to earlier studies the overall nursing staff in the Netherlands might be more homogeneous and of higher intensity. This might explain that we only found that the ratio of intensivists per bed is associated with ICU efficiency. Regarding methodological differences, in the previously mentioned studies, ICUs are classified into efficiency groups based on median SMR and SRU. Straney et al. showed that using the SMR and SRU with their confidence interval to classify ICUs, can lead to a different classification than when using solely the median SMR and SRU [9]. In order to counter the loss of information when classifying ICUs based on median values, we simply used the mean SMR and SRU to represent ICU 8

Journal Pre-proof efficiency. We reanalyzed the data by applying the method described in Rothen et al. [7]. We classified each ICU into one of four groups: most efficient, least efficient, overachieving and underachieving, based on median SMR and SRU. We then applied univariate logistic regression to determine which organizational factor separates the most from least efficient ICUs and could be included as covariates in a multiple, stepwise logistic regression analysis. The results from the univariate analyses showed that none of organizational factors could be selected as covariates in the stepwise logistic regression analyses [ESM eTable 2]. This can be explained by the loss of information

of

by classifying the ICUs into four categories based on medians. Our method to calculate a continuous

ro

efficiency measure based on the average of SMR and SRU provides more power. To check our main

-p

results, we also built a multivariate model with the SMR and the SRU as outcome variables and the intensivist per bed and its propensity score as the independent variables. The results of the

re

Multivariate Analysis of Variance (MANOVA) test showed that the number of intensivist per bed is

lP

significantly associated with both the SMR and the SRU.

Another strength of our study is that we could include a large amount of patient admission

na

data and that almost all ICUs of one country are included.

Jo ur

Previous studies did not take potential confounders into account in their analyses [7,10]. As confounding factors can alter the real effect, it is necessary to control for their effects as much as possible [19]. We tried to eliminate uncertainty regarding confounders by using a robust approach. Despite its strengths, this study also has some limitations. There is no standard way to represent ICU efficiency and we are dependent on surrogate variables such as the length of stay to indicate the ICU’s resource use and therewith efficiency. Furthermore, by representing ICU efficiency as the mean of the SMR and SRU, we assumed that efficient ICUs have a low SMR and low SRU, however, ICUs with a high SMR and low SRU or vice versa, can also have a low average score and are therefore considered efficient. Nevertheless, with our approach we kept the information loss at a minimum.

9

Journal Pre-proof In addition, covariate adjustment using propensity scores strongly assumes that the propensity scores have been modeled correctly [30]. We applied this method to the best of our knowledge with the data we had available. That is not to say that there are no other ICU organizational factors that influence ICU efficiency or that there are no other confounders in the relationships between the organizational factors that we have studied and ICU efficiency. Other ICU related factors were unfortunately not available and therefore not analyzed in this study. Furthermore, in this study, we found that the more intensivists per ICU bed leads to a higher

of

ICU efficiency. However, employing an infinite number of intensivists, will increase efficiency as

ro

defined in our study, but the financial costs will also be very high and might eventually lead to a

-p

negative effect on ICU efficiency. So, there might indeed be a ceiling effect and more research is needed to analyze this.

re

Finally, most of the organizational factors included in this study were ICU related factors,

lP

however, structural and process related factors outside the ICU may also affect an ICU’s performance. Interpretation of the results of this study should therefore be done with caution.

na

Future studies could look into other structural or process related factors in- and outside of the ICU

Conclusion

Jo ur

for a more holistic interpretation of the results for clinical practice.

This study found there was marked variability in efficiency in Dutch ICUs. We applied a robust method and identified one ICU organizational factor, the ratio of intensivists per bed, being associated with ICU efficiency.

Acknowledgements We would like to thank all ICUs participating in the NICE registry for their continuous efforts to collect data and improve their processes based on these data. Funding This research did not receive any specific grant from funding agencies in the public, commercial, or 10

Journal Pre-proof not-for-profit sectors. Conflicts of interest/Competing interests N.F. de Keizer and D.A. Dongelmans are members of the board of the Dutch National Intensive Care Evaluation (NICE) foundation. The NICE foundation pays the department of Medical Informatics, Amsterdam UMC, for processing data of all Dutch ICUs into audit and feedback information. S.A. Wortel, N.F. de Keizer, and F. Bakhshi-Raiez are employees of the department of medical informatics and work on the NICE project.

of

Ethics approval

ro

The Medical Ethics Review Committee of the Amsterdam University Medical Centers reviewed the

-p

research proposal and waived the need for informed consent (reference number W20_192#20.223).

Supplementary data

lP

References

re

Supplementary material

Ridley S. Severity of illness scoring systems and performance appraisal. Anaesthesia. 1998;53(12):1185–94.

2.

Zimmerman JE, Alzola C, Von Rueden KT. The use of benchmarking to identify top performing critical care units: A preliminary assessment of their policies and practices. J Crit Care. 2003;18(2):76–86.

3.

Le Gall JR, Neumann A, Hemery F, Bleriot JP, Fulgencio JP, Garrigues B, et al. Mortality prediction using SAPS II: an update for French intensive care units. Crit Care. 2005;9(6).

4.

Flaatten H. The present use of quality indicators in the intensive care unit. Vol. 56, Acta Anaesthesiologica Scandinavica. 2012. p. 1078–83.

5.

Rapoport J, Teres D, Zhao Y, Lemeshow S. Length of stay data as a guide to hospital economic performance for ICU patients. Med Care. 2003;41(3):386–97.

6.

Stricker K, Rothen HU, Takala J. Resource use in the ICU: short-vs. long-term patients. Acta Anaesthesiol Scand. 2003;(47):508—515.

7.

Rothen HU, Stricker K, Einfalt J, Bauer P, Metnitz PGH, Moreno RP, et al. Variability in outcome and resource use in intensive care units. Intensive Care Med. 2007 Aug;33(8):1329– 36.

8.

Niskanen M, Reinikainen M, Pettilä V. Case-mix-adjusted length of stay and mortality in 23 Finnish ICUs. Intensive Care Med. 2009 Jun;35(6):1060–7.

9.

Straney LD, Clements A, Alexander J, Slater A. Measuring efficiency in Australian and New Zealand paediatric intensive care units. Intensive Care Med. 2010 Aug;36(8):1410–6.

Jo ur

na

1.

11

Journal Pre-proof Soares M, Bozza FA, Angus DC, Japiassú AM, Viana WN, Costa R, et al. Organizational characteristics, outcomes, and resource use in 78 Brazilian intensive care units: the ORCHESTRA study. Intensive Care Med. 2015 Dec 1;41(12):2149–60.

11.

Verburg IWM, Holman R, Dongelmans D, de Jonge E, de Keizer NF. Is patient length of stay associated with intensive care unit characteristics? J Crit Care. 2018 Feb 1;43:114–21.

12.

Nathanson BH, Higgins TL, Teres D, Copes WS, Kramer A, Stark M. A revised method to assess intensive care unit clinical performance and resource utilization. Crit Care Med. 2007;35(8):1853–62.

13.

Weled BJ, Adzhigirey LA, Hodgman TM, Brilli RJ, Spevetz A, Kline AM, et al. Critical care delivery: The importance of process of care and ICU structure to improved outcomes: An update from the American college of critical care medicine task force on models of critical care. Crit Care Med. 2015 Jul 21;43(7):1520–5.

14.

Kahn JM, Rubenfeld GD, Rohrbach J, Fuchs BD. Cost savings attributable to reductions in intensive care unit length of stay for mechanically ventilated patients. Med Care. 2008;46(12):1226–33.

15.

Sakr Y, Moreira CL, Rhodes A, Ferguson ND, Kleinpell R, Pickkers P, et al. The impact of hospital and ICU organizational factors on outcome in critically ill patients: Results from the Extended Prevalence of Infection in Intensive Care Study. Crit Care Med. 2015;43(3):519–26.

16.

Checkley W, Martin GS, Brown SM, Chang SY, Dabbagh O, Fremont RD, et al. Structure, process, and annual ICU mortality across 69 centers: United States critical illness and injury trials group critical illness outcomes study. Crit Care Med. 2014 Feb;42(2):344–56.

17.

Kim MM, Barnato AE, Angus DC, Fleisher LF, Kahn JM. The effect of multidisciplinary care teams on intensive care unit mortality. Arch Intern Med. 2010;170(4):369–76.

18.

Riley RD, Hayden JA, Steyerberg EW, Moons KGM, Abrams K, Kyzas PA, et al. Prognosis Research Strategy (PROGRESS) 2: Prognostic Factor Research. PLoS Med. 2013;10(2).

19.

Jager KJ, Zoccali C, MacLeod A, Dekker FW. Confounding: What it is and how to deal with it. Kidney Int. 2008;73(3):256–60.

20.

Grimes DA, Schulz KF. Bias and causal associations in observational research. Vol. 359, Lancet. Elsevier Limited; 2002. p. 248–52.

21.

Ensing S, Abu-Hanna A, Roseboom TJ, Repping S, Van Der Veen F, Mol BWJ, et al. Risk of poor neonatal outcome at term after medically assisted reproduction: A propensity score-matched study. Fertil Steril. 2015 Aug 1;104(2):384-390.e1.

22.

Shahn Z, Shapiro NI, Tyler PD, Talmor D, Lehman LWH. Fluid-limiting treatment strategies among sepsis patients in the ICU: A retrospective causal analysis. Crit Care. 2020 Feb 22;24(1).

23.

Van De Klundert N, Holman R, Dongelmans DA, De Keizer NF. Data Resource Profile: The Dutch National Intensive Care Evaluation (NICE) Registry of Admissions to Adult Intensive Care Units. Int J Epidemiol. 2015;44(6):1850.

24.

Zimmerman JE, Kramer AA, McNair DS, Malila FM. Acute Physiology and Chronic Health Evaluation (APACHE) IV: Hospital mortality assessment for today’s critically ill patients. Crit Care Med. 2006;34(5):1297–310.

25.

Moreno R, Apolone G. Impact of different customization strategies in the performance of a general severity score. Crit Care Med. 1997;25(12):2001–8.

Jo ur

na

lP

re

-p

ro

of

10.

12

Journal Pre-proof Bakhshi-Raiez F, Peek N, Bosman RJ, De Jonge E, De Keizer NF. The impact of different prognostic models and their customization on institutional comparison of intensive care units. Crit Care Med. 2007;35(11):2553–60.

27.

Maldonado G, Greenland S. Simulation Study of Confounder-Selection Strategies. Vol. 138, American Journal of Epidemiology. 1993.

28.

Greenland S, Mickey RM. The impact of confounder selection criteria on effect estimation. Vol. 130, American Journal of Epidemiology. 1989. p. 1066.

29.

Rosenbaum PR, Rubin DB. The central role of the propensity score in observational studies for causal effects. Vol. 70. Biometrika. 1983. p. 41-55.

30.

Austin PC. An introduction to propensity score methods for reducing the effects of confounding in observational studies. Multivariate Behav Res. 2011 May;46(3):399–424.

31.

Hirano K, Imbens GW. The propensity score with continuous treatments. In: Gelmanand A, Meng X-L, editors. Applied Bayesian Modeling and Causal Inference from Incomplete-Data Perspectives: An Essential Journey with Donald Rubin’s Statistical Family. John Wiley; 2004. p. 73–84.

32.

Kane RL, Shamliyan TA, Mueller C, Duval S, Wilt TJ. The association of registered nurse staffing levels and patient outcomes: Systematic review and meta-analysis. Vol. 45, Medical Care. 2007. p. 1195–204.

33.

Wallace DJ, Angus DC, Barnato AE, Kramer AA, Kahn JM. Nighttime intensivist staffing and mortality among critically ill patients. N Engl J Med. 2012 May 31;366(22):2093–101.

34.

Wilcox ME, Chong CAKY, Niven DJ, Rubenfeld GD, Rowan KM, Wunsch H, et al. Do intensivist staffing patterns influence hospital mortality following icu admission? A systematic review and meta-analyses. Vol. 41, Critical Care Medicine. 2013. p. 2253–74.

35.

Dutch National Intensive Care Evaluation [Internet]. [accessed 2020 Jul 1]. Available from: https://stichting-nice.nl

Jo ur

Figure legend

na

lP

re

-p

ro

of

26.

Figure 1: Selection of Intensive Care Units and patient admissions Figure 2: Standardized Resource Use (SRU) and Standardized Mortality Ratio (SMR) for each ICU. SMR is defined as the ratio between observed and expected hospital mortality. SRU is defined as the ratio between observed and expected length of stay in the Intensive Care Unit. The dashed lines represent the overall mean SMR and SRU.

Tables Table 1: Intensive Care Unit Characteristics Table 2: Covariates in propensity score model Table 3: Coefficients of organizational factors as exposure variables in the unadjusted- and propensity score adjusted model

13

Journal Pre-proof Electronic Supplementary Material Definitions of organizational variables eFigure1: Distribution of the Standardized Mortality Ratio, Standardized Resource Use and ICU efficiency Standardized Mortality Ratio (SMR) is defined as the ratio between observed mortality and expected hospital mortality. Standardized Resource Use (SRU) is defined as the ratio between observed and expected length of stay in the Intensive Care Unit (ICU). ICU efficiency is the average of the SMR and SRU. eTable 1: Patient characteristics

of

eTable 2: Univariate logistic regression analysis with organizational factor as covariate, comparing “most efficient” and “least efficient” Intensive Care Units

ro

Table 1: Intensive Care Unit Characteristics

Jo ur

na

lP

re

-p

Total number of ICUs Outcome measures: ICU Efficiency (mean(SD)) SMR (mean(SD)) SRU (mean(SD)) Organizational factors: 1 Stepdown beds with intensivist supervision = yes(%) 2 Stepdown beds without intensivist supervision = yes(%) 3 CCU beds in separate CCU = yes(%) 4 CCU beds in the ICU = yes(%) 5 PACU beds with mechanical ventilation = yes(%) 6 PACU beds without mechanical ventilation = yes(%) 7 Intensivists in training = yes(%) 8 Qualified ICU nurses to ICU bed ratio (median[IQR]) 9 ICU nurses in training to ICU bed ratio (mean(SD)) 10 ICU physicians (not intensivists) to ICU bed ratio (median[IQR]) 11 Intensivists to ICU bed ratio (median[IQR]) Number of hospital beds (median[IQR]) 12 Number of registration modules (median[IQR]) 13 Number of ICU admissions per ICU bed (mean(SD)) 14 Number of ICU admissions per month (median[IQR])

83 0.99 (0.15) 1.00 (0.18) 0.98 (0.21) 25 (30.12) 7 (8.43) 13 (15.66) 17 (20.48) 16 (19.28) 14 (16.87) 10 (12.05) 3.25 [2.91-3.74] 0.39 (0.23) 0.49 [0.16 - 0.64] 0.46 [0.38 - 0.55] 438 [315 - 626] 3 [2-4] 154 (54) 54 [31-72]

Abbreviations: CCU = Coronary Care Unit, ICU = Intensive Care Unit, IQR = Inter Quartile Range, PACU = Post-Anesthesia Care Unit, SD = Standard Deviation 1

= yes if the hospital has stepdown beds under the supervision of an intensivist, in a separate stepdown unit such as a Medium Care or Special Care or as part of the ICU. Patients can hereby be dismissed from the ICU to the stepdown unit. A CCU bed or PACU bed is not a stepdown bed. 2 = yes if the hospital has stepdown beds not under the supervision of an intensivist, in a separate stepdown unit such as a Medium Care or Special Care or as part of the ICU. Patients can hereby be dismissed from the ICU to the stepdown unit. A CCU bed or PACU bed is not a stepdown bed. 3 = yes if the hospital has CCU beds with ventilation facility in a separate CCU. 4 = yes if the ICU has CCU beds in the ICU itself (a combined ICU/CCU). 5 = yes if the hospital has a separate PACU with 24-hour PACU beds with ventilation facility.

14

Journal Pre-proof 6

of

= yes if the hospital has a separate PACU with 24-hour PACU beds without ventilation facility. PACU is a unit where patients who have just undergone surgery and received anesthesia can regain full consciousness and be observed until their vital functions are stable. 7 = yes if the ICU has medical specialists training as intensivists. 8 Number of FTE qualified ICU nurses in the ICU divided by the total number of ICU beds. 9 Number of FTE trainee ICU nurses working in the ICU divided by the total number of ICU beds. 10 Number of FTE physicians (excluding intensivists and intensivists in training) working in the ICU who provide practical care to ICU patients divided by the total number of ICU beds. 11 Number of FTE intensivists working in the ICU divided by the total number of ICU beds 12 The number of NICE registration modules an ICU participates in. Each Dutch ICU participates in the MDS registration module and may participate in other optional modules such as the Quality Indicator “KIIC” module, the “Complication” module, the Sequential Organ Failure Assessment “SOFA” module, the “Sepsis” module and the “Nursing capacity” module. 13 The total number of ICU admissions during study period (January 1, 2016 and January 1, 2019) divided by the total number of ICU beds 14 The total number of ICU admissions during study period (January 1, 2016 and January 1, 2019) divided by the total number of months that an ICU recorded patient admission records during study period.

1

Stepdown beds with intensivist supervision Stepdown beds without intensivist supervision 3 CCU beds in separate CCU 4 CCU beds in the ICU 5 PACU beds with mechanical ventilation 6 PACU beds without mechanical ventilation 7 Intensivists in training 8 Qualified ICU nurses to ICU bed ratio 9 ICU nurses in training to ICU bed ratio 10 ICU physicians (not intensivists) to ICU bed ratio 11 Intensivists to ICU bed ratio Number of hospital beds 12 Number of registration modules 13 Number of ICU admissions per ICU bed 14 Number of ICU admissions per month 2

X

X X X X

X X

X X X X

X

X

X

X X

X

X

X

X

X X

X X X X

X X X X X X X X X

X

X X

X

X

X

X

Abbreviations: CCU = Coronary Care Unit, ICU = Intensive Care Unit, PACU = Post-Anesthesia Care Unit, X denotes whether the organizational factor in the column is a confounder in the relationship between the organizational factor in the row and ICU efficiency.

15

Number of ICU admissions per month

14

Number of ICU admissions per ICU bed

13

Number of registration modules

12

Number of hospital beds

Intensivists to ICU bed ratio

11

ICU doctor to ICU bed ratio

10

ICU nurses in training to ICU bed ratio

9

Qualified ICU nurses to ICU bed ratio

8

Intensivists in training

7

PACU beds without mechanical ventilation

4

6

PACU beds with mechanical ventilation

ro X

5

-p X

CCU beds in the ICU

re

Stepdown beds without intensivist supervision 3 CCU beds in separate CCU

2

1

Jo ur

na

lP

Organizational factor as exposure variable:

Stepdown beds with intensivist supervision

Table 2: Covariates in propensity score model

Journal Pre-proof 1

re

-p

ro

of

= yes if the hospital has stepdown beds under the supervision of an intensivist, in a separate stepdown unit such as a Medium Care or Special Care or as part of the ICU. Patients can hereby be dismissed from the ICU to the stepdown unit. A CCU bed or PACU bed is not a stepdown bed. 2 = yes if the hospital has stepdown beds not under the supervision of an intensivist, in a separate stepdown unit such as a Medium Care or Special Care or as part of the ICU. Patients can hereby be dismissed from the ICU to the stepdown unit. A CCU bed or PACU bed is not a stepdown bed. 3 = yes if the hospital has CCU beds with ventilation facility in a separate CCU. 4 = yes if the ICU has CCU beds in the ICU itself (a combined ICU/CCU). 5 = yes if the hospital has a separate PACU with 24-hour PACU beds with ventilation facility. 6 = yes if the hospital has a separate PACU with 24-hour PACU beds without ventilation facility. PACU is a unit where patients who have just undergone surgery and received anesthesia can regain full consciousness and be observed until their vital functions are stable. 7 = yes if the ICU has medical specialists training as intensivists. 8 Number of FTE qualified ICU nurses in the ICU divided by the total number of ICU beds. 9 Number of FTE trainee ICU nurses working in the ICU divided by the total number of ICU beds. 10 Number of FTE physicians (excluding intensivists and intensivists in training) working in the ICU who provide practical care to ICU patients divided by the total number of ICU beds. 11 Number of FTE intensivists working in the ICU divided by the total number of ICU beds 12 The number of NICE registration modules an ICU participates in. Each Dutch ICU participates in the MDS registration module and may participate in other optional modules such as the Quality Indicator “KIIC” module, the “Complication” module, the Sequential Organ Failure Assessment “SOFA” module, the “Sepsis” module and the “Nursing capacity” module. 13 The total number of ICU admissions during study period (January 1, 2016 and January 1, 2019) divided by the total number of ICU beds 14 The total number of ICU admissions during study period (January 1, 2016 and January 1, 2019) divided by the total number of months that an ICU recorded patient admission records during study period.

lP

Table 3: Coefficients of organizational factors as exposure variables in the unadjusted- and propensity score adjusted model

Jo ur

na

Organizational factor 1 Stepdown beds with intensivist supervision 2 Stepdown beds without intensivist supervision 3 CCU beds in separate CCU 4 CCU beds in the ICU 5 PACU beds with mechanical ventilation 6 PACU beds without mechanical ventilation 7 Intensivists in training 8 Qualified ICU nurses to ICU bed ratio 9 ICU nurses in training to ICU bed ratio 10 ICU physicians (not intensivists) to ICU bed ratio 11 Intensivists to ICU bed ratio Number of hospital beds 12 Number of registration modules 13 Number of ICU admissions per ICU bed 14 Number of ICU admissions per month

Unadjusted model coefficients Intercept β Pr(>|t|)* 0.98 0.03 0.49 0.98 0.07 0.25 0.98 0.04 0.42 0.99 -0.01 0.86 0.97 0.08 0.05 0.98 0.04 0.44 0.98 0.10 0.06 1.01 -0.01 0.76 0.95 0.09 0.21 0.95 0.09 0.09 1.12 -0.28 0.01 0.94 < 0.001 0.16 0.996 -0.003 0.88 1.07 -0.001 0.08 0.94 0.001 0.13

Propensity Score Adjusted model coefficients Intercept β Pr(>|t|)* 0.97 0.01 0.85 0.95 0.04 0.58 0.98 0.04 0.43 0.98 -0.02 0.65 0.97 0.06 0.22 0.82 0.03 0.47 0.97 0.05 0.4 0.93 -0.004 0.86 0.90 0.10 0.18 0.89 0.09 0.09 1.02 -0.25 0.02 0.92 -0.01 0.76 -

Abbreviations: CCU = Coronary Care Unit, ICU = Intensive Care Unit, PACU = Post-Anesthesia Care Unit *Linear regression 1

= yes if the hospital has stepdown beds under the supervision of an intensivist, in a separate stepdown unit such as a Medium Care or Special Care or as part of the ICU. Patients can hereby be dismissed from the ICU to the stepdown unit. A CCU bed or PACU bed is not a stepdown bed.

16

Journal Pre-proof 2

-p

ro

of

= yes if the hospital has stepdown beds not under the supervision of an intensivist, in a separate stepdown unit such as a Medium Care or Special Care or as part of the ICU. Patients can hereby be dismissed from the ICU to the stepdown unit. A CCU bed or PACU bed is not a stepdown bed. 3 = yes if the hospital has CCU beds with ventilation facility in a separate CCU. 4 = yes if the ICU has CCU beds in the ICU itself (a combined ICU/CCU). 5 = yes if the hospital has a separate PACU with 24-hour PACU beds with ventilation facility. 6 = yes if the hospital has a separate PACU with 24-hour PACU beds without ventilation facility. PACU is a unit where patients who have just undergone surgery and received anesthesia can regain full consciousness and be observed until their vital functions are stable. 7 = yes if the ICU has medical specialists training as intensivists. 8 Number of FTE qualified ICU nurses in the ICU divided by the total number of ICU beds. 9 Number of FTE trainee ICU nurses working in the ICU divided by the total number of ICU beds. 10 Number of FTE physicians (excluding intensivists and intensivists in training) working in the ICU who provide practical care to ICU patients divided by the total number of ICU beds. 11 Number of FTE intensivists working in the ICU divided by the total number of ICU beds 12 The number of NICE registration modules an ICU participates in. Each Dutch ICU participates in the MDS registration module and may participate in other optional modules such as the Quality Indicator “KIIC” module, the “Complication” module, the Sequential Organ Failure Assessment “SOFA” module, the “Sepsis” module and the “Nursing capacity” module. 13 The total number of ICU admissions during study period (January 1, 2016 and January 1, 2019) divided by the total number of ICU beds 14 The total number of ICU admissions during study period (January 1, 2016 and January 1, 2019) divided by the total number of months that an ICU recorded patient admission records during study period.

re

Author statement

lP

Safira A. Wortel: Conceptualization; Data curation; Formal analysis; Methodology; Project administration; Roles/Writing – original draft

na

Nicolette F. de Keizer: Conceptualization; Methodology; Resources; Supervision; Writing – review & editing Ameen Abu-Hanna: Conceptualization; Methodology; Validation; Writing – review & editing

Jo ur

Dave A. Dongelmans: Conceptualization; Methodology; Validation; Writing – review & editing Ferishta Bakhshi-Raiez: Conceptualization; Data Curation; Methodology; Supervision; Writing – review & editing Highlights   

There is marked variability in efficiency between Dutch ICUs. The ratio of employed intensivists per ICU bed has a significant independent association with ICU efficiency, before and after confounder adjustment using propensity scores. Organizational factors regarding nurse staffing, presence of stepdown beds, presence of Post-Anesthesia Care Unit beds, presence of Coronary Care Unit beds or number of admitted patients had no statistically significant association with ICU efficiency.

17

Figure 1

Figure 2