Integrating a safety smart list into the electronic health record decreases intensive care unit length of stay and cost

Journal Pre-proof Integrating a safety smart list into the electronic health record decreases intensive care unit length of stay and cost

Daniel L. Lemkin, Benoit Stryckman, Joel E. Klein, Jason W. Custer, William Bame, Louise Maranda, Kenneth E. Wood, Courtney Paulson, Zachary D.W. Dezman PII:

S0883-9441(19)30784-1

DOI:

https://doi.org/10.1016/j.jcrc.2019.09.016

Reference:

YJCRC 53383

To appear in:

Journal of Critical Care

Please cite this article as: D.L. Lemkin, B. Stryckman, J.E. Klein, et al., Integrating a safety smart list into the electronic health record decreases intensive care unit length of stay and cost, Journal of Critical Care(2018), https://doi.org/10.1016/j.jcrc.2019.09.016

This is a PDF file of an article that has undergone enhancements after acceptance, such as the addition of a cover page and metadata, and formatting for readability, but it is not yet the definitive version of record. This version will undergo additional copyediting, typesetting and review before it is published in its final form, but we are providing this version to give early visibility of the article. Please note that, during the production process, errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

© 2018 Published by Elsevier.

Journal Pre-proof Integrating a Safety Smart List into the Electronic Health Record Decreases Intensive Care Unit Length of Stay and Cost Daniel L. Lemkin, MD, MS a Benoit Stryckman, MAa Joel E. Klein, MD b Jason W. Custer, MD c William Bame, BSd Louise Maranda, PhD e

of

Kenneth E. Wood, DOf

ro

Courtney Paulson, PhD g

-p

Zachary D.W. Dezman, MD, MS, MS a Author Affiliations a

re

Department of Emergency Medicine, University of Maryland School of Medicine, Baltimore, Maryland, USA b

lP

University of Maryland Medical System, Baltimore, Maryland, USA

c

na

Division of Pediatric Critical Care, Department of Pediatrics, University of Maryland School of Medicine, Baltimore, Maryland, USA d

Data & Analytics, University of Maryland Medical System Baltimore, Maryland, USA

e

ur

Department of Quantitative Sciences, University of Massachusetts, Worcester, Massachusetts, USA f

g

Jo

Program in Trauma, R Adams Cowley Shock Trauma Center, University of Maryland School of Medicine, Baltimore, Maryland, USA Department of Decision, Operations, and Information Technologies, University of Maryland Robert H. Smith School of Business, College Park, Maryland, USA Corresponding Author Zachary Dezman, MD, MS, MS Department of Emergency Medicine University of Maryland School of Medicine 110 S Paca Street, 6th Floor, Suite 200 Baltimore, MD 21201 410-328-5085 (phone) 1

Journal Pre-proof 410-328-8028 (fax) [email protected] Conflict of Interest: The authors have no conflicts of interest to declare. Author contribution statement: All authors have reviewed and approved the final draft of the manuscript. Funding statement: This study was unfunded. Word count (text): 3,069

Jo

ur

na

lP

re

-p

ro

of

Word count (abstract): 195

2

Journal Pre-proof ABSTRACT Purpose: To measure how an integrated smartlist developed for critically ill patients would change intensive care units (ICUs) length of stay (LOS), mortality, and charges. Materials and Methods: Propensity-score analysis of adult patients admitted to one of 14 surgical and medical ICUs between June 2017 and May 2018. The smart list aimed to certain preventative measures for all critical patients (e.g., removing unneeded

of

catheters, starting thromboembolic prophylaxis, etc.) and was integrated into the

ro

electronic health record workflows at the hospitals under study.

Results: During the study period, 11,979 patients were treated in the 14 participating

-p

ICUs by 518 unique providers. Patients who had the smart list used during ≥60% of

re

their ICU stay (N=432 patients, 3.6%) were significantly more likely to have a shorter

lP

ICU LOS (HR=1.20, 95% CI:1.0 to 1.4, p=0.015) with an average decrease of -$1,218 (95% CI: -$1,830 to -$607, P<0.001) in the amount charged per day. The intervention

na

cohort had fewer average ventilator days (3.05 vent days, SD=2.55) compared to

ur

propensity score matched controls (3.99, SD=4.68, p=0.015), but no changes in

Jo

mortality (16.7% vs 16.0%, p=0.78). Conclusions: An integrated smart list shortened LOS and lowered charges in a diverse cohort of critically ill patients. Keywords: critical care, checklist, length of stay, mortality, electronic health records

3

Journal Pre-proof INTRODUCTION Checklists are an integral element of high-reliability industries such as aviation and nuclear power. In medicine, they are used to ensure that essential elements of care are addressed on a routine basis, which facilitates standardization and adherence to essential guidelines. The intensive care unit (ICU) is a complex and fast-paced multidisciplinary environment that offers an ideal opportunity to demonstrate the benefits

of

of a checklist (1-3). The implementation of daily checklists in ICUs has yielded mixed

ro

results in regard to patient outcomes. They have decreased the incidence of central

-p

line-associated bloodstream infections (CLABIs) and the number of ventilator days and

re

surgical complications (4,5). After implementing a daily-goals checklist in a pediatric ICU, Agarwal et al reported improved communication between team members but not a

lP

statistically significant reduction in the number days of patient care from admission to

na

discharge from the ICU (length of stay, LOS) (6). Berenholtz et al demonstrated that implementing a best-practice checklist for mechanically ventilated patients reduced their

ur

mortality rate, shortened their LOS, and reduced the cost of care in one adult surgical

Jo

ICU (7). A subsequent study by Provonost (8), looking at implementation of a daily-goal checklist in ICUs, demonstrated a LOS reduction of 50%. A randomized trial of checklist implementation across ICUs in Brazil failed to demonstrate a reduction in hospital mortality rate, LOS, or rate of hospital-acquired infections (9). Traditionally, a checklist is a paper document that prompts the clinician to consider a care process that necessitates an action that changes the process, followed by the execution of an order. Its limitations are that the process involves multiple steps that are not immediately actionable from the checklist. Although a traditional checklist

4

Journal Pre-proof reminds the clinical team to discuss the removal of a central line or urinary catheter, nothing can be done until an order is placed. The care process is not implemented and the risk of harm is not reduced until the central line or catheter is actually removed. The electronic health record (EHR), with integrated nursing documentation, computerized physician order entry, and clinical decision support, has the potential to facilitate action at the point of care (10). In a pediatric ICU, Pageler and colleagues demonstrated that a

of

checklist integrated into the EHR reduced CLABSI rates from 2.6 to 0.7 per 1,000

ro

catheter days (10).

-p

There are many potential reasons for the observed heterogeneity in studies of

re

checklist use outcome: disparate checklist mechanics, varying adherence, accountability and enforcement, and differences in institutional culture. Our team

lP

sought to design a comprehensive ICU patient safety/quality of care checklist, integrate

na

it into the EHR, and subsequently test its effect on patient outcomes. This approach moves a checklist beyond a simple prompt, transforming it into an informative “smart

Jo

METHODS

ur

list” that translates documentation into orders and actions without additional effort.

Development of the Smart List Tool We developed our ICU smart list by seeking consensus about its contents from intensivists who provide surgical, trauma, medical, cardiac surgery, and neurosurgical critical care to both adults and children across a range of academic and community medical facilities. Two paper-based checklists from adult and pediatric medical ICUs served as our starting point. Each specialist brought specific recommendations based on their review of the literature and current practices in their units. After multiple

5

Journal Pre-proof discussions and revisions, the group agreed to focus the checklist on five conditions and actions: 1) Remove unnecessary catheters 2) Verify that deep venous thrombosis (DVT) and gastrointestinal (GI) prophylaxis was addressed 3) Assess and manage sedation, analgesia, and delirium

of

4) Advance enteral diet and mobility

ro

5) Improve communication with family members and people with power of

-p

attorney (POA)

re

These conditions and actions were chosen because they are common in ICUs independent of patient population, and failure to address them has been associated with

lP

hospital-acquired infections (11), venous thromboembolism (VTE), ICU LOS (12), and

na

worse clinical outcomes (13). We then designed a novel tool for integration into the ER (Epic Systems, Verona WI) using the following goals:

ur

1) Present content only if it is relevant to the specific patient (e.g., a clinician

Jo

should be prompted to remove a central venous catheter only if the patient has one).

2) Present supporting data in-line so that users can read it without leaving the smart list. 3) Improve usability by providing visual indicators that highlight optimal patient conditions as well as those requiring attention and intervention (e.g., once VTE prophylaxis has been instituted, its prompt turns green and remains checked on subsequent days, indicating that is has been optimized).

6

Journal Pre-proof 4) Make each item actionable (i.e., selections automatically become orders that immediately affect patient care). 5) Maintain existing operational and nursing workflows to increase acceptance of the smart list and eliminate training requirements for staff. The smart list is organized in clinically relevant sections, as defined by the multidisciplinary steering group. Each filtered item provides a discrete list of options,

of

from which providers make selections as they assess patients on rounds (Figure 1).

ro

When the process is completed, a dialog box opens to confirm the orders for the

-p

specific patient based on smart list selections (Supplemental Figure 1). This dialog box

re

gives clinicians the opportunity to modify or refine the orders. The use of standardized formularies and default order questions streamlines the process for users and

lP

decreases variance between care providers. These selections are then pulled into daily

na

progress notes, reducing duplicative effort (Supplemental Figure 2). Rollout, Study Population, and Environment

ur

The finalized smart list was made available to all participating ICUs using EPIC

Jo

on June 1, 2017. At least one intensivist from each of the study hospitals was involved in the smart list design and implementation. Attending intensivists across the system were educated on the use of the tool during in-person training sessions and via email. Repeat reminders were sent via email during the study period, but smart list use was not mandated at any time. There was no other LOS- or charge-saving interventions being conducted during the study period. Intensivists or any provider working under the intensivist (fellow, resident physician, or mid-level provider) could use the tool at any point during a patient’s stay in any of the 14 medical and surgical intensive care units in

7

Journal Pre-proof the following hospitals: the University of Maryland Medical Center (UMMC), a 757-bed academic and tertiary-care center in downtown Baltimore, Maryland); UMMC: Midtown Campus (MTC), a 179-bed community hospital in Baltimore City); St. Joseph’s Medical Center (SJMC), a 224-bed regional and community hospital in suburban Towson, Maryland); and Baltimore-Washington Medical Center (BWMC), a 288-bed community hospital and referral center in suburban Glen Burnie, Maryland. Subjects in this study

of

were patients who were treated in any of the University of Maryland Medical System

ro

(UMMS) ICUs between June 1, 2017, and May 31, 2018. The institutional review boards

-p

at the University of Maryland, Baltimore, and at all participating institutions approved

re

this study. Outcomes

lP

The study’s key outcome variables were ICU LOS, ICU daily charges, and ICU

na

mortality. Daily charges represent amount charged from fee schedules associated with

“expired.”

ur

resources used during the ICU stay. Mortality was defined as ICU discharge status of

Jo

Data Source and Study Design

This retrospective observational study used EHRs to obtain data relating to each patient’s demographic information, mortality risk, severity of illness, mortality, LOS, disposition, and charges. The intervention (smart list use) was compiled both (a) temporally as a binary time-dependent covariate for each day of the ICU LOS and (b) smart list prevalence, which is the ratio of days of the smart list was applied to the patient over the patient’s total ICU LOS. For example, a patient who stayed in the ICU for ten days and had the smart list used for five of those days would have a smart list

8

Journal Pre-proof prevalence of 0.50. We conducted an initial sensitivity analysis to establish the threshold of smart list prevalence required to significantly decrease a patient’s ICU stay and thereby define cases and controls. Derivation of the Matched Cohort A traditional controlled experiment could not be performed because smart list use was not mandated. We tested the collected observational data in several ways to

of

quantify possible exposure bias in our subsequent analysis. We first performed a

ro

traditional statistical bias analysis using binary logistic and probit regression models

-p

using all available patient, physician, and ICU data. The dependent outcome was

re

whether the smart list was used during the patient’s stay or not (14,15,16). A “closest neighbor” patient match showed that ICU department may have bias effects, so we

lP

conducted a propensity score matched analysis (16).

na

A multivariate logistic regression was used to calculate propensity scores for each ICU stay. The dependent variable was binary and represented a level of smart list

ur

prevalence. The independent variables were all variables that could be associated with

Jo

the smart list use including patient demographics (age, gender), discharge weekday, clinical indicators (mortality, risk of mortality, severity of illness), and instructional characteristics (specific ICU). The resulting propensity score is the predicted probability of having the smart list used at a given prevalence for each ICU encounter, reducing the effect of exposure bias by matching subjects who are closest to their overall characteristics (17). We stratified the study population by smart list prevalence to define the threshold of between cases and controls. The matching algorithm then generated

9

Journal Pre-proof 1:1 best match without replacement based on propensity score. All subsequent results were generated using this propensity-score matched group. Statistical Analysis A Cox’s proportional hazards regression model was used to measure the effect of smart list use on ICU LOS within our propensity-matched cases and controls. The exposure variable was binary time-dependent covariate smart list use for each day the

of

patient was in the ICU. We used mortality as a censor to account for patients for whom

ro

the smart list was used but who did not experience the ICU discharge endpoint. The

-p

model controlled for patient refined diagnosis-related groups (APR-DRGs) risk of

re

mortality and severity of illness (20). We used ICU fixed effects to control for clustering. The time-to-ICU discharge analysis resulted in hazard ratios, which represent the

lP

probability of that patient being discharged at a point in time after the smart list was

na

used, assuming the patient survived to that point (21). We also conducted ordinary least squares regression to measure the effect of

ur

using the smart list on ICU daily charges. Our exposure variable was the daily use of

Jo

smart list during ICU LOS. The model included 864 fixed effects to control for unique patients stays. Analyses were conducted using SAS software version 9.3 (SAS Institute Inc., Cary, NC). RESULTS Study Population The study population consisted of 11,979 patients treated in any of the 14 participating ICUs. The smart list was used at least once during the ICU stay for 2,245 patients (18.7%) and was never used for 7,882 (81.3%). Propensity-score matches

10

Journal Pre-proof were generated for each level of smart list prevalence. Our sensitivity analysis (Table 1) showed that smart list use below 60% use does not have a significant effect on ICU LOS. Smart list use above 60% use has a significant effect on ICU LOS. This effect continues with increasing smart list use, with significant increases in the hazard ratios favoring shorter ICU stays (Table 1). Propensity-score matches were then generated among 432 patients (3.6%) who had a smart list prevalence less than 60% (See

of

CONSORT diagram in Figure 2). The variables associated with smart list use and

ro

retained for calculating the propensity score included 7 ICUs.

-p

Table 2 displays the baseline patient characteristics for patients stratified by two

re

smart list use categories: (a) smart list used <60% (controls) of ICU LOS and (b) smart list used ≥60% of ICU LOS days (cases). The prevalence of urinary catheter use, the

lP

need for mechanical ventilation, and the incidence of GI bleeding was similar among

na

both groups. Cases were on the ventilator less (mean=3.05, SD=2.55 vs mean=3.99, SD=4.68, p = 0.015), their ICU LOS was shorter (4.53 vs. 6.19, P=0.001), and their

ur

charges were less ($26,745 vs. $34,613, P=0.020). There was a trend towards fewer

Jo

cases of ventilator-associated pneumonia among cases (N=3, 0.7% vs N=10, 2.3%, p=0.051). There was no difference in mortality between matched cases and controls (Table 2). Table 3 displays details of the Cox’s proportional hazard regression results for patients for whom the smart list was used for ≥60% of their ICU stay versus matched patients for whom the list was used for <60% of the ICU stay. Regression results suggest that the cases had a significantly greater likelihood of having a shorter LOS than controls (adjusted hazard ratio, 1.20 [95% CI: 1.0 to 1.39], P=0.015).

11

Journal Pre-proof We conducted an ordinary least squares regression to show the differences in charges between cases and matched controls. Regression results suggest that increasing smartlist use is associated with a decrease of -$1,218 (95% CI: -$1,830 to $607, p<0.001) in the daily amount charged. DISCUSSION In this observational study, we measured how an actionable smart list based on

of

best practices and integrated into the EHR affected overall ICU patient LOS, mortality,

ro

and charges. We found that daily use of the smart list more than 60% of the time

-p

decreased ICU LOS and was associated with lower hospital charges without a change

re

the overall mortality rate.

Other investigators have seen similar improvements in care with the use of

lP

checklists. Provonost and colleagues observed that the publication of a simple list of

na

daily care goals for an ICU patient increased the care team’s understanding of those goals and decreased ICU LOS (from 2.2 to 1.1 days) (8). Agarwal and associates found

ur

that providers’ understanding of the goals of care improved when a checklist was

Jo

implemented, decreasing LOS and facilitating communication both within the provider team and with the patient’s family (6). Byrnes et al reported that a surgical ICU checklist listing of prophylactic care increased adherence to standard guidelines, increased physical therapy referrals, decreased the time from admission to the institution of DVT prophylaxis, decreased the number of days a patient had a central venous catheter in place, and enhanced the proportion of patients downgraded from ICU status (22). Integrated checklists have become more prevalent. Investigators found that restricting the ordering of blood tests via an integrated checklist decreased costs without

12

Journal Pre-proof affecting the mortality rate (23). In a series of simulated ICU patient cases, an integrated ICU checklist was found to decrease overall cognitive work load and reduce errors without taking significantly longer than a separate paper checklist. (24) In another series of simulated cases of patients with cardiac arrest, investigators found that attending intensivists with paper checklists missed more critical actions and took longer to complete. (25) In that study, the integrated users felt more prepared to care for patients

of

in cardiac arrest. In an academic tertiary care facility similar to those included in this

ro

study, investigators found that implementation of a checklist integrated into their EMR

-p

increased compliance with standard patient care guidelines, and the variability in care

re

decreased with time.(26) Similar to our results, Duclos and others found that ventilatorfree days increased in their patients when they implemented a checklist integrated into

lP

their EMR. (27) Duclos found their patients suffered fewer cases of ventilator associated

na

pneumonia; our sample shows a clear trend in that direction. There are many barriers to checklist utilization. Some physicians feel it is an

ur

intrusion into physician autonomy, bending the focus away from the patient (28, 29)

Jo

Some checklists are difficult to use, decreasing efficiency and requiring a significant change in provider workflows (30, 31). Integration of checklists into existing EHRs has been shown to address some of these issues (31). A unique aspect of the smart list functionality employed in this study is that it both drove decision making and created the action/order in the EHR, and. Our smart list triggers the order to remove the central line when the decision is made, moving the care team from passive to active decision making. Another unique aspect of the smart list is the extraction of data from the EHR in real time so that the provider does not need to interrogate the chart for data to inform

13

Journal Pre-proof decision making. Presenting pertinent data to the decision-making team at the point of care allows ease of use and may assist in the speed and accuracy of decision making. Checklists might not be suitable or useful for a given patient population, leading physicians to avoid using them (32). In a trial conducted by Weiss et al (33), ICU patients had a lower mortality rate (OR=0.36) and shorter ICU LOS when their physicians were given daily reminders to use an accepted ICU checklist. Our tool

of

overcomes many of these barriers by being integrated into our EHR to facilitate its use,

ro

while providing a comprehensive clinical decision support for a wide array of critically ill

-p

patients without overloading the physician-user with extraneous data.

re

LIMITATIONS

This is a retrospective study and the patients were not randomized to the study

lP

arms, necessitating the use of propensity score matching. A low number of clinicians

na

used the smart list, which may be an epiphenomenon of the providers. The current study was conducted in a wide array of ICUs across a four-hospital system, suggesting

ur

our results may be seen in other hospital systems and populations. The smart list as

Jo

integrated into our EHR would be difficult to translate to a different EHR. We saw a small but statistically insignificant decrease in mortality rate among the cases using propensity score matching. This observation might be due to the increased risk of death among the cases compared with controls and warrants investigation in a randomized trial. A recent large clinical trial, pantoprazole was shown to not to significantly change the likelihood of a composite endpoint of GI bleeding, pneumonia, clostridium difficile infection, or myocardial ischemia. (34) This finding explains why there was no change in

14

Journal Pre-proof the prevalence of bleeding seen between the arms in this study. It also suggests that the checklist would need to be reviewed and updated regularly to reflect changes in the literature. CONCLUSIONS In this retrospective cohort study of critically ill patients admitted to a diverse array of ICUs across a medical system, patients for whom the smart list was used on a

of

regular basis (≥60% of their ICU stay) had a shorter LOS and lower hospital charges

ro

without a significant increase in mortality compared with patients whose physicians did

-p

not use the list. The smart list addresses issues common to all critically ill patients. Its

re

use was strongly associated with further decreases in LOS and hospital charges. ETHCS APPROVAL AND CONSENT TO PARTICIPATE

lP

We obtained human subjects’ approval from the University of Maryland Institutional

na

Review Board. As a retrospective study, we were granted a waiver of consent. CONSENT FOR PUBLICATION

ur

Not applicable.

Jo

AVAILABILITY OF DATA AND MATERIAL The data used in this study is the property of the University of Maryland Medical System and is not currently available to share. COMPETING INTERESTS The authors have no conflicts of interest to declare. We have no financial interest in the results of this work. FUNDING This work was unfunded.

15

Journal Pre-proof AUTHORS’ CONTRIBUTIONS All authors have reviewed and approved the final draft of the manuscript. ACKNOWLEDGEMENTS Not applicable. AUTHORS’ INFORMATION Zachary Dezman, MD, MS, MS

of

Department of Emergency Medicine University of Maryland School of Medicine

ro

110 S Paca Street, 6th Floor, Suite 200 Baltimore, MD 21201

-p

410-328-5085 (phone)

re

410-328-8028 (fax)

REFERENCES 1.

Jo

ur

na

lP

[email protected]

Gawande A, Lloyd JB: The Checklist Manifesto: How to Get Things Right. New York, Metropolitan Books, 2010

2.

Berenholtz SM, Pronovost PJ, Lipsett PA, et al: Eliminating catheter-related bloodstream infections in the intensive care unit. Crit Care Med 2004; 32:20142020

16

Journal Pre-proof 3.

Bergeron N, Dubois MJ, Dumont M, et al: Intensive Care Delirium Screening Checklist: evaluation of a new screening tool. Intensive Care Med 2001; 27:859864

4.

Pronovost P, Needham D, Berenholtz S, et al: An intervention to decrease catheter-related bloodstream infections in the ICU. N Engl J Med 2006; 355:27252732 Haynes AB, Weiser TG, Berry WR, et al: A surgical safety checklist to reduce

of

5.

Agarwal S, Frankel L, Tourner S, et al: Improving communication in a pediatric

-p

6.

ro

morbidity and mortality in a global population. N Engl J Med 2009; 360:491-499

7.

re

intensive care unit using daily patient goal sheets. J Crit Care 2008; 23:227-235 Berenholtz SM, Milanovich S, Faircloth A, et al: Improving care for the ventilated

Pronovost P, Berenholtz S, Dorman T, et al: Improving communication in the ICU

na

8.

lP

patient. Jt Comm J Qual Saf 2004; 30:195-204

using daily goal sheets. J Crit Care 2003; 18:71-75 Damiani LP, Cavalcanti AB, Moreira FR, et al: A cluster-randomised trial of a

ur

9.

Jo

multifaceted quality improvement intervention in Brazilian intensive care units (Checklist-ICU trial): Statistical analysis plan. Crit Care Resusc 2015; 17:113-121 10. Pageler NM, Longhurst CA, Wood M, et al: Use of electronic medical record– enhanced checklist and electronic dashboard to decrease CLABSIs. Pediatrics 2014; 113:e738-746 11. DuBose J, Teixeira PG, Inaba K, et al: Measurable outcomes of quality improvement using a daily quality rounds checklist: One-year analysis in a trauma

17

Journal Pre-proof intensive care unit with sustained ventilator-associated pneumonia reduction. J Trauma Acute Care Surg 2010; 69:855-860 12. Barr J, Fraser GL, Puntillo K, et al: Clinical practice guidelines for the management of pain, agitation, and delirium in adult patients in the intensive care unit. Crit Care Med 2013; 41:263-306 13. Weiser TG, Haynes AB, Dziekan G, et al: Effect of a 19-item surgical safety

of

checklist during urgent operations in a global patient population. Ann Surg 2010;

ro

251:976-980

-p

14. Cohen, E. H., Mowbray, C. T., Bybee, D., Yeich, S., Risbisl, K., & Freddolino, P. P.

re

(1993).Tracking and follow-up methods for research on homelessness. Evaluation Review, 17 (3), 331-352.

lP

15. Lauby, J., Kotranski, L., & Halbert, J. (1996). Effects of intervention attrition and

na

research attrition on the evaluation of an HIV prevention program. Journal of Drug Issues, 26 (3), 663-667.

ur

16. Cuddeback, G., Wilson, E., Orme, J.G., Combs-Orme, T., 2004. Detecting and

Jo

statistically correcting sample selection bias. J. Soc. Serv. Res. 30, 19–33. 17. Ming K, Rosenbaum PR. Substantial gains in bias reduction from matching with a variable number of controls. Biometrics 2000; 56(1):118–124. 18. Austin PC. A critical appraisal of propensity‐score matching in the medical literature between 1996 and 2003. Statistics in Medicine 2008; 27(12):2037–2049. 19. Parsons LS, Ovation Research Group. SUGI 29. Seattle, Washington; 2004. Performing a 1:N Case-Control Match on Propensity Score; pp. 165–29.

18

Journal Pre-proof 20. 3M His: APR™-DRG Classification Software: Overview. Available at https://archive.ahrq.gov/professionals/quality-patient-safety/qualityresources/tools/mortality/Hughessumm.pdf. Accessed June 5, 2018 21. Students 4 Best Evidence: Tutorial about Hazard Ratios. Available at https://www.students4bestevidence.net/tutorial-hazard-ratios/. Accessed June 5, 2018

of

22. Byrnes MC, Schuerer DJ, Schallom ME, et al. Implementation of a mandatory

ro

checklist of protocols and objectives improves compliance with a wide range of

-p

evidence-based intensive care unit practices. Crit Care Med 2009; 37:2775-2781

re

23. Algaze CA, Wood M, Pageler NM, et al. Use of a checklist and clinical decision support tool reduces laboratory use and improves cost. Pediatrics 2016;

lP

137:e20143019

na

24. Thongprayoon C, Harrison AM, O’Horo JC, Berrios RA, Pickering BW, Herasevich V. The effect of an electronic checklist on critical care provider workload, errors,

ur

and performance. Journal of intensive care medicine. 2016 Mar;31(3):205-12.

Jo

25. Sevilla-Berrios R, O’Horo JC, Schmickl CN, Erdogan A, Chen X, Arguello LY, Dong Y, Kilickaya O, Pickering B, Kashyap R, Gajic O. Prompting with electronic checklist improves clinician performance in medical emergencies: a high-fidelity simulation study. International journal of emergency medicine. 2018 Dec;11(1):26. 26. Conroy KM, Elliott D, Burrell AR. Testing the implementation of an electronic process-of-care checklist for use during morning medical rounds in a tertiary intensive care unit: a prospective before–after study. Annals of intensive care. 2015 Dec;5(1):20.

19

Journal Pre-proof 27. Duclos G, Zieleskiewicz L, Antonini F, Mokart D, Paone V, Po MH, Vigne C, Hammad E, Potié F, Martin C, Medam S. Implementation of an electronic checklist in the ICU: Association with improved outcomes. Anaesthesia Critical Care & Pain Medicine. 2018 Feb 1;37(1):25-33. 28. Mathews SC, Pronovost PJ. Physician autonomy and informed decision making: Finding the balance for patient safety and quality. JAMA. 2008; 300:2913-2915

of

29. Thomassen Ø, Brattebø G, Heltne JK, et al. Checklists in the operating room: Help

ro

or hurdle? A qualitative study on health workers' experiences. BMC Health Serv

-p

Res 2010; 10:342.

re

30. Nolan B, Zakirova R, Bridge J, et al. Barriers to implementing the World Health Organization’s Trauma Care Checklist: A Canadian single-center experience. J

lP

Trauma Acute Care Surg 2014; 77:679-683

na

31. Trafton J, Martins S, Michel M, et al. Evaluation of the acceptability and usability of a decision support system to encourage safe and effective use of opioid therapy

ur

for chronic, noncancer pain by primary care providers. Pain Med 2010; 11:575-585

Jo

32. Flottorp SA, Oxman AD, Krause J, et al. A checklist for identifying determinants of practice: A systematic review and synthesis of frameworks and taxonomies of factors that prevent or enable improvements in healthcare professional practice. Implement Sci 2013; 8:35 33. Weiss CH, Moazed F, McEvoy CA, et al. Prompting physicians to address a daily checklist and process of care and clinical outcomes: A single-site study. Am J Respir Crit Care Med 2011; 184:680-686

20

Journal Pre-proof 34. Krag M, Marker S, Perner A, Wetterslev J, Wise MP, Schefold JC, Keus F, Guttormsen AB, Bendel S, Borthwick M, Lange T. Pantoprazole in Patients at Risk for Gastrointestinal Bleeding in the ICU. New England Journal of Medicine. 2018

Jo

ur

na

lP

re

-p

ro

of

Dec 6;379(23):2199-208.

21

Journal Pre-proof Table and Figure Legends Table 1. Sensitivity Analysis of Level of Smart List Use on Length of ICU Stay via Propensity Score Matched Multivariate Cox Regression Model, Controlling for mortality, severity of illness, and location Table 2. Baseline Characteristics of Cases and Propensity-Score Matched Controls Who Were Treated in System ICUs between June 2017 and March 2018 Table 3. Multivariate Cox Regression Model Evaluating Smart List Effect on ICU LOS Figure 1. Checklist showing the various clinically-relevant subsections of prophylactic and preventative care

of

Figure 2. CONSORT diagram for the derivation of the matches used in the final analysis.

ro

Supplemental Figure 1. Review of checklist items prior to submitting changes as physician orders.

Jo

ur

na

lP

re

-p

Supplemental Figure 2. Summary of checklist items that were acted upon and added as an update to the physician’s progress note.

22

na

lP

re

-p

ro

of

Journal Pre-proof

Jo

ur

Figure 2. CONSORT diagram for the derivation of the matches used in the final analysis.

23

Journal Pre-proof Table 1. Sensitivity Analysis of Level of Smart List Use on Length of ICU Stay via Propensity Score Matched Multivariate Cox Regression Model, Controlling for mortality, severity of illness, and location

Level of Smart List Use During ICU Stay

N

Hazard Ratio

>0%

3254

≥10%

P 0.556

3222

0.97

0.88 to 1.06

0.478

≥20%

3166

0.97

0.89 to 1.06

0.443

≥30%

2906

1.01

0.93 to 1.11

0.755

≥40%

2114

1.07

0.97 to 1.17

0.202

≥50%

1840

1.06

0.96 to 1.17

0.277

≥60%

864

1.20

1.04 to 1.39

0.015

≥70%

496

1.30

1.07 to 1.58

0.009

≥80%

352

1.53

1.21 to 1.94

<0.001

1.93

1.46 to 2.56

<0.001

lP

re

-p

ro

of

0.97

95% CI 0.89 to 1.07

≥90%

Jo

ur

na

280

24

Journal Pre-proof Table 2. Baseline Characteristics of Cases and Propensity-Score Matched Controls Who Were Treated in System ICUs between June 2017 and March 2018

61.60 (16.7)

< 60% of ICU Stay Matched Cohort, n=432 62.30 (16.5)

≥ 60% of ICU Stay, n=432 60.90 (16.8)

0.217

345 (39.9)

168 (38.9)

177 (41.0)

0.532

25 (3.0)

12 (2.8)

13 (3.0)

0.839

47 (10.9)

45 (10.4)

0.825

125 (28.9)

130 (30.1)

0.709

243 (56.3)

243 (56.3)

1.000

8 (0.9)

3 (0.7)

5 (1.2)

0.478

lP

Smart List Use

63 (7.3)

30 (6.9)

33 (7.6)

0.695

169 (31.1)

136 (31.5)

133 (30.8)

0.826

522 (60.0)

262 (60.7)

260 (60.2)

0.889

141 (16.3)

72 (16.7)

69 (16.0)

0.782

80 (9.3)

40 (9.3)

40 (9.3)

1.000

82 (9.5)

39 (9.0)

43 (10.0)

0.642

127 (14.7)

62 (14.4)

65 (15.1)

0.773

Wednesday

160 (18.5)

82 (19.0)

78 (18.1)

0.726

Thursday

151 (17.5)

76 (17.6)

75 (17.4)

0.929

Friday

156 (18.1)

77 (17.8)

79 (18.3)

0.860

Saturday

108 (12.5)

56 (13.0)

52 (12.0)

0.681

Length of stay, days (SD)

5.36 (5.7)

6.19 (6.8)

4.53 (4.2)

<0.001

Total charges, $ (SD)

30679.46 (34293.5)

34613.40 (38665.8)

26745.50 (28788.0)

0.001

Characteristics

All, n= 864

Age, yr (SD) Female sex, n (%)

P

Minor Moderate

of

Risk of mortality, n (%)

255 (29.5)

Extreme

486 (56.3)

re

Severity of illness, n (%)

-p

Major

ro

92 (10.6)

Minor Moderate Major

na

Extreme Mortality, n (%)

Monday Tuesday

Jo

Sunday

ur

Weekday discharge, n (%)

Payer, n (%)

25

Commercial

314 (36.3)

153 (35.4)

161 (37.3)

0.572

Medicare 3

460 (53.2)

236 (54.6)

224 (51.9)

0.413

Medicaid 2

50 (5.8)

25 (5.8)

25 (5.8)

1.000

Hospital 1

24 (2.8)

12 (2.8)

12 (2.8)

1.000

Hospital 2

68 (8.0)

34 (7.9)

34 (7.9)

1.000

Hospital 3

421 (48.7)

215 (49.8)

206 (47.7)

0.540

Hospital 4

351 (41.8)

171 (39.6)

180 (41.7)

0.533

of

Journal Pre-proof

12 (2.8)

1.000

34 (7.9)

34 (7.9)

1.000

2 (0.5)

2 (0.5)

1.000

0 (0.0)

0 (0.0)

Hospitals, n (%)

24 (2.8)

ICU 2

69 (8.0)

ICU 3

4 (0.5)

ICU 4

0 (0.0)

ICU 5

20 (2.3)

11 (2.6)

9 (2.1)

0.651

ICU 6

1 (0.1)

0 (0.0)

1 (0.2)

0.317

0 (0.0)

0 (0.0)

0 (0.0)

38 (4.4)

18 (4.2)

20 (4.6)

0 (0.0)

0 (0.0)

0 (0.0)

4 (0.1)

3 (0.7)

1 (0.2)

0 (0.0)

0 (0.0)

0 (0.0)

228 (26.4)

116 (26.9)

112 (25.9)

0.758

126 (14.6)

65 (15.1)

61 (14.1)

0.700

351 (41.0)

171 (39.6)

180 (41.7)

0.533

259 (29.9)

124 (28.8)

135 (31.1)

0.333

2.93 (2.66)

2.92 (2.85)

2.93 (2.46)

0.973

Placement on Mechanical Ventilation, n (%)

234 (27.1)

112 (26.0)

122 (28.3)

0.355

Days on mechanical ventilation, Mean (SD)

3.52 (3.79)

3.99 (4.68)

3.05 (2.55)

0.015

na

ICU 8

ICU 13

Jo

ICU 12

ur

ICU 9

ICU 11

-p

re

ICU 7

ICU 10

12 (2.8)

ro

ICU 1

lP

Intensive Care Units, n (%)

ICU 14 Urinary Catheter Placement, n (%) Days of Urinary Catheter Placement, mean (SD)

0.740

0.316

26

Journal Pre-proof Ventilator-associated pneumonia, n (%)

13 (1.5)

10 (2.3)

3 (0.7)

0.051

Diagnoses of gastrointestinal bleed, n (%)

15 (1.7)

8 (1.9)

7 (1.6)

0.795

Jo

ur

na

lP

re

-p

ro

of

*t-test or chi-squared, where appropriate

27

Journal Pre-proof Table 3. Multivariate Cox Regression Model Evaluating Smart List Effect on Intensive Care Unit (ICU) length of stay (LOS) Patients Smart list used ≥ 60% vs. < 60% during ICU stay matched cohort (n=864) Parameter

Hazard Ratio

P 0.015

Minor

0.13

0.03 to 0.59

0.008

Moderate

0.09

Major

0.07

0.02 to 0.26

<0.001

2.68

1.84 to 3.91

<0.001

2.04

1.63 to 2.54

<0.001

1.53

0.99 to 2.35

0.056

0.51

0.34 to 0.76

0.001

0.53

0.44 to 0.65

<0.001

0.50

0.39 to 0.64

<0.001

Jo

ur

na

Specific Intensive Care Unit*

ICU 13

<0.001

re

Major

lP

Moderate

ICU 12

0.02 to 0.30

0.06

Severity of illness

ICU 8

0.001

-p

Extreme

ICU 1

0.02 to 0.39

ro

Smart List Use (time dependent) Risk of mortality

of

1.20

95% CI 1.04 to 1.39

*ICU 1 is a community medical ICU, ICU 8 is an academic surgical ICU, ICU 12 and 13 are both academic medical ICUs

28

Journal Pre-proof

ur

na

lP

re

-p

ro

of

Checklists enhance care Use decreases length of stay, charges No increase in mortality

Jo

-

29

Figure 1

Figure 2

Figure 3

Figure 4