- Email: [email protected]
Electronically implemented clinical indicators based on a data warehouse in a tertiary hospital: Its clinical benefit and effectiveness
Accepted Manuscript Title: Electronically implemented clinical indicators based on a data warehouse in a tertiary hospital: Its clinical benefit and effectiveness Author: Sooyoung Yoo Seok Kim Kee-Hyuck Lee Chang Wook Jeong Sang Woong Youn Kyoung Un Park So Young Moon Hee Hwang PII: DOI: Reference:
S1386-5056(14)00061-6 http://dx.doi.org/doi:10.1016/j.ijmedinf.2014.04.001 IJB 3082
To appear in:
International Journal of Medical Informatics
Received date: Revised date: Accepted date:
2-9-2013 27-3-2014 20-4-2014
Please cite this article as: S. Yoo, S. Kim, K.-H. Lee, C.W. Jeong, S.W. Youn, K.U. Park, S.Y. Moon, H. Hwang, Electronically implemented clinical indicators based on a data warehouse in a tertiary hospital: its clinical benefit and effectiveness, International Journal of Medical Informatics (2014), http://dx.doi.org/10.1016/j.ijmedinf.2014.04.001 This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Electronically implemented clinical indicators based on a data warehouse in a tertiary hospital: its clinical benefit and effectiveness
ip t
Sooyoung Yoo, PhDa, Seok Kim, MSa, Kee-Hyuck Lee, MD, MSa, Chang Wook Jeong, MD, PhDb, MD, Sang Woong Youn, MDc, Kyoung Un Park, MDd, So Young Moon,
us
cr
RNe, Hee Hwang, MD, PhDa
a
an
Center of Medical Informatics, Seoul National University Bundang Hospital, South
Korea
Department of Urology, College of Medicine, Seoul National University, South Korea
M
b
c
e
Laboratory Medicine, Seoul National University Bundang Hospital, South Korea
Ac ce pt e
d
d
Dermatology, Seoul National University Bundang Hospital, South Korea
Management Innovation Department, Seoul National University Bundang Hospital,
South Korea
Corresponding author: Hee Hwang, MD, PhD Center for Medical Informatics, Department of Pediatrics, Seoul National University Bundang Hospital, 166, Gumi-ro, Bundang-gu, Seongnam-si, Korea (postal code: 436707) E-mail: [email protected] Tel: +82-31-787-7284 / Fax: +82-31-787-4054 1/1
Page 1 of 31
Keywords: hospital information systems, integrated advanced information management
Ac ce pt e
d
M
an
us
cr
ip t
systems, health information management, quality of care, data warehouse
2/2
Page 2 of 31
ABSTRACT OBJECTIVE: Assessing and monitoring care and service using clinical indicators (CIs) can allow the measurement of and lead to improvements in the quality of care.
ip t
However, the management and maintenance of CI data has been shown to be difficult
cr
because the data are usually collected and provided manually. In this study, for the
purpose of efficient managing quality indicators, a data warehouse (DW)-based CI
us
monitoring system was developed. The clinical effectiveness and efficiency of a DWbased CI monitoring was investigated through several case studies of the system’s
an
operation at a tertiary hospital.
M
METHODS: This study analyzed the CIs that have been developed over the past 8 years at a 1,340-bed tertiary general university hospital in South Korea to improve and
d
monitor the quality of care and patient safety. The hospital was opened as a fully digital
Ac ce pt e
hospital in 2003, and the CIs were computerized in 2005 by implementing a DW-based CI monitoring system. We classified the computerized CIs and evaluated the monitoring results for several representative CIs, such as the optimal prescribing of preventive antibiotics, the average length of stay, the mortality rate, and the rehospitalization rate. RESULTS: During the development of the system in 2005, 12 of 19 CIs were computerized, and this number gradually increased until 299 of 335 CIs were computerized by 2012. In addition, among the CIs built computationally through the CI task force team, focal CIs subject to monitoring were selected annually, and the results of this monitoring were shared with all of the staff or the related department and its staff. By providing some examples of our CI monitoring results, we showed the 3/3
Page 3 of 31
feasibility of improving the quality of care, and maintaining the optimum level of patient care with less labor. CONCLUSIONS: The results of this study provide evidence regarding the clinical
ip t
effectiveness and efficiency as well as the systems operation experience of a DW-based CI monitoring system. These findings may aid medical institutions that plan on
cr
computerizing CIs with respect to decision and policy making regarding their systems
Ac ce pt e
d
M
an
us
development and operations.
4/4
Page 4 of 31
INTRODUCTION As interest in healthcare increases in concert with people’s desire to be healthier, the demand for quality medical services also increases [1, 2]. Accordingly, each country is
ip t
devoting significant attention to increasing the quantity and the quality of medical services while keeping treatment costs as low as possible [1, 2]. A clinical indicator (CI)
cr
is an indicator developed for assessment purposes to objectively measure and
us
continually monitor medical quality of care [3, 4]. In 2007, four CI assessment categories that were not included in the 2004 Joint Commission's assessment in Korea,
an
including pneumonia, intensive care, maternity and neonatal and surgical infection prevention divisions were added. In 2004, pneumonia was the 6th leading cause of death
M
for elderly patients over the age of 65 in the United States [5]. This condition has a very high cost of treatment due to its high death and hospitalization rates; however, these
d
rates can potentially be lowered through improvements in treatment processes [5, 6]. As
Ac ce pt e
a result, U.S. institutions such as the Centers for Medicare & Medicaid Services (CMS) and the Joint Commission on Accreditation of Healthcare Organizations (JCAHO) have been trying to improve the quality of pneumonia treatment processes by selecting oxygen saturation measurement, time of blood culture assessment, time of antibiotic administration, and other parameters as the process indicators [7]. In addition, by developing and adjusting indicators of specific diseases, including pneumonia and stroke rehabilitation, several studies exploring the effects of the indicators have been conducted [8-12]. Similarly, implementing CIs in monitoring and evaluating the quality of care and service can lead to better measurement of the quality of care and improvement in 5/5
Page 5 of 31
performance [13, 14]. However, such CIs are usually collected and produced manually, which makes their continual management and maintenance difficult. In other words, through computerized monitoring of specific indicators, the safety of care and patient
ip t
safety activities may be systematically and continuously monitored [15]. In this study, a data warehouse (DW)-based CI monitoring system was developed to
cr
maintain computerized quality indicators. We then evaluated the clinical effectiveness
us
and efficiency of the DW-based CI monitoring system through systems operations at a tertiary hospital. By sharing the operational experiences and knowledge of the systems,
an
the results of this study may aid medical facilities that plan on implementing a DWbased CI monitoring system when making decisions by providing them with sufficient
Ac ce pt e
METHODS
d
M
evidence for efficiency.
Study Site
This study was performed at the Seoul National University Bundang Hospital (SNUBH), located in Seongnam-si, Gyeonggi-do, in the Seoul metropolitan area of Korea. SNUBH is a national tertiary hospital founded in May 2003 as a fully digital hospital equipped with a fully paperless comprehensive electronic health record (EHR) system certified with an Electronic Medical Record Adoption Model (EMRAM) Stage 7. The hospital has 1,340 beds and has been visited by an average of approximately 5,000 outpatients daily as of August 2013. As the hospital had a full electronic medical record (EMR) system, in 2004 it began developing a data warehouse system using the EMR for monitoring performance and clinical indicators as well as to support research. 6/6
Page 6 of 31
Clinical Indicator Development Process To continuously improve the quality of care, we created two major committees: a medical service innovation committee and a hospital safety control committee. The CIs were developed and monitored by a CI task force team (TFT), which was one of the
ip t
TFTs under the medical service innovation committee. The CI TFT was composed of 15
cr
members, including 5 doctors, 1 nurse, 2 staff members from the medical information
team, 1 staff member from the medical records team, 1 pharmacist, 1 staff member from
us
the health insurance team, and 4 staff members from the quality assurance (QA) team.
an
Figure 1 shows our CI development process. The CI TFT examined and approved all of the CI content from selection to monitoring throughout the development process. The
M
team discussed items that could be indicators, shared the content of exact indicators, performed a pilot study on the selected indicators, and reviewed the computerization of
d
those indicators. Once the CIs were computerized, the TFT periodically monitored the
Ac ce pt e
indicators and decided on the method of feedback to all parties concerned.
7/7
Page 7 of 31
Discussion of which items should be chosen by the CI TFT
Pilot study: survey analysis
us
Discussion and review of computerization
cr
Discussion to determine detailed definitions and ranges of the CIs
ip t
Discussion of the appropriateness of each CI
Discussion of monitoring and feedback methods
an
Figure 1. Clinical indicator development process
M
Through monthly meetings, the CI TFT developed the CIs necessary for improving treatment quality. During the period from 2006 to 2007, which was the initial stage of
d
CI development, the meetings were held on a weekly basis. From 2008 onward, the
Ac ce pt e
meetings were held on a biweekly basis.
Electronic Implementation of Clinical Indicators Based on a Data Warehouse. The electronic clinical indicators were implemented into the data warehouse (DW) system so that they did not affect the performance of the operational EMR system. The DW system consisted of operational data storage (ODS), a clinical data warehouse (CDW) and a data mart. The patient information, financial information, and medical data, including diagnoses, prescriptions, tests, medical records, and nursing records, were automatically updated from the operational EMR database to the DW system daily using an extraction, transformation and loading (ETL) tool. Online analytical processing 8/8
Page 8 of 31
(OLAP) was used to develop functions to search, analyze, and visualize the data using a set of ad-hoc query keywords and to manage various indexes regarding quality of care, performance, and customer satisfaction. Figure 2 shows the overall system architecture
Direct Care ETL
Nursing
OLAP
Clinical Data Warehouse (CDW)
Operational Data Storage (ODS)
Patie nt
Dx.
Rx.
Te st
Re cords
OP
CC
Nursing
Data Mart Specific Data for Analysis
-EMR Data Modeling -Data Cle ansing
Supportive Care
Re trie val for Rese arch
cr
Data Warehouse
Free -te xt Re trieval
Tre atme nt Analysis
us
Data Source
ip t
for electronic clinical indicators based on the DW.
Structure d Data
C linical Indicators
ETL
Administration
Insurance Patie nts
Records
Rx.
ETL
OP
CC
Test
Nursing
C ustomer Satisfaction Indicators 6Sigma
Q uality of medical Re cords C linical Pathway Indicators
Memory Insurance Analysis
d
EMR
Dx.
M
Other
an
Memory Platform
Ac ce pt e
Figure 2. Overall architecture of a clinical indicator system based on a data warehouse
Based on the DW system, we developed 4 types of electronic indicators: clinical indicators, performance indicators, safety indicators, and patient experience indicators. The clinical indicators were a collection of indicators related to treatment activities that served the purpose of managing the quality of treatment interactions. The performance indicators consisted of various departmental indicators that were unrelated to treatment. The safety indicators consisted of crucial indicators related to patient safety. The patient
9/9
Page 9 of 31
experience indicators consisted of indicators that managed hospital facilities and program components experienced by patients.
ip t
Monitoring of and Feedback on Clinical Indicators Electronic CIs were reported to the related departments and executives through
cr
discussions and via monitoring and feedback methods. Thus, a committee for selecting
us
focal monitoring indicators was established to select CIs that required priority maintenance at the beginning of each year. The monitoring indicators were selected
an
through evaluation of the five criteria by the CI TFT, and out of approximately 10-11
M
sectors, 79-90 indicators were selected.
The monitoring results for each of the selected CIs were sent to the appropriate
d
department for feedback for each monitoring period (e.g., monthly or quarterly).
Ac ce pt e
Because the CIs were selected from the indicators implemented on the DW system, the heads of the respective departments or practitioners were encouraged to directly access and manage the data by logging onto the system whenever necessary. The results were also provided to the indicator management committee based on requests by the relevant committees (e.g., providing the results of indicators of blood return/disposal to bloodtransfusion management committee that consists of CIO and members from anesthesiology, emergency medicine, hematology & medical oncology, general surgery, nursing unit, laboratory medicine, and QA) or as deemed necessary by the CI TFT.
10 / 10
Page 10 of 31
RESULTS Clinical Indicator Development and Monitoring Results The research facility has been developing and managing CIs through the CI TFT since 2005, and the CIs monitored each year were selected and provided to the appropriate
ip t
departments. Figure 3 presents the cumulative number of CIs developed since 2005,
cr
including the number of CIs computationally developed through the DW system and the
number of CIs subject to monitoring. The results indicate that the number of CIs being
us
developed, computerized, and monitored steadily increased over time.
an
400
335
350
290
300
268
180
150
120
100
79
Ac ce pt e -
19
29
# of computerized CIs # of monitored CIs
134
96
65
# of developed CIs
190
d
Number
235
200
50
264
M
244
250
299
45
82 61
75
43
9
12
2005
2006
2007
2008
2009
2010
2011
2012
Year
Figure 3. Status of CI development and monitoring for eight years
Table 1 shows the common clinical indicators, which were hospital-wide indicators that applied to every clinical department and included indexes for treatment management, emergency management, transfusion management, antibiotics management, CPR management, and clinical quality assessment.
11 / 11
Page 11 of 31
Table 1. Common clinical indicators
Sub-Category
Example indicator
Treatment
Referral response time to other department
Total referral response time
Long-term hospitalization rate
Number of long-term hospitalized patients
Readmission rate of ICU patients
ICU readmission rate
PRN order
PRN prescription order rate
Admissions through the ER
Admissions via the emergency room
ER readmissions
Number of discharged patients returning to ER within 1 week
Statistics on ER patients
Hospital stay statistics by department
Emergency medical institution evaluation index
Adequacy of recanalization therapy for acute myocardial infarction patients
Regular operation start time
Regular operation start time compliance rate
cr
Operation waiting time
Operation waiting time
Blood return/disposal rate
Blood return rate 2: number of blood acceptance RBC Single Unit transfusion rate
d
Transfusion status
Unplanned operation rate
M
Unplanned reoperation rate Transfusion
us
Operation
an
Emergency
ip t
Category
blood
Ac ce pt e
Classification of transfusion reactions
Classification of blood transfusion reactions
Co-administration of drugs and blood
Co-administration of drugs and blood
Blood transfusion time
Post-transfusion vital sign measurement rate
Blood return rate
Ordered blood units return rate after storage
Death Index
Mortality
Inpatient mortality within 48 hours via ambulatory or emergency departments
Patient Care
Number of hospital inpatients per day
Suitability Assessment
stay
Average length of hospital stay
Outpatient reservation rate/noshow rate
Outpatient reservation rate
AMI Index
% of patients receiving thrombolytic treatment within 30 minutes of hospital arrival
CABG Index
Average hospitalization length after isolated CABG surgery
Evaluation relevance
of
medication
Volume-outcome index
Adrenocorticotropic drug prescription rate (osteoarthritis) Operation rate of 7 surgeries (per operation)
12 / 12
Page 12 of 31
Rectal cancer index
Hospital Index
upper
checking rate of TDM
Preventive antibiotics prior to surgery
Preventive antibiotic administration rate within 1 hour before skin incision
Preventive antibiotic use in each department
Rate of aminoglycoside administration
Preventive antibiotics use in clean surgeries
Antibiotic allergy history recording rate
CPR survival rate
CPR survival rate on second try (immediately after procedure)
Time between cardiac arrest and CPR
Time between cardiac arrest and starting CPR
Expertness of intubation with CPR
Number of intubations with CPR
Operation
cr
Waiting time in operating room Waiting period for imaging examination
Reply from referral within 48 hours
Breastfeeding trial rate within 1 hour after normal delivery
Ac ce pt e
d
Maternal/new born
M
Referral
Pneumonia
ip t
TDM monitoring index
Examination
Clinical Quality Assessment Index
Length of antibiotic therapy for acute upper respiratory infections
us
CPR
Antibiotics for acute respiratory infections
an
Antibiotics Index
Pain evaluation rating before surgery
Intensive Care Unit
Rate of SaO2 monitoring Sitting position for preventing pneumonia
Table 2 lists various departmental clinical indicators that were categorized by medical department and care support group. For example, the gastroenterology and surgery departments developed and monitored the use of preventive antibiotics within 24 hours after visit to the hospital due to bleeding varices and re-excision rate of mastectomy, respectively.
13 / 13
Page 13 of 31
Table 2. Clinical indicators for medical departments and care support departments
Category
Sub-Category
Example indicator
Medical Department Index
Neurology
Early antithrombotics
Emergency Medicine
Re-visit rate of APN patient in 5 days
Surgery
Re-excision rate of mastectomy
ENT
ER visit rate in 2 weeks after tonsillectomy
Urology
Average operation time
ip t
Use of preventive antibiotics in 24hours after visit of hospital due to varix bleeding
cr
Surgical Medicine
Gastroenterology
and
Pain
Incidence rate of hypothermia after surgery
an
Anesthesiology Medicin
us
Internal Medicine
Orthopedics
Dislocation rate after THRA Average admission days after free flap
M
Plastic Surgery Care Support Index Pharmacy
Contraindicated drugs for elderly patients
Rate of patient contraindicated drug
Nursing
Medication records
Omitting rate of administration record for medicine
Ac ce pt e
d
administration
Nutrition
prescribed
Pressure sore index
Pressure sore prevalence rate at wards
Maternity and newborn
Ways of newborn baby feeding
Pain index
Pain prevalence rate
Rate of patient management for malnutrition
Ratio of malnutrition patients
Nutrition assessment and the dead patients
Mortality rate due to malnutrition
elderly
Clinical Effectiveness of Clinical Indicators Figure 4 provides an example of one of the many cases in which a CI contributed to improving and monitoring the quality of care. We were able to maintain optimal dosages of preoperative antibiotics by giving feedback of preoperative antibiotic 14 / 14
Page 14 of 31
prescription through continual monitoring. It can lead to an optimal selection of which preoperative antibiotics to prescribe. Following the application of the CI, the administration rate of aminoglycosides and 3rd-generation cephalosporins decreased to
ip t
less than 2%, which was far lower than the recommended rate in Korea of 10%.
d
M
an
us
cr
251658240
Ac ce pt e
Figure 4. A case of clinical indicator application: optimal prescription of preventive antibiotics
In addition, the average length of hospital stay for inpatients has steadily decreased (see Figure 5). As can be observed from the quarterly averages, the average length of hospital stay was 7.9 days in the first quarter of 2005 when the CI development first began, but through continual development and monitoring since 2005, in the first quarter of 2013, the average length of hospital stay was decreased by 2.1 days to a low of 5.8 days. Additionally, compared to the manual CI data collection of the past, the DW-based CI system allowed more efficient management with less labor. The system 15 / 15
Page 15 of 31
contributed to easily monitor and evaluate the average length of hospital stay to make an effort to reduce the hospitalization length. However, it is also considerable that many additional variables would have involved in decreasing the hospitalization length, such as technological development, advanced expertise, and some instances change in the
ip t
policy of the insurers, and change in the hospitalized population profile.
Ac ce pt e
d
M
an
us
cr
251658240
Figure 5. The average length of hospital stays over approximately 10 years
Table 3 shows more examples of the CI monitoring results for ICU patient mortality and rehospitalization rates. Using the DW system, we monitored the ICU patient mortality rate on a quarterly basis beginning in 2003 and the rehospitalization rate beginning in 2008. Table 3 shows several specific indicators and their results during a recent year as an example, highlighting the benefits of developing various specific indicators using the DW system. 16 / 16
Page 16 of 31
Table 3. Examples of CI monitoring results: ICU patient mortality rate, ICU infection rate regarding multidrug resistant microorganisms, and rehospitalization rate through the ER during a recent year
Q4 2012
Q1 2013
Q2 2013
Mortality rate for patients admitted to the ICU once
4.29%
6.05%
5.70%
3.86%
Mortality rate for patients who were readmitted to the ICU more than twice
12.24%
4.44%
Total mortality rate
4.75%
8.93%
us
ICU patient mortality rate
ip t
Q3 2012
cr
Indicators
5.97%
5.90%
4.17% 3.87%
an
ICU Infection rate (the separation rate of multidrug resistant microorganisms among inpatients in ICU) MRAB
5.16%
4.26%
3.38%
3.12%
0.52%
0.16%
0%
0%
0%
0%
0%
0%
0.17%
0.63%
0.59%
0.47%
5.16%
4.26%
3.38%
3.12%
0%
0%
0%
0%
Rate of rehospitalization through the ER within 24 hours
1.91%
1.96%
1.61%
1.73%
Rate of rehospitalization through the ER within 48 hours
3.45%
3.52%
3.13%
3.29%
Rate of rehospitalization through the ER within 96 hours
5.03%
4.91%
4.58%
4.51%
Rate of rehospitalization through the ER within 1 week
6.25%
6.07%
5.79%
5.81%
CRE VRE
M
MRPA
Ac ce pt e
VRSA/VISA
d
MRSA
ER rehospitalization rate
Figure 6 shows an example of a visualization tool that is based on CIs; the CI used for this pie chart and table was the number of PRN orders issued during the selected range per department. 17 / 17
Page 17 of 31
ip t cr us
251658240
an
Figure 6. Example of CI report on the number of PRN orders issued during the selected range, along with a departmental pie chart
M
Benefits of Electronic Clinical Indicators
From our 8 years of experiences in operating electronic CIs based on the DW system,
d
we found that the system has various benefits for managers and users, such as
Ac ce pt e
clinicians, nurses, and administrative staffs. In terms of managers, in the authors’ organization we have only 3 staffs for implementing, maintaining, and managing about 300 CIs, including one program developer, one medical information coordinator, and one QA staff who is responsible for managing all CIs and communicating with user groups. The system helps the manager to efficiently monitor hundreds of CIs, compared with conventional manual management that can handle only a few dozens of CIs required for hospital evaluation. It also enables the manager to analyze and utilize huge amount of data that can be collected since the opening of the hospital, and to provide rapid feedback of monitoring relevant CIs to users using the most recent data, for example, some CIs are monitored daily in the authors’ organization. In addition, it is 18 / 18
Page 18 of 31
useful for collecting accurate data consistently according to the definition of CIs, regardless of changing of the person in charge of the CI management. In terms of users, the DW-based CI monitoring system can help them to immediately utilize the monitoring data for their improvement activities of quality of care and patient
ip t
safety by directly accessing the system and checking the data whenever they need. For
cr
example, our nursing unit is utilizing the monitoring results of their relevant CIs as the
evaluation data on their performance and improvement. The monitoring data has been
us
also utilized for developing new innovative activities of our hospital. This system
an
provides continuous feedbacks to the relevant teams during the activities.
M
DISCUSSION
This study reported our experience developing and applying of a CI monitoring system
d
based on a DW. Studies examining the importance of CIs are currently being conducted
Ac ce pt e
because management of CIs can lead to better patient safety, improved healthcare quality, and increased efficiency of hospital administration [3, 16]. Additionally, a study on the development of not only general indicators for maintaining the quality of healthcare and improving the efficiency of a hospital's administration but also indicators that can be implemented for specific diseases is currently being actively conducted. Fine JM et al. [8] and Mitchiner JC et al. [9] analyzed possible improvements to indicators regarding patients with pneumonia by analyzing the rate of antibiotic administration with increasing numbers of admitted pneumonia patients, the rate of examinations performed based on the number of hospital staff per patient, and the rate of antibiotic administration for patients admitted to the emergency room.
19 / 19
Page 19 of 31
Meehan TP et al. [10] reported that for patients with pneumonia, the 30-day death rate can be reduced by 15% through improvements in the indicator for administration of antibiotics within 8 hours of admittance, and this rate can be reduced by 10% by performing a blood-culture examination within 24 hours. Additionally, Houck PM et al.
ip t
[11] claimed that because improvements in indicators can reduce the length of
cr
hospitalization, the cost of treatment can be reduced as well. This study shows that the
management of computationally built CIs can contribute to improvements in the
us
management of patients and the quality of treatments, as well as the efficiency of the hospital's operations and administration. For example, in this study, we monitored the
an
optimal dosage of preoperative antibiotics, which resulted in the administration rates of
M
aminoglycoside and third-generation cephalosporins decreasing to less than 2%. Because there are no apparent benefits to choosing a third-generation cephalosporin
d
over a first- or second-generation cephalosporin for prophylaxis [17], third-generation
Ac ce pt e
cephalosporins are generally not recommended for surgical prophylaxis. The firstgeneration cephalosporin cefazolin has been widely recommended for prophylaxis [18, 19]. The aminoglycosides could cause nephrotoxicity and ototoxicity [20, 21]; thus, their use should be reserved for specific cases that cannot be treated with other antibiotics [19, 22]. In addition to monitoring the clinical indicators for the optimal prescription of preventive antibiotics, we showed the possible monitoring results of another several CIs, such as the average length of hospital stay, patient mortality rate, infection rate, and the recurrent hospitalization rates, which have been used for continuous improvement activities in the authors’ organization. The DW-based CI system has several advantages in terms of improving of efficiency. 20 / 20
Page 20 of 31
First, the time required for developing a CI can be reduced because it is based on EMR data. Additionally, the DW is operated on a separate server; thus, it is not loading when the program is operating, and it not only simply checks for the values of the indicators but also enables additional multidimensional analyses based on patient characteristics,
ip t
disease, location, and other variables. The DW information is refined upon saving for
cr
effortless analysis, and the most recent information is always available. Compared to
manual data collection, the DW allows for much more efficient maintenance with less
us
labor. Currently, only 1 engineer and 1 medical information coordinator are responsible for the maintenance and computational development of about 300 CIs in our hospital.
an
The system enables to consistently maintain the definition of CIs while manual
M
management of CIs contains problems on the management issues of the definition of various CIs especially when the manger is changed. It also helps users to immediately
d
access the monitoring data and utilize it for their improvement activities of quality of
Ac ce pt e
care, patient safety, and work efficiency.
Computerization alone is not sufficient to efficiently manage and monitor the CIs; rather, efforts to ensure fluent and efficient communications between staff and between departments, as well as efforts to improve the quality of healthcare and patient safety, should become a foundational policy. In our hospital, we shared information about clinical indicator management situations and decision-making matters with respect to hospital quality improvement and patient safety with the whole faculty through various meetings and groupware. To ensure that the physicians are well informed, we hold regular meetings for the department chiefs, professors, residents and fellows. We also hold monthly faculty meetings and use electronic news and groupware to share the CI 21 / 21
Page 21 of 31
monitoring reports and collect feedback with/from hospital staff. As for the limitations, it is possible that the clinical effects of the CIs and the merits of the study results are difficult to generalize because the study was conducted at a specific
ip t
hospital with CIs developed at the hospital. Nevertheless, the experience of this hospital and the results accumulated over the course of 8 years regarding the implementation of
cr
CIs and the system operations can certainly be of help to any hospitals that plan on
us
adopting a similar system.
This study focused on the CIs, however, as we mentioned before, there exists other
an
types of indicators, such as performance indicators, safety indicators, and patient experience indicators. For example, performance indicators include incompleteness rate
M
of medical documents, lead time for various types of medical documents, and the cancellation rate of operation. Safety indicators are related with the management of
Ac ce pt e
d
patient’s decubitus ulcer, falling and medication errors. Examples of patient experience indicators are for monitoring of outpatient’s waiting time for consultation and the net promoter score (NPS). As the patient safety and patient satisfaction are getting more important in the era of patient-centered care, the development of those indicators and continual monitoring of those ones should be further studied and addressed in future work.
CONCLUSIONS Maintaining the quality of healthcare is necessary for evaluating healthcare quality and as a foundation for improving healthcare services. This study shared our 8 years of 22 / 22
Page 22 of 31
experiences in maintaining the quality of healthcare using the DW-based CI monitoring system and showed some samples of various CIs and their monitoring results, such as the administration of the optimal dosage of preoperative antibiotics, the average length of hospitalization, the mortality rate, the infection rate, and the rehospitalization rate.
ip t
The system contributed to efficiently develop, maintain, and monitor approximately 300
cr
CIs with the least labor (e.g., only one CI manager in the authors’ organization of 1,340 beds). With the continuous monitoring and feedback activities, the system enabled to
us
maintain the optimum level of patient care. It is worthy of note that in order to effectively use the system, various communication channels to share the CI monitoring
an
results through the facility and feedback activities with/from users are required. The
M
DW-based CI monitoring system can be utilized as a powerful tool to provide ideas on improvement activities to users by enabling them to directly access relevant CIs.
d
The future work will continue in analyzing the return-on-investment (ROI) of the
Ac ce pt e
system in terms of economic evaluation, investigating more effective data visualization methods for users to easily recognize their objectives and achievement, and focusing on more developing patient-centered indicators such as patient safety indicators and patient satisfaction.
Acknowledgements
This work was partly supported by the IT R&D program of SNUBH and SKT.
Legends Figure 1. Clinical indicator development process 23 / 23
Page 23 of 31
Figure 2. Overall architecture of a clinical indicator system based on a data warehouse Figure 3. Status of CI development and monitoring for eight years Figure 4. A case of clinical indicator application: optimal prescription of preventive antibiotics
ip t
Figure 5. The average length of hospital stays over approximately 10 years
cr
Figure 6. Example of CI report on the number of PRN orders issued during the selected
Ac ce pt e
d
M
an
us
range, along with a departmental pie chart
24 / 24
Page 24 of 31
References [1] J.W. Curtright, S.C. Stolp-Smith, E.S. Edell, Strategic performance management: development of a performance measurement system at the Mayo Clinic, Journal of
ip t
healthcare management / American College of Healthcare Executives, 45 (2000) 58-68.
cr
[2] J. Mainz, Defining and classifying clinical indicators for quality improvement,
us
International journal for quality in health care : journal of the International Society for Quality in Health Care / ISQua, 15 (2003) 523-530.
an
[3] S.M. Shortell, J.L. O'Brien, J.M. Carman, R.W. Foster, E.F. Hughes, H. Boerstler, E.J. O'Connor, Assessing the impact of continuous quality improvement/total
M
quality management: concept versus implementation, Health services research, 30 (1995) 377-401.
d
[4] C.J. Huston, Outcomes measurement in healthcare. New imperatives for
Ac ce pt e
professional nursing practice, Nursing case management : managing the process of patient care, 4 (1999) 188-195.
[5] G.L. Colice, M.A. Morley, C. Asche, H.G. Birnbaum, Treatment costs of community-acquired pneumonia in an employed population, Chest, 125 (2004) 2140-2145.
[6] T.P. Meehan, M.J. Fine, H.M. Krumholz, J.D. Scinto, D.H. Galusha, J.T. Mockalis, G.F. Weber, M.K. Petrillo, P.M. Houck, J.M. Fine, Quality of care, process, and outcomes in elderly patients with pneumonia, JAMA : the journal of the American Medical Association, 278 (1997) 2080-2084. [7] D.W. Bratzler, W. Nsa, P.M. Houck, Performance measures for pneumonia: are they 25 / 25
Page 25 of 31
valuable, and are process measures adequate?, Current opinion in infectious diseases, 20 (2007) 182-189. [8] J.M. Fine, M.J. Fine, D. Galusha, M. Petrillo, T.P. Meehan, Patient and hospital characteristics associated with recommended processes of care for elderly patients
ip t
hospitalized with pneumonia: results from the medicare quality indicator system
cr
pneumonia module, Archives of internal medicine, 162 (2002) 827-833.
[9] J.C. Mitchiner, S.L. Hutto, The effect of selected hospital characteristics on the
us
timeliness of antibiotic administration for pneumonia, American journal of medical quality : the official journal of the American College of Medical Quality, 22 (2007)
an
259-264.
M
[10] T.P. Meehan, S.R. Weingarten, E.S. Holmboe, D. Mathur, Y. Wang, M.K. Petrillo, G.S. Tu, J.M. Fine, A statewide initiative to improve the care of hospitalized
d
pneumonia patients: The Connecticut Pneumonia Pathway Project, The American
Ac ce pt e
journal of medicine, 111 (2001) 203-210. [11] P.M. Houck, D.W. Bratzler, W. Nsa, A. Ma, J.G. Bartlett, Timing of antibiotic administration and outcomes for Medicare patients hospitalized with communityacquired pneumonia, Archives of internal medicine, 164 (2004) 637-644.
[12] M.M. Grube, C. Dohle, D. Djouchadar, P. Rech, K. Bienek, U. Dietz-Fricke, M. Jobges, M. Kohler, I. Missala, B. Schonherr, C. Werner, H. Zeytountchian, J. Wissel, P.U. Heuschmann, Evidence-based quality indicators for stroke rehabilitation, Stroke; a journal of cerebral circulation, 43 (2012) 142-146. [13] J. Mainz, Developing evidence-based clinical indicators: a state of the art methods primer, International journal for quality in health care : journal of the International Society for Quality in Health Care / ISQua, 15 Suppl 1 (2003) i5-11. 26 / 26
Page 26 of 31
[14] J. Mainz, B.R. Krog, B. Bjornshave, P. Bartels, Nationwide continuous quality improvement using clinical indicators: the Danish National Indicator Project, International journal for quality in health care : journal of the International Society for Quality in Health Care / ISQua, 16 Suppl 1 (2004) i45-50.
ip t
[15] S. Kristensen, J. Mainz, P. Bartels, Selection of indicators for continuous
cr
monitoring of patient safety: recommendations of the project 'safety improvement
for patients in Europe', International journal for quality in health care : journal of
us
the International Society for Quality in Health Care / ISQua, 21 (2009) 169-175. [16] S.M. Shortell, D.Z. Levin, J.L. O'Brien, E.F. Hughes, Assessing the evidence on
an
CQI: is the glass half empty or half full?, Hospital & health services administration,
M
40 (1995) 4-24.
[17] S.L. Gorbach, The role of cephalosporins in surgical prophylaxis, Journal of
d
Antimicrobial Chemotherapy, 23 (1989) 61-70.
Ac ce pt e
[18] S. Geroulanos, K. Marathias, J. Kriaras, B. Kadas, Cephalosporins in surgical prophylaxis, Journal of Chemotherapy, 13 (2001) 23-26.
[19] D.W. Bratzler, P.M. Houck, Antimicrobial prophylaxis for surgery: an advisory statement from the National Surgical Infection Prevention Project, The American Journal of Surgery, 189 (2005) 395-404.
[20] G.B. Appel, Aminoglycoside nephrotoxicity, The American journal of medicine, 88 (1990) S16-S20.
[21] E. Selimoglu, Aminoglycoside-induced ototoxicity, Current pharmaceutical design, 13 (2007) 119-126. [22] M.T. Keegan, D.R. Brown, Perioperative antibiotics and practice: Little things that make a big difference, Anesthesiology Clinics of North America, 22 (2004) 47327 / 27
Page 27 of 31
Ac ce pt e
d
M
an
us
cr
ip t
491.
28 / 28
Page 28 of 31
Summary points What was already known: Patients are now demanding a higher quality of medical services due to
ip t
increased interest in health. To improve the quality of treatments, it is necessary to maintain clinical
cr
indicators (CIs), and for continual and efficient maintenance, the CIs must be
us
collected systematically.
an
CI maintenance not only improves the quality of treatments but also contributes to the hospital administration and its operations. However, such CIs are usually
Ac ce pt e
What this study added
d
management and maintenance.
M
collected and produced manually, which presents difficulties for their continual
In a tertiary general hospital, maintenance with DW-based computerized quality indicators resulted in improvements in patient safety, management of treatment quality, and hospital administration. By automatically importing data by managing and monitoring computationally built CIs, less labor is needed to manage the CIs more efficiently than in the past, when the data for CI indicators were collected manually. As one example of the efficacy of computerized CIs, over the course of 8 years, through efficient CI management, the average length of inpatient hospitalization decreased by 2.1 days. 29 / 29
Page 29 of 31
Conflicts of Interest There are no conflicts of interests that could inappropriately influence the authors’ findings.
ip t
Author contributions S. Yoo and S. Kim designed the study and drafted the manuscript. K. Hyuck, C. Wook,
cr
S. Youn, and K. Park contributed to the discussions of data and reviewed the
us
manuscript. S. Moon analyzed and categorized the data. H. Hwang reviewed and
Ac ce pt e
d
M
an
revised the manuscript.
31 / 31
Page 30 of 31
Research Highlights DW-based clinical indicators (CIs) were developed to monitor the quality of care.
ip t
The number of CIs being developed, computerized, and monitored steadily increased over time.
cr
This study showed improvement in patient safety, quality of care and hospital
us
management.
Ac ce pt e
d
M
hospitalization decreased by 2.1 days.
an
Through efficient CI management, the average length of inpatient
32 / 32
Page 31 of 31
S1386-5056(14)00061-6 http://dx.doi.org/doi:10.1016/j.ijmedinf.2014.04.001 IJB 3082
To appear in:
International Journal of Medical Informatics
Received date: Revised date: Accepted date:
2-9-2013 27-3-2014 20-4-2014
Please cite this article as: S. Yoo, S. Kim, K.-H. Lee, C.W. Jeong, S.W. Youn, K.U. Park, S.Y. Moon, H. Hwang, Electronically implemented clinical indicators based on a data warehouse in a tertiary hospital: its clinical benefit and effectiveness, International Journal of Medical Informatics (2014), http://dx.doi.org/10.1016/j.ijmedinf.2014.04.001 This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Electronically implemented clinical indicators based on a data warehouse in a tertiary hospital: its clinical benefit and effectiveness
ip t
Sooyoung Yoo, PhDa, Seok Kim, MSa, Kee-Hyuck Lee, MD, MSa, Chang Wook Jeong, MD, PhDb, MD, Sang Woong Youn, MDc, Kyoung Un Park, MDd, So Young Moon,
us
cr
RNe, Hee Hwang, MD, PhDa
a
an
Center of Medical Informatics, Seoul National University Bundang Hospital, South
Korea
Department of Urology, College of Medicine, Seoul National University, South Korea
M
b
c
e
Laboratory Medicine, Seoul National University Bundang Hospital, South Korea
Ac ce pt e
d
d
Dermatology, Seoul National University Bundang Hospital, South Korea
Management Innovation Department, Seoul National University Bundang Hospital,
South Korea
Corresponding author: Hee Hwang, MD, PhD Center for Medical Informatics, Department of Pediatrics, Seoul National University Bundang Hospital, 166, Gumi-ro, Bundang-gu, Seongnam-si, Korea (postal code: 436707) E-mail: [email protected] Tel: +82-31-787-7284 / Fax: +82-31-787-4054 1/1
Page 1 of 31
Keywords: hospital information systems, integrated advanced information management
Ac ce pt e
d
M
an
us
cr
ip t
systems, health information management, quality of care, data warehouse
2/2
Page 2 of 31
ABSTRACT OBJECTIVE: Assessing and monitoring care and service using clinical indicators (CIs) can allow the measurement of and lead to improvements in the quality of care.
ip t
However, the management and maintenance of CI data has been shown to be difficult
cr
because the data are usually collected and provided manually. In this study, for the
purpose of efficient managing quality indicators, a data warehouse (DW)-based CI
us
monitoring system was developed. The clinical effectiveness and efficiency of a DWbased CI monitoring was investigated through several case studies of the system’s
an
operation at a tertiary hospital.
M
METHODS: This study analyzed the CIs that have been developed over the past 8 years at a 1,340-bed tertiary general university hospital in South Korea to improve and
d
monitor the quality of care and patient safety. The hospital was opened as a fully digital
Ac ce pt e
hospital in 2003, and the CIs were computerized in 2005 by implementing a DW-based CI monitoring system. We classified the computerized CIs and evaluated the monitoring results for several representative CIs, such as the optimal prescribing of preventive antibiotics, the average length of stay, the mortality rate, and the rehospitalization rate. RESULTS: During the development of the system in 2005, 12 of 19 CIs were computerized, and this number gradually increased until 299 of 335 CIs were computerized by 2012. In addition, among the CIs built computationally through the CI task force team, focal CIs subject to monitoring were selected annually, and the results of this monitoring were shared with all of the staff or the related department and its staff. By providing some examples of our CI monitoring results, we showed the 3/3
Page 3 of 31
feasibility of improving the quality of care, and maintaining the optimum level of patient care with less labor. CONCLUSIONS: The results of this study provide evidence regarding the clinical
ip t
effectiveness and efficiency as well as the systems operation experience of a DW-based CI monitoring system. These findings may aid medical institutions that plan on
cr
computerizing CIs with respect to decision and policy making regarding their systems
Ac ce pt e
d
M
an
us
development and operations.
4/4
Page 4 of 31
INTRODUCTION As interest in healthcare increases in concert with people’s desire to be healthier, the demand for quality medical services also increases [1, 2]. Accordingly, each country is
ip t
devoting significant attention to increasing the quantity and the quality of medical services while keeping treatment costs as low as possible [1, 2]. A clinical indicator (CI)
cr
is an indicator developed for assessment purposes to objectively measure and
us
continually monitor medical quality of care [3, 4]. In 2007, four CI assessment categories that were not included in the 2004 Joint Commission's assessment in Korea,
an
including pneumonia, intensive care, maternity and neonatal and surgical infection prevention divisions were added. In 2004, pneumonia was the 6th leading cause of death
M
for elderly patients over the age of 65 in the United States [5]. This condition has a very high cost of treatment due to its high death and hospitalization rates; however, these
d
rates can potentially be lowered through improvements in treatment processes [5, 6]. As
Ac ce pt e
a result, U.S. institutions such as the Centers for Medicare & Medicaid Services (CMS) and the Joint Commission on Accreditation of Healthcare Organizations (JCAHO) have been trying to improve the quality of pneumonia treatment processes by selecting oxygen saturation measurement, time of blood culture assessment, time of antibiotic administration, and other parameters as the process indicators [7]. In addition, by developing and adjusting indicators of specific diseases, including pneumonia and stroke rehabilitation, several studies exploring the effects of the indicators have been conducted [8-12]. Similarly, implementing CIs in monitoring and evaluating the quality of care and service can lead to better measurement of the quality of care and improvement in 5/5
Page 5 of 31
performance [13, 14]. However, such CIs are usually collected and produced manually, which makes their continual management and maintenance difficult. In other words, through computerized monitoring of specific indicators, the safety of care and patient
ip t
safety activities may be systematically and continuously monitored [15]. In this study, a data warehouse (DW)-based CI monitoring system was developed to
cr
maintain computerized quality indicators. We then evaluated the clinical effectiveness
us
and efficiency of the DW-based CI monitoring system through systems operations at a tertiary hospital. By sharing the operational experiences and knowledge of the systems,
an
the results of this study may aid medical facilities that plan on implementing a DWbased CI monitoring system when making decisions by providing them with sufficient
Ac ce pt e
METHODS
d
M
evidence for efficiency.
Study Site
This study was performed at the Seoul National University Bundang Hospital (SNUBH), located in Seongnam-si, Gyeonggi-do, in the Seoul metropolitan area of Korea. SNUBH is a national tertiary hospital founded in May 2003 as a fully digital hospital equipped with a fully paperless comprehensive electronic health record (EHR) system certified with an Electronic Medical Record Adoption Model (EMRAM) Stage 7. The hospital has 1,340 beds and has been visited by an average of approximately 5,000 outpatients daily as of August 2013. As the hospital had a full electronic medical record (EMR) system, in 2004 it began developing a data warehouse system using the EMR for monitoring performance and clinical indicators as well as to support research. 6/6
Page 6 of 31
Clinical Indicator Development Process To continuously improve the quality of care, we created two major committees: a medical service innovation committee and a hospital safety control committee. The CIs were developed and monitored by a CI task force team (TFT), which was one of the
ip t
TFTs under the medical service innovation committee. The CI TFT was composed of 15
cr
members, including 5 doctors, 1 nurse, 2 staff members from the medical information
team, 1 staff member from the medical records team, 1 pharmacist, 1 staff member from
us
the health insurance team, and 4 staff members from the quality assurance (QA) team.
an
Figure 1 shows our CI development process. The CI TFT examined and approved all of the CI content from selection to monitoring throughout the development process. The
M
team discussed items that could be indicators, shared the content of exact indicators, performed a pilot study on the selected indicators, and reviewed the computerization of
d
those indicators. Once the CIs were computerized, the TFT periodically monitored the
Ac ce pt e
indicators and decided on the method of feedback to all parties concerned.
7/7
Page 7 of 31
Discussion of which items should be chosen by the CI TFT
Pilot study: survey analysis
us
Discussion and review of computerization
cr
Discussion to determine detailed definitions and ranges of the CIs
ip t
Discussion of the appropriateness of each CI
Discussion of monitoring and feedback methods
an
Figure 1. Clinical indicator development process
M
Through monthly meetings, the CI TFT developed the CIs necessary for improving treatment quality. During the period from 2006 to 2007, which was the initial stage of
d
CI development, the meetings were held on a weekly basis. From 2008 onward, the
Ac ce pt e
meetings were held on a biweekly basis.
Electronic Implementation of Clinical Indicators Based on a Data Warehouse. The electronic clinical indicators were implemented into the data warehouse (DW) system so that they did not affect the performance of the operational EMR system. The DW system consisted of operational data storage (ODS), a clinical data warehouse (CDW) and a data mart. The patient information, financial information, and medical data, including diagnoses, prescriptions, tests, medical records, and nursing records, were automatically updated from the operational EMR database to the DW system daily using an extraction, transformation and loading (ETL) tool. Online analytical processing 8/8
Page 8 of 31
(OLAP) was used to develop functions to search, analyze, and visualize the data using a set of ad-hoc query keywords and to manage various indexes regarding quality of care, performance, and customer satisfaction. Figure 2 shows the overall system architecture
Direct Care ETL
Nursing
OLAP
Clinical Data Warehouse (CDW)
Operational Data Storage (ODS)
Patie nt
Dx.
Rx.
Te st
Re cords
OP
CC
Nursing
Data Mart Specific Data for Analysis
-EMR Data Modeling -Data Cle ansing
Supportive Care
Re trie val for Rese arch
cr
Data Warehouse
Free -te xt Re trieval
Tre atme nt Analysis
us
Data Source
ip t
for electronic clinical indicators based on the DW.
Structure d Data
C linical Indicators
ETL
Administration
Insurance Patie nts
Records
Rx.
ETL
OP
CC
Test
Nursing
C ustomer Satisfaction Indicators 6Sigma
Q uality of medical Re cords C linical Pathway Indicators
Memory Insurance Analysis
d
EMR
Dx.
M
Other
an
Memory Platform
Ac ce pt e
Figure 2. Overall architecture of a clinical indicator system based on a data warehouse
Based on the DW system, we developed 4 types of electronic indicators: clinical indicators, performance indicators, safety indicators, and patient experience indicators. The clinical indicators were a collection of indicators related to treatment activities that served the purpose of managing the quality of treatment interactions. The performance indicators consisted of various departmental indicators that were unrelated to treatment. The safety indicators consisted of crucial indicators related to patient safety. The patient
9/9
Page 9 of 31
experience indicators consisted of indicators that managed hospital facilities and program components experienced by patients.
ip t
Monitoring of and Feedback on Clinical Indicators Electronic CIs were reported to the related departments and executives through
cr
discussions and via monitoring and feedback methods. Thus, a committee for selecting
us
focal monitoring indicators was established to select CIs that required priority maintenance at the beginning of each year. The monitoring indicators were selected
an
through evaluation of the five criteria by the CI TFT, and out of approximately 10-11
M
sectors, 79-90 indicators were selected.
The monitoring results for each of the selected CIs were sent to the appropriate
d
department for feedback for each monitoring period (e.g., monthly or quarterly).
Ac ce pt e
Because the CIs were selected from the indicators implemented on the DW system, the heads of the respective departments or practitioners were encouraged to directly access and manage the data by logging onto the system whenever necessary. The results were also provided to the indicator management committee based on requests by the relevant committees (e.g., providing the results of indicators of blood return/disposal to bloodtransfusion management committee that consists of CIO and members from anesthesiology, emergency medicine, hematology & medical oncology, general surgery, nursing unit, laboratory medicine, and QA) or as deemed necessary by the CI TFT.
10 / 10
Page 10 of 31
RESULTS Clinical Indicator Development and Monitoring Results The research facility has been developing and managing CIs through the CI TFT since 2005, and the CIs monitored each year were selected and provided to the appropriate
ip t
departments. Figure 3 presents the cumulative number of CIs developed since 2005,
cr
including the number of CIs computationally developed through the DW system and the
number of CIs subject to monitoring. The results indicate that the number of CIs being
us
developed, computerized, and monitored steadily increased over time.
an
400
335
350
290
300
268
180
150
120
100
79
Ac ce pt e -
19
29
# of computerized CIs # of monitored CIs
134
96
65
# of developed CIs
190
d
Number
235
200
50
264
M
244
250
299
45
82 61
75
43
9
12
2005
2006
2007
2008
2009
2010
2011
2012
Year
Figure 3. Status of CI development and monitoring for eight years
Table 1 shows the common clinical indicators, which were hospital-wide indicators that applied to every clinical department and included indexes for treatment management, emergency management, transfusion management, antibiotics management, CPR management, and clinical quality assessment.
11 / 11
Page 11 of 31
Table 1. Common clinical indicators
Sub-Category
Example indicator
Treatment
Referral response time to other department
Total referral response time
Long-term hospitalization rate
Number of long-term hospitalized patients
Readmission rate of ICU patients
ICU readmission rate
PRN order
PRN prescription order rate
Admissions through the ER
Admissions via the emergency room
ER readmissions
Number of discharged patients returning to ER within 1 week
Statistics on ER patients
Hospital stay statistics by department
Emergency medical institution evaluation index
Adequacy of recanalization therapy for acute myocardial infarction patients
Regular operation start time
Regular operation start time compliance rate
cr
Operation waiting time
Operation waiting time
Blood return/disposal rate
Blood return rate 2: number of blood acceptance RBC Single Unit transfusion rate
d
Transfusion status
Unplanned operation rate
M
Unplanned reoperation rate Transfusion
us
Operation
an
Emergency
ip t
Category
blood
Ac ce pt e
Classification of transfusion reactions
Classification of blood transfusion reactions
Co-administration of drugs and blood
Co-administration of drugs and blood
Blood transfusion time
Post-transfusion vital sign measurement rate
Blood return rate
Ordered blood units return rate after storage
Death Index
Mortality
Inpatient mortality within 48 hours via ambulatory or emergency departments
Patient Care
Number of hospital inpatients per day
Suitability Assessment
stay
Average length of hospital stay
Outpatient reservation rate/noshow rate
Outpatient reservation rate
AMI Index
% of patients receiving thrombolytic treatment within 30 minutes of hospital arrival
CABG Index
Average hospitalization length after isolated CABG surgery
Evaluation relevance
of
medication
Volume-outcome index
Adrenocorticotropic drug prescription rate (osteoarthritis) Operation rate of 7 surgeries (per operation)
12 / 12
Page 12 of 31
Rectal cancer index
Hospital Index
upper
checking rate of TDM
Preventive antibiotics prior to surgery
Preventive antibiotic administration rate within 1 hour before skin incision
Preventive antibiotic use in each department
Rate of aminoglycoside administration
Preventive antibiotics use in clean surgeries
Antibiotic allergy history recording rate
CPR survival rate
CPR survival rate on second try (immediately after procedure)
Time between cardiac arrest and CPR
Time between cardiac arrest and starting CPR
Expertness of intubation with CPR
Number of intubations with CPR
Operation
cr
Waiting time in operating room Waiting period for imaging examination
Reply from referral within 48 hours
Breastfeeding trial rate within 1 hour after normal delivery
Ac ce pt e
d
Maternal/new born
M
Referral
Pneumonia
ip t
TDM monitoring index
Examination
Clinical Quality Assessment Index
Length of antibiotic therapy for acute upper respiratory infections
us
CPR
Antibiotics for acute respiratory infections
an
Antibiotics Index
Pain evaluation rating before surgery
Intensive Care Unit
Rate of SaO2 monitoring Sitting position for preventing pneumonia
Table 2 lists various departmental clinical indicators that were categorized by medical department and care support group. For example, the gastroenterology and surgery departments developed and monitored the use of preventive antibiotics within 24 hours after visit to the hospital due to bleeding varices and re-excision rate of mastectomy, respectively.
13 / 13
Page 13 of 31
Table 2. Clinical indicators for medical departments and care support departments
Category
Sub-Category
Example indicator
Medical Department Index
Neurology
Early antithrombotics
Emergency Medicine
Re-visit rate of APN patient in 5 days
Surgery
Re-excision rate of mastectomy
ENT
ER visit rate in 2 weeks after tonsillectomy
Urology
Average operation time
ip t
Use of preventive antibiotics in 24hours after visit of hospital due to varix bleeding
cr
Surgical Medicine
Gastroenterology
and
Pain
Incidence rate of hypothermia after surgery
an
Anesthesiology Medicin
us
Internal Medicine
Orthopedics
Dislocation rate after THRA Average admission days after free flap
M
Plastic Surgery Care Support Index Pharmacy
Contraindicated drugs for elderly patients
Rate of patient contraindicated drug
Nursing
Medication records
Omitting rate of administration record for medicine
Ac ce pt e
d
administration
Nutrition
prescribed
Pressure sore index
Pressure sore prevalence rate at wards
Maternity and newborn
Ways of newborn baby feeding
Pain index
Pain prevalence rate
Rate of patient management for malnutrition
Ratio of malnutrition patients
Nutrition assessment and the dead patients
Mortality rate due to malnutrition
elderly
Clinical Effectiveness of Clinical Indicators Figure 4 provides an example of one of the many cases in which a CI contributed to improving and monitoring the quality of care. We were able to maintain optimal dosages of preoperative antibiotics by giving feedback of preoperative antibiotic 14 / 14
Page 14 of 31
prescription through continual monitoring. It can lead to an optimal selection of which preoperative antibiotics to prescribe. Following the application of the CI, the administration rate of aminoglycosides and 3rd-generation cephalosporins decreased to
ip t
less than 2%, which was far lower than the recommended rate in Korea of 10%.
d
M
an
us
cr
251658240
Ac ce pt e
Figure 4. A case of clinical indicator application: optimal prescription of preventive antibiotics
In addition, the average length of hospital stay for inpatients has steadily decreased (see Figure 5). As can be observed from the quarterly averages, the average length of hospital stay was 7.9 days in the first quarter of 2005 when the CI development first began, but through continual development and monitoring since 2005, in the first quarter of 2013, the average length of hospital stay was decreased by 2.1 days to a low of 5.8 days. Additionally, compared to the manual CI data collection of the past, the DW-based CI system allowed more efficient management with less labor. The system 15 / 15
Page 15 of 31
contributed to easily monitor and evaluate the average length of hospital stay to make an effort to reduce the hospitalization length. However, it is also considerable that many additional variables would have involved in decreasing the hospitalization length, such as technological development, advanced expertise, and some instances change in the
ip t
policy of the insurers, and change in the hospitalized population profile.
Ac ce pt e
d
M
an
us
cr
251658240
Figure 5. The average length of hospital stays over approximately 10 years
Table 3 shows more examples of the CI monitoring results for ICU patient mortality and rehospitalization rates. Using the DW system, we monitored the ICU patient mortality rate on a quarterly basis beginning in 2003 and the rehospitalization rate beginning in 2008. Table 3 shows several specific indicators and their results during a recent year as an example, highlighting the benefits of developing various specific indicators using the DW system. 16 / 16
Page 16 of 31
Table 3. Examples of CI monitoring results: ICU patient mortality rate, ICU infection rate regarding multidrug resistant microorganisms, and rehospitalization rate through the ER during a recent year
Q4 2012
Q1 2013
Q2 2013
Mortality rate for patients admitted to the ICU once
4.29%
6.05%
5.70%
3.86%
Mortality rate for patients who were readmitted to the ICU more than twice
12.24%
4.44%
Total mortality rate
4.75%
8.93%
us
ICU patient mortality rate
ip t
Q3 2012
cr
Indicators
5.97%
5.90%
4.17% 3.87%
an
ICU Infection rate (the separation rate of multidrug resistant microorganisms among inpatients in ICU) MRAB
5.16%
4.26%
3.38%
3.12%
0.52%
0.16%
0%
0%
0%
0%
0%
0%
0.17%
0.63%
0.59%
0.47%
5.16%
4.26%
3.38%
3.12%
0%
0%
0%
0%
Rate of rehospitalization through the ER within 24 hours
1.91%
1.96%
1.61%
1.73%
Rate of rehospitalization through the ER within 48 hours
3.45%
3.52%
3.13%
3.29%
Rate of rehospitalization through the ER within 96 hours
5.03%
4.91%
4.58%
4.51%
Rate of rehospitalization through the ER within 1 week
6.25%
6.07%
5.79%
5.81%
CRE VRE
M
MRPA
Ac ce pt e
VRSA/VISA
d
MRSA
ER rehospitalization rate
Figure 6 shows an example of a visualization tool that is based on CIs; the CI used for this pie chart and table was the number of PRN orders issued during the selected range per department. 17 / 17
Page 17 of 31
ip t cr us
251658240
an
Figure 6. Example of CI report on the number of PRN orders issued during the selected range, along with a departmental pie chart
M
Benefits of Electronic Clinical Indicators
From our 8 years of experiences in operating electronic CIs based on the DW system,
d
we found that the system has various benefits for managers and users, such as
Ac ce pt e
clinicians, nurses, and administrative staffs. In terms of managers, in the authors’ organization we have only 3 staffs for implementing, maintaining, and managing about 300 CIs, including one program developer, one medical information coordinator, and one QA staff who is responsible for managing all CIs and communicating with user groups. The system helps the manager to efficiently monitor hundreds of CIs, compared with conventional manual management that can handle only a few dozens of CIs required for hospital evaluation. It also enables the manager to analyze and utilize huge amount of data that can be collected since the opening of the hospital, and to provide rapid feedback of monitoring relevant CIs to users using the most recent data, for example, some CIs are monitored daily in the authors’ organization. In addition, it is 18 / 18
Page 18 of 31
useful for collecting accurate data consistently according to the definition of CIs, regardless of changing of the person in charge of the CI management. In terms of users, the DW-based CI monitoring system can help them to immediately utilize the monitoring data for their improvement activities of quality of care and patient
ip t
safety by directly accessing the system and checking the data whenever they need. For
cr
example, our nursing unit is utilizing the monitoring results of their relevant CIs as the
evaluation data on their performance and improvement. The monitoring data has been
us
also utilized for developing new innovative activities of our hospital. This system
an
provides continuous feedbacks to the relevant teams during the activities.
M
DISCUSSION
This study reported our experience developing and applying of a CI monitoring system
d
based on a DW. Studies examining the importance of CIs are currently being conducted
Ac ce pt e
because management of CIs can lead to better patient safety, improved healthcare quality, and increased efficiency of hospital administration [3, 16]. Additionally, a study on the development of not only general indicators for maintaining the quality of healthcare and improving the efficiency of a hospital's administration but also indicators that can be implemented for specific diseases is currently being actively conducted. Fine JM et al. [8] and Mitchiner JC et al. [9] analyzed possible improvements to indicators regarding patients with pneumonia by analyzing the rate of antibiotic administration with increasing numbers of admitted pneumonia patients, the rate of examinations performed based on the number of hospital staff per patient, and the rate of antibiotic administration for patients admitted to the emergency room.
19 / 19
Page 19 of 31
Meehan TP et al. [10] reported that for patients with pneumonia, the 30-day death rate can be reduced by 15% through improvements in the indicator for administration of antibiotics within 8 hours of admittance, and this rate can be reduced by 10% by performing a blood-culture examination within 24 hours. Additionally, Houck PM et al.
ip t
[11] claimed that because improvements in indicators can reduce the length of
cr
hospitalization, the cost of treatment can be reduced as well. This study shows that the
management of computationally built CIs can contribute to improvements in the
us
management of patients and the quality of treatments, as well as the efficiency of the hospital's operations and administration. For example, in this study, we monitored the
an
optimal dosage of preoperative antibiotics, which resulted in the administration rates of
M
aminoglycoside and third-generation cephalosporins decreasing to less than 2%. Because there are no apparent benefits to choosing a third-generation cephalosporin
d
over a first- or second-generation cephalosporin for prophylaxis [17], third-generation
Ac ce pt e
cephalosporins are generally not recommended for surgical prophylaxis. The firstgeneration cephalosporin cefazolin has been widely recommended for prophylaxis [18, 19]. The aminoglycosides could cause nephrotoxicity and ototoxicity [20, 21]; thus, their use should be reserved for specific cases that cannot be treated with other antibiotics [19, 22]. In addition to monitoring the clinical indicators for the optimal prescription of preventive antibiotics, we showed the possible monitoring results of another several CIs, such as the average length of hospital stay, patient mortality rate, infection rate, and the recurrent hospitalization rates, which have been used for continuous improvement activities in the authors’ organization. The DW-based CI system has several advantages in terms of improving of efficiency. 20 / 20
Page 20 of 31
First, the time required for developing a CI can be reduced because it is based on EMR data. Additionally, the DW is operated on a separate server; thus, it is not loading when the program is operating, and it not only simply checks for the values of the indicators but also enables additional multidimensional analyses based on patient characteristics,
ip t
disease, location, and other variables. The DW information is refined upon saving for
cr
effortless analysis, and the most recent information is always available. Compared to
manual data collection, the DW allows for much more efficient maintenance with less
us
labor. Currently, only 1 engineer and 1 medical information coordinator are responsible for the maintenance and computational development of about 300 CIs in our hospital.
an
The system enables to consistently maintain the definition of CIs while manual
M
management of CIs contains problems on the management issues of the definition of various CIs especially when the manger is changed. It also helps users to immediately
d
access the monitoring data and utilize it for their improvement activities of quality of
Ac ce pt e
care, patient safety, and work efficiency.
Computerization alone is not sufficient to efficiently manage and monitor the CIs; rather, efforts to ensure fluent and efficient communications between staff and between departments, as well as efforts to improve the quality of healthcare and patient safety, should become a foundational policy. In our hospital, we shared information about clinical indicator management situations and decision-making matters with respect to hospital quality improvement and patient safety with the whole faculty through various meetings and groupware. To ensure that the physicians are well informed, we hold regular meetings for the department chiefs, professors, residents and fellows. We also hold monthly faculty meetings and use electronic news and groupware to share the CI 21 / 21
Page 21 of 31
monitoring reports and collect feedback with/from hospital staff. As for the limitations, it is possible that the clinical effects of the CIs and the merits of the study results are difficult to generalize because the study was conducted at a specific
ip t
hospital with CIs developed at the hospital. Nevertheless, the experience of this hospital and the results accumulated over the course of 8 years regarding the implementation of
cr
CIs and the system operations can certainly be of help to any hospitals that plan on
us
adopting a similar system.
This study focused on the CIs, however, as we mentioned before, there exists other
an
types of indicators, such as performance indicators, safety indicators, and patient experience indicators. For example, performance indicators include incompleteness rate
M
of medical documents, lead time for various types of medical documents, and the cancellation rate of operation. Safety indicators are related with the management of
Ac ce pt e
d
patient’s decubitus ulcer, falling and medication errors. Examples of patient experience indicators are for monitoring of outpatient’s waiting time for consultation and the net promoter score (NPS). As the patient safety and patient satisfaction are getting more important in the era of patient-centered care, the development of those indicators and continual monitoring of those ones should be further studied and addressed in future work.
CONCLUSIONS Maintaining the quality of healthcare is necessary for evaluating healthcare quality and as a foundation for improving healthcare services. This study shared our 8 years of 22 / 22
Page 22 of 31
experiences in maintaining the quality of healthcare using the DW-based CI monitoring system and showed some samples of various CIs and their monitoring results, such as the administration of the optimal dosage of preoperative antibiotics, the average length of hospitalization, the mortality rate, the infection rate, and the rehospitalization rate.
ip t
The system contributed to efficiently develop, maintain, and monitor approximately 300
cr
CIs with the least labor (e.g., only one CI manager in the authors’ organization of 1,340 beds). With the continuous monitoring and feedback activities, the system enabled to
us
maintain the optimum level of patient care. It is worthy of note that in order to effectively use the system, various communication channels to share the CI monitoring
an
results through the facility and feedback activities with/from users are required. The
M
DW-based CI monitoring system can be utilized as a powerful tool to provide ideas on improvement activities to users by enabling them to directly access relevant CIs.
d
The future work will continue in analyzing the return-on-investment (ROI) of the
Ac ce pt e
system in terms of economic evaluation, investigating more effective data visualization methods for users to easily recognize their objectives and achievement, and focusing on more developing patient-centered indicators such as patient safety indicators and patient satisfaction.
Acknowledgements
This work was partly supported by the IT R&D program of SNUBH and SKT.
Legends Figure 1. Clinical indicator development process 23 / 23
Page 23 of 31
Figure 2. Overall architecture of a clinical indicator system based on a data warehouse Figure 3. Status of CI development and monitoring for eight years Figure 4. A case of clinical indicator application: optimal prescription of preventive antibiotics
ip t
Figure 5. The average length of hospital stays over approximately 10 years
cr
Figure 6. Example of CI report on the number of PRN orders issued during the selected
Ac ce pt e
d
M
an
us
range, along with a departmental pie chart
24 / 24
Page 24 of 31
References [1] J.W. Curtright, S.C. Stolp-Smith, E.S. Edell, Strategic performance management: development of a performance measurement system at the Mayo Clinic, Journal of
ip t
healthcare management / American College of Healthcare Executives, 45 (2000) 58-68.
cr
[2] J. Mainz, Defining and classifying clinical indicators for quality improvement,
us
International journal for quality in health care : journal of the International Society for Quality in Health Care / ISQua, 15 (2003) 523-530.
an
[3] S.M. Shortell, J.L. O'Brien, J.M. Carman, R.W. Foster, E.F. Hughes, H. Boerstler, E.J. O'Connor, Assessing the impact of continuous quality improvement/total
M
quality management: concept versus implementation, Health services research, 30 (1995) 377-401.
d
[4] C.J. Huston, Outcomes measurement in healthcare. New imperatives for
Ac ce pt e
professional nursing practice, Nursing case management : managing the process of patient care, 4 (1999) 188-195.
[5] G.L. Colice, M.A. Morley, C. Asche, H.G. Birnbaum, Treatment costs of community-acquired pneumonia in an employed population, Chest, 125 (2004) 2140-2145.
[6] T.P. Meehan, M.J. Fine, H.M. Krumholz, J.D. Scinto, D.H. Galusha, J.T. Mockalis, G.F. Weber, M.K. Petrillo, P.M. Houck, J.M. Fine, Quality of care, process, and outcomes in elderly patients with pneumonia, JAMA : the journal of the American Medical Association, 278 (1997) 2080-2084. [7] D.W. Bratzler, W. Nsa, P.M. Houck, Performance measures for pneumonia: are they 25 / 25
Page 25 of 31
valuable, and are process measures adequate?, Current opinion in infectious diseases, 20 (2007) 182-189. [8] J.M. Fine, M.J. Fine, D. Galusha, M. Petrillo, T.P. Meehan, Patient and hospital characteristics associated with recommended processes of care for elderly patients
ip t
hospitalized with pneumonia: results from the medicare quality indicator system
cr
pneumonia module, Archives of internal medicine, 162 (2002) 827-833.
[9] J.C. Mitchiner, S.L. Hutto, The effect of selected hospital characteristics on the
us
timeliness of antibiotic administration for pneumonia, American journal of medical quality : the official journal of the American College of Medical Quality, 22 (2007)
an
259-264.
M
[10] T.P. Meehan, S.R. Weingarten, E.S. Holmboe, D. Mathur, Y. Wang, M.K. Petrillo, G.S. Tu, J.M. Fine, A statewide initiative to improve the care of hospitalized
d
pneumonia patients: The Connecticut Pneumonia Pathway Project, The American
Ac ce pt e
journal of medicine, 111 (2001) 203-210. [11] P.M. Houck, D.W. Bratzler, W. Nsa, A. Ma, J.G. Bartlett, Timing of antibiotic administration and outcomes for Medicare patients hospitalized with communityacquired pneumonia, Archives of internal medicine, 164 (2004) 637-644.
[12] M.M. Grube, C. Dohle, D. Djouchadar, P. Rech, K. Bienek, U. Dietz-Fricke, M. Jobges, M. Kohler, I. Missala, B. Schonherr, C. Werner, H. Zeytountchian, J. Wissel, P.U. Heuschmann, Evidence-based quality indicators for stroke rehabilitation, Stroke; a journal of cerebral circulation, 43 (2012) 142-146. [13] J. Mainz, Developing evidence-based clinical indicators: a state of the art methods primer, International journal for quality in health care : journal of the International Society for Quality in Health Care / ISQua, 15 Suppl 1 (2003) i5-11. 26 / 26
Page 26 of 31
[14] J. Mainz, B.R. Krog, B. Bjornshave, P. Bartels, Nationwide continuous quality improvement using clinical indicators: the Danish National Indicator Project, International journal for quality in health care : journal of the International Society for Quality in Health Care / ISQua, 16 Suppl 1 (2004) i45-50.
ip t
[15] S. Kristensen, J. Mainz, P. Bartels, Selection of indicators for continuous
cr
monitoring of patient safety: recommendations of the project 'safety improvement
for patients in Europe', International journal for quality in health care : journal of
us
the International Society for Quality in Health Care / ISQua, 21 (2009) 169-175. [16] S.M. Shortell, D.Z. Levin, J.L. O'Brien, E.F. Hughes, Assessing the evidence on
an
CQI: is the glass half empty or half full?, Hospital & health services administration,
M
40 (1995) 4-24.
[17] S.L. Gorbach, The role of cephalosporins in surgical prophylaxis, Journal of
d
Antimicrobial Chemotherapy, 23 (1989) 61-70.
Ac ce pt e
[18] S. Geroulanos, K. Marathias, J. Kriaras, B. Kadas, Cephalosporins in surgical prophylaxis, Journal of Chemotherapy, 13 (2001) 23-26.
[19] D.W. Bratzler, P.M. Houck, Antimicrobial prophylaxis for surgery: an advisory statement from the National Surgical Infection Prevention Project, The American Journal of Surgery, 189 (2005) 395-404.
[20] G.B. Appel, Aminoglycoside nephrotoxicity, The American journal of medicine, 88 (1990) S16-S20.
[21] E. Selimoglu, Aminoglycoside-induced ototoxicity, Current pharmaceutical design, 13 (2007) 119-126. [22] M.T. Keegan, D.R. Brown, Perioperative antibiotics and practice: Little things that make a big difference, Anesthesiology Clinics of North America, 22 (2004) 47327 / 27
Page 27 of 31
Ac ce pt e
d
M
an
us
cr
ip t
491.
28 / 28
Page 28 of 31
Summary points What was already known: Patients are now demanding a higher quality of medical services due to
ip t
increased interest in health. To improve the quality of treatments, it is necessary to maintain clinical
cr
indicators (CIs), and for continual and efficient maintenance, the CIs must be
us
collected systematically.
an
CI maintenance not only improves the quality of treatments but also contributes to the hospital administration and its operations. However, such CIs are usually
Ac ce pt e
What this study added
d
management and maintenance.
M
collected and produced manually, which presents difficulties for their continual
In a tertiary general hospital, maintenance with DW-based computerized quality indicators resulted in improvements in patient safety, management of treatment quality, and hospital administration. By automatically importing data by managing and monitoring computationally built CIs, less labor is needed to manage the CIs more efficiently than in the past, when the data for CI indicators were collected manually. As one example of the efficacy of computerized CIs, over the course of 8 years, through efficient CI management, the average length of inpatient hospitalization decreased by 2.1 days. 29 / 29
Page 29 of 31
Conflicts of Interest There are no conflicts of interests that could inappropriately influence the authors’ findings.
ip t
Author contributions S. Yoo and S. Kim designed the study and drafted the manuscript. K. Hyuck, C. Wook,
cr
S. Youn, and K. Park contributed to the discussions of data and reviewed the
us
manuscript. S. Moon analyzed and categorized the data. H. Hwang reviewed and
Ac ce pt e
d
M
an
revised the manuscript.
31 / 31
Page 30 of 31
Research Highlights DW-based clinical indicators (CIs) were developed to monitor the quality of care.
ip t
The number of CIs being developed, computerized, and monitored steadily increased over time.
cr
This study showed improvement in patient safety, quality of care and hospital
us
management.
Ac ce pt e
d
M
hospitalization decreased by 2.1 days.
an
Through efficient CI management, the average length of inpatient
32 / 32
Page 31 of 31