The effect of health information technology implementation in Veterans Health Administration hospitals on patient outcomes

Healthcare 2 (2014) 40–47

Contents lists available at ScienceDirect

Healthcare journal homepage: www.elsevier.com/locate/hjdsi

The effect of health information technology implementation in Veterans Health Administration hospitals on patient outcomes Joanne Spetz a,n, James F. Burgess b,c, Ciaran S. Phibbs d,e a

University of California, Philip R. Lee Institute for Health Policy Studies, 3333 California Street, Suite 265, San Francisco, CA 94118, United States VA Boston Healthcare System, 150 S Huntington Ave, Jamaica Plain, MA 02130, United States c Boston University, Boston, MA 02215, United States d VA Palo Alto Health Care System, 3801 Miranda Ave, Palo Alto, CA 94304, United States e Stanford University, 450 Serra Mall, Stanford, CA 94305, United States b

art ic l e i nf o

a b s t r a c t

Article history: Received 1 August 2013 Received in revised form 18 November 2013 Accepted 13 December 2013

Background: The impact of health information technology (HIT) in hospitals is dependent in large part on how it is used by nurses. This study examines the impact of HIT on the quality of care in hospitals in the Veterans Health Administration (VA), focusing on nurse-sensitive outcomes from 1995 to 2005. Methods: Data were obtained from VA databases and original data collection. Fixed-effects Poisson regression was used, with the dependent variables measured using the Agency for Healthcare Research and Quality Inpatient Quality Indicators and Patient Safety Indicators software. Dummy variables indicated when each facility began and completed implementation of each type of HIT. Other explanatory variables included hospital volume, patient characteristics, nurse characteristics, and a quadratic time trend. Results: The start of computerized patient record implementation was associated with significantly lower mortality for two diagnoses but significantly higher pressure ulcer rates, and full implementation was associated with significantly more hospital-acquired infections. The start of bar-code medication administration implementation was linked to significantly lower mortality for one diagnosis, but full implementation was not linked to any change in patient outcomes. Conclusions: The commencement of HIT implementation had mixed effects on patient outcomes, and the completion of implementation had little or no effect on outcomes. Implications: This longitudinal study provides little support for the perception of VA staff and leaders that HIT has improved mortality rates or nurse-sensitive patient outcomes. Future research should examine patient outcomes associated with specific care processes affected by HIT. & 2014 Elsevier Inc. All rights reserved.

Keywords: Health information technology Hospitals Patient outcomes

1. Introduction Health information technology (HIT) has been diffusing gradually in the United States for more than a decade.1,2 In the hospital setting, two of the most important technologies are computerized patient records and medication administration systems.3–5 Although the adoption of these systems has accelerated, research on the impact of hospital HIT on patient care, process change, staff time commitment, and staff morale has been inconclusive. While some studies of the effects of hospital HIT on the quality of care have been encouraging,6–13 others have found no or mixed benefits from these systems in acute-care settings.14–22 In fact, some studies indicate that HIT might have negative effects on the process and quality of care.23–31 Differences in the specific details of HIT implementation and process are undoubtedly partly related to differences in conclusions; this

n

Corresponding author. Tel.: þ 1 415 502 4443. E-mail address: [email protected] (J. Spetz).

2213-0764/$ - see front matter & 2014 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.hjdsi.2013.12.009

study focuses on a single HIT implementation and a set of processes important to the largest hospital staff component. The impact of HIT within the hospital setting is likely to have significant dependence on the interactions of nurses with the systems. Nurses are the largest group of staff in hospitals, providing the majority of patient care at the bedside, and they are responsible for inpatient charts and the administration of medications.32,33 They intensively use electronic patient records and charting systems, clinical reminders, and electronic medication administration systems.34 In theory, HIT can enhance nursing care by improving information access, providing automated surveillance for error detection and prevention, facilitating communication among care providers, and standardizing practice patterns.35 HIT implementation also could impact nursing workload in both positive and negative ways, which may influence the ultimate effect of HIT on patient care.33 There is little evidence, however, regarding whether HIT improves the structure, process, and outcomes of nursing care. One prior study found higher rates of nurse-sensitive complications, although there were also lower mortality rates for selected diagnoses.20 Another

J. Spetz et al. / Healthcare 2 (2014) 40–47

study measured improvements in the quality of nursing documentation only after retraining of staff in the use of the system.36 Several studies have found that HIT can affect the process of nursing care and communication among providers negatively.23,33 This study examines the impact of HIT on the quality of care in hospitals, recognizing that nurse staffing is a mediating factor in the determination of patient outcomes. We focus on the implementation of two important types of HIT developed and implemented systematically in the U.S. Veterans Health Administration (VA): the Computerized Patient Record System (CPRS), and Bar Code Medication Administration (BCMA). CPRS is a fully-integrated electronic health record, with computerized physician order entry. BCMA is a bedside medication administration safety system, for which the pharmacy uses computerized physician orders to bar code all prescriptions delivered to patient care units. Nurses then scan the medication bar code and the patient0 s wristband. A mismatch between the patient and the ordered medication (wrong medication, wrong dose, wrong time) results in a warning sound, and the nurse then assesses the source of the error and makes a correction. These systems were developed and then implemented in VA facilities nationwide, with CPRS gradually phased in over a decade starting in the early 1990s, and BCMA implemented over a 2-year period ending in 2001.37,38 These are among the largest investments in information technology in the hospital industry over the past two decades,38–40 but there has been little research objectively assessing the effect of CPRS and BCMA on hospital staff or patients. Early studies of the effects of CPRS indicate that the system improved the specificity of medical orders,41 improved health screening rates with automated alerts,42 decreased the rate of indwelling urinary catheterization,43 reduced redundant laboratory orders,44 and improved overall hospital efficiency.45 A number of studies have demonstrated that BCMA reduced the rate of medication errors,39,46–48 but BCMA has also been associated with some problems, such as lack of reliability,39 difficulty coordinating activities of staff,23 and changing priorities of nursing staff in favor of monitored activities.23 We add to this literature by examining mortality associated with five common diagnoses and four nurse-sensitive adverse inpatient outcomes, while controlling for characteristics of the facility and its nurse staff that may affect the ultimate impact of HIT on nurse-sensitive quality of hospital care.

2. Data and methods 2.1. Data sources We extracted data from multiple VA databases and engaged in original data collection. Our data span the period from 1995 through 2005; we selected this time period to have roughly 5 years of data prior to and following the implementation of BCMA; BCMA was phased in over an intense 2-year period with some beta testers and laggers. All data were aggregated so each observation represents a hospital for one quarter. 2.1.1. IT implementation data Data on the implementation of CPRS and BCMA were obtained through a web-based survey of VA facilities. The VA Chief Nurse0 s office communicated with all VA sites to solicit participation in the survey, resulting in 120 respondents from 147 facilities. These data identify when CPRS and BCMA were launched at each facility, as well as when implementation was started and completed for each major component of CPRS and BCMA. 2.1.2. VA databases To measure the effects of CPRS and BCMA on patient safety in the VA, we used the Agency for Healthcare Research and Quality

41

(AHRQ) Inpatient Quality Indicators (IQIs) and Patient Safety Indicators (PSIs) software, version 3.1b/apr, 4/26/07. The IQIs measure patient volumes and inpatient mortality for specific medical conditions and surgical procedures. We focus on the mortality rates for coronary artery bypass graft (CABG), heart attack (AMI), congestive heart failure (CHF), stroke, and pneumonia. The PSIs measure rates of potentially avoidable complications and iatrogenic events, such nosocomial infections, death in lowmortality DRGs, and pressure ulcers. We focused on four PSIs that have been demonstrated to be sensitive to nursing care: pressure ulcers, mortality following a postsurgical complication (failure to rescue), selected infections due to medical care, and postoperative sepsis.49 Both the PSI and IQI software produce several key measures: the observed number of cases of the adverse event (or death), the number of patients included in the calculation of the adverse event, the expected number of cases based on a riskadjustment algorithm, the observed rate of the adverse event (number of cases divided by patients included), and the riskadjusted rate of the adverse event. The VA collects patient discharge data similar to that in the AHRQ Statewide Inpatient Database in their Patient Treatment File (PTF). The PTF is different from standard discharge abstracts since it is an annual operations database, not a billing system; in addition to the normal discharge record, it includes a census file with one record for every patient in the hospital at the end of the fiscal year, and a “bed section” (unit type) file which has a separate record for each bed section stay. Combined, these files provide a more complete picture of the care provided by each VA facility, and they were used to both compute the PSIs and IQIs, and to adjust for patient case-mix. These data also allowed us to accurately determine the number of inpatient days provided by each VA facility in each quarter. Previous research has analyzed the usefulness of the PSIs for understanding patient outcomes in VA hospitals;50–52 we compared our PSI computations with those of Rosen for the 2000–2001 fiscal year50 and obtained similar estimates of the numbers of affected patients in our calendar year 2001. The PTF also was used to measure characteristics of VA patients. We control for differences in the severity of illness by measuring patient case-mix, measured as the average of patient DRG weights, and the average count of Elixhauser comorbidities.53 The PTF also was used to calculate the number of patient days in inpatient units, and days squared, which accounts for possible economies of scale and/or diminishing returns in the prevention of adverse events.54–56 The VA Payroll data system (PAID) was used to measure some aspects of nurse staffing and the characteristics of nursing staff. The data include hours worked by registered nurses (RNs), licensed practical nurses (LPNs), and aides, as well as nurses0 age and education. From these data we constructed variables that measure the intensity of nursing care provided to patients: the number of nursing hours per patient day and percent of nursing hours worked by RNs. Many studies have linked nurse staffing levels to patient outcomes.56–60 We also measure the human capital of nurses, which has been shown to affect nursing skill and patient outcomes, as the percent of RNs over 50 years old and percent with a bachelor0 s or master0 s degree.61–65 The PAID data do not include the actual work experience of RNs, and thus age of the nurse is used as a proxy. We measure the share of nursing personnel represented by unions, which has been associated with patient outcomes.66 Finally, we measure the share of nurses who work part time, which controls for the exposure of nurses to new technologies during the implementation period. The panel of data is unbalanced; for some hospitals, the PAID data did not provide information about nurse characteristics and thus the observations were not included. We elected to estimate

42

J. Spetz et al. / Healthcare 2 (2014) 40–47

Table 1 Means of explanatory variables, 1995 and 2005. 1995

Variable

Total patient days DRG weight Elixhauser count Nurse hours per patient day Percent RN hours Percent unionized % RNs working part-time % RNs with bachelor’s or higher % RNs age 50 years or older Number of observations (hospital-quarters)

2005

Mean

Standard deviation

Mean Standard deviation

12,138 1.120 1.381 20.54 60.11 23.46 9.09 45.49 56.89

6615 0.172 0.309 10.18 13.31 18.71 7.80 12.10 10.47

6619 1.115 2.022 17.29 61.76 40.81 10.34 43.42 41.12

324

4212 0.144 0.339 9.02 15.20 16.22 9.11 12.74 10.28

366

Table 2 Means of quarterly counts of Inpatient Quality Indicators and Patient Safety Indicators, 1995 and 2005. Variable

1995 Mean Standard deviation

CABG mortality 1.75 1.34 AMI mortality 1.82 1.71 CHF mortality 2.18 2.15 Stroke mortality 1.86 1.96 Pneumonia 4.15 3.63 mortality Pressure ulcer 8.04 6.60 Failure to rescue 11.96 10.34 Selected 1.31 1.89 infections Post-op sepsis 0.84 1.35

2005 # Obs.

Mean Standard deviation

# Obs.

115 287 296 294 296

0.85 1.15 1.54 0.93

133 337 354 344 354

296 296 296 283

1.06 1.83 1.66 1.48

2.49

3.16

8.54 7.71

11.00 6.80

1.36

1.63

0.83

1.16

355 355 355 323

all models with the unbalanced panel to maximize the number of observations in the analysis. Means of all explanatory variables are presented in Table 1, for 1995 and 2005. Each year includes four quarters of data. In 1995, there are 296 observations (hospitalquarters), and in 2005 there are 355. The average number of inpatient days in 1995 is notably larger than in 2005, indicating that hospitals for which all 1995 variables were not available were smaller than those for which all 1995 variables were available. The average count of comorbidities calculated using Elixhauser0 s method was higher in 2005 than in 1995, although there was little difference in the average DRG weight. Nurse hours per patient day were lower in 2005 than in 1995, although there was a higher share of hours worked by RNs in 2005 than in 1995. The percent of nurses represented by a union was higher in 2005 than in 1995, and the share of nurses age 50 years or older was lower in 2005. Table 2 presents the average counts of adverse events per quarter for 1995 and 2005, for each IQI and PSI. The average counts for all of the mortality indicators were higher in 1995 than in 2005. The average counts for the patient safety indicators were similar in 1995 and in 2005, with the exception of the count of failure-to-rescue cases, which was lower in 2005 than in 1995. 2.2. Methods To assess the effects of the implementation of CPRS and BCMA on patient outcomes, we estimated fixed-effects Poisson equations. The dependent variables are the counts of observed adverse

outcomes, as measured using the IQI and PSI software. To control for differential risk of adverse events, we use the count of expected adverse outcomes for each IQI and PSI as a measure of exposure; that is, the natural logarithm of the expected number of adverse events is a regression with its coefficient fixed to 1.0. Dummy variables were used to indicate when each facility implemented CPRS and BCMA. Inclusion of fixed effects improves our estimates of the direct relationship between HIT implementation and patient outcomes by controlling for extraneous time-constant facilityspecific factors. In addition, prior research has found that hospitals0 baseline risk-adjusted PSI rates are important predictors of their later risk-adjusted rates for 8 PSIs.51 The inclusion of fixed effects allows us to estimate the relationships between changes in implementation and changes in outcomes, thus accounting for differences in baseline rates. Three models were estimated for each outcome measure, to examine the possibility the effects of these technologies may have changed during the implementation process. First, we estimated a model with a dummy variable to indicate the quarter in which implementation of each technology began (CPRS, or BCMA in acute care) (Model 1). Other independent variables included total inpatient days, inpatient days squared, case-mix, average number of Elixhauser comorbidities, nurse hours per patient day, percent of nursing hours worked by RNs, percent of RNs represented by a union, percent of RNs working part-time, percent of RNs with a bachelor0 s or master0 s degree, percent of RNs 50 years or older, and a quadratic time trend. Second, we estimated the same model with a dummy variable to indicate the quarter in which implementation of each technology was complete, rather than when it began (Model 2). We then estimated a third model in which we measure the relationships between the beginning of CPRS and BCMA implementation and outcomes separately for “early adopters” and “late adopters” (Model 3). For CPRS, early adopters were hospitals that began CPRS implementation in June 1998 or earlier; for BCMA, early adopters began implementation in acute-care units in 2000 or earlier. For each of these two models, we estimated two equations for each outcome: one for CPRS implementation, and one for BCMA implementation. We tested the sensitivity of our results by estimating several other specifications: (1) models that included indicators for both CPRS and BCMA; (2) logarithmic inpatient days instead of quadratic; (3) time dummy variables rather than a trend; and (4) including lagged implementation dates. The results (not presented) were not substantively different from our main models.

3. Results 3.1. Timeline of CPRS and BCMA implementation CPRS was developed gradually by VA staff, and many VA facilities served as beta-testers of components of CPRS. Thus, there was a wide range of implementation dates for CPRS (Fig. 1). Some facilities were starting their implementation of early versions of CPRS in 1995 and 1996. Most facilities began implementation of CPRS in 1998 or 1999; 45 launched CPRS in 1998, and 40 launched in 1999. Seven facilities reported that they did not begin implementation of CPRS until 2001 or 2002. Data on implementation completion indicates that 51 facilities finished implementation in 1999 or 2000, 14 completed in 2001, 15 ended in 2002, and 10 did not finish until 2003 or 2004. Thirty-four facilities reported that they completed implementation of CPRS within one quarter of launching it. Another 26 facilities completed implementation within one year, and 21 continued implementation over 2 years. A final 34 facilities took more than 2 years to fully implement CPRS. The underlying software base of all implementations is

J. Spetz et al. / Healthcare 2 (2014) 40–47

43

Fig. 1. Distribution of quarters in which CPRS implementation began.

Fig. 2. Distribution of quarters in which BCMA implementation began in acute care units.

identical; however, part of the implementation involved developing local templates for each service, including possibly nursing templates that could differ across facilities. The time period over which BCMA was implemented was more compressed than for CPRS (Fig. 2). VA Headquarters established a requirement that BCMA be implemented by June 30, 2000. It is thus not surprising that most facilities reported that they began implementation of BCMA between the third quarter of 1999 and the second quarter of 2000, with 76 facilities launching BCMA over this time period. Five facilities began BCMA implementation earlier because they were beta-testing sites. Seven facilities did not begin implementation until 2001, 3 delayed until 2002, and one did not launch until 2004. 102 facilities reported no more than

2 quarters between initial implementation and final implementation of BCMA in acute care units.

3.2. Estimates of HIT relationship with patient outcomes Tables 3 and 4 present the average marginal effects of the CPRS and BCMA variables, calculated from the fixed-effects Poisson regressions for patient outcomes. Each row presents marginal effects for one of the IQIs or PSIs, and the columns provide the effects estimated by each of the three models. The effects that are significantly different from zero are identified with asterisks at the levels of significance stated in the footnotes to each table.

44

J. Spetz et al. / Healthcare 2 (2014) 40–47

Table 3 Average marginal effects of CPRS implementation in acute care units, computed from fixed-effects Poisson egression equations. Outcome

CABG mortality AMI mortality CHF mortality Stroke mortality Pneumonia mortality Pressure ulcer Failure to rescue Selected infections Post-op sepsis n

Model 1

Model 2

Model 3

CPRS start

CPRS complete

CPRS start – early adopters

CPRS start – late adopters

 0.077

 0.100

 0.017  0.039 0.024

 0.012  0.117  0.208nn

0.038  0.157nn  0.014

 0.081nn  0.050

 0.113nn

 0.071n

 0.092

0.078

0.027  0.148nn  0.061

0.127nn 0.002

0.034  0.024

0.120  0.001

0.129nn 0.004

 0.061

0.224nn

 0.168

 0.029

 0.118

0.046

 0.219

 0.082

Significant at Pr 0.10. Significant at Pr 0.05.

nn

Table 4 Average marginal effects of BCMA implementation in acute care units, computed from fixed-effects Poisson egression equations. Outcome

CABG mortality AMI mortality CHF mortality Stroke mortality Pneumonia mortality Pressure ulcer Failure to rescue Selected infections Post-op sepsis n

Model 1

Model 2

Model 3

BCMA start

BCMA complete

BCMA start – early BCMA start – late adopter adopter

 0.163nn

0.010

 0.177n

 0.158n

0.088n  0.064  0.021

0.072 0.020  0.015

0.035 0.066  0.028

0.107nn  0.113nn  0.019

 0.007

 0.009

0.033

The last two columns of Table 3 explore whether there are different relationships between the start of CPRS implementation and patient outcomes based on whether the hospital was an early adopter or late adopter. The results do not suggest any pattern of early adopters having consistently different relationships than late adopters. For CHF mortality, the reduction in mortality is significant only for late adopters, whereas the decline in pneumonia mortality is significant for early adopters. The start of CPRS implementation is significantly associated with stroke mortality for early adopters, but not for late adopters and not in general (Model 1). Finally, the increase in pressure ulcer mortality is significantly associated with late adopters but not significantly associated with early adopters. Table 4 presents the results from the equations that focus on BCMA. The first column presents the average marginal effects for the start of BCMA implementation. Its initial implementation is associated with lower mortality for CABG, with 0.163 fewer mortality cases per quarter. Three of the other mortality relationships also indicate lower mortality, but they are not statistically significant. The average marginal effect of the start of BCMA implementation on AMI mortality is positive, although this is significant only at p ¼0.10. The start of BCMA is associated with higher counts of all PSIs, but the relationships are not statistically significant at p ¼0.05. The second column of Table 4 presents the average marginal effects of the completion of BCMA implementation and the patient outcomes; all relationships are small and statistically insignificant. The last two columns consider whether the relationships between the start of BCMA implementation and patient outcomes are different for early adopters and late adopters. Late adoption is associated with a significantly higher count of AMI deaths, and a significantly lower count of CHF deaths. Early adoption is significantly linked to a higher rate of failure to rescue.

4. Discussion  0.004

0.029 0.050n

0.060 0.017

0.007 0.089nn

0.037 0.035

0.065

0.064

0.098

0.057

0.033

0.073

 0.087

0.068

Significant at Pr 0.10. Significant at Pr 0.05.

nn

Table 3 focuses on the effects of CPRS. The first column presents the average marginal effects for the model that includes an indicator for when CPRS implementation began. The second column provides average marginal effects for when CPRS implementation was complete. The start of CPRS implementation is associated with significantly lower rates of mortality for CHF and pneumonia. The other three mortality relationships are not statistically significant. The only PSI for which the start of CPRS implementation has a significant relationship is that of pressure ulcers; CPRS implementation is associated with a higher count of pressure ulcers, with an average of 0.127 more pressure ulcers per quarter. The relationships between the start of CPRS implementation and the other PSIs are statistically insignificant. The results of the model that measures the relationship between complete implementation of CPRS and patient outcomes indicate that there are no statistically significant relationships, with the exception of selected infections due to medical care. The count of such infections is an average of 0.224 higher per quarter when CPRS implementation is complete.

Our analysis found that the start of CPRS implementation was associated with significant improvements in mortality counts for two of the five diagnoses we examined, and was associated with a higher count of pressure ulcers. The completion of CPRS implementation was associated with a higher count of hospital-acquired infections. There was no clear pattern associated with whether a VA was an early adopter of CPRS or a late adopter. The start of BCMA implementation was linked to lower mortality for one diagnosis and a higher rate of failure to rescue after a complication. The completion of BCMA implementation had no statistically significant relationship with any of the outcomes. There was no consistent pattern associated with whether the VA was an early or late adopter of BCMA. The finding that the start of CPRS implementation is associated with significantly lower mortality for congestive heart failure and pneumonia is consistent with the possibility that the ultimate outcome of a hospitalization is affected by the ability of the care team to understand the patient0 s long-term care history in both outpatient and inpatient settings. The information available in CPRS may facilitate care planning in a way that is particularly useful to diagnoses such as CHF and pneumonia. For example, clinicians note the importance of tracking ejection fraction over time for CHF, and CPRS allowed for comprehensive collection of and access to these data. Other research has found some positive relationship between HIT and quality of care for CHF.18 However, the lack of a significant relationship between complete CPRS implementation and mortality for either of these diagnoses raises doubts about the robustness of the association between the start of implementation and these outcomes. The connection between

J. Spetz et al. / Healthcare 2 (2014) 40–47

initial CPRS implementation and higher rates of pressure ulcer, and between completion of CPRS implementation and higher rates of hospital-acquired infections associated with medical care, could reflect better recording of patients0 conditions, rather than a detrimental effect of HIT. The results regarding BCMA implementation were not as positive as those for CPRS. The start of BCMA implementation was associated with a significantly lower rate of CABG mortality. But, it also was linked to a statistically significantly higher rate of failure to rescue after a complication, and the average marginal relationships with other complications were also positive but not statistically significant. Of particular concern in the implementation and use of BCMA has been the possibility of circumvention of the system. One study of a BCMA system reported that nurses overrode system alerts for 4.2 percent of patients.67 Such workarounds may have both reduced the potential for BCMA to have positive effects and also diverted nursing time from more productive patient care activities. The full implementation of BCMA was not significantly associated with any of the patient outcomes examined in this analysis. These mixed findings are consistent with a qualitative study conducted by this team after CPRS and BCMA implementation, which reported that some VA staff and managers expressed concerns that HIT may have reduced nurse time for direct patient care, and thus worsened the quality of care.68 The use of HIT has not been shown to reduce the time required for documenting and retrieving patient information consistently,69 and some studies have found that HIT increases the time demand on nurses.33 This structural change in nursing work may have had a detrimental effect on some aspects of the process of care, thus negatively affecting selected patient outcomes. It is noteworthy that for most of those outcomes with significant effects, the impact was associated with initial HIT implementation, rather than full implementation. This is surprising, because a large body of research documents that the process of implementing HIT systems is important to determining overall success.70–72 Even in the most successful installations of HIT systems, the benefits of HIT may not be immediately realized because healthcare providers must adapt to new processes and methods for recording and communicating information before improvements are achieved.73 Many VA facilities dedicated significant resources to their HIT implementations and they were developed specifically by VA for the VA context, which may have contributed to their relatively rapid impacts on patients.68 4.1. Limitations There are several limitations to this study. First, the VA0 s experience with health information technology may be different than that of other hospitals. Clinicians and managers are employees of the VA and have clear incentives to adhere to policies that were in fact mandated by the national central administration. Because the VA is an integrated system, computerized patient records can transmit information between physician practices, hospitals, long-term care, and other care setting, furthering the benefits of these systems. The difference between the VA and most other hospitals, in both incentives and organization, may limit the generalizability of our findings for other settings. Second, the timing of implementation of CPRS and BCMA at each facility was not likely to have been randomly selected, and thus the results presented here are potentially confounded by selection bias. Some of this selection bias, though, is related to the randomly apportioned internal political power of the managers of each facility. At the same time CPRS and BCMA were being implemented across the VA system, the VA was engaged in other reorganization activities. Some VA facilities were being merged

45

with each other to improve efficiency and centralize services. And, in many VA sites, staff were reorganized so that they reported to managers of each type of service provided (“service line”) rather than leadership within their discipline.74 We could not identify an accurate source of information about which VA facilities were engaged in reorganization, or the timing of such reorganization, and thus this effect is unmeasured. Third, the outcomes we measure may not be sensitive to these types of technologies.18 For example, since BCMA was developed primarily to prevent medication errors, it is plausible that BCMA should not have any significant effect on other outcomes. Quality measures that focus on medication-associated quality of care are needed, but such measures were not readily available for this study.

5. Conclusions The net financial benefit of the VA0 s investment in HIT has been estimated at $3.09 billion, with most of this gain derived from improvement in the quality of outpatient care.75 The analyses presented in this paper suggest that there was little effect of CPRS or BCMA on the quality of inpatient care in the VA. The initial implementation of CPRS is associated with lower mortality for congestive heart failure and pneumonia, but these results are not robust to the full implementation of CPRS. The initial implementation of BCMA is associated with lower mortality for CABG, but there are no significant relationships with mortality for other diagnoses, and the relationships between all patient outcomes are not significantly associated with full BCMA implementation. Other analyses of the impact of hospital HIT systems have considered different patient outcomes, such as length of stay and 30-day mortality and found improvements.19 Recent research using Medicare data found an association between electronic medical records and reduced rates of infections attributable to medical care, but no association with deep-vein thrombosis or postoperative hemorrhage.12 Care processes also have been examined, finding improved process of care for some indicators but no relationship with others.6,11 This study did not assess changes in care processes, nor could it examine outcomes that are directly associated with the purposes of CPRS and BCMA, such as medication errors communication between providers, and coordination between outpatient and inpatient care. Qualitative research has indicated that VA hospital staff and managers view CPRS and BCMA as important, useful IT systems that help ensure that the VA provides excellent care to veterans.68 This perception may be supported in future research that focuses on care processes and different patient outcomes. Studying the direct impact of HIT on nursing workload also is important. Future research could expand limited information from human factors research with more focus on changes related to HIT.76 Our mixed findings with respect to nurse-sensitive patient outcomes and mortality suggest that the impacts on hospital care of the VA0 s HIT investment were complex, and that ongoing monitoring and evaluation are needed to maximize the potential benefit of HIT.77

Appendix A. Supporting information Supplementary data associated with this article can be found in the online version at: http://dx.doi.org/10.1016/j.hjdsi.2013.12.009. References 1. Poon EG, Jha AK, Christino M, Honour M, Fernandopulle R, Middleton B. Assessing the level of healthcare information technology adoption in the United States: a snapshot. BMC Med Inform Decis Mak. 2006;6(1):1–9.

46

J. Spetz et al. / Healthcare 2 (2014) 40–47

2. Jha AK, Burke MF, DesRoches C, et al. Progress toward meaningful use: hospitals0 adoption of electronic health records. Am J Managed Care. 2011;17 (12 Spec no.):SP117–124. 3. Pedersen CA, Schneider PJ, Scheckelhoff DJ. ASHP national survey of pharmacy practice in hospital settings: dispensing and administration—2002. Am J Health Syst Pharm. 2003;60(1):52–68. 4. Coye MJ, Bernstein WS. Improving America’s health care system by investing in information technology. Health Aff (Millwood). 2003;22(4):56–58. 5. Kadas RM. The computer-based patient record is on its way. HMOs, the economy and HIPAA will drive adoption. Healthc Inform. 2002;19(2):57–58. 6. Kazley AS, Ozcan Y. Do hospitals with electronic medical records (EMRs) provide higher quality care? Med Care Res Rev. 2008;65(4):496–513. 7. May S. Bar-code technology. A point-of-care medication management system prevents errors and reduces paperwork. Healthc Inform. 2002;19(5):69–70. 8. Cordero L, Kuehn L, Kumar RR, Mekhjian JS. Impact of computerized physician order entry on clinical practice in a newborn intensive care unit. J Perinatol. 2004;24:88–93. 9. Potts AL, Barr FE, Gregory DF, Lorianne W, Patel NR. Computerized physician order entry and medication errors in a pediatric critical care unit. Pediatrics. 2004;113(1):59–63. 10. Buntin MB, Burke MF, Hoaglin MC, Blumenthal D. The benefits of health information technology: a review of the literature shows predominantly positive results. Health Aff. 2011;30(3):464–471. 11. Restuccia JD, Cohen AB, Horwitt JN, Shwartz M. Hospital implementation of health information technology and quality of care: are they related? BMC Med Inform Decis Mak. 2012;12:109. 12. Parente ST, McCullough JS. Health information technology and patient safety: evidence from panel data. Health Aff (Millwood). 2009;28(2):357–360. 13. Miller AR, Tucker CE. Can health care information technology save babies? J Polit Econ. 2011;119(2):289–324. 14. Asch SM, McGlynn EA, Hogan MM, et al. Comparison of quality of care for patients in the Veterans Health Administration and patients in a national sample. Ann Intern Med. 2004;141(12):938–945. 15. Walsh KE, Landrigan CP, Adams WG, et al. Effect of computer order entry on prevention of serious medication errors in hospitalized children. Pediatrics. 2008;121(3):e421–e427. 16. Metzger J, Welebob E, Bates DW, Lipsitz S, Classen DC. Mixed results in the safety performance of computerized physician order entry. Health Aff. 2010;29 (4):655–663. 17. Menachemi N, Chukmaitov A, Saunders C, Brooks RG. Hospital quality of care: does information technology matter? The relationship between information technology adoption and quality of care Health Care Manage Rev.. 2008;33 (1):51–59. 18. Jones SS, Adams JL, Schneider EC, Ringel JS, McGlynn EA. Electronic health record adoption and quality improvement in US hospitals. Am J Managed Care. 2010;16(12):SP64–SP71 (special issue). 19. Lee J, Kuo Y-F, Goodwin JS. The effect of electronic medical record adoption on outcomes in US hospitals. BMC Health Serv Res. 2013;13:39. 20. Furukawa MF, Raghu TS, Shao BBM. Electronic medical records, nurse staffing, and nurse-sensitive patient outcomes: evidence from California hospitals, 1998–2007. Health Serv Res. 2010;45(4):941–962. 21. Handel DA, Wears RL, Nathanson LA, Pines JM. Using information technology to improve the quality and safety of emergency care. Acad Emerg Med: Off J Soc Acad Emerg Med. 2011;18(6):e45–51. 22. Shekelle PG, Morton SC, Keeler EB. Costs and benefits of health information technology. Rockville, MD: Agency for Healthcare Research and Quality; 2006. 23. Patterson ES, Cook RI, Render ML. Improving patient safety by identifying side effects from introducing bar coding in medication administration. J Am Med Inform Assoc. 2002;9(5):540–553. 24. Nebeker J, Hoffman J, Weir C, Bennett C, Hurdle J. High rates of adverse drug events in a highly computerized hospital. Arch Intern Med. 2005;165:1111–1116. 25. Han YY, Carcillo JA, Venkataraman ST, et al. Unexpected increased morality after implementation of a commercially-sold computerized physician order entry system. Pediatrics. 2005;116:1506–1512. 26. Chaudhry B, Wang J, Wu S, et al. Systematic review: impact of health information technology on quality, efficiency, and costs of medical care. Ann Intern Med. 2006;144:742–752. 27. Spetz J, Keane D. IT. Implementation in a rural hospital: a cautionary tale. J Health Care Manage. 2009;54(5):337–348. 28. Koppel R, Metlay JP, Cohen A, et al. Role of computerized physician order entry systems in facilitating medication errors. JAMA. 2005;293(10):1197–1203. 29. Campbell EM, Guappone KP, Sittig DF, Dykstra RH, Ash JS. Computerized provider order entry adoption: implications for clinical workflow. J Gen Intern Med. 2009;24(1):21–26. 30. Campbell EM, Sittig DF, Guappone KP, Dykstra RH, Ash JS. Overdependence on technology: an unintended adverse consequence of computerized provider order entry. AMIA Annu Symp Proc. 2007:94–98. 31. Ash JS, Sittig DF, Poon EG, Guappone K, Campbell E, Dykstra RH. The extent and importance of unintended consequences related to computerized provider order entry. J Am Med Inform Assoc. 2007;14(4):415–423. 32. Ball MJ, Weaver C, Abbott PA. Enabling technologies promise to revitalize the role of nursing in an era of patient safety. Int J Med Inf. 2003;69(1):29–38. 33. Waneka R, Spetz J. How do hospital information technology systems impact nurses and nursing care? J Nurs Adm. 2010;40(12):509–514. 34. Hamilton AV, Coyle GA, Heinen MG. Applied technology rounds out e-documentation. Nurs Manage. 2004;35(9):44–47.

35. Bakken S, Cimino JJ, Hripcsak G. Promoting patient safety and enabling evidence-based practice through informatics. Med Care. 2004;42(2):II-49–II-56. 36. Larrabee JH, Boldreghini S, Elder-Sorrells K, et al. Evaluation of documentation before and after implementation of a nursing information system in an acute care hospital. Comput Nurs. 2001;19(2):56–65 ([quiz 66-58]). 37. Brown SH, Lincoln MJ, Groen PJ, Kolodner RM. VistA – U.S. Department of Veterans Affairs national-scale HIS. Int J Med Inform. 2003;69:135–156. 38. Gardner J. VA leads the way: computerized patient data an example for private sector. Mod Healthcare. 1997;27(48):24. 39. Coyle GA, Heinen M. Scan your way to a comprehensive electronic medical record. Augment medication administration accuracy and increase documentation efficiency with bar coding technology. (56). Nurs Manage. 2002;33 (12):58–59. 40. Congressional Budget Office. The health care system for veterans: an interim report. Washington, DC: Congressional Budget Office; December 2007. 41. Achtmeyer CE, Payne TH, Anawalt BD. Computer order entry system decreased use of sliding scale insulin regimens. Methods Inf Med. 2002;41(4):277–281. 42. Calabrisi RR, Czarnecki T, Blank C. The impact of clinical reminders and alerts on health screenings. The VA Pittsburgh Healthcare System achieves notable results by enhancing an automated clinical reminder system within its CPR— and has the data to prove it. Health Manage Technol. 2002;23(12):32–34. 43. Cornia PB, Amory JK, Fraser S, Saint S, Lipsky BA. Computer-based order entry decreases duration of indwelling urinary catheterization in hospitalized patients. Am J Med. 2003;114(5):404–407. 44. Stair TO. Reduction of redundant laboratory orders by access to computerized patient records. J Emerg Med. 1998;16(6):895–897. 45. Fletcher RD, Dayhoff RE, Wu CM, Graves A, Jones RE. Computerized medical records in the Department of Veterans Affairs. Cancer. 2001;91(8 Suppl.):1603–1606. 46. Young D. Veterans Affairs bar-code-scanning system reduces medication errors. Am J Health Syst Pharm. 2002;59(7):591–592. 47. Johnson CL, Carlson RA, Tucker CL, Willette C. Using BCMA software to improve patient safety in Veterans Administration Medical Centers. J Healthc Inf Manage. 2002;16(1):46–51. 48. Coyle GA, Heinen M. Evolution of BCMA within the Department of Veterans Affairs. Nurs Adm Q. 2005;29(1):32–38. 49. National Quality Forum. National voluntary consensus standards for nursingsensitive care: an initial performance measure set. Washington, DC: National Quality Forum; 2004. 50. Rosen AK, Rivard P, Zhao S, et al. Evaluating the patient safety indicators: how well do they perform on Veterans Health Administration data? Med Care. 2005;43(9):873–884. 51. Rosen AK, Zhao S, Rivard P, et al. Tracking rates of Patient Safety Indicators over time: lessons from the Veterans Administration. Med Care. 2006;44:850–861. 52. Rivard PE, Elixhauser A, Christiansen CL, Zhao S, Rosen AK. Testing the association between Patient Safety Indicators and hospital structural characteristics in VA and nonfederal hospitals. Med Care Res Rev. 2010;67(3):321–341. 53. Elixhauser A, Steiner C, Harris DR, Coffey RM. Comorbidity measures for use with administrative data. Med Care. 1998;36(1):8–27. 54. Kovner C, Gergen PJ. Nurse staffing levels and adverse events following surgery in U.S. hospitals. Image—J Nurs Scholarsh. 1998;30(4):315–321. 55. Mark BA, Harless DW, McCue M, Xu Y. A longitudinal examination of hospital registered nurse staffing and quality of care. Health Serv Res. 2004;39 (2):279–300. 56. Needleman J, Buerhaus P, Mattke S, Stewart M, Zelevinsky K. Nurse-staffing levels and the quality of care in hospitals. New Engl J Med. 2002;346 (22):1715–1722. 57. Aiken LH, Clarke SP, Sloane DM. Hospital staffing, organizational support, and quality of care: cross-national findings. Int J Qual Health Care. 2002;14:5–13. 58. Kane RL, Shamliyan TA, Mueller C, Duval S, Wilt TJ. The association of registered nurse staffing levels and patient outcomes: systematic review and metaanalysis. Med Care. 2007;45(12):1195–1204. 59. Lang TA, Hodge M, Olson V, Romano PS, Kravitz RL. Nurse-patient ratios: a systematic review on the effects of nurse staffing on patient, nurse employee, and hospital outcomes. J Nurs Adm. 2004;34(7–8):326–337. 60. Unruh LY, Zhang NJ. Nurse staffing and patient safety in hospitals: new variable and longitudinal approaches. Nurs Res. 2012;61(1):3–12. 61. Aiken LH, Cimiotti JP, Sloane DM, Smith HL, Flynn L, Neff DF. Effects of nurse staffing and nurse education on patient deaths in hospitals with different nurse work environments. Med Care. 2011;49(12):1047–1053. 62. Aiken LH, Clarke SP, Cheung RB, Sloane DM, Silber JH. Educational levels of hospital nurses and surgical patient mortality. JAMA. 2003;290(12):1617–1623. 63. Blegen MA, Goode CJ, Park SH, Vaughn T, Spetz J. Baccalaureate education in nursing and patient outcomes. J Nurs Adm. 2013;43(2):89–94. 64. Covell CL, Sidani S. Nursing intellectual capital theory: testing selected propositions. J Adv Nurs. 2013;69(11):2432–2445. 65. Kanai-Pak M, Aiken LH, Sloane DM, Poghosyan L. Poor work environments and nurse inexperience are associated with burnout, job dissatisfaction and quality deficits in Japanese hospitals. J Clin Nurs. 2008;17(24):3324–3329. 66. Seago JA, Ash M. Registered nurse unions and patient outcomes. J Nurs Adm. 2002;32(3):143–151. 67. Koppel R, TWetterneck T, Telles JL, Karsh B-T. Workarounds to barcode medication administration systems: their occurrences, causes, and threats to patient safety. J Am Inform Assoc. 2008;15:408–423. 68. Spetz J, Phibbs CS, Burgess JF. What determines successful implementation of inpatient information technology systems? Am J Managed Care. 2012;19 (3):157–162.

J. Spetz et al. / Healthcare 2 (2014) 40–47

69. Poissant L, Pereira J, Tamblyn R, Kawasumi Y. The impact of electronic health records on time efficiency of physicians and nurses: a systematic review. J Am Inform Assoc. 2005;12(5):505–516. 70. Kraus S, Barber TR, Briggs B, et al. Implementing computerized physician order management at a community hospital. Jt Comm J Qual Patient Saf. 2008;34 (2):74–84. 71. Ovretveit J, Scott T, Rundall TG, Shortell SM, Brommels M. Implementation of electronic medical records in hospitals: two case studies. Health Policy. 2007;84:181–190. 72. Wideman MV, Whittler ME, Anderson TM. Barcode medication administration: lessons learned from an intensive care unit implementation. In: Henriksen K, Battles JB, Marks EB, editors. Advances in patient safety, vol. 3. Rockville, MD: Agency for Healthcare Research and Quality; 2005 (et al.).

47

73. McCullough JS, Casey M, Moscovice I, Prasad S. The effect of health information technology on quality in U.S. hospitals. Health Aff. 2010;29(4):647–654. 74. Byrne MM, Charns MP, Parker VA, Meterko MM, Wray NP. The effects of organization on medical utilization: an analysis of service line organization. Med Care. 2004;42(1):28–37. 75. Byrne CM, Mercincavage LM, Pan EC, Vincent AG, Johnston DS, Middleton B. The value from investments in health information technology at the U.S. Department of Veterans Affairs. Health Aff. 2010;29(4):629–638. 76. Holden RJ, Scanlon MC, Patel NR, et al. A human factors framework and study of the effect of nursing workload on patient safety and employee quality of working life. BMJ Qual Saf. 2011;20(1):15–24. 77. Hebert M. Professional and organizational impact of using patient care information systems. Medinfo. 1998;9(Part 2):849–853.