- Email: [email protected]
Measuring mobile patient safety information system success: An empirical study
i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s 7 7 ( 2 0 0 8 ) 689–697
journal homepage: www.intl.elsevierhealth.com/journals/ijmi
Measuring mobile patient safety information system success: An empirical study Wen-Yuan Jen a,∗ , Chia-Cheng Chao b,c a b c
Department of Information Management, Overseas Chinese Institute of Technology, No. 100, Chiao-Kwang Road, Taichung, Taiwan New Business Development, Department of Taipei Medical University of Wan Fang Hospital, Taiwan Information Science Department, National Taipei University of Education, Taiwan
a r t i c l e
i n f o
a b s t r a c t
Article history:
Objective: The Health Risk Reminders and Surveillance (HRRS) system was designed to deliver
Received 1 April 2007
critical abnormal test results of severely ill patients from Laboratory, Radiology, and Pathol-
Received in revised form
ogy departments to physicians within 5 min using cell phone text messages. This paper
3 March 2008
explores the success of the HRRS system.
Accepted 4 March 2008
Method: This study employed an augmented version of the DeLone and McLean IS success model. Seven variables (system quality, information quality, system use, user satisfaction, mobile healthcare anxiety, impact on the individual and impact on the organization) were
Keywords:
used to evaluate the success of the HRRS system. The interrelationships between the seven
Healthcare
variables were hypothesized and the hypotheses were empirically tested.
Mobile healthcare anxiety
Results: The results indicate that the information quality of the HRRS system is positively
Patient safety service
associated with both system use and user satisfaction. In addition, system use is positively associated with user satisfaction, which is also positively associated with mobile healthcare anxiety. Moreover, results indicate that impact on the individual is positively associated with both user satisfaction and mobile healthcare anxiety. Finally, the impact of the organization is positively associated with impact on the individual. Conclusion: The results of the study provide an expanded understanding of the factors that contribute to mobile patient safety information system (IS) success. Implications of the relationship between system use and physician mobile healthcare anxiety are discussed. © 2008 Elsevier Ireland Ltd. All rights reserved.
1.
Introduction
Mobile technology applications have the capability and potential to improve healthcare service quality as they instantaneously provide critical patient test data, enabling medical staff to render treatment immediately [1–4]. Mobile service emphasizes real-time, full-time information accessibility, and service quality [5–8]; hence, mobile healthcare applications are recognized as emerging and enabling services in some countries [9–11]. The literature points out that mobile healthcare
∗
information systems have the potential to facilitate patient care efficiently and effectively [12–14]. Mobile healthcare information systems have the ability to improve healthcare workflow accuracy and efficiency and to increase patient safety by reducing the risk of human error. Patient safety is an important issue for hospital management, and many hospitals are proactively pursuing greater patient safety as part of their efforts to improve the quality of their service. To improve patient safety, the Municipal WanFang (W.F.) Hospital introduced the Health Risk Reminders
Corresponding author. Tel.: +886 4 22067991x8601; fax: +886 4 27075420. E-mail addresses: [email protected] (W.-Y. Jen), [email protected] (C.-C. Chao). 1386-5056/$ – see front matter © 2008 Elsevier Ireland Ltd. All rights reserved. doi:10.1016/j.ijmedinf.2008.03.003
690
i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s 7 7 ( 2 0 0 8 ) 689–697
Fig. 1 – High-Risk Reminder and Surveillance (HRRS) procedures.
and Surveillance (HRRS) mobile system. The HRRS system was designed to deliver critical abnormal test results of severely ill patients from Laboratory, Radiology, and Pathology departments (see Fig. 1). To measure the success of information systems, DeLone and McLean created a multi-dimensional information system success model [15] that measures system quality, information quality, system use, user satisfaction, impact on the individual and impact on the organization. The current study employs the DeLone–McLean model to measure the success of the HRRS system after an 8-month period of experimental use. However, because the original DeLone–McLean model does not measure the additional stress that might result from system use, and because stress among the physicians who use the HRRS system may contribute greatly to the quality of the services they provide their patients, a seventh variable, “mobile healthcare anxiety” was added to the original model for this study. The results of this study will enable researchers to identify various aspects of mobile patient safety information system success and investigate contributing factors. Healthcare practitioners will also be able to employ the results in assessing anticipated outcomes and justifying medical care service processes in the post-implementation phase. The paper is organized as follows. First, the background of the HRRS system is discussed. Then the HRRS success research model and research hypotheses, method, and research findings are presented in Sections 3–5, respectively. Finally, there is a discussion of the findings of the study and implications for further research and development.
2. The Health Risk Reminders and Surveillance (HRRS) system The HRRS system was launched and implemented by W.F. Hospital, a teaching medical center located in Taipei, Taiwan and affiliated with Taipei Medical University. Ancillary, Pathology, and Radiology were the major service departments employing the HRRS system. Chao et al. [16] describe the stages, workflow and service units involved in the study (see Table 1). Chao et al. [17] identified the outcomes of the experiment as: (1) high-risk laboratory department reports: over the 8month period, there were 339,598 laboratory tests conducted, and of those tests, the results of 6561 (approximately 2%) fell within abnormal ranges. (2) High-risk pathology department reports: there were 329 abnormal test reports (approximately 4%) among the 8367 pathology tests conducted. (3) High-risk radiology department reports: the radiology department sent an average of approximately 23 abnormal test messages to the medical staff each month during the experimental period. These cell phone text messages and Internet e-mails (Fig. 2) were received by the attending physicians within 5 min. As Chao et al. show, by replacing paper notification with the HRRS system, the time required to deliver abnormal test results to physicians was dramatically decreased [17]. The HRRS system clearly contributed to improved service quality and patient safety by improving communication and eliminating the possibility of misplaced reports.
691
i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s 7 7 ( 2 0 0 8 ) 689–697
Table 1 – The stage, workflow and service unit of the HRRS system Stage
Workflow
Recording patient’s test results
Service unit
All patients’ test results of Laboratory, Radiology or Pathology departments were recorded.
• Laboratory technicians
• Radiologists • Pathologists Checking the test results
Delivering alert messages
Employing standard indicators of severely abnormal test results, and the patient’s test results were evaluated. When test results fell within critical abnormal ranges, the HRRS system sent alert messages within 5 min.
• The HRRS system
• The physician’s cell phone text messages and Internet e-mails • Intensive Care Unit (ICU)
Meeting medical care responsibilities
The medical care staffs were assisted by the HRRS system in monitoring the condition of critically ill patients.
• Primary care physicians
• Nurse stations • Department directors • Department secretaries
3. Conceptual model and research hypotheses 3.1.
Information system success model
The DeLone and McLean information system (IS) success model, based on theoretical and empirical IS research, has been employed by a number of researchers since the 1990s [15]. The model posited six major dimensions that can be used to evaluate information systems, including system quality, information quality, system use, user satisfaction, impact on the individual user and impact on the using organization. To measure e-commerce system success, DeLone and McLean later updated their original model, adding “service quality” and “net benefit” dimensions [15,18]. This updating of the model suggests the need to adopt appropriate variables when measuring IS success in different industries. Factors considered in assessing telehealth system success have included the system’s ability to provide: (1) a sound system infrastructure, (2) strong program management based on a needs analysis, and (3) applications that match identified
needs [19]. There is rarely only one factor used to evaluate the success of medical informatics systems and many different performance measures or success factors are commonly applied in evaluating them [20]. Unfortunately, one factor affecting the quality of information systems used in healthcare, the psychological impact on the healthcare professionals using them, is not covered in models previously used to assess system success.
3.2.
Mobile healthcare anxiety
Computer anxiety has been described as “an affective response, such that resistance to and avoidance of computer technology are a function of fear and apprehension, intimidation, hostility, and worries that one will be embarrassed, look stupid, or even damage the equipment” [21]. Wurman defines information anxiety as “the black hole between data and knowledge, and it happens when information does not tell us what we want or need to know” [22] (p. 34). Wilson describes information overload as “a personal problem, a perception on the part of the individual (or observers of that person) that the flow of information associated with work tasks is
Fig. 2 – Sample HRRS subsystem alert message sent via e-mail and cellular phone.
692
i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s 7 7 ( 2 0 0 8 ) 689–697
greater than can be managed effectively,” and organizational information overload as “a situation in which the extent of perceived information overload is sufficiently widespread within an organization so as to reduce the overall effectiveness of management operations” [23] (p. 113). In practice, delivering patients’ critical information to physicians via paper-based patient records or by telephone notifications may sometimes result in delays. The mobile patient safety information service provided by the HRRS system sends both e-mails and cell phone text messages to attending physicians 24 h a day. Included in the present study is an examination of how the instantaneous and uninterrupted character of the mobile patient safety information service might result in information overload or anxiety, mobile healthcare anxiety, and thus negatively affect the quality of care that physicians provide their patients. In this study, we define Mobile Healthcare Anxiety as a “highly anxious response towards interaction or anticipated interaction with the mobile patient safety information system.”
3.3.
As Fig. 2 illustrates, the HRRS was designed to deliver to physicians critical abnormal test results from Laboratory, Radiology, and Pathology departments. Recording patients’ test results, checking the test results, delivering alert messages and meeting medical care responsibilities are the four stages of the HRRS workflow. When abnormal test results were reported, the HRRS delivered an alert via cell phone text message to the attending physician. Unfortunately, because the HRRS message sent alert messages to attending physicians 24/7, each physician’s workload increased in proportion to the number of high-risk patients he or she had. Therefore, we add the hypothesis: H4-1. System Use, User Satisfaction and Mobile Healthcare Anxiety are positively associated with impact on the individual. Thus supplemented to include healthcare anxiety, the IS success model will produce a more comprehensive evaluation of the HRRS system.
Research model and hypotheses
The DeLone and McLean IS success model provided taxonomies of success variables. Based on the characteristics of the HRRS system, this study adds the mobile healthcare anxiety variable to measure the impact of the system on physician’s psychological well being. Following previous IS success literature, the study’s hypotheses are proposed (Fig. 3 presents the research model): H1. System Quality and Information Quality are positively associated with System Use. H2. System Quality and Information Quality are positively associated with User Satisfaction. H3. System Use is positively associated with User Satisfaction. H4. System Use and User Satisfaction are positively associated with impact on the individual. H5. Impact on the individual is positively associated with impact on the organization.
4.
Method
4.1.
Sample
The subjects of this study were W.F. Hospital physicians who had received HRRS cell phone text messages and Internet e-mails during the 8-month experimental period. Questionnaires were distributed to these physicians at their clinics during office hours between Monday and Friday. Although the physicians at W.F. Hospital represent internal medicine, surgery, pediatrics, gynecology and other departments, not all physicians received the HRRS messages frequently. Hence, those physicians who received fewer than 6 HRRS messages in 1 week were excluded from this study.
4.2.
Instrument
The questionnaire was composed of seven variables. All items for each variable were taken from previously validated instruments. In the system quality variable, the measured items were ease of use, user-friendliness, accessibility, and system response time [24,25]. In the information quality variable, the measured items were accuracy, legibility, and currency of information [26,27]. In the system use variable, the measured
Fig. 3 – The HRRS system success model.
693
i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s 7 7 ( 2 0 0 8 ) 689–697
items were motivation to use the system [28], use or nonuse of computer-based decision aids [29], and enlargement of the scope of patient services. The measured items in the user satisfaction variable were overall satisfaction, user attitude toward the information system, improvement of clinical communication and medical record keeping, and effect on decision making [29,30]. Included in the variable measuring impact on the individual were changed work practices, immediate benefits of system use, and job satisfaction [31,32]. The additional variable, mobile healthcare anxiety was composed based on the State-Trait Anxiety Inventory [33]. In this variable, the measured items were job stress increase, information overload anxiety, and quality of life. Finally, the variable measuring impact on the organization included communication efficiency and impact on patient care [32,34], as well as improvement in patient safety. Appendix A contains a summary of the instrument. The questionnaire consisted of two parts. The first part elicited respondent demographics, including age, gender, education, work experience, job title and medical specialization. The second part contained the questionnaire proper, which consisted of 23 statements to which respondents were asked to indicate their degree of agreement. All items employed a five point Likert-type scale with 1 being “strongly disagree,” 3 being neutral, and 5 being “strongly agree.” In order to assess the face validity of the questionnaire, the help of medical professionals and medical informatics researchers was enlisted. After discussion, the initial questionnaire was revised to ensure that there was a good match between the information the questionnaire elicited and the factors upon which the success of the system would be measured.
5.
Questionnaires were sent to the approximately 150 fulltime physicians at W.F. Hospital, and 95 questionnaires were returned. Of these 95 questionnaires, 18 were invalidated because the responding physician too rarely received HRRS notifications, and 5 were returned incomplete. This yielded a total of 72 usable questionnaires, a validation rate of 76%. Respondents of valid questionnaires were 93% male; 44% were between the ages of 30 and 50; 90% were primary physicians; 67% had working experience of more than 10 years, 22% had 5 to 10 years’ experience, and 11% had less than 5 years of working experience. The distribution of specialties was 7% for Family medicine, 3% for Ear, Nose, and Throat, 10% for Pediatrics, 10% for Gynecology, 7% for Orthopedics, 8% for Anesthesiology, 5% for Urology and Dermatology, 40% for Internal Medicine, and 5% for Surgery. Cronbach’s Alpha for the 7 variables in the questionnaire indicated good reliability, taking a reliability value of 0.7 as the minimal standard [36]. The mean, standard deviation, and Cronbach’s Alpha for the independent and dependent variables are listed in Table 2. All variables reported means of above 3.0. Pearson’s correlation coefficients, also shown in Table 2, indicated low and high existent correlations among variables examined for the HRRS system. Most variables were significantly related. Among the seven variables, information quality showed the strongest correlation with system use, user satisfaction, impact on the individual, mobile healthcare anxiety and impact on the organization. The results of the regression analysis are presented in Table 3.
5.1. 4.3.
Research findings
System quality and information quality
Data analysis
A review of the literature showed that prior studies in systems success used structural equation modeling (SEM), which is a regression-based technique rooted in path analysis. Structural equation modeling is often loosely termed a causal modeling technique [35] and requires a large sample size. Because the sample size in the present study was not large, however, linear regression, rather than SEM, was adopted to examine the relationships among the hypotheses. All analyses were carried out using the SPSS statistics program, version 11.0 (SPSS, Inc., Chicago, Illinois).
System quality and information quality (H1) did not behave as hypothesized. For system use, the regression model was significant (R2 = 0.552, p < 0.001). Of the two variables, only information quality was significant with a positive beta (ˇ = 0.720, p < 0.001). Hypothesis H1 was partially supported, as shown in Table 4. Like hypothesis H1, hypothesis H2 was partially supported, as shown in Tables 3 and 4. With regard to user satisfaction, the regression model was significant (R2 = 0.938, p < 0.001); information quality had a significant positive association (ˇ = 0.413, p < 0.01) with user satisfaction, while system quality had no significant association.
Table 2 – Correlation among dimension scores Dimensions
System quality Information quality System use User satisfaction Impact on the individual Mobile healthcare anxiety Impact on the organization ∗ ∗∗
p < 0.05. p < 0.01.
Mean
3.350 3.733 3.710 3.733 3.667 3.710 3.933
S.D.
0.507 0.458 0.414 0.399 0.463 0.634 0.405
Alpha
0.894 0.878 0.852 0.901 0.819 0.697 0.736
Correlation 1
2
3
4
5
6
7
1.00 0.626** 0.487** 0.502** 0.414* 0.255 0.258
1.00 0.743** 0.855** 0.722** 0.435* 0.582**
1.00 0.894** 0.699** 0.241 0.347
1.00 0.762** 0.318 0.420*
1.00 0.606** 0.552**
1.00 0.559**
1.00
694
i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s 7 7 ( 2 0 0 8 ) 689–697
Table 3 – Regression analysis table Independent variable
Dependent variable System use
Beta System quality Information quality System use User satisfaction Mobile healthcare anxiety Impact on the individual R2 Adjusted R2 F value
User satisfaction
0.036 0.720***
Impact on the individual
0.024 0.413** 0.575***
0.089 0.682*
Impact on the organization
0.179 0.467* 0.414** 0.552**
0.552 0.519 16.663***
0.938 0.880 63.815***
0.582 0.551 18.801***
0.734 0.704 23.958***
0.304 0.279 12.239**
∗
p < 0.05. p < 0.01. ∗∗∗ p < 0.001. ∗∗
5.2. System use, user satisfaction and mobile healthcare anxiety
5.3. Impact on the individual and impact on the organization
Hypothesis H3 was supported by the study, as is shown in Table 4. System use was positively associated with user satisfaction, as indicated by the positive beta (ˇ = 0.575, p < 0.001). Hypothesis H4 was partially supported, as shown in Table 4. For impact on the individual, the regression model was significant (R2 = 0.582, p < 0.001). Of the two variables, user satisfaction had a positive beta (ˇ = 0.682, p < 0.05), while system use had no significant association with impact on the individual. Hypothesis H4-1 was partially supported, as shown in Table 4. For impact on the individual, the regression model was significant (R2 = 0.734, p < 0.001). Of the three variables, only system use did not have a positive association with impact on the individual. User satisfaction had a positive beta (ˇ = 0.467, p < 0.05), and mobile healthcare anxiety had a positive beta (ˇ = 0.414, p < 0.01). Both user satisfaction and mobile healthcare anxiety were positively associated with impact on the individual.
Hypothesis 5 also behaved as hypothesized, as shown in Table 4. For impact on the organization, the regression model was significant (R2 = 0.304, p < 0.01). These results show that impact on the individual had a significantly positive association (ˇ = 0.552, p < 0.01) with impact on the organization.
Table 4 – Regression comparison analysis
*
p < 0.05; ** p < 0.01; and *** p < 0.001.
6.
Discussion
In discussing their Technology Acceptance Model (TAM), Wu et al. [37] pointed out that compatibility, perceived usefulness and perceived ease of use significantly affected healthcare professional behavioral intent in adopting a mobile healthcare information system. The DeLone and McLean information system success model (IS) is a multi-dimensional tool that can be used to assess an information system at various levels. In continuing the evolution of information system success model
i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s 7 7 ( 2 0 0 8 ) 689–697
use in the healthcare industry, the present study employed a mobile patient safety information system success model with a mobile healthcare anxiety. The empirical results provide considerable support for the model. The results show that, first, information quality is positively associated with system use and user satisfaction; second, system use is positively associated with user satisfaction; third, user satisfaction and mobile healthcare anxiety are positively associated with impact on the individual; and, finally, that impact on the individual is positively associated with impact on the organization. Information quality was significantly associated with both system use and user satisfaction, but system quality of the HRRS system did not have a significant association with system use or user satisfaction, results that are inconsistent with most prior MIS research [15]. That system quality is not significantly associated with user satisfaction might indicate that the respondents saw cellular phones and e-mail merely as a media source through which they receive patient data, and that system quality was either “yes” when it was working or “no” when their system was down, rather than a sliding scale of quality. We found that system use had a positive association with user satisfaction. This might be explained by the fact that the HRRS system contributed to physician satisfaction because the mobility of the HRRS system did help the physician to make timely medical decisions in emergent cases. With regard to the impact of HRRS system use on individual physicians, the results for H4 showed that only user satisfaction had a positive association. However, the results for H4-1 showed that both user satisfaction and mobile healthcare anxiety were positively associated with impact on the individual. It is clear from our study that mobile healthcare anxiety is one facet of the impact that the use of the HRRS has on individual physicians. When people feel anxiety using a new product or service, they generally rate that product or service negatively. However, even though they did suffer from mobile healthcare anxiety, the physicians in this study rated the HRRS system positively because it increased their efficiency by helping them to make timely medical decisions. From a practical perspective, physicians may feel exhausted if they continue to receive updates on emergent cases while they are off duty. Hence, hospitals should develop guidelines for HRRS use which minimize any negative impact system use might have on physicians and, indirectly, on their patients. Although previous studies have rarely discussed mobile service anxiety among healthcare professionals, it is an issue worthy of future discussion. Mobile patient safety information services should not become a burden to medical professionals’ health. Impact on the organization was significantly associated with impact on the individual. In general, most respondents agreed HRRS played a role in improving medical service quality. Lack of access to information during the decision-making process and ineffective communication among patient care team members were proximal causes of medical errors and other adverse events in patient care [38]. The traditional system of delivering critical test results and information about critical patients to physicians via paper-based patient records or by telephone notifications often resulted in delays, compromising patient safety. Employing the advantages of mobile
695
Summary points What was known before the study? • Laboratories commonly send important test results to both patients and physicians by paper-based record. The misplacement of test results compromises patient safety in clinical practice. • Laboratory, radiology, and pathology reports may have difficulty reaching patients, attending physicians or administrators immediately after detection of critical results. • Because the patient is the most important player in healthcare service and safety, most of the professional literature focuses on the benefits that patients derive from innovative mobile technology use. However, the well-being of the proximal healthcare professional, the physician, is critical to effective healthcare delivery, so how their work environment affects them should also be examined. • The success of information systems and information technology is often measured by the DeLone and McLean IS success model. What has the study added to the body of knowledge? • The augmented model is based on the DeLone and McLean IS success model and includes an additional mobile healthcare anxiety that delineates the attending physician’s dilemma of how to balance the additional efficiencies derived from HRRS system use against the increased anxiety that accompanies it. • The results indicated that information quality, system use, user satisfaction, mobile healthcare anxiety, impact on the individual, and impact on the organization are factors significantly associated with mobile patient safety service system success. Some implications and suggestions have been delineated in this study. • The study found that, while mobile patient safety information systems contribute to an improvement in patient services and a reduction in patient risk (through increased administrative efficiency), use of these communication systems may inadvertently contribute to physician anxiety. Continued research into how to relieve physician anxiety associated with mobile patient safety information system use is proposed.
technology and the Internet, the HRRS system helped physicians make fast, accurate and appropriate decisions, reducing patient risk and improving patient service. In the near future, the results of this study may lead to an expansion of HRRS use from laboratory, radiology, and pathology to emergency departments, where it could reduce ER patient risk and improve patient safety [39]. The HRRS system might even move out of the hospital and into private residences, where it might be utilized in patient health ser-
696
i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s 7 7 ( 2 0 0 8 ) 689–697
vice management for home-bound, chronically ill patients. Moreover, if technological advances proceed apace, HRRS service domains will soon let physicians query patient databases to view up-to-date documents and images for interactive diagnoses, and cellular phones may soon be integrated into PDAs with larger screens, web cams and high speed data transmission capability. That the HRRS system improves both administrative efficiency and the quality of healthcare that patients receive is no longer just rhetoric, but a growing reality that will continue to expand in the foreseeable future.
Acknowledgements The authors thank Dr. J. Talmon and the anonymous reviewers for their insightful suggestions. This study is most grateful to
Construct
System quality
Information quality
Items
Mobile healthcare anxiety
Impact on the individual
Appendix A. Summary of measures for the variables
Measure
SQ1
The operation menu of the HRRS is easy to use.
SQ2
The use of the HRRS is user-friendly.
SQ3
The HRRS system is stable.
SQ4
The response time of the HRRS is speedy.
IQ1
The information of the HRRS is accurate.
IQ2
The information of the HRRS is legible.
IQ3
The information of the HRRS is up-to-date.
SU1
I agree to continue to use the HRRS alert messages for high-risk patients.
SU2
The alert messages of the HRRS can help me to save high-risk patients.
SU3
I agree the scope of the HRRS alert message should be expanded.
US1
The use of the HRRS is helpful for my medical decision making.
US2
The HRRS improves clinical communication and medical record keeping.
US3
I keep a positive attitude toward the HRRS.
US4
Over all, the service of the HRRS makes me feel satisfied.
MH1
The HRRS increases my job strain.
MH2
The HRRS makes me feel information overload anxiety.
MH3
The HRRS affects the quality of my life.
II1
The HRRS changed my work practices.
II2
I experienced the immediate benefits of the HRRS.
II3
The HRRS improves my job satisfaction.
System use
User satisfaction
both the Patient Safety Institute staff and the physicians of W.F. Hospital. Your participation and help was essential to this study. Authors’ contributions: The first author is the primary author of this study. The second author participated in both questionnaire design and data acquisition, and his abundant experience in the practice of hospital management was of great help to the first author in drafting this paper. Competing interests: The authors declare that they have no competing interests or other interests that might be perceived to influence the results and/or discussion reported in this paper. The HRRS system was implemented by the Patient Safety Institute of W.F. Hospital and is published with the permission of W.F. Hospital.
OA1
The HRRS improves the communication efficiency of our hospital.
OA2
The HRRS increases the patient safety capability of our hospital.
OA3
The HRRS improves the quality of services to high-risk patients.
Impact on the organization
i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s 7 7 ( 2 0 0 8 ) 689–697
references
[1] E.A. Mendonc¸a, E.S. Chen, P.D. Stetson, L.K. McKnight, Approach to mobile information and communication for healthcare, Int. J. Med. Inform. 73 (2004) 631–638. [2] Y.C. Lu, Y. Xiao, A. Sears, J.A. Jacko, A review and a framework of handheld computer adoption in healthcare, Int. J. Med. Inform. 74 (2005) 409–422. [3] A. Krause, D. Hartl, F. Theis, M. Stangl, K.E. Gerauer, A.T. Mehlhorn, Mobile decision support for transplantation patient data, Int. J. Med. Inform. 73 (2004) 461–464. [4] J.R. Barrett, S.M. Strayer, J.R. Schubart, Assessing medical residents’ usage and perceived needs for personal digital assistants, Int. J. Med. Inform. 73 (2004) 25–34. [5] N. Samuli, H. Lamminen, K. Niemi, H. Mattila, S. Kalli, A cost study of new media supported near oral anticoagulant treatment follow-up, Int. J. Med. Inform. 70 (2003) 19–29. [6] N. Maglaveras, V. Koutkias, I. Chouvarda, D.G. Goulis, A. Avramides, D. Adamidis, G. Louridas, E.A. Balas, Home care delivery through the mobile telecommunications platform: the Citizen Health System (CHS) perspective, Int. J. Med. Inform. 68 (2002) 99–111. [7] U. Sax, I. Kohane, K.D. Mandl, Wireless technology infrastructures for authentication of patients: PKI that rings, J. Am. Med. Inform. Assoc. 12 (2005) 263–268. [8] C.A. Sneiderman, M.J. Ackerman, Cellular radio telecommunication for healthcare: benefits and risks, a brief review, J. Am. Med. Inform. Assoc. 11 (2005) 479–481. [9] H.J. Ten Duis, C. Van der Werken, Trauma care systems in the Netherlands, Injury 34 (9) (2003) 722–727. [10] E. Ammenwerth, S. Graber, G. Herrman, T. Burkle, J. Konig, Evaluation of health information systems-problems and challenges, Int. J. Med. Inform. 71 (2/3) (2003) 125–135. [11] R. Haux, Health information systems—past, present, future, Int. J. Med. Inform. 75 (3–4) (2006) 268–281. [12] M. Bang, A. Larsson, E. Berglund, H. Eriksson, Distributed user interfaces for clinical ubiquitous computing applications, Int. J. Med. Inform. 74 (7–8) (2005) 545–551. [13] Y.C. Lu, Y. Xiao, A. Sears, J.A. Jacko, A review and a framework of handheld computer adoption in healthcare, Int. J. Med. Inform. 74 (5) (2005) 409–422. [14] E. Ammenwerth, A. Buchauer, B. Bludau, R. Haux, Mobile information and communication tools in the hospital, Int. J. Med. Inform. 57 (1) (2000) 21–40. [15] W.H. DeLone, E.R. McLean, The DeLone and McLean model of information system success: a 10-year update, J. Manag. Inform. Syst. 19 (4) (2003) 9–30. [16] C.C. Chao, W.Y. Jen, Y.P. Chi, B. Lin, Improving patient safety with RFID and mobile technology, Int. J. Electron. Healthc. 3 (2) (2007) 175–192. [17] C.C. Chao, W.Y. Jen, M.C. Hung, Y.C. Li, Y.P. Chi, An innovative mobile approach for patient safety services. The case of a Taiwan healthcare provider, Technovation 27 (5) (2007) 268–279. [18] W.H. DeLone, E.R. McLean, Measuring e-commerce success: applying DeLone & McLean information systems success model, Int. J. Electron. Commun. 9 (1) (2004) 31–47.
697
[19] J.R. Moehra, J. Schaafsmab, C. Anglinc, S.V. Pantazia, N.A. Grimma, S. Anglinc, Success factors for telehealth—a case study, Int. J. Med. Inform. 75 (2006) 755–763. [20] P.Y.K. Chau, P.J.H. Hu, Investigating healthcare professionals’ decisions to accept telemedicine technology: an empirical test of competing theories, Inform. Manag. 39 (4) (2002) 297–311. [21] R.K. Heinssen, C.R. Glass, L.A. Knight, Assessing computer anxiety: development and validation of the computer anxiety rating scale, Comput. Hum. Behav. 3 (1987) 49–59. [22] R.S. Wurman, Information Anxiety, Doubleday, New York, 1989. [23] T.D. Wilson, Information overload: implication for healthcare service, Health Inform. 7 (2001) 112–117. [24] J.E. Bailey, S.W. Pearson, Development of a tool for measuring and analyzing computer user satisfaction, Manage. Sci. 29 (5) (1983) 530–545. [25] H. Barth, S. Huff, Change attitude to change, and decision support system success, Inform. Manag. 9 (5) (1985) 261–268. [26] W.R. King, B.J. Epstein, Assessing information system value, Decision Sci. 14 (1) (1983) 34–45. [27] A. Srinivasan, Alternative measures of system effectiveness: associations and implications, MIS Q 9 (3) (1985) 243–253. [28] G. DeSanctis, An examination of an expectancy theory model of decision support system use, in: Proceeding of the 3rd International Conference on Information system, 1982, pp. 121–135. [29] M. Alavi, J.C. Henderson, An evolutionary strategy for implementing a decision support system, Manage. Sci. 27 (11) (1981) 1309–1322. [30] B. Kaplan, H. Lundsgaarde, Toward an evaluation of an integrated clinical imaging system: identifying clinical benefits, Method Inform. Med. 35 (3) (1996) 221–229. [31] C. Sicotte, J. Denis, P. Lehoux, F. Champagne, The computer-based patient record challenges towards timeless and spaceless medical practice, J. Med. Syst. 22 (4) (1998) 237–256. [32] T. Ostbye, A. Moen, G. Erikssen, P. Hurlen, Introducing a module for laboratory test order entry and reporting of results at a hospital ward: an evaluation study using a multi-method approach, J. Med. Syst. 21 (2) (1997) 107–117. [33] C.D. Spielberger, Manual for the State-Trait Anxiety Inventory, Consulting Psychologists Press, Palo Alto, CA, 1983. [34] J.E. Allan, Patient-centered documentation: an effective and efficient use of clinical information systems, J. Nurs. Adm. 30 (2) (2000) 90–95. [35] W.W. Chin, P. Todd, On the use, usefulness, and ease of use of structural equation modeling in MIS Research: a note of caution, MIS Q 19 (2) (1995) 237–246. [36] J.C. Nunnally, Psychometric Theory, McGraw-Hill, New York, 1978. [37] J.H. Wu, S.C. Wang, L.M. Lin, Mobile computing acceptance factors in the healthcare industry: a structural equation model, Int. J. Med. Inform. 76 (1) (2007) 66–77. [38] C.M. Ruland, Improving patient safety through informatics tools for shared decision making and risk communication, Int. J. Med. Inform. 73 (7–8) (2004) 551–557. [39] M.N. Miller, R.K. McGowen, The painful truth: physicians are not invincible, South Med. J. 93 (10) (2000) 966–973.
journal homepage: www.intl.elsevierhealth.com/journals/ijmi
Measuring mobile patient safety information system success: An empirical study Wen-Yuan Jen a,∗ , Chia-Cheng Chao b,c a b c
Department of Information Management, Overseas Chinese Institute of Technology, No. 100, Chiao-Kwang Road, Taichung, Taiwan New Business Development, Department of Taipei Medical University of Wan Fang Hospital, Taiwan Information Science Department, National Taipei University of Education, Taiwan
a r t i c l e
i n f o
a b s t r a c t
Article history:
Objective: The Health Risk Reminders and Surveillance (HRRS) system was designed to deliver
Received 1 April 2007
critical abnormal test results of severely ill patients from Laboratory, Radiology, and Pathol-
Received in revised form
ogy departments to physicians within 5 min using cell phone text messages. This paper
3 March 2008
explores the success of the HRRS system.
Accepted 4 March 2008
Method: This study employed an augmented version of the DeLone and McLean IS success model. Seven variables (system quality, information quality, system use, user satisfaction, mobile healthcare anxiety, impact on the individual and impact on the organization) were
Keywords:
used to evaluate the success of the HRRS system. The interrelationships between the seven
Healthcare
variables were hypothesized and the hypotheses were empirically tested.
Mobile healthcare anxiety
Results: The results indicate that the information quality of the HRRS system is positively
Patient safety service
associated with both system use and user satisfaction. In addition, system use is positively associated with user satisfaction, which is also positively associated with mobile healthcare anxiety. Moreover, results indicate that impact on the individual is positively associated with both user satisfaction and mobile healthcare anxiety. Finally, the impact of the organization is positively associated with impact on the individual. Conclusion: The results of the study provide an expanded understanding of the factors that contribute to mobile patient safety information system (IS) success. Implications of the relationship between system use and physician mobile healthcare anxiety are discussed. © 2008 Elsevier Ireland Ltd. All rights reserved.
1.
Introduction
Mobile technology applications have the capability and potential to improve healthcare service quality as they instantaneously provide critical patient test data, enabling medical staff to render treatment immediately [1–4]. Mobile service emphasizes real-time, full-time information accessibility, and service quality [5–8]; hence, mobile healthcare applications are recognized as emerging and enabling services in some countries [9–11]. The literature points out that mobile healthcare
∗
information systems have the potential to facilitate patient care efficiently and effectively [12–14]. Mobile healthcare information systems have the ability to improve healthcare workflow accuracy and efficiency and to increase patient safety by reducing the risk of human error. Patient safety is an important issue for hospital management, and many hospitals are proactively pursuing greater patient safety as part of their efforts to improve the quality of their service. To improve patient safety, the Municipal WanFang (W.F.) Hospital introduced the Health Risk Reminders
Corresponding author. Tel.: +886 4 22067991x8601; fax: +886 4 27075420. E-mail addresses: [email protected] (W.-Y. Jen), [email protected] (C.-C. Chao). 1386-5056/$ – see front matter © 2008 Elsevier Ireland Ltd. All rights reserved. doi:10.1016/j.ijmedinf.2008.03.003
690
i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s 7 7 ( 2 0 0 8 ) 689–697
Fig. 1 – High-Risk Reminder and Surveillance (HRRS) procedures.
and Surveillance (HRRS) mobile system. The HRRS system was designed to deliver critical abnormal test results of severely ill patients from Laboratory, Radiology, and Pathology departments (see Fig. 1). To measure the success of information systems, DeLone and McLean created a multi-dimensional information system success model [15] that measures system quality, information quality, system use, user satisfaction, impact on the individual and impact on the organization. The current study employs the DeLone–McLean model to measure the success of the HRRS system after an 8-month period of experimental use. However, because the original DeLone–McLean model does not measure the additional stress that might result from system use, and because stress among the physicians who use the HRRS system may contribute greatly to the quality of the services they provide their patients, a seventh variable, “mobile healthcare anxiety” was added to the original model for this study. The results of this study will enable researchers to identify various aspects of mobile patient safety information system success and investigate contributing factors. Healthcare practitioners will also be able to employ the results in assessing anticipated outcomes and justifying medical care service processes in the post-implementation phase. The paper is organized as follows. First, the background of the HRRS system is discussed. Then the HRRS success research model and research hypotheses, method, and research findings are presented in Sections 3–5, respectively. Finally, there is a discussion of the findings of the study and implications for further research and development.
2. The Health Risk Reminders and Surveillance (HRRS) system The HRRS system was launched and implemented by W.F. Hospital, a teaching medical center located in Taipei, Taiwan and affiliated with Taipei Medical University. Ancillary, Pathology, and Radiology were the major service departments employing the HRRS system. Chao et al. [16] describe the stages, workflow and service units involved in the study (see Table 1). Chao et al. [17] identified the outcomes of the experiment as: (1) high-risk laboratory department reports: over the 8month period, there were 339,598 laboratory tests conducted, and of those tests, the results of 6561 (approximately 2%) fell within abnormal ranges. (2) High-risk pathology department reports: there were 329 abnormal test reports (approximately 4%) among the 8367 pathology tests conducted. (3) High-risk radiology department reports: the radiology department sent an average of approximately 23 abnormal test messages to the medical staff each month during the experimental period. These cell phone text messages and Internet e-mails (Fig. 2) were received by the attending physicians within 5 min. As Chao et al. show, by replacing paper notification with the HRRS system, the time required to deliver abnormal test results to physicians was dramatically decreased [17]. The HRRS system clearly contributed to improved service quality and patient safety by improving communication and eliminating the possibility of misplaced reports.
691
i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s 7 7 ( 2 0 0 8 ) 689–697
Table 1 – The stage, workflow and service unit of the HRRS system Stage
Workflow
Recording patient’s test results
Service unit
All patients’ test results of Laboratory, Radiology or Pathology departments were recorded.
• Laboratory technicians
• Radiologists • Pathologists Checking the test results
Delivering alert messages
Employing standard indicators of severely abnormal test results, and the patient’s test results were evaluated. When test results fell within critical abnormal ranges, the HRRS system sent alert messages within 5 min.
• The HRRS system
• The physician’s cell phone text messages and Internet e-mails • Intensive Care Unit (ICU)
Meeting medical care responsibilities
The medical care staffs were assisted by the HRRS system in monitoring the condition of critically ill patients.
• Primary care physicians
• Nurse stations • Department directors • Department secretaries
3. Conceptual model and research hypotheses 3.1.
Information system success model
The DeLone and McLean information system (IS) success model, based on theoretical and empirical IS research, has been employed by a number of researchers since the 1990s [15]. The model posited six major dimensions that can be used to evaluate information systems, including system quality, information quality, system use, user satisfaction, impact on the individual user and impact on the using organization. To measure e-commerce system success, DeLone and McLean later updated their original model, adding “service quality” and “net benefit” dimensions [15,18]. This updating of the model suggests the need to adopt appropriate variables when measuring IS success in different industries. Factors considered in assessing telehealth system success have included the system’s ability to provide: (1) a sound system infrastructure, (2) strong program management based on a needs analysis, and (3) applications that match identified
needs [19]. There is rarely only one factor used to evaluate the success of medical informatics systems and many different performance measures or success factors are commonly applied in evaluating them [20]. Unfortunately, one factor affecting the quality of information systems used in healthcare, the psychological impact on the healthcare professionals using them, is not covered in models previously used to assess system success.
3.2.
Mobile healthcare anxiety
Computer anxiety has been described as “an affective response, such that resistance to and avoidance of computer technology are a function of fear and apprehension, intimidation, hostility, and worries that one will be embarrassed, look stupid, or even damage the equipment” [21]. Wurman defines information anxiety as “the black hole between data and knowledge, and it happens when information does not tell us what we want or need to know” [22] (p. 34). Wilson describes information overload as “a personal problem, a perception on the part of the individual (or observers of that person) that the flow of information associated with work tasks is
Fig. 2 – Sample HRRS subsystem alert message sent via e-mail and cellular phone.
692
i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s 7 7 ( 2 0 0 8 ) 689–697
greater than can be managed effectively,” and organizational information overload as “a situation in which the extent of perceived information overload is sufficiently widespread within an organization so as to reduce the overall effectiveness of management operations” [23] (p. 113). In practice, delivering patients’ critical information to physicians via paper-based patient records or by telephone notifications may sometimes result in delays. The mobile patient safety information service provided by the HRRS system sends both e-mails and cell phone text messages to attending physicians 24 h a day. Included in the present study is an examination of how the instantaneous and uninterrupted character of the mobile patient safety information service might result in information overload or anxiety, mobile healthcare anxiety, and thus negatively affect the quality of care that physicians provide their patients. In this study, we define Mobile Healthcare Anxiety as a “highly anxious response towards interaction or anticipated interaction with the mobile patient safety information system.”
3.3.
As Fig. 2 illustrates, the HRRS was designed to deliver to physicians critical abnormal test results from Laboratory, Radiology, and Pathology departments. Recording patients’ test results, checking the test results, delivering alert messages and meeting medical care responsibilities are the four stages of the HRRS workflow. When abnormal test results were reported, the HRRS delivered an alert via cell phone text message to the attending physician. Unfortunately, because the HRRS message sent alert messages to attending physicians 24/7, each physician’s workload increased in proportion to the number of high-risk patients he or she had. Therefore, we add the hypothesis: H4-1. System Use, User Satisfaction and Mobile Healthcare Anxiety are positively associated with impact on the individual. Thus supplemented to include healthcare anxiety, the IS success model will produce a more comprehensive evaluation of the HRRS system.
Research model and hypotheses
The DeLone and McLean IS success model provided taxonomies of success variables. Based on the characteristics of the HRRS system, this study adds the mobile healthcare anxiety variable to measure the impact of the system on physician’s psychological well being. Following previous IS success literature, the study’s hypotheses are proposed (Fig. 3 presents the research model): H1. System Quality and Information Quality are positively associated with System Use. H2. System Quality and Information Quality are positively associated with User Satisfaction. H3. System Use is positively associated with User Satisfaction. H4. System Use and User Satisfaction are positively associated with impact on the individual. H5. Impact on the individual is positively associated with impact on the organization.
4.
Method
4.1.
Sample
The subjects of this study were W.F. Hospital physicians who had received HRRS cell phone text messages and Internet e-mails during the 8-month experimental period. Questionnaires were distributed to these physicians at their clinics during office hours between Monday and Friday. Although the physicians at W.F. Hospital represent internal medicine, surgery, pediatrics, gynecology and other departments, not all physicians received the HRRS messages frequently. Hence, those physicians who received fewer than 6 HRRS messages in 1 week were excluded from this study.
4.2.
Instrument
The questionnaire was composed of seven variables. All items for each variable were taken from previously validated instruments. In the system quality variable, the measured items were ease of use, user-friendliness, accessibility, and system response time [24,25]. In the information quality variable, the measured items were accuracy, legibility, and currency of information [26,27]. In the system use variable, the measured
Fig. 3 – The HRRS system success model.
693
i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s 7 7 ( 2 0 0 8 ) 689–697
items were motivation to use the system [28], use or nonuse of computer-based decision aids [29], and enlargement of the scope of patient services. The measured items in the user satisfaction variable were overall satisfaction, user attitude toward the information system, improvement of clinical communication and medical record keeping, and effect on decision making [29,30]. Included in the variable measuring impact on the individual were changed work practices, immediate benefits of system use, and job satisfaction [31,32]. The additional variable, mobile healthcare anxiety was composed based on the State-Trait Anxiety Inventory [33]. In this variable, the measured items were job stress increase, information overload anxiety, and quality of life. Finally, the variable measuring impact on the organization included communication efficiency and impact on patient care [32,34], as well as improvement in patient safety. Appendix A contains a summary of the instrument. The questionnaire consisted of two parts. The first part elicited respondent demographics, including age, gender, education, work experience, job title and medical specialization. The second part contained the questionnaire proper, which consisted of 23 statements to which respondents were asked to indicate their degree of agreement. All items employed a five point Likert-type scale with 1 being “strongly disagree,” 3 being neutral, and 5 being “strongly agree.” In order to assess the face validity of the questionnaire, the help of medical professionals and medical informatics researchers was enlisted. After discussion, the initial questionnaire was revised to ensure that there was a good match between the information the questionnaire elicited and the factors upon which the success of the system would be measured.
5.
Questionnaires were sent to the approximately 150 fulltime physicians at W.F. Hospital, and 95 questionnaires were returned. Of these 95 questionnaires, 18 were invalidated because the responding physician too rarely received HRRS notifications, and 5 were returned incomplete. This yielded a total of 72 usable questionnaires, a validation rate of 76%. Respondents of valid questionnaires were 93% male; 44% were between the ages of 30 and 50; 90% were primary physicians; 67% had working experience of more than 10 years, 22% had 5 to 10 years’ experience, and 11% had less than 5 years of working experience. The distribution of specialties was 7% for Family medicine, 3% for Ear, Nose, and Throat, 10% for Pediatrics, 10% for Gynecology, 7% for Orthopedics, 8% for Anesthesiology, 5% for Urology and Dermatology, 40% for Internal Medicine, and 5% for Surgery. Cronbach’s Alpha for the 7 variables in the questionnaire indicated good reliability, taking a reliability value of 0.7 as the minimal standard [36]. The mean, standard deviation, and Cronbach’s Alpha for the independent and dependent variables are listed in Table 2. All variables reported means of above 3.0. Pearson’s correlation coefficients, also shown in Table 2, indicated low and high existent correlations among variables examined for the HRRS system. Most variables were significantly related. Among the seven variables, information quality showed the strongest correlation with system use, user satisfaction, impact on the individual, mobile healthcare anxiety and impact on the organization. The results of the regression analysis are presented in Table 3.
5.1. 4.3.
Research findings
System quality and information quality
Data analysis
A review of the literature showed that prior studies in systems success used structural equation modeling (SEM), which is a regression-based technique rooted in path analysis. Structural equation modeling is often loosely termed a causal modeling technique [35] and requires a large sample size. Because the sample size in the present study was not large, however, linear regression, rather than SEM, was adopted to examine the relationships among the hypotheses. All analyses were carried out using the SPSS statistics program, version 11.0 (SPSS, Inc., Chicago, Illinois).
System quality and information quality (H1) did not behave as hypothesized. For system use, the regression model was significant (R2 = 0.552, p < 0.001). Of the two variables, only information quality was significant with a positive beta (ˇ = 0.720, p < 0.001). Hypothesis H1 was partially supported, as shown in Table 4. Like hypothesis H1, hypothesis H2 was partially supported, as shown in Tables 3 and 4. With regard to user satisfaction, the regression model was significant (R2 = 0.938, p < 0.001); information quality had a significant positive association (ˇ = 0.413, p < 0.01) with user satisfaction, while system quality had no significant association.
Table 2 – Correlation among dimension scores Dimensions
System quality Information quality System use User satisfaction Impact on the individual Mobile healthcare anxiety Impact on the organization ∗ ∗∗
p < 0.05. p < 0.01.
Mean
3.350 3.733 3.710 3.733 3.667 3.710 3.933
S.D.
0.507 0.458 0.414 0.399 0.463 0.634 0.405
Alpha
0.894 0.878 0.852 0.901 0.819 0.697 0.736
Correlation 1
2
3
4
5
6
7
1.00 0.626** 0.487** 0.502** 0.414* 0.255 0.258
1.00 0.743** 0.855** 0.722** 0.435* 0.582**
1.00 0.894** 0.699** 0.241 0.347
1.00 0.762** 0.318 0.420*
1.00 0.606** 0.552**
1.00 0.559**
1.00
694
i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s 7 7 ( 2 0 0 8 ) 689–697
Table 3 – Regression analysis table Independent variable
Dependent variable System use
Beta System quality Information quality System use User satisfaction Mobile healthcare anxiety Impact on the individual R2 Adjusted R2 F value
User satisfaction
0.036 0.720***
Impact on the individual
0.024 0.413** 0.575***
0.089 0.682*
Impact on the organization
0.179 0.467* 0.414** 0.552**
0.552 0.519 16.663***
0.938 0.880 63.815***
0.582 0.551 18.801***
0.734 0.704 23.958***
0.304 0.279 12.239**
∗
p < 0.05. p < 0.01. ∗∗∗ p < 0.001. ∗∗
5.2. System use, user satisfaction and mobile healthcare anxiety
5.3. Impact on the individual and impact on the organization
Hypothesis H3 was supported by the study, as is shown in Table 4. System use was positively associated with user satisfaction, as indicated by the positive beta (ˇ = 0.575, p < 0.001). Hypothesis H4 was partially supported, as shown in Table 4. For impact on the individual, the regression model was significant (R2 = 0.582, p < 0.001). Of the two variables, user satisfaction had a positive beta (ˇ = 0.682, p < 0.05), while system use had no significant association with impact on the individual. Hypothesis H4-1 was partially supported, as shown in Table 4. For impact on the individual, the regression model was significant (R2 = 0.734, p < 0.001). Of the three variables, only system use did not have a positive association with impact on the individual. User satisfaction had a positive beta (ˇ = 0.467, p < 0.05), and mobile healthcare anxiety had a positive beta (ˇ = 0.414, p < 0.01). Both user satisfaction and mobile healthcare anxiety were positively associated with impact on the individual.
Hypothesis 5 also behaved as hypothesized, as shown in Table 4. For impact on the organization, the regression model was significant (R2 = 0.304, p < 0.01). These results show that impact on the individual had a significantly positive association (ˇ = 0.552, p < 0.01) with impact on the organization.
Table 4 – Regression comparison analysis
*
p < 0.05; ** p < 0.01; and *** p < 0.001.
6.
Discussion
In discussing their Technology Acceptance Model (TAM), Wu et al. [37] pointed out that compatibility, perceived usefulness and perceived ease of use significantly affected healthcare professional behavioral intent in adopting a mobile healthcare information system. The DeLone and McLean information system success model (IS) is a multi-dimensional tool that can be used to assess an information system at various levels. In continuing the evolution of information system success model
i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s 7 7 ( 2 0 0 8 ) 689–697
use in the healthcare industry, the present study employed a mobile patient safety information system success model with a mobile healthcare anxiety. The empirical results provide considerable support for the model. The results show that, first, information quality is positively associated with system use and user satisfaction; second, system use is positively associated with user satisfaction; third, user satisfaction and mobile healthcare anxiety are positively associated with impact on the individual; and, finally, that impact on the individual is positively associated with impact on the organization. Information quality was significantly associated with both system use and user satisfaction, but system quality of the HRRS system did not have a significant association with system use or user satisfaction, results that are inconsistent with most prior MIS research [15]. That system quality is not significantly associated with user satisfaction might indicate that the respondents saw cellular phones and e-mail merely as a media source through which they receive patient data, and that system quality was either “yes” when it was working or “no” when their system was down, rather than a sliding scale of quality. We found that system use had a positive association with user satisfaction. This might be explained by the fact that the HRRS system contributed to physician satisfaction because the mobility of the HRRS system did help the physician to make timely medical decisions in emergent cases. With regard to the impact of HRRS system use on individual physicians, the results for H4 showed that only user satisfaction had a positive association. However, the results for H4-1 showed that both user satisfaction and mobile healthcare anxiety were positively associated with impact on the individual. It is clear from our study that mobile healthcare anxiety is one facet of the impact that the use of the HRRS has on individual physicians. When people feel anxiety using a new product or service, they generally rate that product or service negatively. However, even though they did suffer from mobile healthcare anxiety, the physicians in this study rated the HRRS system positively because it increased their efficiency by helping them to make timely medical decisions. From a practical perspective, physicians may feel exhausted if they continue to receive updates on emergent cases while they are off duty. Hence, hospitals should develop guidelines for HRRS use which minimize any negative impact system use might have on physicians and, indirectly, on their patients. Although previous studies have rarely discussed mobile service anxiety among healthcare professionals, it is an issue worthy of future discussion. Mobile patient safety information services should not become a burden to medical professionals’ health. Impact on the organization was significantly associated with impact on the individual. In general, most respondents agreed HRRS played a role in improving medical service quality. Lack of access to information during the decision-making process and ineffective communication among patient care team members were proximal causes of medical errors and other adverse events in patient care [38]. The traditional system of delivering critical test results and information about critical patients to physicians via paper-based patient records or by telephone notifications often resulted in delays, compromising patient safety. Employing the advantages of mobile
695
Summary points What was known before the study? • Laboratories commonly send important test results to both patients and physicians by paper-based record. The misplacement of test results compromises patient safety in clinical practice. • Laboratory, radiology, and pathology reports may have difficulty reaching patients, attending physicians or administrators immediately after detection of critical results. • Because the patient is the most important player in healthcare service and safety, most of the professional literature focuses on the benefits that patients derive from innovative mobile technology use. However, the well-being of the proximal healthcare professional, the physician, is critical to effective healthcare delivery, so how their work environment affects them should also be examined. • The success of information systems and information technology is often measured by the DeLone and McLean IS success model. What has the study added to the body of knowledge? • The augmented model is based on the DeLone and McLean IS success model and includes an additional mobile healthcare anxiety that delineates the attending physician’s dilemma of how to balance the additional efficiencies derived from HRRS system use against the increased anxiety that accompanies it. • The results indicated that information quality, system use, user satisfaction, mobile healthcare anxiety, impact on the individual, and impact on the organization are factors significantly associated with mobile patient safety service system success. Some implications and suggestions have been delineated in this study. • The study found that, while mobile patient safety information systems contribute to an improvement in patient services and a reduction in patient risk (through increased administrative efficiency), use of these communication systems may inadvertently contribute to physician anxiety. Continued research into how to relieve physician anxiety associated with mobile patient safety information system use is proposed.
technology and the Internet, the HRRS system helped physicians make fast, accurate and appropriate decisions, reducing patient risk and improving patient service. In the near future, the results of this study may lead to an expansion of HRRS use from laboratory, radiology, and pathology to emergency departments, where it could reduce ER patient risk and improve patient safety [39]. The HRRS system might even move out of the hospital and into private residences, where it might be utilized in patient health ser-
696
i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s 7 7 ( 2 0 0 8 ) 689–697
vice management for home-bound, chronically ill patients. Moreover, if technological advances proceed apace, HRRS service domains will soon let physicians query patient databases to view up-to-date documents and images for interactive diagnoses, and cellular phones may soon be integrated into PDAs with larger screens, web cams and high speed data transmission capability. That the HRRS system improves both administrative efficiency and the quality of healthcare that patients receive is no longer just rhetoric, but a growing reality that will continue to expand in the foreseeable future.
Acknowledgements The authors thank Dr. J. Talmon and the anonymous reviewers for their insightful suggestions. This study is most grateful to
Construct
System quality
Information quality
Items
Mobile healthcare anxiety
Impact on the individual
Appendix A. Summary of measures for the variables
Measure
SQ1
The operation menu of the HRRS is easy to use.
SQ2
The use of the HRRS is user-friendly.
SQ3
The HRRS system is stable.
SQ4
The response time of the HRRS is speedy.
IQ1
The information of the HRRS is accurate.
IQ2
The information of the HRRS is legible.
IQ3
The information of the HRRS is up-to-date.
SU1
I agree to continue to use the HRRS alert messages for high-risk patients.
SU2
The alert messages of the HRRS can help me to save high-risk patients.
SU3
I agree the scope of the HRRS alert message should be expanded.
US1
The use of the HRRS is helpful for my medical decision making.
US2
The HRRS improves clinical communication and medical record keeping.
US3
I keep a positive attitude toward the HRRS.
US4
Over all, the service of the HRRS makes me feel satisfied.
MH1
The HRRS increases my job strain.
MH2
The HRRS makes me feel information overload anxiety.
MH3
The HRRS affects the quality of my life.
II1
The HRRS changed my work practices.
II2
I experienced the immediate benefits of the HRRS.
II3
The HRRS improves my job satisfaction.
System use
User satisfaction
both the Patient Safety Institute staff and the physicians of W.F. Hospital. Your participation and help was essential to this study. Authors’ contributions: The first author is the primary author of this study. The second author participated in both questionnaire design and data acquisition, and his abundant experience in the practice of hospital management was of great help to the first author in drafting this paper. Competing interests: The authors declare that they have no competing interests or other interests that might be perceived to influence the results and/or discussion reported in this paper. The HRRS system was implemented by the Patient Safety Institute of W.F. Hospital and is published with the permission of W.F. Hospital.
OA1
The HRRS improves the communication efficiency of our hospital.
OA2
The HRRS increases the patient safety capability of our hospital.
OA3
The HRRS improves the quality of services to high-risk patients.
Impact on the organization
i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s 7 7 ( 2 0 0 8 ) 689–697
references
[1] E.A. Mendonc¸a, E.S. Chen, P.D. Stetson, L.K. McKnight, Approach to mobile information and communication for healthcare, Int. J. Med. Inform. 73 (2004) 631–638. [2] Y.C. Lu, Y. Xiao, A. Sears, J.A. Jacko, A review and a framework of handheld computer adoption in healthcare, Int. J. Med. Inform. 74 (2005) 409–422. [3] A. Krause, D. Hartl, F. Theis, M. Stangl, K.E. Gerauer, A.T. Mehlhorn, Mobile decision support for transplantation patient data, Int. J. Med. Inform. 73 (2004) 461–464. [4] J.R. Barrett, S.M. Strayer, J.R. Schubart, Assessing medical residents’ usage and perceived needs for personal digital assistants, Int. J. Med. Inform. 73 (2004) 25–34. [5] N. Samuli, H. Lamminen, K. Niemi, H. Mattila, S. Kalli, A cost study of new media supported near oral anticoagulant treatment follow-up, Int. J. Med. Inform. 70 (2003) 19–29. [6] N. Maglaveras, V. Koutkias, I. Chouvarda, D.G. Goulis, A. Avramides, D. Adamidis, G. Louridas, E.A. Balas, Home care delivery through the mobile telecommunications platform: the Citizen Health System (CHS) perspective, Int. J. Med. Inform. 68 (2002) 99–111. [7] U. Sax, I. Kohane, K.D. Mandl, Wireless technology infrastructures for authentication of patients: PKI that rings, J. Am. Med. Inform. Assoc. 12 (2005) 263–268. [8] C.A. Sneiderman, M.J. Ackerman, Cellular radio telecommunication for healthcare: benefits and risks, a brief review, J. Am. Med. Inform. Assoc. 11 (2005) 479–481. [9] H.J. Ten Duis, C. Van der Werken, Trauma care systems in the Netherlands, Injury 34 (9) (2003) 722–727. [10] E. Ammenwerth, S. Graber, G. Herrman, T. Burkle, J. Konig, Evaluation of health information systems-problems and challenges, Int. J. Med. Inform. 71 (2/3) (2003) 125–135. [11] R. Haux, Health information systems—past, present, future, Int. J. Med. Inform. 75 (3–4) (2006) 268–281. [12] M. Bang, A. Larsson, E. Berglund, H. Eriksson, Distributed user interfaces for clinical ubiquitous computing applications, Int. J. Med. Inform. 74 (7–8) (2005) 545–551. [13] Y.C. Lu, Y. Xiao, A. Sears, J.A. Jacko, A review and a framework of handheld computer adoption in healthcare, Int. J. Med. Inform. 74 (5) (2005) 409–422. [14] E. Ammenwerth, A. Buchauer, B. Bludau, R. Haux, Mobile information and communication tools in the hospital, Int. J. Med. Inform. 57 (1) (2000) 21–40. [15] W.H. DeLone, E.R. McLean, The DeLone and McLean model of information system success: a 10-year update, J. Manag. Inform. Syst. 19 (4) (2003) 9–30. [16] C.C. Chao, W.Y. Jen, Y.P. Chi, B. Lin, Improving patient safety with RFID and mobile technology, Int. J. Electron. Healthc. 3 (2) (2007) 175–192. [17] C.C. Chao, W.Y. Jen, M.C. Hung, Y.C. Li, Y.P. Chi, An innovative mobile approach for patient safety services. The case of a Taiwan healthcare provider, Technovation 27 (5) (2007) 268–279. [18] W.H. DeLone, E.R. McLean, Measuring e-commerce success: applying DeLone & McLean information systems success model, Int. J. Electron. Commun. 9 (1) (2004) 31–47.
697
[19] J.R. Moehra, J. Schaafsmab, C. Anglinc, S.V. Pantazia, N.A. Grimma, S. Anglinc, Success factors for telehealth—a case study, Int. J. Med. Inform. 75 (2006) 755–763. [20] P.Y.K. Chau, P.J.H. Hu, Investigating healthcare professionals’ decisions to accept telemedicine technology: an empirical test of competing theories, Inform. Manag. 39 (4) (2002) 297–311. [21] R.K. Heinssen, C.R. Glass, L.A. Knight, Assessing computer anxiety: development and validation of the computer anxiety rating scale, Comput. Hum. Behav. 3 (1987) 49–59. [22] R.S. Wurman, Information Anxiety, Doubleday, New York, 1989. [23] T.D. Wilson, Information overload: implication for healthcare service, Health Inform. 7 (2001) 112–117. [24] J.E. Bailey, S.W. Pearson, Development of a tool for measuring and analyzing computer user satisfaction, Manage. Sci. 29 (5) (1983) 530–545. [25] H. Barth, S. Huff, Change attitude to change, and decision support system success, Inform. Manag. 9 (5) (1985) 261–268. [26] W.R. King, B.J. Epstein, Assessing information system value, Decision Sci. 14 (1) (1983) 34–45. [27] A. Srinivasan, Alternative measures of system effectiveness: associations and implications, MIS Q 9 (3) (1985) 243–253. [28] G. DeSanctis, An examination of an expectancy theory model of decision support system use, in: Proceeding of the 3rd International Conference on Information system, 1982, pp. 121–135. [29] M. Alavi, J.C. Henderson, An evolutionary strategy for implementing a decision support system, Manage. Sci. 27 (11) (1981) 1309–1322. [30] B. Kaplan, H. Lundsgaarde, Toward an evaluation of an integrated clinical imaging system: identifying clinical benefits, Method Inform. Med. 35 (3) (1996) 221–229. [31] C. Sicotte, J. Denis, P. Lehoux, F. Champagne, The computer-based patient record challenges towards timeless and spaceless medical practice, J. Med. Syst. 22 (4) (1998) 237–256. [32] T. Ostbye, A. Moen, G. Erikssen, P. Hurlen, Introducing a module for laboratory test order entry and reporting of results at a hospital ward: an evaluation study using a multi-method approach, J. Med. Syst. 21 (2) (1997) 107–117. [33] C.D. Spielberger, Manual for the State-Trait Anxiety Inventory, Consulting Psychologists Press, Palo Alto, CA, 1983. [34] J.E. Allan, Patient-centered documentation: an effective and efficient use of clinical information systems, J. Nurs. Adm. 30 (2) (2000) 90–95. [35] W.W. Chin, P. Todd, On the use, usefulness, and ease of use of structural equation modeling in MIS Research: a note of caution, MIS Q 19 (2) (1995) 237–246. [36] J.C. Nunnally, Psychometric Theory, McGraw-Hill, New York, 1978. [37] J.H. Wu, S.C. Wang, L.M. Lin, Mobile computing acceptance factors in the healthcare industry: a structural equation model, Int. J. Med. Inform. 76 (1) (2007) 66–77. [38] C.M. Ruland, Improving patient safety through informatics tools for shared decision making and risk communication, Int. J. Med. Inform. 73 (7–8) (2004) 551–557. [39] M.N. Miller, R.K. McGowen, The painful truth: physicians are not invincible, South Med. J. 93 (10) (2000) 966–973.