- Email: [email protected]
Towards an evaluation framework for Laboratory Information Systems
JIPH-627; No. of Pages 8
ARTICLE IN PRESS
Journal of Infection and Public Health (2016) xxx, xxx—xxx
Towards an evaluation framework for Laboratory Information Systems Maryati M. Yusof ∗, Azila Arifin Centre for Software Technology & Management, Faculty of Information Science and Technology, Universiti Kebangsaan Malaysia, 43600 Bangi, Selangor, Malaysia Received 24 June 2016 ; received in revised form 20 July 2016; accepted 24 August 2016
KEYWORDS Error; Evaluation; Framework; Total Testing Process; Laboratory Information Systems
Summary Introduction: Laboratory testing and reporting are error-prone and redundant due to repeated, unnecessary requests and delayed or missed reactions to laboratory reports. Occurring errors may negatively affect the patient treatment process and clinical decision making. Evaluation on laboratory testing and Laboratory Information System (LIS) may explain the root cause to improve the testing process and enhance LIS in supporting the process. This paper discusses a new evaluation framework for LIS that encompasses the laboratory testing cycle and the socio-technical part of LIS. Methodology: Literature review on discourses, dimensions and evaluation methods of laboratory testing and LIS. A critical appraisal of the Total Testing Process (TTP) and the human, organization, technology-fit factors (HOT-fit) evaluation frameworks was undertaken in order to identify error incident, its contributing factors and preventive action pertinent to laboratory testing process and LIS. Result: A new evaluation framework for LIS using a comprehensive and sociotechnical approach is outlined. Positive relationship between laboratory and clinical staff resulted in a smooth laboratory testing process, reduced errors and increased process efficiency whilst effective use of LIS streamlined the testing processes. Conclusion: The TTP-LIS framework could serve as an assessment as well as a problem-solving tool for the laboratory testing process and system. © 2016 King Saud Bin Abdulaziz University for Health Sciences. Published by Elsevier Limited. All rights reserved.
Introduction ∗
Corresponding author. E-mail addresses: [email protected] (M.M. Yusof), [email protected] (A. Arifin).
Laboratory testing errors can happen at any stage of the testing process, from the pre-analytic
http://dx.doi.org/10.1016/j.jiph.2016.08.014 1876-0341/© 2016 King Saud Bin Abdulaziz University for Health Sciences. Published by Elsevier Limited. All rights reserved.
Please cite this article in press as: Yusof MM, Arifin A. Towards an evaluation framework for Laboratory Information Systems. J Infect Public Health (2016), http://dx.doi.org/10.1016/j.jiph.2016.08.014
JIPH-627; No. of Pages 8
ARTICLE IN PRESS
2
M.M. Yusof, A. Arifin
steps (for example, test selection and ordering, specimen collection) to post-analytic steps (for example, reporting and interpreting results, notifying patients) [1]. Errors in laboratory testing generally include unreasonable testing order (e.g.: additional test copy, duplicate test); wrong patient identification/specimen/labelling; unidentified failure in quality control; problems in handling, storing and transporting test sample; wrong validation of data analysis; and data entry error [1—5]. Various strategies have been used to reduce error and monitor workflow performance in laboratory including quality control programme and Information Systems/Technology [2,3]. The use of Health Information Systems (HIS), particularly Laboratory Information Systems (LIS) to validate, manage, deliver, process, and store data should reduce problems and ease process implementation in the laboratory testing workflow [6]. LIS facilitate smooth and fast interaction between medical practitioners and laboratory staff, specifically in ordering tests and delivering test reports [7—9]. However, numerous error factors related to LIS have also been reported including wrong data entry and access; poor system interface and reporting; limited system functionality; and incompetent users [1,10,11]. The involvement of multiple units in a workflow requires effective methods to monitor the task performance as the method would ensure process smoothness and ease error detection. The paper aims to discuss influencing factors for errors in laboratory testing processes and LIS as well as to present our proposed framework known as TTP-LIS which combined laboratory process and socio-technical factors. The framework makes use of the original Total Testing Process (TTP) framework and combines it with Human, Organization, Technology-fit (HOT-fit) framework [12]. Laboratory related errors are briefly described in this introduction section. Section two discusses the theoretical background of TTP and HOT-fit frameworks; the basis of our proposed framework. The third section illustrates the new framework whilst the discussion and conclusion are included in the last section.
Theoretical background LIS supports laboratory requirements [13] and integrates multiple laboratories [8]. However, the LIS role in preventing recurring error in laboratory testing process is still a work in progress. Platform heterogeneity in lab-clinical settings [14,15] which involve system development, software use, discrepancy in technology management and infor-
mation systems used in both settings, contribute to error incidents. In order to identify root causes, error incident in laboratory testing processes needs to be evaluated rigorously. Laboratory testing generally consists of nine steps: (1) test request; (2) sample collection; (3) sample labelling; (4) transportation of labelled sample to the lab; (5) preparation of raw specimen; (6) analysis of specimen testing; (7) interpretation of test results; (8) reporting of test interpretation; and (9) archiving of test results [16]. These steps are represented in a framework known as Total Testing Process (TTP); it can be used to evaluate laboratory process while the socio-technical aspects of LIS require another evaluation framework called the HOT-fit framework which takes a socio-technical approach to represent the interaction of social and technical in IS. The following section elaborates TTP and HOT-fit framework and their relationship that formed TTPLIS.
Total Testing Process (TTP) framework TTP is used as a basic guideline in the testing process of medical laboratories. It is a unique framework for analysing and minimising error risk not only in laboratory test centre but also in other clinical units [17]. TTP encompasses internal and external laboratory activities that comprise of one or more procedures and require interaction between internal and external laboratory staff. Failure in any TTP activity can affect patient care as doctors make decisions based on clinical results obtained from laboratory [18]. The original TTP framework was introduced by Lundberg [19], known as brain-to-brain loop concept (Fig. 1). The concept has been used by medical practitioners in conducting lab testing processes; from a triggered idea to testing patient samples to taking action in treating a patient. The simplified Lundberg concept in Fig. 1 illustrates the thoroughness of laboratory processes, from ordering tests to generating and utilising laboratory test results. The evidence of implementation effectiveness in each step indicates error reduction in patient care and treatment. TTP workflow in medical laboratories also focuses on process smoothness as smooth and systematic process yield to effective quality control. Process thoroughness based on productive and ethical work culture is critical in maintaining and improving workflow quality as it contributes to minimising error and subsequently ensuring patient safety [20]. The testing steps in Fig. 1 can be aggregated into five phases namely pre-pre-analytic, pre-analytic, analytic, post-analytic and post-post-analytic, as
Please cite this article in press as: Yusof MM, Arifin A. Towards an evaluation framework for Laboratory Information Systems. J Infect Public Health (2016), http://dx.doi.org/10.1016/j.jiph.2016.08.014
ARTICLE IN PRESS
JIPH-627; No. of Pages 8
Towards a framework for Laboratory Information System evaluation
3
Discussion for laboratory tests requirement
Laboratory tests selection
Laboratory test request
Specimen / patient identification
Analyiss of the laboratory test result
Laboratory test preparation
Specimen transportation
Specimen collection
Laboratory test results
Reports interpretation
Action / treatment
Figure 1 Brain-to-brain loop concept [19].
Pre-pre-analytical
Pre- analytical Analytical
Post-post- analytical
a wider definition for laboratory error. To ensure high quality laboratory service, error risk must be minimised, particularly before and after laboratory testing process [17].
Post- analytical
HOT-fit framework Figure 2 TTP workflow [3].
modelled in Hawkins [3] (Fig. 2). Most studies did not introduce the first and last phases in the TTP framework to classify activities based on the brainto-brain loop concept. Pre-pre-analytic phase takes place outside clinical laboratory and post-postanalytic phase involved activities within laboratory [17,21]. Phase based activity approach can be used to identify whether an error initiates before, during, or after the laboratory test [3]. Early error detection could potentially prevent the same error from recurring. The brain-to-brain loop concept and phase chain from related, disparate studies were combined and illustrated in Fig. 3. Each activity can be identified based on these phases; this structure eased error identification and classification as well as facilitated doctor, clinical specialist, and lab staff to perform and monitor lab activity smoothly and thoroughly [3]. Most studies on TTP identified error incidents that occur in all TTP phases; however, the first and last phase have the highest error percentage [3,22] due to the absence of monitoring on external laboratory processes. An error that occurs in TTP is regarded as a laboratory error although it happens outside the laboratory control. Conditions that contributed to those errors include poor communication; action taken by individual involved in laboratory testing process (doctor, nurse, and phlebotomists) such as role confusion; and ineffective process flow such as incomplete and redundant process steps. Therefore, the International Standardisation Organisation recommends
The HOT-fit evaluation framework [12,23] for HIS featured comprehensive dimensions and measures of technology, human, and organisation factors (Fig. 4). The adaption of two IS models in HOTfit framework, namely IS Success Model [24] and IT-Organization Fit Model [25] enables HOT-fit to become a comprehensive evaluation tool for various HIS, including LIS. The framework is comprised of nine interrelated dimensions, namely system quality (information processing quality), information quality (IS output), service quality (technical and service support), system development, system use, user satisfaction, organisational structure (related to management, strategy, organisation plan), organisation environment (related to politics, finance, inter organisation systems) and net benefits (overall IS impact). The fit concept between technology, human and organisation in the HOT-fit framework is complex, subjective, and abstract [12,23]. Based on its comprehensive dimension, HOT-fit is not only used to evaluate HIS performance, efficiency, and HIS impact. It could also guide error evaluation systematically according to process phase and level from the three factors.
The proposed TTP-LIS framework The proposed framework aims to provide better illustration of systematic, coordinated, and optimised laboratory testing process and LIS flow as well to facilitate a rigorous error evaluation. The evaluation dimension, process and their relationships are depicted in Fig. 5. The study focused only on pre-
Please cite this article in press as: Yusof MM, Arifin A. Towards an evaluation framework for Laboratory Information Systems. J Infect Public Health (2016), http://dx.doi.org/10.1016/j.jiph.2016.08.014
JIPH-627; No. of Pages 8
ARTICLE IN PRESS
4
M.M. Yusof, A. Arifin
Figure 3 Combination of brain-to-brain loop concept and TTP phase.
Figure 4 HOT-fit framework [12,26].
pre-analytic and post-post-analytic phases of the TTP framework. Pre-pre-analytic phase • The first two processes, namely ‘discussion of laboratory test requirement’ and ‘laboratory test selection’ involved a ‘human’ role that depends on knowledge, training, commitment, credibility, and patient condition. • A ‘Laboratory test order’ process is made by a doctor/medical specialist/nurse through ‘system use’ by entering the applicant information, patient information and specimen record. • The ‘Identify laboratory test’ process should be carefully performed by a doctor/nurse before laboratory test information and specimen are brought to the laboratory.
Pre-analytic phase • Laboratory test order information is accessed by laboratory staff through ‘system use’ for further action. • The ‘Identify/check laboratory test’ process is performed by laboratory staff involved in specimen collection, specimen identification creation using bar code, management of laboratory test use, and monitoring duration. Post-analytic phase • Laboratory staff enter laboratory test results through ‘system use’ based on a matching laboratory test code. Post-post-analytic phase
Please cite this article in press as: Yusof MM, Arifin A. Towards an evaluation framework for Laboratory Information Systems. J Infect Public Health (2016), http://dx.doi.org/10.1016/j.jiph.2016.08.014
JIPH-627; No. of Pages 8
ARTICLE IN PRESS
Towards a framework for Laboratory Information System evaluation
5
Figure 5 The proposed TTP-LIS framework.
• Doctor/medical specialist accesses laboratory test result from ‘system use’ and interpret the results in a report form. • Doctor reports laboratory test and uses it to determine further ‘treatment’ on patient. Human and technology category • ‘System quality’, ‘information quality’ and ‘service quality’ influence ‘system use’ and ‘user satisfaction’. • ‘System use’ and ‘information quality’ influence each other as the generation of laboratory test result, report, and image from system depends on user knowledge, skill, and training [23]. • Level of ‘information quality’ also influences ‘user satisfaction’ and vice versa. Feedbacks
from users pertinent to information quality should improve the level of information quality. • ‘System use’ and ‘user satisfaction’ influence each other. Effective ‘system use’ that include conditions such as LIS-task fit, low error rate, and user friendly interface could encourage users to optimise system use, which subsequently increases ‘user satisfaction’. • ‘System use’ and ‘user satisfaction’ result in direct or indirect ‘net benefits’ negatively or positively. Likewise, intensive ‘system use’ results in positive ‘net benefits’ while ineffective system use yields to negative ‘net benefits’. The enhancement in the TTP-LIS framework aims to facilitate basic laboratory test procedures; coor-
Please cite this article in press as: Yusof MM, Arifin A. Towards an evaluation framework for Laboratory Information Systems. J Infect Public Health (2016), http://dx.doi.org/10.1016/j.jiph.2016.08.014
JIPH-627; No. of Pages 8
ARTICLE IN PRESS
6
M.M. Yusof, A. Arifin
dinating the process with system use to reduce human and technological error as well as to produce smoother workflow. Therefore, evaluation of error incidents in laboratory test processes can be performed by analyzing factors related to human, organisation and technology—–such as the ease of system use and learning, system flexibility, relevant information, user attitude, planning, strategy, management and communication between doctor and laboratory staff.
Technology System quality is related to system performance and interface [6]. Elements of system quality in TTP-LIS measure system performance from system design, system function, communication, and synchronisation between systems in clinical and laboratory settings. In the system quality context, error incidents are contributed by system performance such as system development incompatibility, misfit of system function with task requirement, and poor log system and communication [7,8,27,28]. Incompatible system platform and software yield to disrupted system interaction. As a result, information and image are inaccessible and unreadable, causing difficulty for doctor to make decision on patient diagnosis or treatment. Poor system function is attributed to various factors, including management disputes and unclear/missing user requirements. A poor log system hindered monitoring user activity such as unauthorised system use or user negligence to log out. An automated telephone log is available in LIS to track and monitor laboratory test results that may have been discussed over the telephone [27]. High system quality is associated with its ease of use; for example, instant reference for system functions through tooltips. System training also helps user to become competent. System flexibility refers to the ability of a system to adapt to a work setting and integrate with other systems [23]. For example, a patient history, treatment plan or plan prepared by medical specialist are also made accessible to other specialists involved with the same patient case [29]. Information quality is measured from systems displays in various forms such as patient record, report, image, and prescription. Information quality is subjected to user perspective on information accuracy, completeness, consistency, and legibility [23]. Poor information quality can originated from users with limited information literacy, education, and communication skills.
Service quality is measured from responsiveness, assurance, empathy, and follow up service [12]. Most users think that management unit responses (responsiveness) to the requirements of system use and function are based on their own perspectives instead of those of user. A service provider is responsible for adhering to agreed system features and functions and assurance to support the requirement of user tasks. Personal and individual interests can foster in management lack of empathy in user requirements that are critical in supporting them to perform their daily tasks smoothly.
Human The potential of system impact and overall user experience in using systems is defined as user satisfaction [23]. User satisfaction influences the level of system use; it also affects patient treatment and organisational performance in the long run. Reduced error due to system use could increase user satisfaction. Users who receive good quality of service and information show higher satisfaction levels through increased system use. [30] System use could be viewed as a benchmark to assess service, system, and information quality [31—33]. Measures of system use include frequency of use, output information and volunteered or mandatory use [12,23]. Our main evaluation focus includes frequency, type, and number of error incident attributed to system function, relevant module, and frequency of system use.
Organization structure Clinical process is one of the measures in organisation structure in the HOT-fit framework and matches with TTP-LIS, renamed as laboratory test process. Human related error can occur intentionally or accidentally. The error usually needs to be corrected or mitigated through rigorous evaluation to avoid recurring incidents and adverse effects that demand time, cost, and manpower. Error evaluation can be identified and categorised according to knowledge or procedure. Knowledge is not only limited to health and medical discipline, it is also related to knowledge in information systems and technology, effective communication, and process flow. Any process, including a laboratory test, should follow specific procedures that can be categorised into good and bad states. A smooth process indicates the practicality of its procedure while a disrupted process shows the need for procedure improvement. A disrupted process is frequently associated with lack of user training or exposure. Knowledge related
Please cite this article in press as: Yusof MM, Arifin A. Towards an evaluation framework for Laboratory Information Systems. J Infect Public Health (2016), http://dx.doi.org/10.1016/j.jiph.2016.08.014
JIPH-627; No. of Pages 8
ARTICLE IN PRESS
Towards a framework for Laboratory Information System evaluation error generally occurs when user or medical practitioner faces a rare situation that requires an urgent solution. As a consequence, the problem is solved through reasoning or assumption and estimation that are prone to error risk due to limited knowledge sources, reliance on the current situation, and the use of individual intuition or hypothesis [34].
Net benefits LIS is seen as a system that eases laboratory tasks and facilitates communication between laboratory and clinical units to enable faster delivery of laboratory test orders and reports. Comprehensive system use in laboratory and clinical units lead to continuous system improvement as system weakness can be identified earlier. This would enable system developers to analyse problems that trigger the error occurrence. Error evaluation that involves process flow outside and within laboratories shows the importance of cooperation between laboratory and clinical units. System impacts on the initial and final phase of laboratory testing process can be evaluated in their instruction/procedure compliance, task performance, efficiency, effectiveness, accuracy, synchronisation (related to system development in terms of platform, software and tool), information access, decision quality, and time.
Discussion and conclusion Aligning IS use with clinical workflow to fulfil actual work reality in healthcare is a challenging task. The relationship between and combination of TTP and LIS formed our proposed framework, known as TTPLIS that aims to facilitate the evaluation of error incident for laboratory setting. We analysed the existing frameworks to identify their suitability in addressing error incidents related to TTP and LIS through their strengths and limitations and subsequently extend TTP to construct a new framework. Its comprehensive measures that encompass the overall socio-technical HOT-fit framework enabled a rigorous evaluation. The combination of factor and dimension in the HOT-fit and TTP models resulted in a comprehensive laboratory test process flow and HIS evaluation dimensions. LIS plays an important role in managing laboratory test process in clinical unit and laboratory. However, the misfit of technology with health organisation structure and clinical practice in laboratory testing process resulted in various errors. Therefore, continuous evaluation in the overall
7
laboratory testing process is crucial in addressing system problems and creating user awareness of system potentials and advantages to overall laboratory and clinical units. To validate its usefulness, TTP-LIS can be tested in clinical and laboratory settings, preferably with baseline data. Access to relevant documents such as incident report, error management report, and procedure process flow could be very useful in evaluating error incident factor. However, obtaining access to these documents may be challenging due to their sensitivity, privacy and confidentiality issues. Synergy and cooperation between clinical, laboratory, and IT, and support from management units, is required to improve laboratory testing process and LIS usefulness. Evaluation measures in TTP-LIS could be extended to evaluate factors that contributes to error in laboratory testing processes and LIS that are caused by external factors such as system incompatibility that affect LIS capabilities or other factor related to organisation management (latent failure).
Funding No funding sources.
Competing interests None declared.
Ethical approval Not required.
Acknowledgements We gratefully acknowledge the funding received from the Malaysia Exploratory Research Grant SchemeERGS/1/2011/STG/UKM/02/46 and the Japan Sumitomo Foundation Grant that helped sponsor this study. The project was scientifically supported by King Saud University, Deanship of Scientific research, research chairs and Research Chair of Informatics and Promotion.
References [1] Fernald D, Hamer M, James K, Tutt B, West D. Launching a laboratory testing process quality improvement
Please cite this article in press as: Yusof MM, Arifin A. Towards an evaluation framework for Laboratory Information Systems. J Infect Public Health (2016), http://dx.doi.org/10.1016/j.jiph.2016.08.014
JIPH-627; No. of Pages 8
ARTICLE IN PRESS
8
M.M. Yusof, A. Arifin
[2]
[3] [4] [5]
[6]
[7]
[8]
[9]
[10]
[11]
[12]
[13]
[14]
[15]
[16]
toolkit: from the Shared Networks of Colorado Ambulatory Practices and Partners (SNOCAP). J Am Board Fam Med 2015;28:576—83. Carraro P, Plebani M. Errors in a stat laboratory: types and frequencies 10 years later. Clin Chem: Lab Manag 2007;53:1338—42. Hawkins R. Managing the pre-and post-analytical phases of the total testing process. Ann Lab Med 2012;32:5—16. Plebani M, Piva E. Medical errors: pre-analytical issue in patient safety. J Med Biochem 2010;29:310—4. Plebani M. Errors in clinical laboratories or errors in laboratory medicine? Clin Chem Lab Med: CCLM/FESCC 2006;44:750—9. Yusof MM, Papazafeiropoulou A, Paul RJ, Stergioulas LK. Investigating evaluation frameworks for health information systems. Int J Med Inform 2008;77:377—85. Lichenstein R, O’Connell K, Funai T, Blumberg S, Shaw K, Ruddy R, et al. Laboratory errors in a Pediatric Emergency Department Network—–an analysis of incident reports. Pediatr Emerg Care Appl Res Netw 2015;00:1—5. Blaya JA, Shin SS, Yagui M, Contreras C, Cegielski P, Yale G, et al. Reducing communication delays and improving quality of care with a tuberculosis laboratory information system in resource poor environments: a cluster randomized controlled trial. Open Access J: PLoS One 2014;9:1—7. Kristensen GB, Moberg Aakre K, Kristoffersen AH, Sandberg S. How to conduct external quality assessment schemes for the pre-analytical phase? Biochem Med 2014;24:114—22. Takian A, Sheikh A, Barber N. We are bitter, but we are better off: case study of the implementation of an electronic health record system into a mental health hospital in England. BMC Health Serv Res 2012;12. Ash JS, Berg M, Coiera E. Some unintended consequences of information technology in health care: the nature of patient care information system-related errors. J Am Med Inform Assoc 2004;11:104—12. Yusof MM. A case study evaluation of a Critical Care Information System adoption using the socio-technical and fit approach. Int J Med Inform 2015;84:486—99. McGrowder D, Bishop R. An evaluation of laboratory information systems in medical laboratories in Jamaica. In: Moumtzoglou A, Kastania A, Archondakis S, editors. Laboratory management information systems: current requirements and future perspectives. Hershey, PA; 2015. Plebani M. Harmonization in laboratory medicine: the complete picture. Clin Chem Lab Med: CCLM/FESCC 2013;51:741—51. Plebani M, Panteghini M. Promoting clinical and laboratory interaction by harmonization. Clin Chim Acta: Int J Clin Chem 2013;432:15—21. Hopp WJ, Lovejoy WS, Myers J. Diagnostic services. In: Hopp WJ, Lovejoy WS, editors. Hospital operations: principles of high efficiency health care. New Jersey: Pearson Education; 2014. p. 339—462.
[17] Plebani M. The detection and prevention of errors in laboratory medicine. Ann Clin Biochem 2010;47:101—10. [18] O’Kane M. The reporting, classification and grading of quality failures in the medical laboratory. Clin Chim Acta 2009;404:28—31. [19] Lundberg GD. How clinicians should use the diagnostic laboratory in a changing medical world. Clin Chim Acta 1999:3—11. [20] Ellwood PM, Lundberg GD. Managed care: a work in progress. JAMA 1996;276:1083—6. [21] Laposata M, Dighe A. ‘‘Pre-pre’’ and ‘‘post-post’’ analytical error: high-incidence patient safety hazards involving the clinical laboratory. Clin Chem Lab Med 2007;45:712—9. [22] Deetz CO, Nolan DK, Scott MG. An examination of the usefulness of repeat testing practices in a large hospital clinical chemistry laboratory. Am J Clin Pathol 2012;137:20—5. [23] Yusof MM, Kuljis J, Papazafeiropoulou A, Stergioulas LK. An evaluation framework for health information systems: human, organization and technology-fit factors (HOT-fit). Int J Med Inform 2008:1—14. [24] Delone WH, McLean ER. The DeLone and McLean model of information systems success: a ten-year update. J Manag Inf Syst 2003;19:9—30. [25] Scott Morton MS. The corporation of the 1990. New York: Oxford University Press; 1991. [26] Yusof MM. HOT-fit evaluation framework: validation using case studies and qualitative systematic review in health information systems evaluation adoption. In: Proceedings of the 5rd European conference on information management and evaluation. 2011. p. 359—65. [27] Nguyen MT, Fuhrer P, Pasquier-Rocha J. Enhancing E-health information systems with agent technology. Int J Telemed Appl 2009:1—13. [28] Plebani M, Sciacovelli L, Aita A, Padoan A, Chiozza ML. Quality indicators to detect pre-analytical errors in laboratory testing. Clin Chim Acta: Int J Clin Chem 2013:1—5. [29] Becher EC, Chassin MR. Improving quality, minimizing error: making it happen. Health Aff 2001;3:68—81. [30] Markovi´ c S, Lonˇ cari´ c D, Lonˇ cari´ c D. Service quality and customer satisfaction in the health care industry—–towards health tourism market. Tour Hosp Manag 2014;20:155—70. [31] Antreas D, Opoulos A. Modeling customer satisfaction in telecommunication: assessing the multiple transaction points on perceived over all performance of the provider. Prod Oper Manag 2003;12:224—45. [32] Cronin Jr JJ, Taylor SA. Measuring service quality: a reexamination and extension. J Mark 1992;56:55—68. [33] Spreng RA, Mackoy RD. An empirical examination of a model of perceived service quality and satisfaction. J Retail 1996;72:201—14. [34] Reason J. Safety in the operating theatre—–Part 2: human error and organisational failure. Qual Saf Health Care 2005;14:56—60.
Available online at www.sciencedirect.com
ScienceDirect
Please cite this article in press as: Yusof MM, Arifin A. Towards an evaluation framework for Laboratory Information Systems. J Infect Public Health (2016), http://dx.doi.org/10.1016/j.jiph.2016.08.014
ARTICLE IN PRESS
Journal of Infection and Public Health (2016) xxx, xxx—xxx
Towards an evaluation framework for Laboratory Information Systems Maryati M. Yusof ∗, Azila Arifin Centre for Software Technology & Management, Faculty of Information Science and Technology, Universiti Kebangsaan Malaysia, 43600 Bangi, Selangor, Malaysia Received 24 June 2016 ; received in revised form 20 July 2016; accepted 24 August 2016
KEYWORDS Error; Evaluation; Framework; Total Testing Process; Laboratory Information Systems
Summary Introduction: Laboratory testing and reporting are error-prone and redundant due to repeated, unnecessary requests and delayed or missed reactions to laboratory reports. Occurring errors may negatively affect the patient treatment process and clinical decision making. Evaluation on laboratory testing and Laboratory Information System (LIS) may explain the root cause to improve the testing process and enhance LIS in supporting the process. This paper discusses a new evaluation framework for LIS that encompasses the laboratory testing cycle and the socio-technical part of LIS. Methodology: Literature review on discourses, dimensions and evaluation methods of laboratory testing and LIS. A critical appraisal of the Total Testing Process (TTP) and the human, organization, technology-fit factors (HOT-fit) evaluation frameworks was undertaken in order to identify error incident, its contributing factors and preventive action pertinent to laboratory testing process and LIS. Result: A new evaluation framework for LIS using a comprehensive and sociotechnical approach is outlined. Positive relationship between laboratory and clinical staff resulted in a smooth laboratory testing process, reduced errors and increased process efficiency whilst effective use of LIS streamlined the testing processes. Conclusion: The TTP-LIS framework could serve as an assessment as well as a problem-solving tool for the laboratory testing process and system. © 2016 King Saud Bin Abdulaziz University for Health Sciences. Published by Elsevier Limited. All rights reserved.
Introduction ∗
Corresponding author. E-mail addresses: [email protected] (M.M. Yusof), [email protected] (A. Arifin).
Laboratory testing errors can happen at any stage of the testing process, from the pre-analytic
http://dx.doi.org/10.1016/j.jiph.2016.08.014 1876-0341/© 2016 King Saud Bin Abdulaziz University for Health Sciences. Published by Elsevier Limited. All rights reserved.
Please cite this article in press as: Yusof MM, Arifin A. Towards an evaluation framework for Laboratory Information Systems. J Infect Public Health (2016), http://dx.doi.org/10.1016/j.jiph.2016.08.014
JIPH-627; No. of Pages 8
ARTICLE IN PRESS
2
M.M. Yusof, A. Arifin
steps (for example, test selection and ordering, specimen collection) to post-analytic steps (for example, reporting and interpreting results, notifying patients) [1]. Errors in laboratory testing generally include unreasonable testing order (e.g.: additional test copy, duplicate test); wrong patient identification/specimen/labelling; unidentified failure in quality control; problems in handling, storing and transporting test sample; wrong validation of data analysis; and data entry error [1—5]. Various strategies have been used to reduce error and monitor workflow performance in laboratory including quality control programme and Information Systems/Technology [2,3]. The use of Health Information Systems (HIS), particularly Laboratory Information Systems (LIS) to validate, manage, deliver, process, and store data should reduce problems and ease process implementation in the laboratory testing workflow [6]. LIS facilitate smooth and fast interaction between medical practitioners and laboratory staff, specifically in ordering tests and delivering test reports [7—9]. However, numerous error factors related to LIS have also been reported including wrong data entry and access; poor system interface and reporting; limited system functionality; and incompetent users [1,10,11]. The involvement of multiple units in a workflow requires effective methods to monitor the task performance as the method would ensure process smoothness and ease error detection. The paper aims to discuss influencing factors for errors in laboratory testing processes and LIS as well as to present our proposed framework known as TTP-LIS which combined laboratory process and socio-technical factors. The framework makes use of the original Total Testing Process (TTP) framework and combines it with Human, Organization, Technology-fit (HOT-fit) framework [12]. Laboratory related errors are briefly described in this introduction section. Section two discusses the theoretical background of TTP and HOT-fit frameworks; the basis of our proposed framework. The third section illustrates the new framework whilst the discussion and conclusion are included in the last section.
Theoretical background LIS supports laboratory requirements [13] and integrates multiple laboratories [8]. However, the LIS role in preventing recurring error in laboratory testing process is still a work in progress. Platform heterogeneity in lab-clinical settings [14,15] which involve system development, software use, discrepancy in technology management and infor-
mation systems used in both settings, contribute to error incidents. In order to identify root causes, error incident in laboratory testing processes needs to be evaluated rigorously. Laboratory testing generally consists of nine steps: (1) test request; (2) sample collection; (3) sample labelling; (4) transportation of labelled sample to the lab; (5) preparation of raw specimen; (6) analysis of specimen testing; (7) interpretation of test results; (8) reporting of test interpretation; and (9) archiving of test results [16]. These steps are represented in a framework known as Total Testing Process (TTP); it can be used to evaluate laboratory process while the socio-technical aspects of LIS require another evaluation framework called the HOT-fit framework which takes a socio-technical approach to represent the interaction of social and technical in IS. The following section elaborates TTP and HOT-fit framework and their relationship that formed TTPLIS.
Total Testing Process (TTP) framework TTP is used as a basic guideline in the testing process of medical laboratories. It is a unique framework for analysing and minimising error risk not only in laboratory test centre but also in other clinical units [17]. TTP encompasses internal and external laboratory activities that comprise of one or more procedures and require interaction between internal and external laboratory staff. Failure in any TTP activity can affect patient care as doctors make decisions based on clinical results obtained from laboratory [18]. The original TTP framework was introduced by Lundberg [19], known as brain-to-brain loop concept (Fig. 1). The concept has been used by medical practitioners in conducting lab testing processes; from a triggered idea to testing patient samples to taking action in treating a patient. The simplified Lundberg concept in Fig. 1 illustrates the thoroughness of laboratory processes, from ordering tests to generating and utilising laboratory test results. The evidence of implementation effectiveness in each step indicates error reduction in patient care and treatment. TTP workflow in medical laboratories also focuses on process smoothness as smooth and systematic process yield to effective quality control. Process thoroughness based on productive and ethical work culture is critical in maintaining and improving workflow quality as it contributes to minimising error and subsequently ensuring patient safety [20]. The testing steps in Fig. 1 can be aggregated into five phases namely pre-pre-analytic, pre-analytic, analytic, post-analytic and post-post-analytic, as
Please cite this article in press as: Yusof MM, Arifin A. Towards an evaluation framework for Laboratory Information Systems. J Infect Public Health (2016), http://dx.doi.org/10.1016/j.jiph.2016.08.014
ARTICLE IN PRESS
JIPH-627; No. of Pages 8
Towards a framework for Laboratory Information System evaluation
3
Discussion for laboratory tests requirement
Laboratory tests selection
Laboratory test request
Specimen / patient identification
Analyiss of the laboratory test result
Laboratory test preparation
Specimen transportation
Specimen collection
Laboratory test results
Reports interpretation
Action / treatment
Figure 1 Brain-to-brain loop concept [19].
Pre-pre-analytical
Pre- analytical Analytical
Post-post- analytical
a wider definition for laboratory error. To ensure high quality laboratory service, error risk must be minimised, particularly before and after laboratory testing process [17].
Post- analytical
HOT-fit framework Figure 2 TTP workflow [3].
modelled in Hawkins [3] (Fig. 2). Most studies did not introduce the first and last phases in the TTP framework to classify activities based on the brainto-brain loop concept. Pre-pre-analytic phase takes place outside clinical laboratory and post-postanalytic phase involved activities within laboratory [17,21]. Phase based activity approach can be used to identify whether an error initiates before, during, or after the laboratory test [3]. Early error detection could potentially prevent the same error from recurring. The brain-to-brain loop concept and phase chain from related, disparate studies were combined and illustrated in Fig. 3. Each activity can be identified based on these phases; this structure eased error identification and classification as well as facilitated doctor, clinical specialist, and lab staff to perform and monitor lab activity smoothly and thoroughly [3]. Most studies on TTP identified error incidents that occur in all TTP phases; however, the first and last phase have the highest error percentage [3,22] due to the absence of monitoring on external laboratory processes. An error that occurs in TTP is regarded as a laboratory error although it happens outside the laboratory control. Conditions that contributed to those errors include poor communication; action taken by individual involved in laboratory testing process (doctor, nurse, and phlebotomists) such as role confusion; and ineffective process flow such as incomplete and redundant process steps. Therefore, the International Standardisation Organisation recommends
The HOT-fit evaluation framework [12,23] for HIS featured comprehensive dimensions and measures of technology, human, and organisation factors (Fig. 4). The adaption of two IS models in HOTfit framework, namely IS Success Model [24] and IT-Organization Fit Model [25] enables HOT-fit to become a comprehensive evaluation tool for various HIS, including LIS. The framework is comprised of nine interrelated dimensions, namely system quality (information processing quality), information quality (IS output), service quality (technical and service support), system development, system use, user satisfaction, organisational structure (related to management, strategy, organisation plan), organisation environment (related to politics, finance, inter organisation systems) and net benefits (overall IS impact). The fit concept between technology, human and organisation in the HOT-fit framework is complex, subjective, and abstract [12,23]. Based on its comprehensive dimension, HOT-fit is not only used to evaluate HIS performance, efficiency, and HIS impact. It could also guide error evaluation systematically according to process phase and level from the three factors.
The proposed TTP-LIS framework The proposed framework aims to provide better illustration of systematic, coordinated, and optimised laboratory testing process and LIS flow as well to facilitate a rigorous error evaluation. The evaluation dimension, process and their relationships are depicted in Fig. 5. The study focused only on pre-
Please cite this article in press as: Yusof MM, Arifin A. Towards an evaluation framework for Laboratory Information Systems. J Infect Public Health (2016), http://dx.doi.org/10.1016/j.jiph.2016.08.014
JIPH-627; No. of Pages 8
ARTICLE IN PRESS
4
M.M. Yusof, A. Arifin
Figure 3 Combination of brain-to-brain loop concept and TTP phase.
Figure 4 HOT-fit framework [12,26].
pre-analytic and post-post-analytic phases of the TTP framework. Pre-pre-analytic phase • The first two processes, namely ‘discussion of laboratory test requirement’ and ‘laboratory test selection’ involved a ‘human’ role that depends on knowledge, training, commitment, credibility, and patient condition. • A ‘Laboratory test order’ process is made by a doctor/medical specialist/nurse through ‘system use’ by entering the applicant information, patient information and specimen record. • The ‘Identify laboratory test’ process should be carefully performed by a doctor/nurse before laboratory test information and specimen are brought to the laboratory.
Pre-analytic phase • Laboratory test order information is accessed by laboratory staff through ‘system use’ for further action. • The ‘Identify/check laboratory test’ process is performed by laboratory staff involved in specimen collection, specimen identification creation using bar code, management of laboratory test use, and monitoring duration. Post-analytic phase • Laboratory staff enter laboratory test results through ‘system use’ based on a matching laboratory test code. Post-post-analytic phase
Please cite this article in press as: Yusof MM, Arifin A. Towards an evaluation framework for Laboratory Information Systems. J Infect Public Health (2016), http://dx.doi.org/10.1016/j.jiph.2016.08.014
JIPH-627; No. of Pages 8
ARTICLE IN PRESS
Towards a framework for Laboratory Information System evaluation
5
Figure 5 The proposed TTP-LIS framework.
• Doctor/medical specialist accesses laboratory test result from ‘system use’ and interpret the results in a report form. • Doctor reports laboratory test and uses it to determine further ‘treatment’ on patient. Human and technology category • ‘System quality’, ‘information quality’ and ‘service quality’ influence ‘system use’ and ‘user satisfaction’. • ‘System use’ and ‘information quality’ influence each other as the generation of laboratory test result, report, and image from system depends on user knowledge, skill, and training [23]. • Level of ‘information quality’ also influences ‘user satisfaction’ and vice versa. Feedbacks
from users pertinent to information quality should improve the level of information quality. • ‘System use’ and ‘user satisfaction’ influence each other. Effective ‘system use’ that include conditions such as LIS-task fit, low error rate, and user friendly interface could encourage users to optimise system use, which subsequently increases ‘user satisfaction’. • ‘System use’ and ‘user satisfaction’ result in direct or indirect ‘net benefits’ negatively or positively. Likewise, intensive ‘system use’ results in positive ‘net benefits’ while ineffective system use yields to negative ‘net benefits’. The enhancement in the TTP-LIS framework aims to facilitate basic laboratory test procedures; coor-
Please cite this article in press as: Yusof MM, Arifin A. Towards an evaluation framework for Laboratory Information Systems. J Infect Public Health (2016), http://dx.doi.org/10.1016/j.jiph.2016.08.014
JIPH-627; No. of Pages 8
ARTICLE IN PRESS
6
M.M. Yusof, A. Arifin
dinating the process with system use to reduce human and technological error as well as to produce smoother workflow. Therefore, evaluation of error incidents in laboratory test processes can be performed by analyzing factors related to human, organisation and technology—–such as the ease of system use and learning, system flexibility, relevant information, user attitude, planning, strategy, management and communication between doctor and laboratory staff.
Technology System quality is related to system performance and interface [6]. Elements of system quality in TTP-LIS measure system performance from system design, system function, communication, and synchronisation between systems in clinical and laboratory settings. In the system quality context, error incidents are contributed by system performance such as system development incompatibility, misfit of system function with task requirement, and poor log system and communication [7,8,27,28]. Incompatible system platform and software yield to disrupted system interaction. As a result, information and image are inaccessible and unreadable, causing difficulty for doctor to make decision on patient diagnosis or treatment. Poor system function is attributed to various factors, including management disputes and unclear/missing user requirements. A poor log system hindered monitoring user activity such as unauthorised system use or user negligence to log out. An automated telephone log is available in LIS to track and monitor laboratory test results that may have been discussed over the telephone [27]. High system quality is associated with its ease of use; for example, instant reference for system functions through tooltips. System training also helps user to become competent. System flexibility refers to the ability of a system to adapt to a work setting and integrate with other systems [23]. For example, a patient history, treatment plan or plan prepared by medical specialist are also made accessible to other specialists involved with the same patient case [29]. Information quality is measured from systems displays in various forms such as patient record, report, image, and prescription. Information quality is subjected to user perspective on information accuracy, completeness, consistency, and legibility [23]. Poor information quality can originated from users with limited information literacy, education, and communication skills.
Service quality is measured from responsiveness, assurance, empathy, and follow up service [12]. Most users think that management unit responses (responsiveness) to the requirements of system use and function are based on their own perspectives instead of those of user. A service provider is responsible for adhering to agreed system features and functions and assurance to support the requirement of user tasks. Personal and individual interests can foster in management lack of empathy in user requirements that are critical in supporting them to perform their daily tasks smoothly.
Human The potential of system impact and overall user experience in using systems is defined as user satisfaction [23]. User satisfaction influences the level of system use; it also affects patient treatment and organisational performance in the long run. Reduced error due to system use could increase user satisfaction. Users who receive good quality of service and information show higher satisfaction levels through increased system use. [30] System use could be viewed as a benchmark to assess service, system, and information quality [31—33]. Measures of system use include frequency of use, output information and volunteered or mandatory use [12,23]. Our main evaluation focus includes frequency, type, and number of error incident attributed to system function, relevant module, and frequency of system use.
Organization structure Clinical process is one of the measures in organisation structure in the HOT-fit framework and matches with TTP-LIS, renamed as laboratory test process. Human related error can occur intentionally or accidentally. The error usually needs to be corrected or mitigated through rigorous evaluation to avoid recurring incidents and adverse effects that demand time, cost, and manpower. Error evaluation can be identified and categorised according to knowledge or procedure. Knowledge is not only limited to health and medical discipline, it is also related to knowledge in information systems and technology, effective communication, and process flow. Any process, including a laboratory test, should follow specific procedures that can be categorised into good and bad states. A smooth process indicates the practicality of its procedure while a disrupted process shows the need for procedure improvement. A disrupted process is frequently associated with lack of user training or exposure. Knowledge related
Please cite this article in press as: Yusof MM, Arifin A. Towards an evaluation framework for Laboratory Information Systems. J Infect Public Health (2016), http://dx.doi.org/10.1016/j.jiph.2016.08.014
JIPH-627; No. of Pages 8
ARTICLE IN PRESS
Towards a framework for Laboratory Information System evaluation error generally occurs when user or medical practitioner faces a rare situation that requires an urgent solution. As a consequence, the problem is solved through reasoning or assumption and estimation that are prone to error risk due to limited knowledge sources, reliance on the current situation, and the use of individual intuition or hypothesis [34].
Net benefits LIS is seen as a system that eases laboratory tasks and facilitates communication between laboratory and clinical units to enable faster delivery of laboratory test orders and reports. Comprehensive system use in laboratory and clinical units lead to continuous system improvement as system weakness can be identified earlier. This would enable system developers to analyse problems that trigger the error occurrence. Error evaluation that involves process flow outside and within laboratories shows the importance of cooperation between laboratory and clinical units. System impacts on the initial and final phase of laboratory testing process can be evaluated in their instruction/procedure compliance, task performance, efficiency, effectiveness, accuracy, synchronisation (related to system development in terms of platform, software and tool), information access, decision quality, and time.
Discussion and conclusion Aligning IS use with clinical workflow to fulfil actual work reality in healthcare is a challenging task. The relationship between and combination of TTP and LIS formed our proposed framework, known as TTPLIS that aims to facilitate the evaluation of error incident for laboratory setting. We analysed the existing frameworks to identify their suitability in addressing error incidents related to TTP and LIS through their strengths and limitations and subsequently extend TTP to construct a new framework. Its comprehensive measures that encompass the overall socio-technical HOT-fit framework enabled a rigorous evaluation. The combination of factor and dimension in the HOT-fit and TTP models resulted in a comprehensive laboratory test process flow and HIS evaluation dimensions. LIS plays an important role in managing laboratory test process in clinical unit and laboratory. However, the misfit of technology with health organisation structure and clinical practice in laboratory testing process resulted in various errors. Therefore, continuous evaluation in the overall
7
laboratory testing process is crucial in addressing system problems and creating user awareness of system potentials and advantages to overall laboratory and clinical units. To validate its usefulness, TTP-LIS can be tested in clinical and laboratory settings, preferably with baseline data. Access to relevant documents such as incident report, error management report, and procedure process flow could be very useful in evaluating error incident factor. However, obtaining access to these documents may be challenging due to their sensitivity, privacy and confidentiality issues. Synergy and cooperation between clinical, laboratory, and IT, and support from management units, is required to improve laboratory testing process and LIS usefulness. Evaluation measures in TTP-LIS could be extended to evaluate factors that contributes to error in laboratory testing processes and LIS that are caused by external factors such as system incompatibility that affect LIS capabilities or other factor related to organisation management (latent failure).
Funding No funding sources.
Competing interests None declared.
Ethical approval Not required.
Acknowledgements We gratefully acknowledge the funding received from the Malaysia Exploratory Research Grant SchemeERGS/1/2011/STG/UKM/02/46 and the Japan Sumitomo Foundation Grant that helped sponsor this study. The project was scientifically supported by King Saud University, Deanship of Scientific research, research chairs and Research Chair of Informatics and Promotion.
References [1] Fernald D, Hamer M, James K, Tutt B, West D. Launching a laboratory testing process quality improvement
Please cite this article in press as: Yusof MM, Arifin A. Towards an evaluation framework for Laboratory Information Systems. J Infect Public Health (2016), http://dx.doi.org/10.1016/j.jiph.2016.08.014
JIPH-627; No. of Pages 8
ARTICLE IN PRESS
8
M.M. Yusof, A. Arifin
[2]
[3] [4] [5]
[6]
[7]
[8]
[9]
[10]
[11]
[12]
[13]
[14]
[15]
[16]
toolkit: from the Shared Networks of Colorado Ambulatory Practices and Partners (SNOCAP). J Am Board Fam Med 2015;28:576—83. Carraro P, Plebani M. Errors in a stat laboratory: types and frequencies 10 years later. Clin Chem: Lab Manag 2007;53:1338—42. Hawkins R. Managing the pre-and post-analytical phases of the total testing process. Ann Lab Med 2012;32:5—16. Plebani M, Piva E. Medical errors: pre-analytical issue in patient safety. J Med Biochem 2010;29:310—4. Plebani M. Errors in clinical laboratories or errors in laboratory medicine? Clin Chem Lab Med: CCLM/FESCC 2006;44:750—9. Yusof MM, Papazafeiropoulou A, Paul RJ, Stergioulas LK. Investigating evaluation frameworks for health information systems. Int J Med Inform 2008;77:377—85. Lichenstein R, O’Connell K, Funai T, Blumberg S, Shaw K, Ruddy R, et al. Laboratory errors in a Pediatric Emergency Department Network—–an analysis of incident reports. Pediatr Emerg Care Appl Res Netw 2015;00:1—5. Blaya JA, Shin SS, Yagui M, Contreras C, Cegielski P, Yale G, et al. Reducing communication delays and improving quality of care with a tuberculosis laboratory information system in resource poor environments: a cluster randomized controlled trial. Open Access J: PLoS One 2014;9:1—7. Kristensen GB, Moberg Aakre K, Kristoffersen AH, Sandberg S. How to conduct external quality assessment schemes for the pre-analytical phase? Biochem Med 2014;24:114—22. Takian A, Sheikh A, Barber N. We are bitter, but we are better off: case study of the implementation of an electronic health record system into a mental health hospital in England. BMC Health Serv Res 2012;12. Ash JS, Berg M, Coiera E. Some unintended consequences of information technology in health care: the nature of patient care information system-related errors. J Am Med Inform Assoc 2004;11:104—12. Yusof MM. A case study evaluation of a Critical Care Information System adoption using the socio-technical and fit approach. Int J Med Inform 2015;84:486—99. McGrowder D, Bishop R. An evaluation of laboratory information systems in medical laboratories in Jamaica. In: Moumtzoglou A, Kastania A, Archondakis S, editors. Laboratory management information systems: current requirements and future perspectives. Hershey, PA; 2015. Plebani M. Harmonization in laboratory medicine: the complete picture. Clin Chem Lab Med: CCLM/FESCC 2013;51:741—51. Plebani M, Panteghini M. Promoting clinical and laboratory interaction by harmonization. Clin Chim Acta: Int J Clin Chem 2013;432:15—21. Hopp WJ, Lovejoy WS, Myers J. Diagnostic services. In: Hopp WJ, Lovejoy WS, editors. Hospital operations: principles of high efficiency health care. New Jersey: Pearson Education; 2014. p. 339—462.
[17] Plebani M. The detection and prevention of errors in laboratory medicine. Ann Clin Biochem 2010;47:101—10. [18] O’Kane M. The reporting, classification and grading of quality failures in the medical laboratory. Clin Chim Acta 2009;404:28—31. [19] Lundberg GD. How clinicians should use the diagnostic laboratory in a changing medical world. Clin Chim Acta 1999:3—11. [20] Ellwood PM, Lundberg GD. Managed care: a work in progress. JAMA 1996;276:1083—6. [21] Laposata M, Dighe A. ‘‘Pre-pre’’ and ‘‘post-post’’ analytical error: high-incidence patient safety hazards involving the clinical laboratory. Clin Chem Lab Med 2007;45:712—9. [22] Deetz CO, Nolan DK, Scott MG. An examination of the usefulness of repeat testing practices in a large hospital clinical chemistry laboratory. Am J Clin Pathol 2012;137:20—5. [23] Yusof MM, Kuljis J, Papazafeiropoulou A, Stergioulas LK. An evaluation framework for health information systems: human, organization and technology-fit factors (HOT-fit). Int J Med Inform 2008:1—14. [24] Delone WH, McLean ER. The DeLone and McLean model of information systems success: a ten-year update. J Manag Inf Syst 2003;19:9—30. [25] Scott Morton MS. The corporation of the 1990. New York: Oxford University Press; 1991. [26] Yusof MM. HOT-fit evaluation framework: validation using case studies and qualitative systematic review in health information systems evaluation adoption. In: Proceedings of the 5rd European conference on information management and evaluation. 2011. p. 359—65. [27] Nguyen MT, Fuhrer P, Pasquier-Rocha J. Enhancing E-health information systems with agent technology. Int J Telemed Appl 2009:1—13. [28] Plebani M, Sciacovelli L, Aita A, Padoan A, Chiozza ML. Quality indicators to detect pre-analytical errors in laboratory testing. Clin Chim Acta: Int J Clin Chem 2013:1—5. [29] Becher EC, Chassin MR. Improving quality, minimizing error: making it happen. Health Aff 2001;3:68—81. [30] Markovi´ c S, Lonˇ cari´ c D, Lonˇ cari´ c D. Service quality and customer satisfaction in the health care industry—–towards health tourism market. Tour Hosp Manag 2014;20:155—70. [31] Antreas D, Opoulos A. Modeling customer satisfaction in telecommunication: assessing the multiple transaction points on perceived over all performance of the provider. Prod Oper Manag 2003;12:224—45. [32] Cronin Jr JJ, Taylor SA. Measuring service quality: a reexamination and extension. J Mark 1992;56:55—68. [33] Spreng RA, Mackoy RD. An empirical examination of a model of perceived service quality and satisfaction. J Retail 1996;72:201—14. [34] Reason J. Safety in the operating theatre—–Part 2: human error and organisational failure. Qual Saf Health Care 2005;14:56—60.
Available online at www.sciencedirect.com
ScienceDirect
Please cite this article in press as: Yusof MM, Arifin A. Towards an evaluation framework for Laboratory Information Systems. J Infect Public Health (2016), http://dx.doi.org/10.1016/j.jiph.2016.08.014