- Email: [email protected]
Improving Electronic Medical Records with Support of Human Computer Interaction in Medical Information Systems
ScienceDirect
Available online at www.sciencedirect.com Procedia Computer Science 00 (2017) 000–000
ScienceDirect
www.elsevier.com/locate/procedia
Available online at www.sciencedirect.com Procedia Computer Science 00 (2017) 000–000
ScienceDirect
www.elsevier.com/locate/procedia
Procedia Computer Science 121 (2017) 469–474 CENTERIS - International Conference on ENTERprise Information Systems / ProjMAN International Conference on Project MANagement / HCist - International Conference on Health and Social Care Information Systems and Technologies, CENTERIS / ProjMAN / HCist 2017, 8-10 CENTERIS - International November Conference2017, on ENTERprise Information Systems / ProjMAN Barcelona, Spain International Conference on Project MANagement / HCist - International Conference on Health and Social Care Information Systems and Technologies, CENTERIS / ProjMAN / HCist 2017, 8-10 Improving Electronic Medical November 2017,Records Barcelona, with Spain Support of Human
Computer Interaction in Medical Information Systems Improving Electronic Medical Records with Support of Human Ekaterina V. Bolgvaa, Nadezhda E. Zvartaua,b,Sergey V. Kovalchuka, Marina A. Computer Interaction in Medical Information Systems Balakhontceva a,*, Oleg G. Metskera a University, 49 Kronverksky Pr., St. a ITMO Russia Ekaterina V. Bolgva , Nadezhda E. Zvartaua,bPetersburg, ,Sergey197101, V. Kovalchuk , Marina A. Federal Almazov North-West Medical Research Centre, 2 Akkuratova street, St. Petersburg 197341, Russia. a, a Balakhontceva *, Oleg G. Metsker a
b
ITMO University, 49 Kronverksky Pr., St. Petersburg, 197101, Russia Federal Almazov North-West Medical Research Centre, 2 Akkuratova street, St. Petersburg 197341, Russia. a
Abstract
b
This study investigated the most common challenges of human computer interaction (HCI) while using electronic medical records (EMR) based on the experience of a large Russian medical research center. The paper presents the results of testing DSS Abstract implemented in the mode of an additional interface with the EMR. The percentage of erroneous data for two groups of users (with and notifications) is presented the entire periodcomputer of the experiment thewhile weekly changes. The This without study investigated the most common for challenges of human interaction and (HCI) usingdynamics electronicofmedical records implementation CDSS in the supplemented interfacemedical mode ofresearch the maincenter. MIS has hadpaper a positive effect reducing errors in (EMR) based onof the experience of a large Russian The presents thein results of user testing DSS the data. The results of theofusers survey areinterface presented, a satisfactory evaluation of the implemented system.ofThis study is implemented in the mode an additional withshowing the EMR. The percentage of erroneous data for two groups users (with part of a largernotifications) project to develop complexfor CDSS on cardiovascular disorders for medical research centers. and without is presented the entire period of the experiment and the weekly dynamics of changes. The implementation of CDSS in the supplemented interface mode of the main MIS has had a positive effect in reducing user errors in © 2017 Elsevier B.V. the data.The TheAuthors. results ofPublished the users by survey are presented, showing a satisfactory evaluation of the implemented system. This study is Peer-review under responsibility the scientific of the CENTERIS - International Conference part of a larger project to developof complex CDSScommittee on cardiovascular disorders for medical research centers.on ENTERprise Information Systems / ProjMAN - International Conference on Project MANagement / HCist - International Conference on © 2017and The Authors. by Elsevier B.V. Health CarePublished Information SystemsB.V. and Technologies. © 2017 TheSocial Authors. Published by Elsevier Peer-review under responsibility of the scientific committee of the CENTERIS - International Conference on ENTERprise Peer-review under responsibility of the scientific of the - International Conference on ENTERprise Information Systems / ProjMAN - Internationalcommittee Conference on CENTERIS Project MANagement / HCist - International Conference on Information Systems / ProjMAN International Conference on Project MANagement / HCist International Conference on Health and Social Care Information Systems and Technologies. Health and Social Care Information Systems and Technologies.
* Corresponding author. Tel.: +7-911-102-0556. E-mail address: [email protected] 1877-0509 © 2017 author. The Authors. Published by Elsevier B.V. * Corresponding Tel.: +7-911-102-0556. Peer-review under responsibility of the scientific committee of the CENTERIS - International Conference on ENTERprise Information Systems / E-mail address: [email protected] ProjMAN - International Conference on Project MANagement / HCist - International Conference on Health and Social Care Information Systems and Technologies. 1877-0509 © 2017 The Authors. Published by Elsevier B.V. Peer-review under responsibility of the scientific committee of the CENTERIS - International Conference on ENTERprise Information Systems / ProjMAN - International Conference on Project MANagement / HCist - International Conference on Health and Social Care Information Systems and Technologies.
1877-0509 © 2017 The Authors. Published by Elsevier B.V. Peer-review under responsibility of the scientific committee of the CENTERIS - International Conference on ENTERprise Information Systems / ProjMAN - International Conference on Project MANagement / HCist - International Conference on Health and Social Care Information Systems and Technologies. 10.1016/j.procs.2017.11.063
470 2
Ekaterina V. Bolgva et al. / Procedia Computer Science 121 (2017) 469–474 Author name / Procedia Computer Science 00 (2017) 000–000
Keywords: electronic health records; healthcare quality; clinical disicion support systems; medical data analysis; human-coumputer interaction
1. Introduction Recent years, so-called personalized medicine has become widespread and developed 1. The transition to personalized medicine within the paradigm of P4 medicine (Predictive, Preventive, Participatory and Personalized) 2 is inextricably linked to the transition from evidence-based (or volume-based) medicine to value-based approach, which can be expressed as the ratio of the change in the quality of patient life and the number of resources spent on the treatment (the number of tests, procedures, prescription drugs, medical hours etc.) 3. The most valuable is the care delivery, which is based on rigorous scientific knowledge and has the minimum cost with maximum benefit for patients. Costs are determined not so much by the monetary costs of treatment, as by the time and effort the patient spends on the treatment, and also by the quantity of man-hours. That is why, to provide the quality healthcare delivery, it is it is necessary to carry out comprehensive efforts that will reduce costs and improve the quality of treatment. One of such area is improvement of human-computer interaction between physicians and medical information system. On the one hand, satisfaction with the system will allow physicians to enter more correctly and quickly all required information, and on the other hand it will improve the quality of the data itself for the purpose of their subsequent analysis. The Western world invests significant resources to digitize healthcare with special emphasis on the creation of an integrated electronic medical records (EMR) to improve the efficiency and quality of care 4. EMR offers several key advantages over paper medical records (PMR) related to quality of care, efficiency and high level of patient safety5. In addition, EMR is a valuable source of quality assurance of medical practice and research 6. Effective use of EMR requires structured data entry; which can be a challenge for users due to EMR method of interaction, which does not coincide with their mental models and do not meet the requirements of document flow 6,7,8. Poorly designed and cumbersome user interfaces of EMR input data can complicate the structured data-entry that will lead to a deterioration of data quality and incompleteness of data9,10. Consequently, this can lead to suboptimal functioning of information systems of medical technology, integrated into the EMR, for example, computerized support for making clinical decisions (CDSS). CDSS is one of the most effective strategies for improving clinical decisions9,11. CDSS often requires a large amount of data about the patient (demographic data, data on complaints, symptoms, medical history, physical examination, laboratory and other tests). Despite the fact that the researches aim is improving the quality of service, most of researches reported only about the improvement of the professional performance11,12 and attempts to identify the critical success factors for CDSS systems have provided conflicting results11. System CDSS take their information from forms were filled in EMR and can provide inadequate advice due to incomplete and unstructured EMR data13. However, often application of the existing approaches to design DSS health care and medical is faced with significant difficulties for several reasons considered further with respect to Russian hospital practice. First of all, medical (clinical) information systems (MIS) in use often do not provide the functionality of DSS or the possibility to add such options. DSS deployment with existing MIS will complicate doctors’ work because with filling paper records and entering data into MIS they will have to double the data in the CDSS. Meanwhile, as mentioned above, improvement of human-computer interaction in EMR demands not only technical solutions, but also facilitation of physicians’ understanding of the importance of such systems for their routine practice and further use of data stored in such systems. 2. Experiment description The results of the current experiment are determined by previous studies. In particular, papers 14, 15 describe the analysis of data from MIS for the most common mistakes and it was suggested that the developing of a CDSS that integrates with an existing MIS can solve the problem. Authors of article16 provide a general approach that was taken as the basis for the introduction of such a CDSS. As a result, an experimental sample of an integrated CDSS with a limited functional was developed and implemented for the current experiment (the CDSS architecture is shown in Fig. 1) The functionality of CDSS was intended to be limited for fixing errors and notifying users of a particular error. This was done to focus on the effect in the work and the perception of users from the operation of such a system.
Ekaterina V. Bolgva et al. / Procedia Computer Science 121 (2017) 469–474 Author name / Procedia Computer Science 00 (2017) 000–000
471 3
Fig. 1. The architecture of integrated CDSS
Two EMR subsection were allocated for the experiment: 1) Subsection “General inspection”, in which the fields are distinguished: height, weight, waist circumference (WCirc), heart rate (HR), arterial blood pressure (ABP). For this section, two error classes are taken into account: miss - if the field is left blank and error - if the field contains erroneous data (for example, letters in a numeric field, or the value is out of bounds). 2) Subsection "Recommendations", which verifies the compatibility of prescription drugs. 3. Experiment Results The testing period of the experimental sample of the CDSS was 10 full weeks. In the testing involved 7 physicians who were divided into two focus groups: 4 doctors worked with included mistakes notifications (with triggers), and 3 of them in the "fixing actions" mode, that is, notifications were not shown (without triggers). The results are shown in Table 1. Table 1. General results of system testing
At least with one mistake Remained wrong
With triggers (%) Subsection “General inspection” 49,1* 8,7 Subsection "Recommendations" 25,5 **
Without triggers (%) 63,7 42,2
Recorded messages about drugs 24 *** incompatibility Information changed after the message 7 0 * 84.7% of mistakes were committed in the first weeks. ** Some of the reports were about "Mutual intensification of the hypotensive effect" (64.3%) and "Reduction of the effect of one drug in interaction with another" (21.4%) *** Из of the reports were about "Mutual intensification of the hypotensive effect" (37,5%) and "Reduction of the effect of one drug in interaction with another" (12,5%)
Statistics for specific mistakes are presented in Table 2. Particular attention was paid to the "General inspection", since the vast majority of errors was previously associated with either the lack of data in specific fields, or the erroneous writing of them. The table shows the percentage of missing or erroneous data that was recorded in the system, and the percentage of missing or erroneous data that remained in the system after the form was closed. Table 2. Statistics for specific mistakes Height Weight WCirc HR
Missing Remained wrong Missing Remained wrong Missing Remained wrong Missing
With triggers (%) 7,3 2,9 24,6 8,7 20,3 7,25 23,2
Without triggers (%) 25,5 23,5 27,5 21,6 49,1 41,2 39,2
Ekaterina V. Bolgva et al. / Procedia Computer Science 121 (2017) 469–474 Author name / Procedia Computer Science 00 (2017) 000–000
472 4
ABP Height Weight WCirc HR ABP
Remained wrong Missing Remained wrong
10,14 13 1,5
29,5 15,7 11,7
Error Remained wrong Error Remained wrong Error Remained wrong Error Remained wrong Error Remained wrong
1,5 0 7,25 1,5 0 0 8,7 0 18,8 4,35
0 0 9,8 9,8 0 0 5,9 5,9 30,3 21,2
In Figure 2, the results from Table 2 are presented graphically. The graphs show that the number of omissions and erroneous data remaining in the MIS has decreased noticeably in the focus group that works with enabled notifications (with triggers).
Fig. 2. The percentage of user mistakes during the system testing period: (a) the total number of recorded mistakes; (b) the number of mistakes remaining in the MIS (after corrections)
Figure 3 shows the percentage of the initial omissions or user errors recorded in the system for each of the 10 weeks.
(a)
(b)
Ekaterina V. Bolgva et al. / Procedia Computer Science 121 (2017) 469–474 Author name / Procedia Computer Science 00 (2017) 000–000
473 5
Fig. 3. The percentage of user mistakes by week: (a) the focus group with enabled notifications; (b) focus group with notifications turned off
In addition to analyzing the results of mistakes, a survey was carried out by physicians who worked with the system with enabled notifications. The purpose of the survey was to find out how the system is convenient and useful in the work of a doctor. The results of the survey showed that most physicians were able to adapt to the new mode of operation, all noted that notifications were useful to them in their work. However, almost all respondents noted that it was not more convenient for the system to work. This is due to the fact that there are additional pop-up windows and the adaptation to them requires a slightly longer time interval. 4. Conclusion and Future Work In Fig. 3 it can be seen that the curves noticeably decrease, and then increase slightly, but at the moment it is difficult to predict their behavior in the future, so additional testing is necessary. Figure 4 shows the average mistakes value (blue solid line). As mentioned in the previous work14, we use the DMAIC cycle17 to develop the system, and we have already passed one turn, then we need to clarify the recommendations of users, make some changes to the system and conduct further testing. We assume that without changes the behavior of the curve will not change much later (Fig. 4, the blue dashed line). However, when following the DMIAC cycle and taking into account the interests and requirements of users, the behavior of the curve can be a similar (Fig. 4 green line). Thus, we need to achieve the Fig. 4. The average mistakes value and the further behavior shown in Figure 4. behavior. That in its turn, when approximated, will give a curve very similar to the Fogg curve18, shown in fig. 5. And if we can get this result, we can use the Fogg model to predict behavior and to control behavior through motivation, training, and triggers. In conclusion, it is important to note that the implementation of CDSS in the supplemented interface mode of the main MIS has had a positive effect in reducing user errors in the data. As for HCI, so far we have not been able to achieve a great positive effect, however, there are some small shifts in the positive direction now and further close work with the physicians as a users of the system will improve this result. Fig. 5. The Fogg curve18 Acknowledgements This research is financially supported by The Russian Scientific Foundation, Agreement #14-11-00823 (15.07.2014). References 1. Hamburg M.A., Collins F.S. The path to personalized medicine. N Engl J Med 2010; 363(4): 301-304. 2. P. Sobradillo, F. Pozo and Á. Agustí, "P4 medicine: the future around the corner," Archivos de Bronconeumología (English Edition), vol. 47, no. 1, pp. 35-40, 2011. 3. Bae JM. Value-based medicine: concepts and application. Epidemiol Health. 2015;37:e2015018. Published online 2015 Mar 4. doi: 10.4178/epih/e2015014 4. Fitzpatrick, G., and Ellingsen, G. A review of 25 years of cscw research in healthcare: contributions, challenges and future agendas. Computer Supported Cooperative Work (CSCW) 22, 4-6 (2013), 609–665. 5. Chantler, C., Clarke, T., and Granger, R. Information technology in the english national health service. JAMA 296, 18 (2006), 2255–2258. 6. S.J. Stack, G. Botstein, J. Mattison, G. Melton-Meaux, B. Middleton, R. Ratwani, et al. Improving Care: Priorities to Improve Electronic Health Record Usability American Medical Association (AMA) (2014)
474 6
Ekaterina V. Bolgva et al. / Procedia Computer Science 121 (2017) 469–474 Author name / Procedia Computer Science 00 (2017) 000–000
7. J.L. Belden, R. Grayson, J. Barnes, On behalf of the HIMSS EHR Usability Taskforce Defining and Testing EMR Usability: Principles and Proposed Methods of EMR Usability Evaluation and Rating M. Brodnik, H. Volger (Eds.)Healthcare Information and Management Systems Society (HIMSS), The Ohio State University Columbus, Ohio (2009) 8. M.W. Friedberg, P.G. Chen, K.R.V. Busum, F.M. Aunon, C. Pham, J.P. Caloyeras, et al. Factors Affecting Physician Professional Satisfaction and Their Implications for Patient Care, Health Systems, and Health Policy Research Report RAND Corporation (2013) 9. R.N. Shiffman, Y. Liaw, C.A. Brandt, G.J. Corb Computer-based guideline implementation systems: a systematic review of functionality and effectiveness J. Am. Med. Inform. Assoc., 6 (2) (1999), pp. 104–114 10. R. Khajouei, N. Peek, P.C. Wierenga, M.J. Kersten, M.W. Jaspers Effect of predefined order sets and usability problems on efficiency of computerized medication ordering Int. J. Med. Inform., 79 (10) (2010), pp. 690–698 11. P.S. Roshanov, N. Fernandes, J.M. Wilczynski, B.J. Hemens, J.J. You, S.M. Handler, et al. Features of effective computerised clinical decision support systems: meta-regression of 162 randomised trials BMJ, 346 (2013), p. F657 12. T.J. Bright, A. Wong, R. Dhurjati, E. Bristow, L. Bastian, R.R. Coeytaux, et al. Effect of clinical decision-support systems: a systematic review Ann. Intern. Med., 157 (1) (2012), pp. 29–43 13. M.W. Jaspers, M. Smeulers, H. Vermeulen, L.W. Peute Effects of clinical decision-support systems on practitioner performance and patient outcomes: a synthesis of high-quality systematic review findings J. Am. Med. Inform. Assoc., 18 (3) (2011), pp. 327–334 14. Bolgova E.V., Krikunov A.V., Prokusheva D.I., Zvartau N.E. /Human-Computer Interaction in Electronic Medical Records: from the Perspectives of Physicians and Data Scientists // Procedia Computer Science. — 2016. — Vol. 100. — pp. 915–920 15. Bolgova E.V., Konradi A. , Semakova A., Zvartau N.E. /Quality of Hypertensive Patients' Electronic Health Records in Specialized Cardiological Centre:6-year Trends // Communications in Computer and Information Science . — 2016. — Vol. 674. — pp. 544-552. 16. Bolgova E.V., Syomov I.I., Krikunov A.V., Kovalchuk S.V., Moiseeva O.M., Simakova M.A. /Towards Infrastructure for Knowledge-based Decision Support in Clinical Practice // Procedia Computer Science. — 2016. — Vol. 100. — pp. 907 – 914. 17. George, Michael; Rowlands, David; Price, Mark; Maxey, John. Using DMAIC to improve speed, quality, and cost // The Lean Six Sigma Pocket Toolbook: A Quick Reference Guide to Nearly 100 Tools for Improving Process Quality, Speed, and Complexity. — McGraw-Hill, 2005. — P. 1-26. — 282 p. — ISBN 978-0-07-144199-3 18. BJ Fogg. Persuasive computers: perspectives and research directions. CHI '98 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. Pages 225-232. Los Angeles, California, USA — April 18 - 23, 1998
Available online at www.sciencedirect.com Procedia Computer Science 00 (2017) 000–000
ScienceDirect
www.elsevier.com/locate/procedia
Available online at www.sciencedirect.com Procedia Computer Science 00 (2017) 000–000
ScienceDirect
www.elsevier.com/locate/procedia
Procedia Computer Science 121 (2017) 469–474 CENTERIS - International Conference on ENTERprise Information Systems / ProjMAN International Conference on Project MANagement / HCist - International Conference on Health and Social Care Information Systems and Technologies, CENTERIS / ProjMAN / HCist 2017, 8-10 CENTERIS - International November Conference2017, on ENTERprise Information Systems / ProjMAN Barcelona, Spain International Conference on Project MANagement / HCist - International Conference on Health and Social Care Information Systems and Technologies, CENTERIS / ProjMAN / HCist 2017, 8-10 Improving Electronic Medical November 2017,Records Barcelona, with Spain Support of Human
Computer Interaction in Medical Information Systems Improving Electronic Medical Records with Support of Human Ekaterina V. Bolgvaa, Nadezhda E. Zvartaua,b,Sergey V. Kovalchuka, Marina A. Computer Interaction in Medical Information Systems Balakhontceva a,*, Oleg G. Metskera a University, 49 Kronverksky Pr., St. a ITMO Russia Ekaterina V. Bolgva , Nadezhda E. Zvartaua,bPetersburg, ,Sergey197101, V. Kovalchuk , Marina A. Federal Almazov North-West Medical Research Centre, 2 Akkuratova street, St. Petersburg 197341, Russia. a, a Balakhontceva *, Oleg G. Metsker a
b
ITMO University, 49 Kronverksky Pr., St. Petersburg, 197101, Russia Federal Almazov North-West Medical Research Centre, 2 Akkuratova street, St. Petersburg 197341, Russia. a
Abstract
b
This study investigated the most common challenges of human computer interaction (HCI) while using electronic medical records (EMR) based on the experience of a large Russian medical research center. The paper presents the results of testing DSS Abstract implemented in the mode of an additional interface with the EMR. The percentage of erroneous data for two groups of users (with and notifications) is presented the entire periodcomputer of the experiment thewhile weekly changes. The This without study investigated the most common for challenges of human interaction and (HCI) usingdynamics electronicofmedical records implementation CDSS in the supplemented interfacemedical mode ofresearch the maincenter. MIS has hadpaper a positive effect reducing errors in (EMR) based onof the experience of a large Russian The presents thein results of user testing DSS the data. The results of theofusers survey areinterface presented, a satisfactory evaluation of the implemented system.ofThis study is implemented in the mode an additional withshowing the EMR. The percentage of erroneous data for two groups users (with part of a largernotifications) project to develop complexfor CDSS on cardiovascular disorders for medical research centers. and without is presented the entire period of the experiment and the weekly dynamics of changes. The implementation of CDSS in the supplemented interface mode of the main MIS has had a positive effect in reducing user errors in © 2017 Elsevier B.V. the data.The TheAuthors. results ofPublished the users by survey are presented, showing a satisfactory evaluation of the implemented system. This study is Peer-review under responsibility the scientific of the CENTERIS - International Conference part of a larger project to developof complex CDSScommittee on cardiovascular disorders for medical research centers.on ENTERprise Information Systems / ProjMAN - International Conference on Project MANagement / HCist - International Conference on © 2017and The Authors. by Elsevier B.V. Health CarePublished Information SystemsB.V. and Technologies. © 2017 TheSocial Authors. Published by Elsevier Peer-review under responsibility of the scientific committee of the CENTERIS - International Conference on ENTERprise Peer-review under responsibility of the scientific of the - International Conference on ENTERprise Information Systems / ProjMAN - Internationalcommittee Conference on CENTERIS Project MANagement / HCist - International Conference on Information Systems / ProjMAN International Conference on Project MANagement / HCist International Conference on Health and Social Care Information Systems and Technologies. Health and Social Care Information Systems and Technologies.
* Corresponding author. Tel.: +7-911-102-0556. E-mail address: [email protected] 1877-0509 © 2017 author. The Authors. Published by Elsevier B.V. * Corresponding Tel.: +7-911-102-0556. Peer-review under responsibility of the scientific committee of the CENTERIS - International Conference on ENTERprise Information Systems / E-mail address: [email protected] ProjMAN - International Conference on Project MANagement / HCist - International Conference on Health and Social Care Information Systems and Technologies. 1877-0509 © 2017 The Authors. Published by Elsevier B.V. Peer-review under responsibility of the scientific committee of the CENTERIS - International Conference on ENTERprise Information Systems / ProjMAN - International Conference on Project MANagement / HCist - International Conference on Health and Social Care Information Systems and Technologies.
1877-0509 © 2017 The Authors. Published by Elsevier B.V. Peer-review under responsibility of the scientific committee of the CENTERIS - International Conference on ENTERprise Information Systems / ProjMAN - International Conference on Project MANagement / HCist - International Conference on Health and Social Care Information Systems and Technologies. 10.1016/j.procs.2017.11.063
470 2
Ekaterina V. Bolgva et al. / Procedia Computer Science 121 (2017) 469–474 Author name / Procedia Computer Science 00 (2017) 000–000
Keywords: electronic health records; healthcare quality; clinical disicion support systems; medical data analysis; human-coumputer interaction
1. Introduction Recent years, so-called personalized medicine has become widespread and developed 1. The transition to personalized medicine within the paradigm of P4 medicine (Predictive, Preventive, Participatory and Personalized) 2 is inextricably linked to the transition from evidence-based (or volume-based) medicine to value-based approach, which can be expressed as the ratio of the change in the quality of patient life and the number of resources spent on the treatment (the number of tests, procedures, prescription drugs, medical hours etc.) 3. The most valuable is the care delivery, which is based on rigorous scientific knowledge and has the minimum cost with maximum benefit for patients. Costs are determined not so much by the monetary costs of treatment, as by the time and effort the patient spends on the treatment, and also by the quantity of man-hours. That is why, to provide the quality healthcare delivery, it is it is necessary to carry out comprehensive efforts that will reduce costs and improve the quality of treatment. One of such area is improvement of human-computer interaction between physicians and medical information system. On the one hand, satisfaction with the system will allow physicians to enter more correctly and quickly all required information, and on the other hand it will improve the quality of the data itself for the purpose of their subsequent analysis. The Western world invests significant resources to digitize healthcare with special emphasis on the creation of an integrated electronic medical records (EMR) to improve the efficiency and quality of care 4. EMR offers several key advantages over paper medical records (PMR) related to quality of care, efficiency and high level of patient safety5. In addition, EMR is a valuable source of quality assurance of medical practice and research 6. Effective use of EMR requires structured data entry; which can be a challenge for users due to EMR method of interaction, which does not coincide with their mental models and do not meet the requirements of document flow 6,7,8. Poorly designed and cumbersome user interfaces of EMR input data can complicate the structured data-entry that will lead to a deterioration of data quality and incompleteness of data9,10. Consequently, this can lead to suboptimal functioning of information systems of medical technology, integrated into the EMR, for example, computerized support for making clinical decisions (CDSS). CDSS is one of the most effective strategies for improving clinical decisions9,11. CDSS often requires a large amount of data about the patient (demographic data, data on complaints, symptoms, medical history, physical examination, laboratory and other tests). Despite the fact that the researches aim is improving the quality of service, most of researches reported only about the improvement of the professional performance11,12 and attempts to identify the critical success factors for CDSS systems have provided conflicting results11. System CDSS take their information from forms were filled in EMR and can provide inadequate advice due to incomplete and unstructured EMR data13. However, often application of the existing approaches to design DSS health care and medical is faced with significant difficulties for several reasons considered further with respect to Russian hospital practice. First of all, medical (clinical) information systems (MIS) in use often do not provide the functionality of DSS or the possibility to add such options. DSS deployment with existing MIS will complicate doctors’ work because with filling paper records and entering data into MIS they will have to double the data in the CDSS. Meanwhile, as mentioned above, improvement of human-computer interaction in EMR demands not only technical solutions, but also facilitation of physicians’ understanding of the importance of such systems for their routine practice and further use of data stored in such systems. 2. Experiment description The results of the current experiment are determined by previous studies. In particular, papers 14, 15 describe the analysis of data from MIS for the most common mistakes and it was suggested that the developing of a CDSS that integrates with an existing MIS can solve the problem. Authors of article16 provide a general approach that was taken as the basis for the introduction of such a CDSS. As a result, an experimental sample of an integrated CDSS with a limited functional was developed and implemented for the current experiment (the CDSS architecture is shown in Fig. 1) The functionality of CDSS was intended to be limited for fixing errors and notifying users of a particular error. This was done to focus on the effect in the work and the perception of users from the operation of such a system.
Ekaterina V. Bolgva et al. / Procedia Computer Science 121 (2017) 469–474 Author name / Procedia Computer Science 00 (2017) 000–000
471 3
Fig. 1. The architecture of integrated CDSS
Two EMR subsection were allocated for the experiment: 1) Subsection “General inspection”, in which the fields are distinguished: height, weight, waist circumference (WCirc), heart rate (HR), arterial blood pressure (ABP). For this section, two error classes are taken into account: miss - if the field is left blank and error - if the field contains erroneous data (for example, letters in a numeric field, or the value is out of bounds). 2) Subsection "Recommendations", which verifies the compatibility of prescription drugs. 3. Experiment Results The testing period of the experimental sample of the CDSS was 10 full weeks. In the testing involved 7 physicians who were divided into two focus groups: 4 doctors worked with included mistakes notifications (with triggers), and 3 of them in the "fixing actions" mode, that is, notifications were not shown (without triggers). The results are shown in Table 1. Table 1. General results of system testing
At least with one mistake Remained wrong
With triggers (%) Subsection “General inspection” 49,1* 8,7 Subsection "Recommendations" 25,5 **
Without triggers (%) 63,7 42,2
Recorded messages about drugs 24 *** incompatibility Information changed after the message 7 0 * 84.7% of mistakes were committed in the first weeks. ** Some of the reports were about "Mutual intensification of the hypotensive effect" (64.3%) and "Reduction of the effect of one drug in interaction with another" (21.4%) *** Из of the reports were about "Mutual intensification of the hypotensive effect" (37,5%) and "Reduction of the effect of one drug in interaction with another" (12,5%)
Statistics for specific mistakes are presented in Table 2. Particular attention was paid to the "General inspection", since the vast majority of errors was previously associated with either the lack of data in specific fields, or the erroneous writing of them. The table shows the percentage of missing or erroneous data that was recorded in the system, and the percentage of missing or erroneous data that remained in the system after the form was closed. Table 2. Statistics for specific mistakes Height Weight WCirc HR
Missing Remained wrong Missing Remained wrong Missing Remained wrong Missing
With triggers (%) 7,3 2,9 24,6 8,7 20,3 7,25 23,2
Without triggers (%) 25,5 23,5 27,5 21,6 49,1 41,2 39,2
Ekaterina V. Bolgva et al. / Procedia Computer Science 121 (2017) 469–474 Author name / Procedia Computer Science 00 (2017) 000–000
472 4
ABP Height Weight WCirc HR ABP
Remained wrong Missing Remained wrong
10,14 13 1,5
29,5 15,7 11,7
Error Remained wrong Error Remained wrong Error Remained wrong Error Remained wrong Error Remained wrong
1,5 0 7,25 1,5 0 0 8,7 0 18,8 4,35
0 0 9,8 9,8 0 0 5,9 5,9 30,3 21,2
In Figure 2, the results from Table 2 are presented graphically. The graphs show that the number of omissions and erroneous data remaining in the MIS has decreased noticeably in the focus group that works with enabled notifications (with triggers).
Fig. 2. The percentage of user mistakes during the system testing period: (a) the total number of recorded mistakes; (b) the number of mistakes remaining in the MIS (after corrections)
Figure 3 shows the percentage of the initial omissions or user errors recorded in the system for each of the 10 weeks.
(a)
(b)
Ekaterina V. Bolgva et al. / Procedia Computer Science 121 (2017) 469–474 Author name / Procedia Computer Science 00 (2017) 000–000
473 5
Fig. 3. The percentage of user mistakes by week: (a) the focus group with enabled notifications; (b) focus group with notifications turned off
In addition to analyzing the results of mistakes, a survey was carried out by physicians who worked with the system with enabled notifications. The purpose of the survey was to find out how the system is convenient and useful in the work of a doctor. The results of the survey showed that most physicians were able to adapt to the new mode of operation, all noted that notifications were useful to them in their work. However, almost all respondents noted that it was not more convenient for the system to work. This is due to the fact that there are additional pop-up windows and the adaptation to them requires a slightly longer time interval. 4. Conclusion and Future Work In Fig. 3 it can be seen that the curves noticeably decrease, and then increase slightly, but at the moment it is difficult to predict their behavior in the future, so additional testing is necessary. Figure 4 shows the average mistakes value (blue solid line). As mentioned in the previous work14, we use the DMAIC cycle17 to develop the system, and we have already passed one turn, then we need to clarify the recommendations of users, make some changes to the system and conduct further testing. We assume that without changes the behavior of the curve will not change much later (Fig. 4, the blue dashed line). However, when following the DMIAC cycle and taking into account the interests and requirements of users, the behavior of the curve can be a similar (Fig. 4 green line). Thus, we need to achieve the Fig. 4. The average mistakes value and the further behavior shown in Figure 4. behavior. That in its turn, when approximated, will give a curve very similar to the Fogg curve18, shown in fig. 5. And if we can get this result, we can use the Fogg model to predict behavior and to control behavior through motivation, training, and triggers. In conclusion, it is important to note that the implementation of CDSS in the supplemented interface mode of the main MIS has had a positive effect in reducing user errors in the data. As for HCI, so far we have not been able to achieve a great positive effect, however, there are some small shifts in the positive direction now and further close work with the physicians as a users of the system will improve this result. Fig. 5. The Fogg curve18 Acknowledgements This research is financially supported by The Russian Scientific Foundation, Agreement #14-11-00823 (15.07.2014). References 1. Hamburg M.A., Collins F.S. The path to personalized medicine. N Engl J Med 2010; 363(4): 301-304. 2. P. Sobradillo, F. Pozo and Á. Agustí, "P4 medicine: the future around the corner," Archivos de Bronconeumología (English Edition), vol. 47, no. 1, pp. 35-40, 2011. 3. Bae JM. Value-based medicine: concepts and application. Epidemiol Health. 2015;37:e2015018. Published online 2015 Mar 4. doi: 10.4178/epih/e2015014 4. Fitzpatrick, G., and Ellingsen, G. A review of 25 years of cscw research in healthcare: contributions, challenges and future agendas. Computer Supported Cooperative Work (CSCW) 22, 4-6 (2013), 609–665. 5. Chantler, C., Clarke, T., and Granger, R. Information technology in the english national health service. JAMA 296, 18 (2006), 2255–2258. 6. S.J. Stack, G. Botstein, J. Mattison, G. Melton-Meaux, B. Middleton, R. Ratwani, et al. Improving Care: Priorities to Improve Electronic Health Record Usability American Medical Association (AMA) (2014)
474 6
Ekaterina V. Bolgva et al. / Procedia Computer Science 121 (2017) 469–474 Author name / Procedia Computer Science 00 (2017) 000–000
7. J.L. Belden, R. Grayson, J. Barnes, On behalf of the HIMSS EHR Usability Taskforce Defining and Testing EMR Usability: Principles and Proposed Methods of EMR Usability Evaluation and Rating M. Brodnik, H. Volger (Eds.)Healthcare Information and Management Systems Society (HIMSS), The Ohio State University Columbus, Ohio (2009) 8. M.W. Friedberg, P.G. Chen, K.R.V. Busum, F.M. Aunon, C. Pham, J.P. Caloyeras, et al. Factors Affecting Physician Professional Satisfaction and Their Implications for Patient Care, Health Systems, and Health Policy Research Report RAND Corporation (2013) 9. R.N. Shiffman, Y. Liaw, C.A. Brandt, G.J. Corb Computer-based guideline implementation systems: a systematic review of functionality and effectiveness J. Am. Med. Inform. Assoc., 6 (2) (1999), pp. 104–114 10. R. Khajouei, N. Peek, P.C. Wierenga, M.J. Kersten, M.W. Jaspers Effect of predefined order sets and usability problems on efficiency of computerized medication ordering Int. J. Med. Inform., 79 (10) (2010), pp. 690–698 11. P.S. Roshanov, N. Fernandes, J.M. Wilczynski, B.J. Hemens, J.J. You, S.M. Handler, et al. Features of effective computerised clinical decision support systems: meta-regression of 162 randomised trials BMJ, 346 (2013), p. F657 12. T.J. Bright, A. Wong, R. Dhurjati, E. Bristow, L. Bastian, R.R. Coeytaux, et al. Effect of clinical decision-support systems: a systematic review Ann. Intern. Med., 157 (1) (2012), pp. 29–43 13. M.W. Jaspers, M. Smeulers, H. Vermeulen, L.W. Peute Effects of clinical decision-support systems on practitioner performance and patient outcomes: a synthesis of high-quality systematic review findings J. Am. Med. Inform. Assoc., 18 (3) (2011), pp. 327–334 14. Bolgova E.V., Krikunov A.V., Prokusheva D.I., Zvartau N.E. /Human-Computer Interaction in Electronic Medical Records: from the Perspectives of Physicians and Data Scientists // Procedia Computer Science. — 2016. — Vol. 100. — pp. 915–920 15. Bolgova E.V., Konradi A. , Semakova A., Zvartau N.E. /Quality of Hypertensive Patients' Electronic Health Records in Specialized Cardiological Centre:6-year Trends // Communications in Computer and Information Science . — 2016. — Vol. 674. — pp. 544-552. 16. Bolgova E.V., Syomov I.I., Krikunov A.V., Kovalchuk S.V., Moiseeva O.M., Simakova M.A. /Towards Infrastructure for Knowledge-based Decision Support in Clinical Practice // Procedia Computer Science. — 2016. — Vol. 100. — pp. 907 – 914. 17. George, Michael; Rowlands, David; Price, Mark; Maxey, John. Using DMAIC to improve speed, quality, and cost // The Lean Six Sigma Pocket Toolbook: A Quick Reference Guide to Nearly 100 Tools for Improving Process Quality, Speed, and Complexity. — McGraw-Hill, 2005. — P. 1-26. — 282 p. — ISBN 978-0-07-144199-3 18. BJ Fogg. Persuasive computers: perspectives and research directions. CHI '98 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. Pages 225-232. Los Angeles, California, USA — April 18 - 23, 1998