- Email: [email protected]
SEWA: A framework for sociotechnical analysis of electronic health record system workarounds
International Journal of Medical Informatics 125 (2019) 71–78
Contents lists available at ScienceDirect
International Journal of Medical Informatics journal homepage: www.elsevier.com/locate/ijmedinf
SEWA: A framework for sociotechnical analysis of electronic health record system workarounds
T
Vincent Blijlevena,b, , Kitty Koelemeijera, Monique Jaspersb ⁎
a b
Center for Marketing and Supply Chain Management, Nyenrode Business University, Straatweg 25, 3621 BG, Breukelen, the Netherlands Department of Medical Informatics, Academic Medical Center, Meibergdreef 15, 1105 AZ, Amsterdam, the Netherlands
ARTICLE INFO
ABSTRACT
Keywords: Electronic health records Framework Patient safety Unintended consequences Workarounds Workflow
Objective: To develop a conceptual framework, SEWA, to address challenges of studying workarounds emerging from Electronic Health Record (EHR) system usage. Materials and methods: SEWA is based on direct observations and follow-up interviews with physicians, nurses and clerks using their EHR at a large academic hospital. SEWA was developed by an iterative process: each new version was reviewed by experts (case study participants, hospital management, EHR developers) and refined accordingly till deemed final. Results: SEWA defines the work system and its five components constituting the context in which EHR workarounds are created. It also contains 15 rationales for creating EHR workarounds. Furthermore, four attributes are included that define EHR workarounds: cascadedness, anticipatedness, avoidability, and repetitiveness. Finally, SEWA lists the possible effects of workarounds on outcomes of clinical processes in terms of scope and impact. Discussion: SEWA provides a grounded foundation for performing sociotechnical analyses of EHR workarounds based on components of the work system. SEWA can likewise be supportive in planning redesign efforts of the work system. Finally, workarounds are subject to gradual change caused by e.g. changes in one’s knowledge of the EHR, hospital policies, care directives, and system updates. Snapshots of SEWA can be taken over time and compared to gain insights into the evolution of workarounds. Conclusion: Given the absence of a sociotechnical framework to study EHR workarounds, SEWA could aid researchers and practitioners to identify, analyze and resolve workarounds, and thereby contribute to improved patient safety, effectiveness of care and efficiency of care.
1. Background and significance Electronic health record systems (EHRs) are increasingly being adopted by healthcare organizations worldwide [1–3], spurred by government-led incentives [4] and in pursuit of favorable outcomes related to patient safety [5–7], quality of care [7,8], efficiency of care [7,9,10], and reduced costs [11,12]. However, realizing the expected benefits of adopting EHRs remains challenging. Examples are EHR users experiencing ‘alert fatigue’ [13], not all clinical work being supported by EHRs [14], unavailability of complete clinical information at the point of care [15], difficulties in finding the right information in the EHR [16], and significantly disrupted workflows due to modified timing, sequence of work practices and revised professional responsibilities [17–19]. Many causes of unintended consequences of EHRs can be traced
⁎
back to discrepancies between the behavior, intentions and expectations of EHR users, and the workflows dictated by EHRs [18,20–23]. When users experience workflow mismatches they create workarounds [24]. Workarounds have been defined as “informal temporary practices for handling exceptions to normal workflow” [25] that “do not follow explicit or implicit rules, assumptions, workflow regulations, or intentions of systems designers” [26]. The impact of workarounds may extend far beyond solely allowing EHR users to proceed with their workflow to get their clinical tasks done. Existing studies on EHR workarounds tend to report their findings from a single out of four angles we identified. First, EHR users have distinct rationales for creating workarounds such as a lack of declarative knowledge (i.e. not knowing how to use (a part of) the EHR), memory aids (e.g. writing patient data down on paper during an outpatient consultation session and entering this data into the EHR later
Corresponding author at: Straatweg 25, 3621 BG, Breukelen, the Netherlands. E-mail addresses: [email protected], [email protected] (V. Blijleven).
https://doi.org/10.1016/j.ijmedinf.2019.02.012 Received 11 May 2018; Received in revised form 28 September 2018; Accepted 28 February 2019 1386-5056/ © 2019 Elsevier B.V. All rights reserved.
International Journal of Medical Informatics 125 (2019) 71–78
V. Blijleven, et al.
Fig. 1. SEWA: Sociotechnical Electronic health record Workaround Analysis framework. *No identified workaround rationales could be associated with the physical environment.
on), and technical issues (e.g. EHR halting, crashing or slowing down) [14,27–31]. Second, the impact of workarounds may be limited to the EHR user or extend beyond the EHR user to the patient, the overall organization, or a combination of the foregoing stakeholders [27]. Third, EHR workarounds may lead to safer, higher quality and more efficient care and thus be considered favorable [21,27,32,33], but may likewise lead to less safe, lower quality and less efficient care and thus be considered unfavorable [24,26,27,34]. Fourth, workarounds are defined by several attributes, such as being temporarily versus routinely used, avoidable versus unavoidable, and/or deliberately chosen versus unplanned [35]. A sociotechnical systems perspective incorporating all these four angles could be supportive in examining, typifying, and subsequently resolving EHR workarounds. To this end we present a comprehensive framework – SEWA (Sociotechnical Ehr Workaround Analysis) – based
on a study on EHR workarounds in an academic hospital setting. The framework is inspired by the Systems Engineering Initiative for Patient Safety (SEIPS) framework [36]. 2. Materials and methods We followed a qualitative approach consisting of non-participant direct observations combined with semi-structured follow-up interviews with 31 physicians, 13 nurses and 3 clerks using their EHR while performing daily (clinical) tasks at a large academic hospital. Case study participants were recruited via the director of medical staff, the director of operations, and participants suggesting other participants. No institutional board review was required. All healthcare professionals and patients participating in the study were asked for an informed consent before any recording would take place.
72
International Journal of Medical Informatics 125 (2019) 71–78
V. Blijleven, et al.
may act simultaneously to shape processes and generate outcomes [36]. As shown in Fig. 1, these components are:
All observations and interviews were audiovisually recorded and analyzed following an inductive bottom-up coding strategy [37]. The direct observations allowed us to observe workarounds while work practices and EHR use by healthcare professionals unfolded. The interviews allowed us to gain greater insight into each observed workaround, more specifically to determine their rationale, scope, impact and attributes. The EHR is used by over 8000 hospital staff and concerns a hospital-wide integrated suite of healthcare software supporting functions related to patient care and management, registration and scheduling, clinical systems for health care providers, ancillary laboratory, pharmacy, and radiology systems and a billing system. Around 200 h of audiovisual material were captured. Data analysis consisted of four main steps, with each step focusing on one of the four angles of the framework. The first step concerned determining the rationale of each workaround. Each rationale was then associated with one out of five work system components derived from the SEIPS framework [36]. In the second step the scope of each workaround was determined, more specifically which stakeholders (i.e. the patient, healthcare professional, the overall organization or a combination thereof) were or could be affected by the workaround. The third step concerned determining the impact of each workaround on patient safety, effectiveness of care and efficiency of care. Finally, the fourth step concerned revealing attributes of EHR workarounds: inherent properties defining EHR workarounds of which their value (e.g. incidental versus routinized) may change over time. An extensive description of the data collection and analysis procedures has been published in [38]. Results of the first three steps have been described in [27]. This article reports on the fourth step and consolidates the results of all four steps into a comprehensive framework to analyze and typify EHR workarounds utilizing a sociotechnical view. Attributes of workarounds were defined through analyzing each identified workaround one-by-one with the purpose of revealing recurring properties appearing more than once and being applicable to multiple workarounds. The properties were identified by one of the authors and discussed in meetings with the members of the research team having expertise in software development, cognitive psychology, biomedical informatics, and behavioral economics. Case study participants, hospital management and EHR developers were then consulted to discuss, validate and refine the identified properties. Our framework, SEWA, was designed following an iterative process with the aim of converging the research results. The results of all four analysis steps were first combined to create an initial version. Followup versions were created over time with each version being reviewed and refined by experts (case study participants, hospital management and EHR developers) and validated accordingly.
• Person(s): healthcare professionals developing and using EHR workarounds • Technology and tools: the EHR and related information technology used by healthcare professionals • Task(s): clinical tasks performed by healthcare professionals • Organization: organizational conditions (e.g. care directives, hos•
pital policies) under which clinical tasks and EHR usage are performed Physical environment: the environment (e.g. outpatient examination room, inpatient ward) and its conditions (e.g. lighting, noise) in which clinical tasks are carried out by healthcare professionals
The work system components together constitute the context in which EHR workarounds are created by EHR users. We identified 15 rationales (as described in [27]): declarative knowledge, procedural knowledge, memory aid, awareness, social norms, usability, technical issues, data presentation, patient data specificity, task interference, commitment to patient interaction, efficiency, data migration policy, enforced data entry, and required data entry option missing. A definition and several examples per workaround rationale are provided in Appendix A. Each of the 15 workaround rationales has been associated with one of the five work system components that reflects its source of origin [27]. The sources of origin provide clues about how related workarounds could be resolved. For example, workaround rationales associated with the Persons component may be resolved most effectively through personal training to promote optimal and proper EHR and Technology and Tools-associated rationales through EHR (re)design initiatives. 5. Attributes of EHR workarounds At the center of SEWA in Fig. 1 are EHR workarounds. Four distinct attributes were identified that define EHR workarounds: cascadedness, avoidability, anticipatedness and repetitiveness. Each attribute may be either ‘true’ or ‘false’ for each workaround. A definition and several examples of each are provided in Appendix B. 5.1. Cascading versus non-cascading workarounds Workarounds can be ‘cascading’, meaning their usage initiates the creation of additional workarounds. For example, the EHR studied offers an extensive standardized data entry template for entering patient data. However, because of a perceived low usability of the standardized data entry template, nearly all users preferred copy-pasting the majority of patient data from previous progress notes into a new progress note. Clinicians argued that a disadvantage of this workaround is that it causes patient data to get lost in the system. For instance, over 25 progress notes for a single patient were created within a week’s time. Clinicians rarely copy-pasted all information from the available progress notes into a new progress note, causing patient data harder to be found as more progress notes are created over time. Clinicians created other workarounds such as customized progress note labeling systems or added markup such as boldfacing or underlining text parts within progress notes in an attempt to counter this problem. Most workarounds we observed are non-cascading. Examples are physicians writing down keywords in a patient’s progress note in
3. Results SEWA illustrated in Fig. 1 consists of two major parts: (I) the work system and its components (inspired by the SEIPS framework) [36] constituting the context in which EHR workarounds are created including 15 associated rationales for workaround creation and EHR workarounds themselves defined by four attributes, and (II) the scope of workarounds in terms of stakeholders affected and their impact on patient safety, effectiveness of care and efficiency of care. 4. Work system components and associated workaround rationales The work system consists of five interconnected components that
73
International Journal of Medical Informatics 125 (2019) 71–78
V. Blijleven, et al.
physicians would write down patient data on paper and enter this data in the EHR after the EHR was booted up again. Such workarounds became obsolete as soon as the bug was resolved. In contrast, routinized workarounds are ingrained into the workflows of EHR users and have become a common way of dealing with workflow constraints. For example, a group of hemophilia nurses would enter certain patient data such as treatment regimens in the EHR and a standalone tailor-made database. This database was in use before the introduction of the EHR and perceived to be superior to the EHR with regard to quickly looking up certain patient data. When asked why they kept on using the database, nurses indicated that they always had worked this way, although they could abandon use of the database and only work with the EHR.
advance of a patient consultation as a personal reminder, physicians skipping data fields in the standardized data entry template because they considered these inapplicable, or a physician entering ‘x’ in a mandatory data field in order to proceed in the EHR because the supposed entry in the data field is not known or beyond his expertise. None of these workarounds initiated the creation of other workarounds. 5.2. Avoidable versus unavoidable workarounds Workarounds that can be avoided are not required to proceed with one’s workflow. Although avoidable workarounds can be easily abandoned, they remain in use primarily because of a lack of time, disinterest, or limited expected returns of changing one’s way of working. For example, a physician repetitively adjusted a frequently used predefined drug order set because it contained known mistakes. When asked if the drug order set could simply not be corrected in the backend of the EHR, she indicated that although she had been trained and authorized to configure some of the tools and workflows such as documentation and ordering, management did not give her the time to use these skills in practice. Unavoidable workarounds result from EHR users having no other way to proceed with their workflow unless they resort to a workaround. The creation of unavoidable workarounds is often driven by circumstances outside of the EHR user’s direct area of influence. Examples are enforced hospital policies or technical EHR constraints causing unintended workflow mismatches. For example, a physician who aimed to prescribe 1.5 tablets of 2.5 mg prednisone (3.75 mg total) for a patient per day was enforced to choose between 2.5 mg (Table A1) or 5 mg (Table B1). This was due to the drugs list being directly linked to the inventory of the hospital pharmacy (whereas certain drugs can be easily split in half by patients). The physician ordered 1 tablet (2.5 mg) per day but entered a textual description in multiple free (unstructured) text fields that the supposed dosage should be 1.5 tablets (3.75 mg) per day.
6. Scope and impact of EHR workarounds Ultimately, each EHR workaround has a scope and impact on care processes. Scope concerns the stakeholders (i.e. the patient, healthcare professional, organization, or a combination thereof) affected by the workaround. Impact concerns the consequence(s) each workaround has on patient safety, effectiveness of care and efficiency of care. 6.1. Scope Workarounds may solely affect the single EHR user who devised it, extend beyond the EHR user to the patient(s) under diagnosis, treatment or in follow-up, the work context of other healthcare providers or even to the organizational level, or a combination of these. For example, a physician re-entering and copy-pasting patient data from the EHR into a letter due to not knowing how to use the automatic letter generation tool solely affects himself. Other workarounds solely affect patients, such as a physician purposefully ordering a too large quantity of drugs (e.g. two tubes instead of one) for a patient due to being unsure what quantity the hospital pharmacy would eventually deliver. Some workarounds solely affect the organization ‘at large’, such as decreased financial reimbursement for the hospital resulting from clinicians not registering certain treatments due to not knowing which ones are to be registered in the EHR to have them reimbursed through a so-called ‘Diagnosis and Treatment Combination’. Most workarounds affected two or three stakeholders. For example, nurses would register batches of around 15 bleedings per hemophilia patient in a tailor-made standalone database due to an unintended technical issue of the EHR accepting only one bleeding registration per minute (with the system used prior to the EHR, registering a bleeding would only take several seconds). This workaround affected all three stakeholders. First, the clinicians themselves had to maintain two systems by keeping e.g. patient contact details up to date in both systems. This was considered an additional administrative burden. Second, data stored in the tailor-made database is inaccessible to both internal and external clinicians not working at that specialty. Third, as a result, patient safety was jeopardized as the tailor-made database became an inaccurate and unreliable data source in both ordinary and emergency situations.
5.3. Anticipated versus unanticipated workarounds Unanticipated workarounds are created when EHR users face unexpected workflow constraints. Examples of such unexpected constraints are EHR users lacking the knowledge on how to use an EHR functionality for the first time, sudden technical issues such as the EHR system crashing, or a required data entry option not being available in the displayed list of options. In contrast, EHR users apply anticipated workarounds in situations where they know beforehand they will experience workflow constraints. For example, a physician indicated it is impossible to examine patients on a treatment table while simultaneously entering patient data into the EHR from behind his desk. In these situations, the physician wrote relevant patient data on paper while examining a patient and entered this data in the EHR after the patient consultation session. 5.4. Incidental versus routinized workarounds Incidental workarounds are temporarily used to overcome workflow constraints mainly caused by unconventional events and have little reason to be repeated. For example, after a system update, multiple physicians occasionally experienced EHR crashes when they opened the ‘growth analyzer’ functionality used to document and monitor patient growth curves. Whenever such crashes occurred, for the time being,
6.2. Impact Workarounds were analyzed on three impact-related dimensions: patient safety, effectiveness of care and efficiency of care [27]. Two properties of workaround impact were identified.
74
International Journal of Medical Informatics 125 (2019) 71–78
V. Blijleven, et al.
defines the work system and its five components constituting the context in which EHR workarounds are created, rationales for workaround creation, EHR workaround attributes, and the possible effects of a workaround on clinical processes. Design flaws or incompatibilities between work system components caused clinicians to experience obstacles in performing their tasks. As a result, they created workarounds to enable completion of clinical tasks. We identified 15 rationales, each relating to one of the five work system components. We further found that each workaround has a scope indicating whether the workaround solely affects the healthcare professional who created the workaround or extends to the patient, organization or a combination thereof, and impact indicating consequences of the workaround for patient safety, effectiveness of care and efficiency of care including related favorability and time-bound properties. Finally, we identified four attributes defining workarounds: cascadedness (whether a workaround can initiate a series of additional workarounds being created or not), avoidability (whether the application of a workaround is inevitable in order to proceed with one’s workflow or is avertible), anticipatedness (whether the application of a workaround is expected beforehand by its user or unexpected), and repetitiveness (whether the workaround being applied is an incidental occurrence or has become ingrained into one’s daily work routine). The four attributes defining EHR workarounds that we identified resemble the ‘features’ of EHR workarounds used in small-to-medium size primary care practices: ‘avoidable versus unavoidable’, ‘temporary versus routinized’ and ‘deliberately chosen versus unplanned’ [35]. Our attribute avoidability (avoidable versus unavoidable) is literally and semantically identical to [35]. Our attribute repetitiveness (incidental versus routinized) partly resonates with the feature ‘temporary versus routinized’ in [35]: although routinized has an identical meaning, temporary in [35] is determined by a certain period of time whereas incidental is determined by frequency of use. Our attribute anticipatedness (anticipated versus unanticipated) is syntactically different from ‘deliberately chosen versus unplanned’ in [35]. However, an unanticipated workaround could be considered ‘unplanned’ in [35], whereas an ‘anticipated’ workaround need not necessarily be ‘deliberately chosen’. Finally, we identified an attribute not described in [35]: cascading versus non-cascading. Cascading workarounds initiate the creation of additional workarounds. This attribute has also been described in [25] although not specifically related to EHR usage. Furthermore, we saw that these four attributes are dynamic and may take a different value (yes/no) at varying moments in time. For example, an unanticipated workaround may become anticipated if the situation that triggered the workaround persists, an unavoidable workaround may become avoidable after a bug that enforced the workaround has been resolved through a system update, and an incidental workaround to become routinized if its application is sustained over a longer period of time. Furthermore, multiple of the foregoing attributes can be simultaneously applicable to a workaround as also described in [35]. For instance, incidental workarounds may occur at unexpected moments, also making them unanticipated workarounds. Likewise, cascading workarounds may have become part of daily workflows, thereby also making them routinized workarounds. All the foregoing properties illustrate the multifaceted nature of workarounds. Studies on workarounds should therefore not solely focus on analyzing and categorizing workarounds at face value, but also take into account their situational perspectives as well as short-term and long-term consequences. Our framework, SEWA, provides a grounded foundation for performing comprehensive sociotechnical analyses of EHR workarounds. A
First, the impact of workarounds can be favorable, unfavorable, or neutral. Favorable workarounds improve patient safety, effectiveness of care and/or efficiency of care, whereas unfavorable workarounds have the opposite effect. Neutral workarounds have a negligible impact. For example, a favorable workaround concerned a physician writing patient data from other EHR tabs down on paper as a memory aid to avoid excessive toggling between tabs while writing a progress note. This improved the efficiency of the physician’s workflow with a negligible impact on patient safety or effectiveness of care. An observed unfavorable workaround concerned physicians entering the same patient data in two near-identical EHR data fields as they were unsure which data field entry would be forwarded to their colleague. This negatively impacted efficiency of care, but had negligible consequences for patient safety or effectiveness of care. However, the impact of most workarounds is rarely entirely favorable, unfavorable, or neutral. Tradeoffs also frequently exist between their impact on patient safety, effectiveness and/or efficiency of care. Second, the impact of workarounds on patient safety, effectiveness of care and efficiency of care may be seen directly, after a time lag, or both. For example, the impact of a physician drawing graphs on paper as he could not generate the desired chart or graph in the EHR (e.g. line chart instead of desired pie chart) was immediately visible: effectiveness of care increased, efficiency of care decreased, and patient safety was unaffected. In contrast, in another case, a physician noticed that the left foot of a patient was swollen. Entering ‘swelling’ or ‘swollen’ into the EHR’s Problem List search field yielded no results, nor did entering ‘edema’. She eventually settled with ‘arthralgia’ from the predefined list of symptoms displayed by the EHR as it most closely matched her diagnosis. Although this workaround allowed her to proceed with her workflow (i.e. visible increase in efficiency of care), she argued that if the EHR forced users to keep on entering data not matching the actual diagnosis, in the long run, a patient’s record may simply no longer provide an accurate representation of a patient’s medical history (i.e. time lagged decrease in patient safety). 7. Discussion and conclusion Thus far, a conceptual framework to address challenges inherent in studying workarounds emerging from EHR usage has not been developed yet. Previous studies that analyzed these workarounds may have been limited by a lack of such a conceptual framework. Well-known technology evaluation frameworks such as the Technology Acceptance Model (TAM) [39], Unified Theory of Acceptance and Usage of Technology (UTAUT) [40], and Human Organization Technology Fit (HOTFit) [41] offer great means to gain insight into the determinants of adoption of information technology by their users. However, they lack the domain specificity of (academic) hospitals and EHRs due to their generic nature. In addition, the creation of workarounds follows adoption of an EHR. EHR workaround analysis does not focus on intentions and degree of acceptance of using the EHR (usage is generally mandatory), but on rationales and in particular the consequences of EHR usage on healthcare providers’ workflows (and in our case resulting consequences for patient safety, effectiveness of care and efficiency of care). Moreover, a framework to support EHR workaround analysis should incorporate a clear sociotechnical perspective which is necessary since we found that workarounds are not solely the result of technical EHR-related factors, but also of human, organizational and task-related factors. To overcome this limitation, we introduce a framework designed to support sociotechnical analyses of EHR workarounds in healthcare settings: SEWA. SEWA, based on SEIPS [36],
75
International Journal of Medical Informatics 125 (2019) 71–78
V. Blijleven, et al.
major assumption of SEWA is, as with SEIPS [36], that the various components of the work system cannot be viewed as independently influencing the creation of EHR workarounds. As with other components of complex adaptive systems, key to SEWA is that the five work system components interact and depend on one another. Depending on the conditions, a change in one aspect of a work system component may lead to small but likewise large changes in other parts of the work system. As such, the five components must be studied in relation to each other in analyzing underlying rationales for EHR workarounds, their attributes, scope and impact on clinical processes. The value of SEWA is threefold. First, SEWA can be used by other healthcare organizations to analyze and resolve EHR workarounds. Specifically, multidisciplinary teams consisting of e.g. physicians, nurses, hospital management, EHR developers and quality assurance staff may jointly analyze workarounds by using SEWA to determine their rationales, scope, impact, and attributes. A priority for the workaround to be resolved may subsequently be assigned, including the perceived most effective way to resolve the workaround. SEWA may similarly be used to reflect on the overall current configuration of the work system to prevent unfavorable workarounds from occurring, as well as discuss possible future work system configurations to see how a redesign of the work system would positively or negatively impact the interaction between the components and resulting outcomes. Finally, workarounds are not set in stone but subject to gradual change caused by e.g. changes in one’s knowledge of the EHR, hospital policies, care directives or system updates. Multiple snapshots of SEWA can be taken over time and compared to gain insights in the evolution of workarounds. This study has several limitations. First, this study was performed at a large academic hospital. EHRs in academic hospitals differ from their non-academic counterparts [42], hence the results should be interpreted with care. Second, the EHR had been in use for little over half a year since our first observations began. Several workaround rationales became increasingly or decreasingly prevalent over time as time progressed. For instance, knowledge-related workarounds occurred more frequently at the start of our observations, whereas the need to enter patient data with greater or lesser specificity increasingly came to the fore at the end of our observations. Not all workarounds therefore seem to be equally represented throughout the EHR’s adoption timeframe. Third, we may not have captured all workarounds used in practice. However, observations continued till the research team agreed that data saturation was achieved. This was confirmed by the large number and broad variety of workarounds observed. Fourth, to limit the influence of the Hawthorne effect, we clearly communicated to participants ‘what is in it for them’, that the EHR was observed rather than the participant, that all data is made fully anonymous. In addition, the audiovisual camera was permanently and unobtrusively installed and observers positioned at a safe distance from the healthcare professional using the EHR. Future research should explore whether SEWA applies to settings other than academic hospitals. Furthermore, SEWA could be extended by a more comprehensive set of indicators for patient safety, effectiveness of care and efficiency of care to more accurately measure how (un)favorably a workaround affects each of these dimensions. Finally, SEWA should be subject to ongoing refinement through application in practice. The identification of hitherto unknown workaround rationales and workaround attributes could further strengthen SEWA as a foundation to analyze EHR workarounds and their impact to achieve safer,
more effective and more efficient care. Authors’ contributions Conceived and designed the study: VB, KK,MJ. Collected the data: VB. Analyzed the data: VB, KK, MJ. Wrote the manuscript: VB. Edited the manuscript: KK, MJ. All authors read and approved the final manuscript. Conflicts of interests The authors declare that they have no conflicting interests. Funding This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors. Summary points What was already known on the topic:
• Discrepancies frequently occur between electronic health re•
cord (EHR) system-dictated workflows and actual workflows of EHR users, causing impeded workflows that lead to the development and usage of EHR workarounds Although EHR workarounds may seem favorable at first sight, they are generally suboptimal and may jeopardize patient safety, effectiveness of care and efficiency of care What this study added to our knowledge:
• A conceptual framework to address challenges inherent in
•
• •
76
studying workarounds emerging from EHR usage had hitherto not been developed. Existing technology evaluation frameworks offer great means to gain insight into the determinants of adoption of information technology by their users, but have a strong technology focus and lack the domain specificity of (academic) hospitals and EHRs Our framework, SEWA, provides a grounded foundation for performing sociotechnical analyses of EHR workarounds. It defines the work system and its components (human, organization, technology, task, physical environment) in which EHR workarounds are created, 15 rationales for EHR workaround creation, 4 EHR workaround attributes, and the possible effects of EHR workarounds on clinical processes and outcomes Contrary to existing frameworks, SEWA has a clear sociotechnical perspective that is necessary since we found that EHR workarounds are not solely the result of technical EHRrelated factors, but also of human, organization and taskrelated factors SEWA can be used by researchers and practitioners to identify, analyze and resolve workarounds, and thereby contribute to safer, higher quality and more efficient delivery of care.
International Journal of Medical Informatics 125 (2019) 71–78
V. Blijleven, et al.
Appendix A. Identified Rationales for EHR Workarounds Table A1 Overview of identified rationales for EHR workarounds, from [27]. Rationale for EHR Workaround
Definition
Declarative knowledge Procedural knowledge Memory aid
Not knowing how to use (a part of) the EHR to accomplish a task. Knowing how but not being proficient enough to use a part of the EHR to accomplish a task. Writing patient data down on paper (e.g. keywords) or adding visual elements to parts of text in a progress note (e.g. boldfacing, italicizing or underlining) to remind oneself. Storing patient data that is perceived important by the EHR user for other colleagues to be noticed in a data field other than the intended field in the EHR. Informal understandings among healthcare professionals leading to the creation and dissemination of workarounds (e.g. mimicking workarounds devised by colleagues to accomplish a task, or working around the system upon as friendly requested or enforced by a fellow clinician). High behavioral user cost in accomplishing a task. (A part of the) EHR halting, crashing or slowing down, hampering the EHR user in accomplishing a task. Preferring a different data view (e.g. visualization by means of charts or graphs rather than plain text). Needing to enter or request patient data with greater or lesser specificity than offered or enforced by the EHR. Inability to perform multiple tasks at once (e.g. simultaneously treating a patient on a treatment table as well as entering patient data into the EHR). Valuing patient interaction over computer interaction (i.e. writing things down on paper and afterwards entering this into the EHR). Using an alternative way to accomplish a task that improves actual efficiency. Not having (direct) access to required historical data due to data not having been imported from previously used systems to the current EHR EHR enforcing user to enter patient data of which neither the user nor the patient has knowledge of. EHR not offering the required data entry option (e.g. 3.75 mg rather than the available options 2.5 mg or 5 mg).
Awareness Social norms Usability Technical issues Data presentation Patient data specificity Task interference Commitment to patient interaction Efficiency Data migration policy Enforced data entry Required data entry option missing
Appendix B. Examples of Observed Workarounds per Workaround Attribute
Table B1 Examples of observed workarounds per workaround attribute. Workaround Attribute
Description
Cascading
Workaround initiating the creation of one or multiple additional workarounds.
Examples from direct observations
the intended dosage (e.g. 1.5 tablets per day) in a separate progress note as • Describing EHR forced user to choose between 1 and 2 tablets in the medication order form (where
• Non-cascading
Workaround not initiating the creation of additional workarounds.
Avoidable
Workaround that is not required to proceed with one’s workflow.
Unavoidable
Workaround that is required to proceed with one’s workflow.
Anticipated
Workaround that is used when upcoming workflow constraints are known beforehand.
Unanticipated
Workaround that is created when facing unexpected workflow constraints.
Incidental
Workaround that is temporarily used to overcome workflow constraints.
Routinized
Workaround that is ingrained into workflows is a common way of dealing with workflow constraints.
• • • • • • • • • • • • • • •
the user entered 1 tablet). Additional workarounds occurred when entering data in a separate progress note. Creating a patient progress note labeling system to structure a great number of progress notes as a result of not using a required standardized data entry template (e.g. patient progress notes labeled with red contain critical data, notes labeled with yellow contain important but not critical data, etc.). Skipping data fields in a standardized data entry template that are considered inapplicable (e.g. smoking history when seeing toddlers). Entering the same patient data in two near-identical data fields due to not knowing which data field is forwarded to a colleague. Not registering treatments as required effort to do so is perceived to outweigh benefits. Editing automatically generated patient letters because of e.g. undesirable font type, size, or color. Knowingly writing down patient data in data fields that are inappropriate for the data that have to be entered, because these fields are easier to locate than the appropriate data fields. Entering a diagnosis that most closely matches the actual diagnosis as the intended data entry option is missing. Leaving data field blank when the right option for reason for stopping medication is not offered in the drop-down list. Requesting lab results from longer than 5 years ago via an online form, as physician knew that the required lab results for more than 5 years ago have not been migrated to the EHR. Manually checking if patients had arrived in the waiting room as the arrival notification system was known to be down. Lacking knowledge of how to use an EHR functionality for the first time. Asking colleagues how to order a certain drug in an emergency situation due to not knowing how to do so. Writing down patient data on paper when seeing a patient potentially carrying MRSA in a different room (out of precaution to prevent spread of MRSA) where the EHR could not be accessed. This data was reentered into the EHR as soon as the physician returned to her former location. Writing down patient data on paper when the EHR crashed and reentering this data in the EHR after the EHR rebooted. Habitually clicking away excessive inappropriate or redundant alerts due to EHR generating alerts on a too low specificity level. Habitually correcting mistakes in automated EHR-generated output (e.g. letters to GPs or medication instructions) as output contained lay-out mistakes.
77
International Journal of Medical Informatics 125 (2019) 71–78
V. Blijleven, et al.
References [22]
[1] A. Boonstra, A. Versluis, J.F. Vos, Implementing electronic health records in hospitals: a systematic literature review, BMC Health Serv. Res. 14 (2014) 370 PMID: 25190184. [2] J. Henry, Y. Pylypchuk, T. Searcy, V. Patel, Adoption of Electronic Health Record Systems Among U.S. Non-Federal Acute Care Hospitals: 2008-2015, [Internet] Available from: ONC Data Brief, 2016, https://www.healthit.gov/sites/default/ files/briefs/2015_hospital_adoption_db_v17.pdf. [3] A. Robertson, K. Cresswell, A. Takian, D. Petrakaki, S. Crowe, T. Cornford, et al., Implementation and adoption of nationwide electronic health records in secondary care in England: qualitative analysis of interim results from a prospective national evaluation, BMJ 341 (2010) c4564PMID: 20813822. [4] D. Blumenthal, M. Tavenner, The “Meaningful Use” regulation for electronic health records, N. Engl. J. Med. 363 (6) (2010) 501–504, https://doi.org/10.1056/ NEJMp1006114 PMID: 20647183. [5] D. Blumenthal, J.P. Glaser, Information technology comes to medicine, N. Engl. J. Med. 356 (24) (2007) 2527–2534 PMID: 17568035. [6] J.D. Chaparro, D.C. Classen, M. Danforth, D.C. Stockwell, C.A. Longhurst, National trends in safety performance of electronic health record systems in children’s hospitals, J. Am. Med. Inform. Assoc. 24 (2) (2017) 268–274, https://doi.org/10.1093/ jamia/ocw134. [7] J. King, V. Patel, E.W. Jamoom, M.F. Furukawa, Clinical benefits of electronic health record use: national findings, Health Serv. Res. 49 (1 Pt 2) (2014) 392–404 PMID: 24359580. [8] M. Plantier, N. Havet, T. Durand, N. Caquot, C. Amaz, P. Biron, et al., Does adoption of electronic health records improve the quality of care management in France? Results from the French e-SI (PREPS-SIPS) study, Int. J. Med. Inform. 102 (6) (2017) 156–165, https://doi.org/10.1016/j.ijmedinf.2017.04.002. [9] A.D. Cheriff, A.G. Kapur, M. Qiu, C.L. Cole, Physician productivity and the ambulatory EHR in a large academic multi-specialty physician group, Int. J. Med. Inform. 79 (7) (2010) 492–500 PMID: 20478738. [10] J. Howard, E.C. Clark, A. Friedman, J.C. Crosson, M. Pellerano, B.F. Crabtree, et al., Electronic health record impact on work burden in small, unaffiliated, communitybased primary care practices, J. Gen. Intern. Med. 28 (1) (2013) 107–113 PMID: 22926633. [11] J.A. Zlabek, J.W. Wickus, M.A. Mathiason, Early cost and safety benefits of an inpatient electronic health record, J. Am. Med. Inform. Assoc. 18 (2) (2011) 169–172 PMID: 21292703. [12] J. Adler-Milstein, C. Salzberg, C. Franz, E.J. Orav, J.P. Newhouse, D.W. Bates, Effect of electronic health records on health care costs: longitudinal comparative evidence from community practices, Ann. Intern. Med. 159 (2) (2013) 97–104 PMID: 23856682. [13] C.W. Carspecken, P.J. Sharek, C. Longhurst, N.M. Pageler, A clinical case of electronic health record drug alert fatigue: consequences for patient outcome, Pediatrics 131 (6) (2013) e1970–3 PMID: 23713099. [14] M.E. Flanagan, J.J. Saleem, L.G. Millitello, A.L. Russ, B.N. Doebbeling, Paper- and computer-based workarounds to electronic health record use at three benchmark institutions, J. Am. Med. Inform. Assoc. 20 (e1) (2013) e59–66 PMID: 23492593. [15] D.F. Sittig, A. Wright, J.S. Ash, H. Singh, New unintended adverse consequences of electronic health records, Yearb Med. Inform. (1) (2016) 7. [16] J.S. Ash, P.N. Gorman, M. Lavelle, T.H. Payne, T.A. Massaro, G.L. Frantz, et al., A cross-site qualitative study of physician order entry, J. Am. Med. Inform. Assoc. 10 (2) (2003) 188–200 PMID: 12595408. [17] J. Aarts, J.S. Ash, M. Berg, Extending the understanding of computerized physician order entry: implications for professional collaboration, workflow and quality of care, Int. J. Med. Inform. 76 (2007) S4–S13. [18] E.M. Campbell, K.P. Guappone, D.F. Sittig, R.H. Dykstra, J.S. Ash, Computerized provider order entry adoption: implications for clinical workflow, J. Gen. Intern. Med. 24 (1) (2009) 21–26. [19] K.M. Cresswell, A. Worth, A. Sheikh, Integration of a nationally procured electronic health record system into user work practices, BMC Med. Inform. Decis. Mak. 12 (1) (2012) 1–12. [20] E.M. Campbell, D.F. Sittig, J. Ash, K.P. Guappone, R.H. Dykstra, Types of unintended consequences related to computerized provider order entry, J. Am. Med. Inform. Assoc. 13 (5) (2006) 547–556 PMID: 16799128. [21] J.S. Ash, D.F. Sittig, R.H. Dykstra, K.P. Guappone, J.D. Carpenter, V. Seshadri,
[23] [24] [25] [26] [27]
[28] [29] [30]
[31] [32] [33] [34] [35] [36]
[37] [38] [39] [40] [41]
[42]
78
Categorizing the unintended sociotechnical consequences of computerized provider order entry, Int. J. Med. Inform. 76 (Suppl 1) (2007) S21–7 PMID: 16793330. J.S. Ash, D.F. Sittig, E.G. Poon, K.P. Guappone, E.M. Campbell, R.H. Dykstra, The extent and importance of unintended consequences related to computerized provider order entry, J. Am. Med. Inform. Assoc. 14 (4) (2007) 415–423. T. Greenhalgh, H.W. Potts, G. Wong, P. Bark, D. Swinglehurst, Tensions and paradoxes in electronic patient record research: a systematic literature review using the meta‐narrative method, Milbank Q. 87 (4) (2009) 729–788. J.R.B. Halbesleben, D.S. Wakefield, B.J. Wakefield, Work-arounds in health care settings: literature review and research agenda, Health Care Manage Rev. 33 (1) (2008) 2–12. M. Kobayashi, S.R. Fussell, Y. Xiao, F.J. Seagull, Work Coordination, Workflow, and Workarounds in a Medical Context. CHI’ 05 Extended Abstracts on Human Factors in Computing Systems, ACM, Portland, OR, USA, 2005, pp. 1561–1564. R. Koppel, T. Wetterneck, J.L. Telles, B.T. Karsh, Workarounds to barcode medication administration systems: their occurrences, causes, and threats to patient safety, J. Am. Med. Inform. Assoc. 15 (4) (2008) 408–423 PMID: 18436903. V. Blijleven, K. Koelemeijer, M.A. Wetzels, M.W.M. Jaspers, Workarounds emerging from electronic health record system usage: consequences for patient safety, effectiveness of care and efficiency of care, J. Med. Intern. Res.: Hum. Factors 4 (4) (2017) e27, https://doi.org/10.2196/humanfactors.7978. S. Menon, D.R. Murphy, H. Singh, D. Sittig, Workarounds and test results follow-up in electronic health record-based primary care, Appl. Clin. Inform. 7 (2) (2016) 543–559. J.J. Saleem, A.L. Russ, C.F. Justice, H. Hagg, P.R. Ebright, P.A. Woodbridge, et al., Exploring the persistence of paper with the electronic health record, Int. J. Med. Inform. 78 (9) (2009) 618–628 PMID: 19464231. J.J. Saleem, A.L. Russ, A. Neddo, P.T. Blades, B.N. Doebbeling, B.H. Foresman, Paper persistence, workarounds, and communication breakdowns in computerized consultation management, Int. J. Med. Inform. 80 (7) (2011) 466–479 PMID: 21530383. G. Ser, A. Robertson, A. Sheikh, A qualitative exploration of workarounds related to the implementation of national electronic health records in early adopter mental health hospitals, PLoS One 9 (1) (2014) e77669. Z. Niazkhani, H. Pirnejad, H. van der Sijs, J. Aarts, Evaluating the medication process in the context of CPOE use: the significance of working around the system, Int. J. Med. Inform. 80 (7) (2011) 490–506. H. Van Der Sijs, I. Rootjes, J. Aarts, The shift in workarounds upon implementation of computerized physician order entry, Stud. Health Technol. Inform. 169 (2010) 290–294. O. Whooley, Diagnostic ambivalence: psychiatric workarounds and the diagnostic and statistical manual of mental disorders, Sociol. Health Illn. 32 (3) (2010) 452–469 PMID: 20415790. A. Friedman, J.C. Crosson, J. Howard, E.C. Clark, M. Pellerano, B.-T. Karsh, et al., A typology of electronic health record workarounds in small-to-medium size primary care practices, J. Am. Med. Inform. Assoc. 21 (e1) (2014) e78–e83. R.J. Holden, P. Carayon, A.P. Gurses, P. Hoonakker, A.S. Hundt, A.A. Ozok, et al., SEIPS 2.0: a human factors framework for studying and improving the work of healthcare professionals and patients, Ergonomics 56 (11) (2013) 1669–1686 PMID: 24088063. B.G. Glaser, A.L. Strauss, The Discovery of Grounded Theory: Strategies for Qualitative Research, Transaction Publishers, 2009. V. Blijleven, K. Koelemeijer, M.W.M. Jaspers, Exploring workarounds related to electronic health record system usage: a study protocol, JMIR Res. Protocols 6 (4) (2017) e72, https://doi.org/10.2196/resprot.6766. F.D. Davis, Perceived usefulness, perceived ease of use, and user acceptance of information technology, MIS Q. 13 (3) (1989) 319–340, https://doi.org/10.2307/ 249008. V. Venkatesh, M.G. Morris, G.B. Davis, F.D. Davis, User acceptance of information technology: toward a unified view, MIS Q. 27 (3) (2003) 425–478. M.M. Yusof, J. Kuljis, A. Papazafeiropoulou, L.K. Stergioulas, An evaluation framework for health information systems: human, organization and technology-fit factors (HOT-fit), Int. J. Med. Inform. 77 (June 6) (2008) 386–398, https://doi.org/ 10.1016/j.ijmedinf.2007.08.011 PMID: 17964851. S.O. Zandieh, K. Yoon-Flannery, G.J. Kuperman, D.J. Langsam, D. Hyman, R. Kaushal, Challenges to EHR implementation in electronic- versus paper-based office practices, J. Gen. Intern. Med. 23 (6) (2008) 755–761 PMID: 18369679.
Contents lists available at ScienceDirect
International Journal of Medical Informatics journal homepage: www.elsevier.com/locate/ijmedinf
SEWA: A framework for sociotechnical analysis of electronic health record system workarounds
T
Vincent Blijlevena,b, , Kitty Koelemeijera, Monique Jaspersb ⁎
a b
Center for Marketing and Supply Chain Management, Nyenrode Business University, Straatweg 25, 3621 BG, Breukelen, the Netherlands Department of Medical Informatics, Academic Medical Center, Meibergdreef 15, 1105 AZ, Amsterdam, the Netherlands
ARTICLE INFO
ABSTRACT
Keywords: Electronic health records Framework Patient safety Unintended consequences Workarounds Workflow
Objective: To develop a conceptual framework, SEWA, to address challenges of studying workarounds emerging from Electronic Health Record (EHR) system usage. Materials and methods: SEWA is based on direct observations and follow-up interviews with physicians, nurses and clerks using their EHR at a large academic hospital. SEWA was developed by an iterative process: each new version was reviewed by experts (case study participants, hospital management, EHR developers) and refined accordingly till deemed final. Results: SEWA defines the work system and its five components constituting the context in which EHR workarounds are created. It also contains 15 rationales for creating EHR workarounds. Furthermore, four attributes are included that define EHR workarounds: cascadedness, anticipatedness, avoidability, and repetitiveness. Finally, SEWA lists the possible effects of workarounds on outcomes of clinical processes in terms of scope and impact. Discussion: SEWA provides a grounded foundation for performing sociotechnical analyses of EHR workarounds based on components of the work system. SEWA can likewise be supportive in planning redesign efforts of the work system. Finally, workarounds are subject to gradual change caused by e.g. changes in one’s knowledge of the EHR, hospital policies, care directives, and system updates. Snapshots of SEWA can be taken over time and compared to gain insights into the evolution of workarounds. Conclusion: Given the absence of a sociotechnical framework to study EHR workarounds, SEWA could aid researchers and practitioners to identify, analyze and resolve workarounds, and thereby contribute to improved patient safety, effectiveness of care and efficiency of care.
1. Background and significance Electronic health record systems (EHRs) are increasingly being adopted by healthcare organizations worldwide [1–3], spurred by government-led incentives [4] and in pursuit of favorable outcomes related to patient safety [5–7], quality of care [7,8], efficiency of care [7,9,10], and reduced costs [11,12]. However, realizing the expected benefits of adopting EHRs remains challenging. Examples are EHR users experiencing ‘alert fatigue’ [13], not all clinical work being supported by EHRs [14], unavailability of complete clinical information at the point of care [15], difficulties in finding the right information in the EHR [16], and significantly disrupted workflows due to modified timing, sequence of work practices and revised professional responsibilities [17–19]. Many causes of unintended consequences of EHRs can be traced
⁎
back to discrepancies between the behavior, intentions and expectations of EHR users, and the workflows dictated by EHRs [18,20–23]. When users experience workflow mismatches they create workarounds [24]. Workarounds have been defined as “informal temporary practices for handling exceptions to normal workflow” [25] that “do not follow explicit or implicit rules, assumptions, workflow regulations, or intentions of systems designers” [26]. The impact of workarounds may extend far beyond solely allowing EHR users to proceed with their workflow to get their clinical tasks done. Existing studies on EHR workarounds tend to report their findings from a single out of four angles we identified. First, EHR users have distinct rationales for creating workarounds such as a lack of declarative knowledge (i.e. not knowing how to use (a part of) the EHR), memory aids (e.g. writing patient data down on paper during an outpatient consultation session and entering this data into the EHR later
Corresponding author at: Straatweg 25, 3621 BG, Breukelen, the Netherlands. E-mail addresses: [email protected], [email protected] (V. Blijleven).
https://doi.org/10.1016/j.ijmedinf.2019.02.012 Received 11 May 2018; Received in revised form 28 September 2018; Accepted 28 February 2019 1386-5056/ © 2019 Elsevier B.V. All rights reserved.
International Journal of Medical Informatics 125 (2019) 71–78
V. Blijleven, et al.
Fig. 1. SEWA: Sociotechnical Electronic health record Workaround Analysis framework. *No identified workaround rationales could be associated with the physical environment.
on), and technical issues (e.g. EHR halting, crashing or slowing down) [14,27–31]. Second, the impact of workarounds may be limited to the EHR user or extend beyond the EHR user to the patient, the overall organization, or a combination of the foregoing stakeholders [27]. Third, EHR workarounds may lead to safer, higher quality and more efficient care and thus be considered favorable [21,27,32,33], but may likewise lead to less safe, lower quality and less efficient care and thus be considered unfavorable [24,26,27,34]. Fourth, workarounds are defined by several attributes, such as being temporarily versus routinely used, avoidable versus unavoidable, and/or deliberately chosen versus unplanned [35]. A sociotechnical systems perspective incorporating all these four angles could be supportive in examining, typifying, and subsequently resolving EHR workarounds. To this end we present a comprehensive framework – SEWA (Sociotechnical Ehr Workaround Analysis) – based
on a study on EHR workarounds in an academic hospital setting. The framework is inspired by the Systems Engineering Initiative for Patient Safety (SEIPS) framework [36]. 2. Materials and methods We followed a qualitative approach consisting of non-participant direct observations combined with semi-structured follow-up interviews with 31 physicians, 13 nurses and 3 clerks using their EHR while performing daily (clinical) tasks at a large academic hospital. Case study participants were recruited via the director of medical staff, the director of operations, and participants suggesting other participants. No institutional board review was required. All healthcare professionals and patients participating in the study were asked for an informed consent before any recording would take place.
72
International Journal of Medical Informatics 125 (2019) 71–78
V. Blijleven, et al.
may act simultaneously to shape processes and generate outcomes [36]. As shown in Fig. 1, these components are:
All observations and interviews were audiovisually recorded and analyzed following an inductive bottom-up coding strategy [37]. The direct observations allowed us to observe workarounds while work practices and EHR use by healthcare professionals unfolded. The interviews allowed us to gain greater insight into each observed workaround, more specifically to determine their rationale, scope, impact and attributes. The EHR is used by over 8000 hospital staff and concerns a hospital-wide integrated suite of healthcare software supporting functions related to patient care and management, registration and scheduling, clinical systems for health care providers, ancillary laboratory, pharmacy, and radiology systems and a billing system. Around 200 h of audiovisual material were captured. Data analysis consisted of four main steps, with each step focusing on one of the four angles of the framework. The first step concerned determining the rationale of each workaround. Each rationale was then associated with one out of five work system components derived from the SEIPS framework [36]. In the second step the scope of each workaround was determined, more specifically which stakeholders (i.e. the patient, healthcare professional, the overall organization or a combination thereof) were or could be affected by the workaround. The third step concerned determining the impact of each workaround on patient safety, effectiveness of care and efficiency of care. Finally, the fourth step concerned revealing attributes of EHR workarounds: inherent properties defining EHR workarounds of which their value (e.g. incidental versus routinized) may change over time. An extensive description of the data collection and analysis procedures has been published in [38]. Results of the first three steps have been described in [27]. This article reports on the fourth step and consolidates the results of all four steps into a comprehensive framework to analyze and typify EHR workarounds utilizing a sociotechnical view. Attributes of workarounds were defined through analyzing each identified workaround one-by-one with the purpose of revealing recurring properties appearing more than once and being applicable to multiple workarounds. The properties were identified by one of the authors and discussed in meetings with the members of the research team having expertise in software development, cognitive psychology, biomedical informatics, and behavioral economics. Case study participants, hospital management and EHR developers were then consulted to discuss, validate and refine the identified properties. Our framework, SEWA, was designed following an iterative process with the aim of converging the research results. The results of all four analysis steps were first combined to create an initial version. Followup versions were created over time with each version being reviewed and refined by experts (case study participants, hospital management and EHR developers) and validated accordingly.
• Person(s): healthcare professionals developing and using EHR workarounds • Technology and tools: the EHR and related information technology used by healthcare professionals • Task(s): clinical tasks performed by healthcare professionals • Organization: organizational conditions (e.g. care directives, hos•
pital policies) under which clinical tasks and EHR usage are performed Physical environment: the environment (e.g. outpatient examination room, inpatient ward) and its conditions (e.g. lighting, noise) in which clinical tasks are carried out by healthcare professionals
The work system components together constitute the context in which EHR workarounds are created by EHR users. We identified 15 rationales (as described in [27]): declarative knowledge, procedural knowledge, memory aid, awareness, social norms, usability, technical issues, data presentation, patient data specificity, task interference, commitment to patient interaction, efficiency, data migration policy, enforced data entry, and required data entry option missing. A definition and several examples per workaround rationale are provided in Appendix A. Each of the 15 workaround rationales has been associated with one of the five work system components that reflects its source of origin [27]. The sources of origin provide clues about how related workarounds could be resolved. For example, workaround rationales associated with the Persons component may be resolved most effectively through personal training to promote optimal and proper EHR and Technology and Tools-associated rationales through EHR (re)design initiatives. 5. Attributes of EHR workarounds At the center of SEWA in Fig. 1 are EHR workarounds. Four distinct attributes were identified that define EHR workarounds: cascadedness, avoidability, anticipatedness and repetitiveness. Each attribute may be either ‘true’ or ‘false’ for each workaround. A definition and several examples of each are provided in Appendix B. 5.1. Cascading versus non-cascading workarounds Workarounds can be ‘cascading’, meaning their usage initiates the creation of additional workarounds. For example, the EHR studied offers an extensive standardized data entry template for entering patient data. However, because of a perceived low usability of the standardized data entry template, nearly all users preferred copy-pasting the majority of patient data from previous progress notes into a new progress note. Clinicians argued that a disadvantage of this workaround is that it causes patient data to get lost in the system. For instance, over 25 progress notes for a single patient were created within a week’s time. Clinicians rarely copy-pasted all information from the available progress notes into a new progress note, causing patient data harder to be found as more progress notes are created over time. Clinicians created other workarounds such as customized progress note labeling systems or added markup such as boldfacing or underlining text parts within progress notes in an attempt to counter this problem. Most workarounds we observed are non-cascading. Examples are physicians writing down keywords in a patient’s progress note in
3. Results SEWA illustrated in Fig. 1 consists of two major parts: (I) the work system and its components (inspired by the SEIPS framework) [36] constituting the context in which EHR workarounds are created including 15 associated rationales for workaround creation and EHR workarounds themselves defined by four attributes, and (II) the scope of workarounds in terms of stakeholders affected and their impact on patient safety, effectiveness of care and efficiency of care. 4. Work system components and associated workaround rationales The work system consists of five interconnected components that
73
International Journal of Medical Informatics 125 (2019) 71–78
V. Blijleven, et al.
physicians would write down patient data on paper and enter this data in the EHR after the EHR was booted up again. Such workarounds became obsolete as soon as the bug was resolved. In contrast, routinized workarounds are ingrained into the workflows of EHR users and have become a common way of dealing with workflow constraints. For example, a group of hemophilia nurses would enter certain patient data such as treatment regimens in the EHR and a standalone tailor-made database. This database was in use before the introduction of the EHR and perceived to be superior to the EHR with regard to quickly looking up certain patient data. When asked why they kept on using the database, nurses indicated that they always had worked this way, although they could abandon use of the database and only work with the EHR.
advance of a patient consultation as a personal reminder, physicians skipping data fields in the standardized data entry template because they considered these inapplicable, or a physician entering ‘x’ in a mandatory data field in order to proceed in the EHR because the supposed entry in the data field is not known or beyond his expertise. None of these workarounds initiated the creation of other workarounds. 5.2. Avoidable versus unavoidable workarounds Workarounds that can be avoided are not required to proceed with one’s workflow. Although avoidable workarounds can be easily abandoned, they remain in use primarily because of a lack of time, disinterest, or limited expected returns of changing one’s way of working. For example, a physician repetitively adjusted a frequently used predefined drug order set because it contained known mistakes. When asked if the drug order set could simply not be corrected in the backend of the EHR, she indicated that although she had been trained and authorized to configure some of the tools and workflows such as documentation and ordering, management did not give her the time to use these skills in practice. Unavoidable workarounds result from EHR users having no other way to proceed with their workflow unless they resort to a workaround. The creation of unavoidable workarounds is often driven by circumstances outside of the EHR user’s direct area of influence. Examples are enforced hospital policies or technical EHR constraints causing unintended workflow mismatches. For example, a physician who aimed to prescribe 1.5 tablets of 2.5 mg prednisone (3.75 mg total) for a patient per day was enforced to choose between 2.5 mg (Table A1) or 5 mg (Table B1). This was due to the drugs list being directly linked to the inventory of the hospital pharmacy (whereas certain drugs can be easily split in half by patients). The physician ordered 1 tablet (2.5 mg) per day but entered a textual description in multiple free (unstructured) text fields that the supposed dosage should be 1.5 tablets (3.75 mg) per day.
6. Scope and impact of EHR workarounds Ultimately, each EHR workaround has a scope and impact on care processes. Scope concerns the stakeholders (i.e. the patient, healthcare professional, organization, or a combination thereof) affected by the workaround. Impact concerns the consequence(s) each workaround has on patient safety, effectiveness of care and efficiency of care. 6.1. Scope Workarounds may solely affect the single EHR user who devised it, extend beyond the EHR user to the patient(s) under diagnosis, treatment or in follow-up, the work context of other healthcare providers or even to the organizational level, or a combination of these. For example, a physician re-entering and copy-pasting patient data from the EHR into a letter due to not knowing how to use the automatic letter generation tool solely affects himself. Other workarounds solely affect patients, such as a physician purposefully ordering a too large quantity of drugs (e.g. two tubes instead of one) for a patient due to being unsure what quantity the hospital pharmacy would eventually deliver. Some workarounds solely affect the organization ‘at large’, such as decreased financial reimbursement for the hospital resulting from clinicians not registering certain treatments due to not knowing which ones are to be registered in the EHR to have them reimbursed through a so-called ‘Diagnosis and Treatment Combination’. Most workarounds affected two or three stakeholders. For example, nurses would register batches of around 15 bleedings per hemophilia patient in a tailor-made standalone database due to an unintended technical issue of the EHR accepting only one bleeding registration per minute (with the system used prior to the EHR, registering a bleeding would only take several seconds). This workaround affected all three stakeholders. First, the clinicians themselves had to maintain two systems by keeping e.g. patient contact details up to date in both systems. This was considered an additional administrative burden. Second, data stored in the tailor-made database is inaccessible to both internal and external clinicians not working at that specialty. Third, as a result, patient safety was jeopardized as the tailor-made database became an inaccurate and unreliable data source in both ordinary and emergency situations.
5.3. Anticipated versus unanticipated workarounds Unanticipated workarounds are created when EHR users face unexpected workflow constraints. Examples of such unexpected constraints are EHR users lacking the knowledge on how to use an EHR functionality for the first time, sudden technical issues such as the EHR system crashing, or a required data entry option not being available in the displayed list of options. In contrast, EHR users apply anticipated workarounds in situations where they know beforehand they will experience workflow constraints. For example, a physician indicated it is impossible to examine patients on a treatment table while simultaneously entering patient data into the EHR from behind his desk. In these situations, the physician wrote relevant patient data on paper while examining a patient and entered this data in the EHR after the patient consultation session. 5.4. Incidental versus routinized workarounds Incidental workarounds are temporarily used to overcome workflow constraints mainly caused by unconventional events and have little reason to be repeated. For example, after a system update, multiple physicians occasionally experienced EHR crashes when they opened the ‘growth analyzer’ functionality used to document and monitor patient growth curves. Whenever such crashes occurred, for the time being,
6.2. Impact Workarounds were analyzed on three impact-related dimensions: patient safety, effectiveness of care and efficiency of care [27]. Two properties of workaround impact were identified.
74
International Journal of Medical Informatics 125 (2019) 71–78
V. Blijleven, et al.
defines the work system and its five components constituting the context in which EHR workarounds are created, rationales for workaround creation, EHR workaround attributes, and the possible effects of a workaround on clinical processes. Design flaws or incompatibilities between work system components caused clinicians to experience obstacles in performing their tasks. As a result, they created workarounds to enable completion of clinical tasks. We identified 15 rationales, each relating to one of the five work system components. We further found that each workaround has a scope indicating whether the workaround solely affects the healthcare professional who created the workaround or extends to the patient, organization or a combination thereof, and impact indicating consequences of the workaround for patient safety, effectiveness of care and efficiency of care including related favorability and time-bound properties. Finally, we identified four attributes defining workarounds: cascadedness (whether a workaround can initiate a series of additional workarounds being created or not), avoidability (whether the application of a workaround is inevitable in order to proceed with one’s workflow or is avertible), anticipatedness (whether the application of a workaround is expected beforehand by its user or unexpected), and repetitiveness (whether the workaround being applied is an incidental occurrence or has become ingrained into one’s daily work routine). The four attributes defining EHR workarounds that we identified resemble the ‘features’ of EHR workarounds used in small-to-medium size primary care practices: ‘avoidable versus unavoidable’, ‘temporary versus routinized’ and ‘deliberately chosen versus unplanned’ [35]. Our attribute avoidability (avoidable versus unavoidable) is literally and semantically identical to [35]. Our attribute repetitiveness (incidental versus routinized) partly resonates with the feature ‘temporary versus routinized’ in [35]: although routinized has an identical meaning, temporary in [35] is determined by a certain period of time whereas incidental is determined by frequency of use. Our attribute anticipatedness (anticipated versus unanticipated) is syntactically different from ‘deliberately chosen versus unplanned’ in [35]. However, an unanticipated workaround could be considered ‘unplanned’ in [35], whereas an ‘anticipated’ workaround need not necessarily be ‘deliberately chosen’. Finally, we identified an attribute not described in [35]: cascading versus non-cascading. Cascading workarounds initiate the creation of additional workarounds. This attribute has also been described in [25] although not specifically related to EHR usage. Furthermore, we saw that these four attributes are dynamic and may take a different value (yes/no) at varying moments in time. For example, an unanticipated workaround may become anticipated if the situation that triggered the workaround persists, an unavoidable workaround may become avoidable after a bug that enforced the workaround has been resolved through a system update, and an incidental workaround to become routinized if its application is sustained over a longer period of time. Furthermore, multiple of the foregoing attributes can be simultaneously applicable to a workaround as also described in [35]. For instance, incidental workarounds may occur at unexpected moments, also making them unanticipated workarounds. Likewise, cascading workarounds may have become part of daily workflows, thereby also making them routinized workarounds. All the foregoing properties illustrate the multifaceted nature of workarounds. Studies on workarounds should therefore not solely focus on analyzing and categorizing workarounds at face value, but also take into account their situational perspectives as well as short-term and long-term consequences. Our framework, SEWA, provides a grounded foundation for performing comprehensive sociotechnical analyses of EHR workarounds. A
First, the impact of workarounds can be favorable, unfavorable, or neutral. Favorable workarounds improve patient safety, effectiveness of care and/or efficiency of care, whereas unfavorable workarounds have the opposite effect. Neutral workarounds have a negligible impact. For example, a favorable workaround concerned a physician writing patient data from other EHR tabs down on paper as a memory aid to avoid excessive toggling between tabs while writing a progress note. This improved the efficiency of the physician’s workflow with a negligible impact on patient safety or effectiveness of care. An observed unfavorable workaround concerned physicians entering the same patient data in two near-identical EHR data fields as they were unsure which data field entry would be forwarded to their colleague. This negatively impacted efficiency of care, but had negligible consequences for patient safety or effectiveness of care. However, the impact of most workarounds is rarely entirely favorable, unfavorable, or neutral. Tradeoffs also frequently exist between their impact on patient safety, effectiveness and/or efficiency of care. Second, the impact of workarounds on patient safety, effectiveness of care and efficiency of care may be seen directly, after a time lag, or both. For example, the impact of a physician drawing graphs on paper as he could not generate the desired chart or graph in the EHR (e.g. line chart instead of desired pie chart) was immediately visible: effectiveness of care increased, efficiency of care decreased, and patient safety was unaffected. In contrast, in another case, a physician noticed that the left foot of a patient was swollen. Entering ‘swelling’ or ‘swollen’ into the EHR’s Problem List search field yielded no results, nor did entering ‘edema’. She eventually settled with ‘arthralgia’ from the predefined list of symptoms displayed by the EHR as it most closely matched her diagnosis. Although this workaround allowed her to proceed with her workflow (i.e. visible increase in efficiency of care), she argued that if the EHR forced users to keep on entering data not matching the actual diagnosis, in the long run, a patient’s record may simply no longer provide an accurate representation of a patient’s medical history (i.e. time lagged decrease in patient safety). 7. Discussion and conclusion Thus far, a conceptual framework to address challenges inherent in studying workarounds emerging from EHR usage has not been developed yet. Previous studies that analyzed these workarounds may have been limited by a lack of such a conceptual framework. Well-known technology evaluation frameworks such as the Technology Acceptance Model (TAM) [39], Unified Theory of Acceptance and Usage of Technology (UTAUT) [40], and Human Organization Technology Fit (HOTFit) [41] offer great means to gain insight into the determinants of adoption of information technology by their users. However, they lack the domain specificity of (academic) hospitals and EHRs due to their generic nature. In addition, the creation of workarounds follows adoption of an EHR. EHR workaround analysis does not focus on intentions and degree of acceptance of using the EHR (usage is generally mandatory), but on rationales and in particular the consequences of EHR usage on healthcare providers’ workflows (and in our case resulting consequences for patient safety, effectiveness of care and efficiency of care). Moreover, a framework to support EHR workaround analysis should incorporate a clear sociotechnical perspective which is necessary since we found that workarounds are not solely the result of technical EHR-related factors, but also of human, organizational and task-related factors. To overcome this limitation, we introduce a framework designed to support sociotechnical analyses of EHR workarounds in healthcare settings: SEWA. SEWA, based on SEIPS [36],
75
International Journal of Medical Informatics 125 (2019) 71–78
V. Blijleven, et al.
major assumption of SEWA is, as with SEIPS [36], that the various components of the work system cannot be viewed as independently influencing the creation of EHR workarounds. As with other components of complex adaptive systems, key to SEWA is that the five work system components interact and depend on one another. Depending on the conditions, a change in one aspect of a work system component may lead to small but likewise large changes in other parts of the work system. As such, the five components must be studied in relation to each other in analyzing underlying rationales for EHR workarounds, their attributes, scope and impact on clinical processes. The value of SEWA is threefold. First, SEWA can be used by other healthcare organizations to analyze and resolve EHR workarounds. Specifically, multidisciplinary teams consisting of e.g. physicians, nurses, hospital management, EHR developers and quality assurance staff may jointly analyze workarounds by using SEWA to determine their rationales, scope, impact, and attributes. A priority for the workaround to be resolved may subsequently be assigned, including the perceived most effective way to resolve the workaround. SEWA may similarly be used to reflect on the overall current configuration of the work system to prevent unfavorable workarounds from occurring, as well as discuss possible future work system configurations to see how a redesign of the work system would positively or negatively impact the interaction between the components and resulting outcomes. Finally, workarounds are not set in stone but subject to gradual change caused by e.g. changes in one’s knowledge of the EHR, hospital policies, care directives or system updates. Multiple snapshots of SEWA can be taken over time and compared to gain insights in the evolution of workarounds. This study has several limitations. First, this study was performed at a large academic hospital. EHRs in academic hospitals differ from their non-academic counterparts [42], hence the results should be interpreted with care. Second, the EHR had been in use for little over half a year since our first observations began. Several workaround rationales became increasingly or decreasingly prevalent over time as time progressed. For instance, knowledge-related workarounds occurred more frequently at the start of our observations, whereas the need to enter patient data with greater or lesser specificity increasingly came to the fore at the end of our observations. Not all workarounds therefore seem to be equally represented throughout the EHR’s adoption timeframe. Third, we may not have captured all workarounds used in practice. However, observations continued till the research team agreed that data saturation was achieved. This was confirmed by the large number and broad variety of workarounds observed. Fourth, to limit the influence of the Hawthorne effect, we clearly communicated to participants ‘what is in it for them’, that the EHR was observed rather than the participant, that all data is made fully anonymous. In addition, the audiovisual camera was permanently and unobtrusively installed and observers positioned at a safe distance from the healthcare professional using the EHR. Future research should explore whether SEWA applies to settings other than academic hospitals. Furthermore, SEWA could be extended by a more comprehensive set of indicators for patient safety, effectiveness of care and efficiency of care to more accurately measure how (un)favorably a workaround affects each of these dimensions. Finally, SEWA should be subject to ongoing refinement through application in practice. The identification of hitherto unknown workaround rationales and workaround attributes could further strengthen SEWA as a foundation to analyze EHR workarounds and their impact to achieve safer,
more effective and more efficient care. Authors’ contributions Conceived and designed the study: VB, KK,MJ. Collected the data: VB. Analyzed the data: VB, KK, MJ. Wrote the manuscript: VB. Edited the manuscript: KK, MJ. All authors read and approved the final manuscript. Conflicts of interests The authors declare that they have no conflicting interests. Funding This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors. Summary points What was already known on the topic:
• Discrepancies frequently occur between electronic health re•
cord (EHR) system-dictated workflows and actual workflows of EHR users, causing impeded workflows that lead to the development and usage of EHR workarounds Although EHR workarounds may seem favorable at first sight, they are generally suboptimal and may jeopardize patient safety, effectiveness of care and efficiency of care What this study added to our knowledge:
• A conceptual framework to address challenges inherent in
•
• •
76
studying workarounds emerging from EHR usage had hitherto not been developed. Existing technology evaluation frameworks offer great means to gain insight into the determinants of adoption of information technology by their users, but have a strong technology focus and lack the domain specificity of (academic) hospitals and EHRs Our framework, SEWA, provides a grounded foundation for performing sociotechnical analyses of EHR workarounds. It defines the work system and its components (human, organization, technology, task, physical environment) in which EHR workarounds are created, 15 rationales for EHR workaround creation, 4 EHR workaround attributes, and the possible effects of EHR workarounds on clinical processes and outcomes Contrary to existing frameworks, SEWA has a clear sociotechnical perspective that is necessary since we found that EHR workarounds are not solely the result of technical EHRrelated factors, but also of human, organization and taskrelated factors SEWA can be used by researchers and practitioners to identify, analyze and resolve workarounds, and thereby contribute to safer, higher quality and more efficient delivery of care.
International Journal of Medical Informatics 125 (2019) 71–78
V. Blijleven, et al.
Appendix A. Identified Rationales for EHR Workarounds Table A1 Overview of identified rationales for EHR workarounds, from [27]. Rationale for EHR Workaround
Definition
Declarative knowledge Procedural knowledge Memory aid
Not knowing how to use (a part of) the EHR to accomplish a task. Knowing how but not being proficient enough to use a part of the EHR to accomplish a task. Writing patient data down on paper (e.g. keywords) or adding visual elements to parts of text in a progress note (e.g. boldfacing, italicizing or underlining) to remind oneself. Storing patient data that is perceived important by the EHR user for other colleagues to be noticed in a data field other than the intended field in the EHR. Informal understandings among healthcare professionals leading to the creation and dissemination of workarounds (e.g. mimicking workarounds devised by colleagues to accomplish a task, or working around the system upon as friendly requested or enforced by a fellow clinician). High behavioral user cost in accomplishing a task. (A part of the) EHR halting, crashing or slowing down, hampering the EHR user in accomplishing a task. Preferring a different data view (e.g. visualization by means of charts or graphs rather than plain text). Needing to enter or request patient data with greater or lesser specificity than offered or enforced by the EHR. Inability to perform multiple tasks at once (e.g. simultaneously treating a patient on a treatment table as well as entering patient data into the EHR). Valuing patient interaction over computer interaction (i.e. writing things down on paper and afterwards entering this into the EHR). Using an alternative way to accomplish a task that improves actual efficiency. Not having (direct) access to required historical data due to data not having been imported from previously used systems to the current EHR EHR enforcing user to enter patient data of which neither the user nor the patient has knowledge of. EHR not offering the required data entry option (e.g. 3.75 mg rather than the available options 2.5 mg or 5 mg).
Awareness Social norms Usability Technical issues Data presentation Patient data specificity Task interference Commitment to patient interaction Efficiency Data migration policy Enforced data entry Required data entry option missing
Appendix B. Examples of Observed Workarounds per Workaround Attribute
Table B1 Examples of observed workarounds per workaround attribute. Workaround Attribute
Description
Cascading
Workaround initiating the creation of one or multiple additional workarounds.
Examples from direct observations
the intended dosage (e.g. 1.5 tablets per day) in a separate progress note as • Describing EHR forced user to choose between 1 and 2 tablets in the medication order form (where
• Non-cascading
Workaround not initiating the creation of additional workarounds.
Avoidable
Workaround that is not required to proceed with one’s workflow.
Unavoidable
Workaround that is required to proceed with one’s workflow.
Anticipated
Workaround that is used when upcoming workflow constraints are known beforehand.
Unanticipated
Workaround that is created when facing unexpected workflow constraints.
Incidental
Workaround that is temporarily used to overcome workflow constraints.
Routinized
Workaround that is ingrained into workflows is a common way of dealing with workflow constraints.
• • • • • • • • • • • • • • •
the user entered 1 tablet). Additional workarounds occurred when entering data in a separate progress note. Creating a patient progress note labeling system to structure a great number of progress notes as a result of not using a required standardized data entry template (e.g. patient progress notes labeled with red contain critical data, notes labeled with yellow contain important but not critical data, etc.). Skipping data fields in a standardized data entry template that are considered inapplicable (e.g. smoking history when seeing toddlers). Entering the same patient data in two near-identical data fields due to not knowing which data field is forwarded to a colleague. Not registering treatments as required effort to do so is perceived to outweigh benefits. Editing automatically generated patient letters because of e.g. undesirable font type, size, or color. Knowingly writing down patient data in data fields that are inappropriate for the data that have to be entered, because these fields are easier to locate than the appropriate data fields. Entering a diagnosis that most closely matches the actual diagnosis as the intended data entry option is missing. Leaving data field blank when the right option for reason for stopping medication is not offered in the drop-down list. Requesting lab results from longer than 5 years ago via an online form, as physician knew that the required lab results for more than 5 years ago have not been migrated to the EHR. Manually checking if patients had arrived in the waiting room as the arrival notification system was known to be down. Lacking knowledge of how to use an EHR functionality for the first time. Asking colleagues how to order a certain drug in an emergency situation due to not knowing how to do so. Writing down patient data on paper when seeing a patient potentially carrying MRSA in a different room (out of precaution to prevent spread of MRSA) where the EHR could not be accessed. This data was reentered into the EHR as soon as the physician returned to her former location. Writing down patient data on paper when the EHR crashed and reentering this data in the EHR after the EHR rebooted. Habitually clicking away excessive inappropriate or redundant alerts due to EHR generating alerts on a too low specificity level. Habitually correcting mistakes in automated EHR-generated output (e.g. letters to GPs or medication instructions) as output contained lay-out mistakes.
77
International Journal of Medical Informatics 125 (2019) 71–78
V. Blijleven, et al.
References [22]
[1] A. Boonstra, A. Versluis, J.F. Vos, Implementing electronic health records in hospitals: a systematic literature review, BMC Health Serv. Res. 14 (2014) 370 PMID: 25190184. [2] J. Henry, Y. Pylypchuk, T. Searcy, V. Patel, Adoption of Electronic Health Record Systems Among U.S. Non-Federal Acute Care Hospitals: 2008-2015, [Internet] Available from: ONC Data Brief, 2016, https://www.healthit.gov/sites/default/ files/briefs/2015_hospital_adoption_db_v17.pdf. [3] A. Robertson, K. Cresswell, A. Takian, D. Petrakaki, S. Crowe, T. Cornford, et al., Implementation and adoption of nationwide electronic health records in secondary care in England: qualitative analysis of interim results from a prospective national evaluation, BMJ 341 (2010) c4564PMID: 20813822. [4] D. Blumenthal, M. Tavenner, The “Meaningful Use” regulation for electronic health records, N. Engl. J. Med. 363 (6) (2010) 501–504, https://doi.org/10.1056/ NEJMp1006114 PMID: 20647183. [5] D. Blumenthal, J.P. Glaser, Information technology comes to medicine, N. Engl. J. Med. 356 (24) (2007) 2527–2534 PMID: 17568035. [6] J.D. Chaparro, D.C. Classen, M. Danforth, D.C. Stockwell, C.A. Longhurst, National trends in safety performance of electronic health record systems in children’s hospitals, J. Am. Med. Inform. Assoc. 24 (2) (2017) 268–274, https://doi.org/10.1093/ jamia/ocw134. [7] J. King, V. Patel, E.W. Jamoom, M.F. Furukawa, Clinical benefits of electronic health record use: national findings, Health Serv. Res. 49 (1 Pt 2) (2014) 392–404 PMID: 24359580. [8] M. Plantier, N. Havet, T. Durand, N. Caquot, C. Amaz, P. Biron, et al., Does adoption of electronic health records improve the quality of care management in France? Results from the French e-SI (PREPS-SIPS) study, Int. J. Med. Inform. 102 (6) (2017) 156–165, https://doi.org/10.1016/j.ijmedinf.2017.04.002. [9] A.D. Cheriff, A.G. Kapur, M. Qiu, C.L. Cole, Physician productivity and the ambulatory EHR in a large academic multi-specialty physician group, Int. J. Med. Inform. 79 (7) (2010) 492–500 PMID: 20478738. [10] J. Howard, E.C. Clark, A. Friedman, J.C. Crosson, M. Pellerano, B.F. Crabtree, et al., Electronic health record impact on work burden in small, unaffiliated, communitybased primary care practices, J. Gen. Intern. Med. 28 (1) (2013) 107–113 PMID: 22926633. [11] J.A. Zlabek, J.W. Wickus, M.A. Mathiason, Early cost and safety benefits of an inpatient electronic health record, J. Am. Med. Inform. Assoc. 18 (2) (2011) 169–172 PMID: 21292703. [12] J. Adler-Milstein, C. Salzberg, C. Franz, E.J. Orav, J.P. Newhouse, D.W. Bates, Effect of electronic health records on health care costs: longitudinal comparative evidence from community practices, Ann. Intern. Med. 159 (2) (2013) 97–104 PMID: 23856682. [13] C.W. Carspecken, P.J. Sharek, C. Longhurst, N.M. Pageler, A clinical case of electronic health record drug alert fatigue: consequences for patient outcome, Pediatrics 131 (6) (2013) e1970–3 PMID: 23713099. [14] M.E. Flanagan, J.J. Saleem, L.G. Millitello, A.L. Russ, B.N. Doebbeling, Paper- and computer-based workarounds to electronic health record use at three benchmark institutions, J. Am. Med. Inform. Assoc. 20 (e1) (2013) e59–66 PMID: 23492593. [15] D.F. Sittig, A. Wright, J.S. Ash, H. Singh, New unintended adverse consequences of electronic health records, Yearb Med. Inform. (1) (2016) 7. [16] J.S. Ash, P.N. Gorman, M. Lavelle, T.H. Payne, T.A. Massaro, G.L. Frantz, et al., A cross-site qualitative study of physician order entry, J. Am. Med. Inform. Assoc. 10 (2) (2003) 188–200 PMID: 12595408. [17] J. Aarts, J.S. Ash, M. Berg, Extending the understanding of computerized physician order entry: implications for professional collaboration, workflow and quality of care, Int. J. Med. Inform. 76 (2007) S4–S13. [18] E.M. Campbell, K.P. Guappone, D.F. Sittig, R.H. Dykstra, J.S. Ash, Computerized provider order entry adoption: implications for clinical workflow, J. Gen. Intern. Med. 24 (1) (2009) 21–26. [19] K.M. Cresswell, A. Worth, A. Sheikh, Integration of a nationally procured electronic health record system into user work practices, BMC Med. Inform. Decis. Mak. 12 (1) (2012) 1–12. [20] E.M. Campbell, D.F. Sittig, J. Ash, K.P. Guappone, R.H. Dykstra, Types of unintended consequences related to computerized provider order entry, J. Am. Med. Inform. Assoc. 13 (5) (2006) 547–556 PMID: 16799128. [21] J.S. Ash, D.F. Sittig, R.H. Dykstra, K.P. Guappone, J.D. Carpenter, V. Seshadri,
[23] [24] [25] [26] [27]
[28] [29] [30]
[31] [32] [33] [34] [35] [36]
[37] [38] [39] [40] [41]
[42]
78
Categorizing the unintended sociotechnical consequences of computerized provider order entry, Int. J. Med. Inform. 76 (Suppl 1) (2007) S21–7 PMID: 16793330. J.S. Ash, D.F. Sittig, E.G. Poon, K.P. Guappone, E.M. Campbell, R.H. Dykstra, The extent and importance of unintended consequences related to computerized provider order entry, J. Am. Med. Inform. Assoc. 14 (4) (2007) 415–423. T. Greenhalgh, H.W. Potts, G. Wong, P. Bark, D. Swinglehurst, Tensions and paradoxes in electronic patient record research: a systematic literature review using the meta‐narrative method, Milbank Q. 87 (4) (2009) 729–788. J.R.B. Halbesleben, D.S. Wakefield, B.J. Wakefield, Work-arounds in health care settings: literature review and research agenda, Health Care Manage Rev. 33 (1) (2008) 2–12. M. Kobayashi, S.R. Fussell, Y. Xiao, F.J. Seagull, Work Coordination, Workflow, and Workarounds in a Medical Context. CHI’ 05 Extended Abstracts on Human Factors in Computing Systems, ACM, Portland, OR, USA, 2005, pp. 1561–1564. R. Koppel, T. Wetterneck, J.L. Telles, B.T. Karsh, Workarounds to barcode medication administration systems: their occurrences, causes, and threats to patient safety, J. Am. Med. Inform. Assoc. 15 (4) (2008) 408–423 PMID: 18436903. V. Blijleven, K. Koelemeijer, M.A. Wetzels, M.W.M. Jaspers, Workarounds emerging from electronic health record system usage: consequences for patient safety, effectiveness of care and efficiency of care, J. Med. Intern. Res.: Hum. Factors 4 (4) (2017) e27, https://doi.org/10.2196/humanfactors.7978. S. Menon, D.R. Murphy, H. Singh, D. Sittig, Workarounds and test results follow-up in electronic health record-based primary care, Appl. Clin. Inform. 7 (2) (2016) 543–559. J.J. Saleem, A.L. Russ, C.F. Justice, H. Hagg, P.R. Ebright, P.A. Woodbridge, et al., Exploring the persistence of paper with the electronic health record, Int. J. Med. Inform. 78 (9) (2009) 618–628 PMID: 19464231. J.J. Saleem, A.L. Russ, A. Neddo, P.T. Blades, B.N. Doebbeling, B.H. Foresman, Paper persistence, workarounds, and communication breakdowns in computerized consultation management, Int. J. Med. Inform. 80 (7) (2011) 466–479 PMID: 21530383. G. Ser, A. Robertson, A. Sheikh, A qualitative exploration of workarounds related to the implementation of national electronic health records in early adopter mental health hospitals, PLoS One 9 (1) (2014) e77669. Z. Niazkhani, H. Pirnejad, H. van der Sijs, J. Aarts, Evaluating the medication process in the context of CPOE use: the significance of working around the system, Int. J. Med. Inform. 80 (7) (2011) 490–506. H. Van Der Sijs, I. Rootjes, J. Aarts, The shift in workarounds upon implementation of computerized physician order entry, Stud. Health Technol. Inform. 169 (2010) 290–294. O. Whooley, Diagnostic ambivalence: psychiatric workarounds and the diagnostic and statistical manual of mental disorders, Sociol. Health Illn. 32 (3) (2010) 452–469 PMID: 20415790. A. Friedman, J.C. Crosson, J. Howard, E.C. Clark, M. Pellerano, B.-T. Karsh, et al., A typology of electronic health record workarounds in small-to-medium size primary care practices, J. Am. Med. Inform. Assoc. 21 (e1) (2014) e78–e83. R.J. Holden, P. Carayon, A.P. Gurses, P. Hoonakker, A.S. Hundt, A.A. Ozok, et al., SEIPS 2.0: a human factors framework for studying and improving the work of healthcare professionals and patients, Ergonomics 56 (11) (2013) 1669–1686 PMID: 24088063. B.G. Glaser, A.L. Strauss, The Discovery of Grounded Theory: Strategies for Qualitative Research, Transaction Publishers, 2009. V. Blijleven, K. Koelemeijer, M.W.M. Jaspers, Exploring workarounds related to electronic health record system usage: a study protocol, JMIR Res. Protocols 6 (4) (2017) e72, https://doi.org/10.2196/resprot.6766. F.D. Davis, Perceived usefulness, perceived ease of use, and user acceptance of information technology, MIS Q. 13 (3) (1989) 319–340, https://doi.org/10.2307/ 249008. V. Venkatesh, M.G. Morris, G.B. Davis, F.D. Davis, User acceptance of information technology: toward a unified view, MIS Q. 27 (3) (2003) 425–478. M.M. Yusof, J. Kuljis, A. Papazafeiropoulou, L.K. Stergioulas, An evaluation framework for health information systems: human, organization and technology-fit factors (HOT-fit), Int. J. Med. Inform. 77 (June 6) (2008) 386–398, https://doi.org/ 10.1016/j.ijmedinf.2007.08.011 PMID: 17964851. S.O. Zandieh, K. Yoon-Flannery, G.J. Kuperman, D.J. Langsam, D. Hyman, R. Kaushal, Challenges to EHR implementation in electronic- versus paper-based office practices, J. Gen. Intern. Med. 23 (6) (2008) 755–761 PMID: 18369679.