- Email: [email protected]
Design of a safety dashboard for patients
Journal Pre-proof Design of a Safety Dashboard for Patients Bryan Gibson, Jorie Butler, Kumiko Schnock, David Bates, David Classen
PII:
S0738-3991(19)30478-1
DOI:
https://doi.org/10.1016/j.pec.2019.10.021
Reference:
PEC 6437
To appear in:
Patient Education and Counseling
Received Date:
24 January 2019
Revised Date:
24 October 2019
Accepted Date:
26 October 2019
Please cite this article as: Gibson B, Butler J, Schnock K, Bates D, Classen D, Design of a Safety Dashboard for Patients, Patient Education and Counseling (2019), doi: https://doi.org/10.1016/j.pec.2019.10.021
This is a PDF file of an article that has undergone enhancements after acceptance, such as the addition of a cover page and metadata, and formatting for readability, but it is not yet the definitive version of record. This version will undergo additional copyediting, typesetting and review before it is published in its final form, but we are providing this version to give early visibility of the article. Please note that, during the production process, errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain. © 2019 Published by Elsevier.
Title: Design of a Safety Dashboard for Patients
Authors: Bryan Gibson, 1 Jorie Butler , 2 Kumiko Schnock,3 David Bates ,3 David Classen 4 Affiliations: 1 University of Utah ,Department of Biomedical Informatics, Salt Lake City, USA ; 2 University of Utah, Department of Geriatrics, Salt Lake City, USA 3 Harvard University, Division of General Internal Medicine and Primary care, Boston, USA,4 University of Utah, Division of Clinical Epidemiology, Salt Lake City, USA
-p
ro
of
Corresponding Author at: Bryan S. Gibson University of Utah Department of Biomedical Informatics 421 Wakara Way Salt Lake City, UT 84132 Tel: (801) 585-0929 Email: [email protected]
Highlights
We co-designed a dashboard to engage patients in safety efforts The framing of the tool's purpose and appropriate use is critical User-centered design suggests an action oriented interface Health literacy and numeracy is a fundamental design consideration
lP
re
ur na
Abstract Background: Nearly one third of hospitalized patients suffer harm from medical errors in U.S. hospitals each year. Objective: Our goal was to design a patient-facing application that is intended to engage patients and their caregivers in reviewing and responding to clinical issues that may have safety implications. Patient Involvement: We conducted user-centered design sessions with recently hospitalized individuals and /or informal caregivers.
Jo
Methods: We conducted five user-centered design sessions with total of 37 individuals. Sessions began with individuals sharing personal stories of recent hospitalizations and any experienced safety events. We then solicited feedback on the current iteration of the patient-facing safety application. The design of the app was updated between sessions. Results: The design of our app centers around three key findings. First, involving patients in safety promotion is novel to most patients and their caregivers: therefore the framing of the tool's purpose and appropriate use is critical to engage potential users and manage expectations, this messaging was carefully crafted with patient input. Second, since most patients do not associate specific safety issues with appropriate remedial or preventative actions, the centerpiece of the application is a table that connects safety issues with related “Questions you should ask” and “Things you can do”. Third,
patients need understandable explanations of medical terms and concepts as well as explanation of changes in risk; the tool includes plain language “translations” of all medical terms used, links to curated patient education materials, and simplified graphics to visualize changes in risk. Discussion: Our findings may generalize to other efforts to engage patients in their care. Practice Value: Designing for patient engagement requires patients' perspective both on their current role and their ideal role; framing for expectations, action-oriented design, and clarity of presented information may address that gap in patient engagement.
of
Key words: Patient safety, Patient education, and Patient engagement
Jo
ur na
lP
re
-p
ro
1. Introduction Eighteen years after the Institute of Medicine (IOM) report, "To Err is Human," brought it to international attention, preventable injury to hospitalized patients remains a significant problem. [1] Medical error was recently cited as the third-leading cause of death amongst hospitalized patients in the U.S. [2], leading to estimates that medical errors account for more than 400,000 deaths and eight million patient injuries per year. [3] In this study we sought to design an electronic application that would engage patients and their caregivers in the process of improving patient safety. The patient-facing dashboard we designed in this study is an extension of a clinician-facing system, the Patient Safety Active Management system (PSAM). The PSAM mines data on a wide variety of harms and precursors of harm (e.g. falls, pressure ulcers, infection, medication errors, etc.) and presents them to clinicians for further assessment and treatment.[4,5] It is increasingly recognized that patients have a critical role to play in promoting safe care and interventions have been developed to promote patients and caregivers' engagement.[6] Safety guides are one promising type of patient-directed safety campaign. These documents suggest questions that patients should ask related to their care ( i.e. question-prompt lists) , issues to look out for, and specific actions they can take to improve their health and safety in specific circumstances. [7,8] A limitation of these interventions is that they are text documents, and therefore they need to be brought to the hospital or clinic and manually searched by the patient or caregiver to find the relevant questions to ask and actions to take in a given situation. Another more recent type of intervention are electronic patient portals which provide the patient with access to data from their electronic medical record and, in some cases, provide additional functionality such as providing names and photos of the care team and the ability to record questions for the care team. [9,10,11,12] The goal of this project was to engage patients and informal caregivers in co-designing an electronic application that would integrate the functionality of the PSAM with the information provided by the patient safety guides described above into a dashboard accessed through a patient portal. The purpose of this new system would be to engage patients and caregivers in reviewing and responding to clinical issues that may have safety implications. In preparation for a pilot test, our short-term objective was to design a system for use by patients while in the hospital, our long-term goal is for the system to be designed for use across the continuum of care. A clinician facing electronic dashboard was the starting point for the design of the patient-facing dashboard. [13] This system was designed for operation under a federally certified Patient Safety Organization (PSO; Pascal Metrics). A PSO is a legal entity that allows its members to share safety related information from their institution's medical records, in order to learn from each other's practices and improve safety. Data stored and analyzed under a PSO is not subject to discovery or subpoena in court proceedings, thus PSOs are intended to promote data sharing in a non-punitive culture of safety. [5]
The Patient Safety Active Management system (PSAM) is the clinician facing safety dashboard contained within the PSO. The PSAM provides surveillance, and prediction of individual patients' risk of experiencing an adverse event (the global safety risk score). Figure 1 depicts a screenshot of the main page of the PSAM.
ur na
lP
re
-p
ro
of
2. Methods Prior to study initiation approval from the University of Utah Institutional review board was obtained. To recruit individuals to participate in these user centered design sessions, we conducted inperson recruitment and posted fliers and in clinics and hospital waiting areas. All participants completed informed consent via review of a consent cover letter and discussion as needed with research staff prior to participating in the design sessions. Individuals between 18 and 80 years old, who are fluent English speakers, and had been hospitalized within the preceding 24 months were invited to participate in the sessions. Informal caregivers of patients who fit these criteria were also recruited. An interdisciplinary team of a health psychologist, an informaticist, and a clinician conducted the five user-centered design sessions between February and June 2016. The first four sessions were held at the University of Utah healthcare system in Salt Lake City, Utah. The fifth session was held with the patient family advisory council at Partners Healthcare in Boston, Massachusetts. The design of our dashboard was performed iteratively. Each moderated session began with individuals sharing personal stories of recent hospitalizations and any events that occurred. The intention of this storytelling section was to prompt participants to be thinking concretely and specifically about safety related problems and potential solutions. After participants told their stories, we showed them provider-facing patient safety dashboard and the contemporaneous design iteration of the patient-facing dashboard. We then solicited input on the patient-facing dashboard's purpose, functionality and visual design. Our research team used the Design thinking approach described by Brown as a framework for this work.[14] The key features of this approach are placing the user at the center of design, repeated iteration of design and feedback, and a careful consideration of the context in which the innovation will be implemented and the view of different stakeholders on the innovation. This paper describes one aspect of the design process: potential end-user's input on the innovation. Additional work to evaluate these designs and examine its effects in context is ongoing and will be discussed below. All design sessions were audio-recorded and transcribed. Transcriptions were loaded into ATLAS.ti and analyzed for themes relevant to design and functionality according to standard thematic analysis methods.[15] We developed themes iteratively using consensus amongst three investigators (BG, JB and DC).
Jo
3. Results Thirty-seven recently hospitalized patients and/or caregivers participated in the design sessions. Twenty five participants were male and twelve were female. We collected additional demographic data at these sessions but due to a technical error this data was subsequently lost. Based on the iterative input we received in the design sessions, we arrived at a final design of My Safety Advisor. This design centers around three key findings and considerations: the framing of the tool's purpose and appropriate use is critical to engage potential users and manage expectations, Usercentered design suggest an action oriented design, and Health literacy and numeracy is a fundamental design consideration. 3.1 Design requirement 1: The framing of the tool's purpose and appropriate use is critical to engage potential users and manage expectations
ro
of
In our initial session, it was apparent that most participants did not, without prompting, think of patients and caregivers as having an active role in patient safety efforts; thereafter we spent considerable time at the start of each session discussing this with participants. Therefore, we concluded that there was a clear need for the introduction to our tool to address this issue or risk being misunderstood and/or unused. This introduction should address why a patient or caregiver would use the tool and then how to use the tool. After several iterations, messaging to explain the importance of the patient's role in safety was developed (Figure 2.). It should be noted that the redundancy in the two descriptions provided in this screen is intentional: the first is intended to provide a big picture of the app's purpose, the second explanation is intended to orient the user to the important point that issues the systems identifies may be warning signs of preventable events or may be adverse events that have already occurred. For the next screen we use the metaphor of credit card monitoring to present the slightly different idea that information presented by the system is not necessarily cause for alarm (Figure 3.). The framing and specific wording for each of these screens was developed with significant participant input. We believe this framing, along with link to videos from the Agency for Healthcare Research and Quality (AHRQ), on the importance of asking questions to healthcare providers [16] should help users understand why they might want to use our system and how to use it effectively. Quote In order to make your stay more pleasant and your health care more successful these are the things we’re looking at and would like your assistance in, in helping us Maybe to phrase it as part of the healing; this is the healing process I think it means to have something that says watching for these events are at least partially the responsibilities on the patient. How about something like, you’re an important part of your healthcare team....: In preventing safety problems or whatever we call it, we need your help
ur na
lP
Framing - patients have a key role in safety
re
-p
Subtheme Framing - purpose of tool
Jo
3.2 Design Requirement 2: User-centered design suggests an action oriented design The PSAM provider-facing dashboard is intended for healthcare providers. As result of their training and experience, clinicians automatically mentally associate appropriate actions (i.e. further evaluation or treatment) with specific safety triggers (e.g. hematocrit = 21.7% in Figure 1.) Naturally, patients and caregivers do not automatically makes these associations, so they need the system to provide assistance in what to do in response to specific safety triggers. Our design solution to this issue was to provide a table that links specific safety issues with "questions you should ask" and "things you can do". In addition, in our sessions, it was clear that patients and caregivers wanted the ability to ask free form questions of their care team. Finally, several participants expressed desire for more information about the conditions mentioned by the tool, based on this request we mapped each of the triggers that the system detects to relevant content on a patient education website (e.g. Medline Plus), this mapping covers almost all the safety events that the PSAM system references and is accessed within the patient-facing system via a "more information" link in the rightmost column of the table (Figure 4).
Subtheme Support for prepackaged questions
ro
of
Desire for adding questions
Quote Well, a lot of it, too, is you may not know what the question is that you should ask Does the patient have a say in any of this or are they just saying this is what’s wrong and this is how we will help you I like that it gives you, these are questions that you should ask. Because sometimes I don’t know the right questions...it’s nice that you’re going to give me some ideas. Is there a place for an open-ended one? I mean someone might think that they have a particular problem that most people don’t know about I’ve seen videos about the importance of asking questions. I mean if they were specifically videos directed towards me Here, I want to know more, I still want to know more, I still want to know more
-p
Desire for additional information
Jo
ur na
lP
re
3.3 Design Requirement 3: Health literacy and numeracy is a fundamental design consideration In sharing their stories of recent medical experiences, several participants spontaneously complained that, in the past information had been provided to them in difficult to understand medical jargon. To address this need, we created plain language “translations” of all terms included in the dashboard. These translations were initially developed by the first author (e.g. " Possible acute Kidney (creatinine >2*baseline OR BUN 2* baseline" = " Test results indicate that your kidneys are not working as well they used to”). We then conducted an online survey with study team members and fifteen patient education and health literacy experts to collect feedback on the translations and solicit potential improvements.[17] The finalized plain language translations are included in online Appendix 1. When presented with the graphic of the global safety risk score as it changed over time, several participants expressed confusion over the meaning of the graph, participants wondered: where was I when this happened? What caused the change? What is the importance of changes in the score? To address these issues: we added icons for the patients location (e.g. outpatient, emergency department, inpatient ward, intensive care unit). We also created a functional requirement that when an individual clicks and/or hovers over any node in the graph the related safety triggers would be highlighted in the table immediately below the graph (linking cause and effect), finally we modified the display of the risk score from a continuous risk scale to a categorical scale ( low, medium, and high risk), (Figure 5). Based on prior literature, these design decisions should help lower numerate users understand the graph. [18] Subtheme Quote Need for simple language in explaining trigger, I think you need to keep it as simple as possible. questions, actions Now when I had my knee taken out and they put a concrete spacer in, the surgeon said, if you put weight on this, you will never walk... Consequently, they put a knee in, I’m walking great, but I think you need to keep it in very simple terms that not only could an advocate
understand, but a patient could understand. That for you as medical staff to be able to tell us as John Doe citizen what the issue is and how we can understand it in plain English and be able to share that as we go forward Icons could be changed over time, but icons could grab the attention with just a little bit of text below, you know, helps you zero in on it, not much. And they help you remember so when you’re not looking at the website and you’re doing your everyday stuff you can more visually remember. What would be really nice is, what was done at this point that made the curve change, right?
Icon
of
Graph
Jo
ur na
lP
re
-p
ro
4. Discussion and Conclusion 4.1 Discussion In this study, we conducted user-centered sessions to design a novel patient-facing application intended to promote patients' and their caregivers’ engagement in safety efforts. Based on the user input collected in these sessions, our application's design seeks to address three key needs: the need for a clear framing of the patients' role in patients safety efforts and the purpose and use of the tool, the need for action oriented design to help patients engage in safety efforts, and the need to use plain language and simple graphical display of how an individuals' risk has changed over time and by location. We believe the findings of this study may apply to the design of other efforts to engage patients and caregivers in care processes. The resulting design of the patient-facing dashboard of this project is in line with similar research in providing safety related information to patients. For example, O'Leary et al. found that patients in the hospital who were given access to medical data wanted the ability to record questions for their care team.[9] Similarly Prey et al. found that while patients perceived the information in their medical records as highly useful, they often did not understand the medical terms used and wanted additional information linked to medical information provided by the portal. [9,10] We hypothesize that by addressing these needs our design may facilitate increased understanding and improve patient- care team communication. This study has strengths; The integration of user-centered design sessions with content developed by others (e.g. questions and actions from safety guides , links to patient education) into the design of the dashboard is a strength. There are also limitations to this project: The fact that most participants were recruited from a single university hospital is a limitation; The use of patients that had been hospitalized in the last 2 years reflects a potential bias in our selection of patients, the small number of patients involved, and the missing demographic data also limit the generalizability of our findings. Finally, showing participants the existing PSAM before discussing the design of the patientfacing dashboard likely constrained this "ideation" phase of the design process. The rationale for this approach was practical; this project was conceived and funded as the design and evaluation of a patientfacing safety dashboard. Further research which elicits open-ended input from patients and caregivers on engaging them in safety efforts may elicit innovative concepts we did not address. The design developed in this project is undergoing pilot testing. We have integrated the patient safety dashboard with a patient portal to examine the effects of the system on risk perceptions, patient activation, and question asking. We will also be examining any associations between usage of the system and clinical outcomes. Finally we are collecting qualitative feedback that should provide further
insights on the design of the dashboard and actual use in context, a critical aspect of Design Thinking.[19] 4.2 Conclusion We present the user-centered design of a real-time patient safety dashboard. We believe our findings generalize to other efforts to engage patients in their care: framing for engagement and expectations, action-oriented design, and clarity of presented information.
of
4.3 Practice Implications Designing for patients to increase their engagement in safety efforts requires that designers elicit patients' perspective both on their current role and their ideal role. In this study we found that patients were initially unclear their role in safety, and once this was clear, they wanted context-specific actions to take and questions to ask in order to engage in safety.
ro
Funding: Robert Wood Johnson Foundation
-p
Acknowledgements We thank all the patient and caregiver participants in this project whose input was invaluable. This project was funded with the generous support of the Robert Wood Johnson Foundation
Jo
ur na
lP
re
Conflict of interest statements: Bryan Gibson, none declared, Jorie Butler none declared, etc. Kumiko Schnock non declared. David Bates consults for EarlySense, which makes patient safety monitoring systems. He receives cash compensation from CDI (Negev), Ltd, which is a not-for-profit incubator for health IT startups. He receives equity from ValeraHealth which makes software to help patients with chronic diseases. He receives equity from Clew which makes software to support clinical decisionmaking in intensive care. He receives equity from MDClone which takes clinical data and produces deidentified versions of it. His financial interests have been reviewed by Brigham and Women’s Hospital and Partners HealthCare in accordance with their institutional policies. David Classen is Chief medical Information Officer for Pascal metrics
Jo
ur na
lP
re
-p
ro
of
References 1. Institute of Medicine. To Err Is Human: Building a Safer Health System. Linda TK, Janet MC, Molla SD, editors. Washington, DC: The National Academies Press; 2000. 2. Makary MA, Daniel M. Medical error the third leading cause of death in the US. Brit Med J 2016;353.i2139 3. Adler L, Yi D, Li M, McBroom B, Hauck L, Sammer C, et al. Impact of Inpatient Harms on Hospital Finances and Patient Clinical Outcomes. J Patient Saf 2015; 14:67-73 4. Classen DC, Griffin FA, Berwick DM. Measuring patient safety in real time: An essential method for effectively improving the safety of care. Ann Intern Med 2017;167:882–3. 5. Sammer C, Jones C, Nelson A, Garrett P, Classen D, Stockwell D MS. Developing and evaluating an automated all cause harm trigger system. Jt Comm J Qual Patient Saf 2017;43:155–65. 6. AHQR. Guide to Patient and Family Engagement in Hospital Quality and Safety. Rockville, MD; 2017. 7. CampaignZero.Your Hospital Safety Checklists 2019 Available from: http://www.campaignzero.org/patient-safety-checklists/ 8. Louise H Batz Foundation. Batz Guide for Bedside Advocacy. 2013. Available from: http://www.louisebatz.org/patients/the-batz-guide/ 9. O’Leary KJ, Sharma RK, Killarney A, O’Hara LS, Lohman ME, Culver E, et al. Patients’ and healthcare providers’ perceptions of a mobile portal application for hospitalized patients. BMC Med Inform Decis Mak 2016 ;16:123-31. 10. Prey JE, Restaino S, Vawdrey DK. Providing hospital patients with access to their medical records. AMIA . Annu Symp proceedings AMIA Symp . American Medical Informatics Association; 2014:1884–93. 11. Greysen SR, Harrison JD, Rareshide C, Magan Y, Seghal N, Rosenthal J, et al. A randomized controlled trial to improve engagement of hospitalized patients with their patient portals. J Am Med Informatics Assoc [Internet]. 2018;25:1626–33. 12. Masterson Creber RM, Grossman L V, Ryan B, Qian M, Polubriaginof FCG, Restaino S, et al. Engaging hospitalized patients with personalized health information: a randomized trial of an inpatient portal. J Am Med Informatics Assoc 2019 ; 26:115-23 13. Classen D, Li M, Miller S, Ladner D. An Electronic Health Record–Based Real-Time Analytics Program For Patient Safety Surveillance And Improvement. Health Affairs 2018;37:1805-12. 14. Brown T. Design Thinking. Harvard Business Review 2008;86:84-92. 15. Patton M. Qualitative Research and Evaluation. Thousand Oaks, CA: Sage; 2002. 16. AHRQ. Questions to ask your doctor . Available from: https://www.ahrq.gov/patientsconsumers/patient-involvement/ask-your-doctor/index.html 17. Gibson B, Butler J, Classen D. User Centered Design of a Real-Time Patient Safety Dashboard. Annals of Behavioral Medicine 2017; 51:S777 18. Fagerlin A, BJ Z-F, PA U. Helping patients decide: ten steps to better risk communication. J Natl Cancer Inst. 2011;103:1436–43. 19. Brown T, Martin R. Design for Action. Harvard Business Review. 2015;93(9):56-1
re
-p
ro
of
Figure 1
Jo
ur na
lP
Figure 2
-p
ro
of
Figure 3
Jo
ur na
lP
re
Figure 4
of
ro
-p
re
lP
ur na
Jo Figure 5
PII:
S0738-3991(19)30478-1
DOI:
https://doi.org/10.1016/j.pec.2019.10.021
Reference:
PEC 6437
To appear in:
Patient Education and Counseling
Received Date:
24 January 2019
Revised Date:
24 October 2019
Accepted Date:
26 October 2019
Please cite this article as: Gibson B, Butler J, Schnock K, Bates D, Classen D, Design of a Safety Dashboard for Patients, Patient Education and Counseling (2019), doi: https://doi.org/10.1016/j.pec.2019.10.021
This is a PDF file of an article that has undergone enhancements after acceptance, such as the addition of a cover page and metadata, and formatting for readability, but it is not yet the definitive version of record. This version will undergo additional copyediting, typesetting and review before it is published in its final form, but we are providing this version to give early visibility of the article. Please note that, during the production process, errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain. © 2019 Published by Elsevier.
Title: Design of a Safety Dashboard for Patients
Authors: Bryan Gibson, 1 Jorie Butler , 2 Kumiko Schnock,3 David Bates ,3 David Classen 4 Affiliations: 1 University of Utah ,Department of Biomedical Informatics, Salt Lake City, USA ; 2 University of Utah, Department of Geriatrics, Salt Lake City, USA 3 Harvard University, Division of General Internal Medicine and Primary care, Boston, USA,4 University of Utah, Division of Clinical Epidemiology, Salt Lake City, USA
-p
ro
of
Corresponding Author at: Bryan S. Gibson University of Utah Department of Biomedical Informatics 421 Wakara Way Salt Lake City, UT 84132 Tel: (801) 585-0929 Email: [email protected]
Highlights
We co-designed a dashboard to engage patients in safety efforts The framing of the tool's purpose and appropriate use is critical User-centered design suggests an action oriented interface Health literacy and numeracy is a fundamental design consideration
lP
re
ur na
Abstract Background: Nearly one third of hospitalized patients suffer harm from medical errors in U.S. hospitals each year. Objective: Our goal was to design a patient-facing application that is intended to engage patients and their caregivers in reviewing and responding to clinical issues that may have safety implications. Patient Involvement: We conducted user-centered design sessions with recently hospitalized individuals and /or informal caregivers.
Jo
Methods: We conducted five user-centered design sessions with total of 37 individuals. Sessions began with individuals sharing personal stories of recent hospitalizations and any experienced safety events. We then solicited feedback on the current iteration of the patient-facing safety application. The design of the app was updated between sessions. Results: The design of our app centers around three key findings. First, involving patients in safety promotion is novel to most patients and their caregivers: therefore the framing of the tool's purpose and appropriate use is critical to engage potential users and manage expectations, this messaging was carefully crafted with patient input. Second, since most patients do not associate specific safety issues with appropriate remedial or preventative actions, the centerpiece of the application is a table that connects safety issues with related “Questions you should ask” and “Things you can do”. Third,
patients need understandable explanations of medical terms and concepts as well as explanation of changes in risk; the tool includes plain language “translations” of all medical terms used, links to curated patient education materials, and simplified graphics to visualize changes in risk. Discussion: Our findings may generalize to other efforts to engage patients in their care. Practice Value: Designing for patient engagement requires patients' perspective both on their current role and their ideal role; framing for expectations, action-oriented design, and clarity of presented information may address that gap in patient engagement.
of
Key words: Patient safety, Patient education, and Patient engagement
Jo
ur na
lP
re
-p
ro
1. Introduction Eighteen years after the Institute of Medicine (IOM) report, "To Err is Human," brought it to international attention, preventable injury to hospitalized patients remains a significant problem. [1] Medical error was recently cited as the third-leading cause of death amongst hospitalized patients in the U.S. [2], leading to estimates that medical errors account for more than 400,000 deaths and eight million patient injuries per year. [3] In this study we sought to design an electronic application that would engage patients and their caregivers in the process of improving patient safety. The patient-facing dashboard we designed in this study is an extension of a clinician-facing system, the Patient Safety Active Management system (PSAM). The PSAM mines data on a wide variety of harms and precursors of harm (e.g. falls, pressure ulcers, infection, medication errors, etc.) and presents them to clinicians for further assessment and treatment.[4,5] It is increasingly recognized that patients have a critical role to play in promoting safe care and interventions have been developed to promote patients and caregivers' engagement.[6] Safety guides are one promising type of patient-directed safety campaign. These documents suggest questions that patients should ask related to their care ( i.e. question-prompt lists) , issues to look out for, and specific actions they can take to improve their health and safety in specific circumstances. [7,8] A limitation of these interventions is that they are text documents, and therefore they need to be brought to the hospital or clinic and manually searched by the patient or caregiver to find the relevant questions to ask and actions to take in a given situation. Another more recent type of intervention are electronic patient portals which provide the patient with access to data from their electronic medical record and, in some cases, provide additional functionality such as providing names and photos of the care team and the ability to record questions for the care team. [9,10,11,12] The goal of this project was to engage patients and informal caregivers in co-designing an electronic application that would integrate the functionality of the PSAM with the information provided by the patient safety guides described above into a dashboard accessed through a patient portal. The purpose of this new system would be to engage patients and caregivers in reviewing and responding to clinical issues that may have safety implications. In preparation for a pilot test, our short-term objective was to design a system for use by patients while in the hospital, our long-term goal is for the system to be designed for use across the continuum of care. A clinician facing electronic dashboard was the starting point for the design of the patient-facing dashboard. [13] This system was designed for operation under a federally certified Patient Safety Organization (PSO; Pascal Metrics). A PSO is a legal entity that allows its members to share safety related information from their institution's medical records, in order to learn from each other's practices and improve safety. Data stored and analyzed under a PSO is not subject to discovery or subpoena in court proceedings, thus PSOs are intended to promote data sharing in a non-punitive culture of safety. [5]
The Patient Safety Active Management system (PSAM) is the clinician facing safety dashboard contained within the PSO. The PSAM provides surveillance, and prediction of individual patients' risk of experiencing an adverse event (the global safety risk score). Figure 1 depicts a screenshot of the main page of the PSAM.
ur na
lP
re
-p
ro
of
2. Methods Prior to study initiation approval from the University of Utah Institutional review board was obtained. To recruit individuals to participate in these user centered design sessions, we conducted inperson recruitment and posted fliers and in clinics and hospital waiting areas. All participants completed informed consent via review of a consent cover letter and discussion as needed with research staff prior to participating in the design sessions. Individuals between 18 and 80 years old, who are fluent English speakers, and had been hospitalized within the preceding 24 months were invited to participate in the sessions. Informal caregivers of patients who fit these criteria were also recruited. An interdisciplinary team of a health psychologist, an informaticist, and a clinician conducted the five user-centered design sessions between February and June 2016. The first four sessions were held at the University of Utah healthcare system in Salt Lake City, Utah. The fifth session was held with the patient family advisory council at Partners Healthcare in Boston, Massachusetts. The design of our dashboard was performed iteratively. Each moderated session began with individuals sharing personal stories of recent hospitalizations and any events that occurred. The intention of this storytelling section was to prompt participants to be thinking concretely and specifically about safety related problems and potential solutions. After participants told their stories, we showed them provider-facing patient safety dashboard and the contemporaneous design iteration of the patient-facing dashboard. We then solicited input on the patient-facing dashboard's purpose, functionality and visual design. Our research team used the Design thinking approach described by Brown as a framework for this work.[14] The key features of this approach are placing the user at the center of design, repeated iteration of design and feedback, and a careful consideration of the context in which the innovation will be implemented and the view of different stakeholders on the innovation. This paper describes one aspect of the design process: potential end-user's input on the innovation. Additional work to evaluate these designs and examine its effects in context is ongoing and will be discussed below. All design sessions were audio-recorded and transcribed. Transcriptions were loaded into ATLAS.ti and analyzed for themes relevant to design and functionality according to standard thematic analysis methods.[15] We developed themes iteratively using consensus amongst three investigators (BG, JB and DC).
Jo
3. Results Thirty-seven recently hospitalized patients and/or caregivers participated in the design sessions. Twenty five participants were male and twelve were female. We collected additional demographic data at these sessions but due to a technical error this data was subsequently lost. Based on the iterative input we received in the design sessions, we arrived at a final design of My Safety Advisor. This design centers around three key findings and considerations: the framing of the tool's purpose and appropriate use is critical to engage potential users and manage expectations, Usercentered design suggest an action oriented design, and Health literacy and numeracy is a fundamental design consideration. 3.1 Design requirement 1: The framing of the tool's purpose and appropriate use is critical to engage potential users and manage expectations
ro
of
In our initial session, it was apparent that most participants did not, without prompting, think of patients and caregivers as having an active role in patient safety efforts; thereafter we spent considerable time at the start of each session discussing this with participants. Therefore, we concluded that there was a clear need for the introduction to our tool to address this issue or risk being misunderstood and/or unused. This introduction should address why a patient or caregiver would use the tool and then how to use the tool. After several iterations, messaging to explain the importance of the patient's role in safety was developed (Figure 2.). It should be noted that the redundancy in the two descriptions provided in this screen is intentional: the first is intended to provide a big picture of the app's purpose, the second explanation is intended to orient the user to the important point that issues the systems identifies may be warning signs of preventable events or may be adverse events that have already occurred. For the next screen we use the metaphor of credit card monitoring to present the slightly different idea that information presented by the system is not necessarily cause for alarm (Figure 3.). The framing and specific wording for each of these screens was developed with significant participant input. We believe this framing, along with link to videos from the Agency for Healthcare Research and Quality (AHRQ), on the importance of asking questions to healthcare providers [16] should help users understand why they might want to use our system and how to use it effectively. Quote In order to make your stay more pleasant and your health care more successful these are the things we’re looking at and would like your assistance in, in helping us Maybe to phrase it as part of the healing; this is the healing process I think it means to have something that says watching for these events are at least partially the responsibilities on the patient. How about something like, you’re an important part of your healthcare team....: In preventing safety problems or whatever we call it, we need your help
ur na
lP
Framing - patients have a key role in safety
re
-p
Subtheme Framing - purpose of tool
Jo
3.2 Design Requirement 2: User-centered design suggests an action oriented design The PSAM provider-facing dashboard is intended for healthcare providers. As result of their training and experience, clinicians automatically mentally associate appropriate actions (i.e. further evaluation or treatment) with specific safety triggers (e.g. hematocrit = 21.7% in Figure 1.) Naturally, patients and caregivers do not automatically makes these associations, so they need the system to provide assistance in what to do in response to specific safety triggers. Our design solution to this issue was to provide a table that links specific safety issues with "questions you should ask" and "things you can do". In addition, in our sessions, it was clear that patients and caregivers wanted the ability to ask free form questions of their care team. Finally, several participants expressed desire for more information about the conditions mentioned by the tool, based on this request we mapped each of the triggers that the system detects to relevant content on a patient education website (e.g. Medline Plus), this mapping covers almost all the safety events that the PSAM system references and is accessed within the patient-facing system via a "more information" link in the rightmost column of the table (Figure 4).
Subtheme Support for prepackaged questions
ro
of
Desire for adding questions
Quote Well, a lot of it, too, is you may not know what the question is that you should ask Does the patient have a say in any of this or are they just saying this is what’s wrong and this is how we will help you I like that it gives you, these are questions that you should ask. Because sometimes I don’t know the right questions...it’s nice that you’re going to give me some ideas. Is there a place for an open-ended one? I mean someone might think that they have a particular problem that most people don’t know about I’ve seen videos about the importance of asking questions. I mean if they were specifically videos directed towards me Here, I want to know more, I still want to know more, I still want to know more
-p
Desire for additional information
Jo
ur na
lP
re
3.3 Design Requirement 3: Health literacy and numeracy is a fundamental design consideration In sharing their stories of recent medical experiences, several participants spontaneously complained that, in the past information had been provided to them in difficult to understand medical jargon. To address this need, we created plain language “translations” of all terms included in the dashboard. These translations were initially developed by the first author (e.g. " Possible acute Kidney (creatinine >2*baseline OR BUN 2* baseline" = " Test results indicate that your kidneys are not working as well they used to”). We then conducted an online survey with study team members and fifteen patient education and health literacy experts to collect feedback on the translations and solicit potential improvements.[17] The finalized plain language translations are included in online Appendix 1. When presented with the graphic of the global safety risk score as it changed over time, several participants expressed confusion over the meaning of the graph, participants wondered: where was I when this happened? What caused the change? What is the importance of changes in the score? To address these issues: we added icons for the patients location (e.g. outpatient, emergency department, inpatient ward, intensive care unit). We also created a functional requirement that when an individual clicks and/or hovers over any node in the graph the related safety triggers would be highlighted in the table immediately below the graph (linking cause and effect), finally we modified the display of the risk score from a continuous risk scale to a categorical scale ( low, medium, and high risk), (Figure 5). Based on prior literature, these design decisions should help lower numerate users understand the graph. [18] Subtheme Quote Need for simple language in explaining trigger, I think you need to keep it as simple as possible. questions, actions Now when I had my knee taken out and they put a concrete spacer in, the surgeon said, if you put weight on this, you will never walk... Consequently, they put a knee in, I’m walking great, but I think you need to keep it in very simple terms that not only could an advocate
understand, but a patient could understand. That for you as medical staff to be able to tell us as John Doe citizen what the issue is and how we can understand it in plain English and be able to share that as we go forward Icons could be changed over time, but icons could grab the attention with just a little bit of text below, you know, helps you zero in on it, not much. And they help you remember so when you’re not looking at the website and you’re doing your everyday stuff you can more visually remember. What would be really nice is, what was done at this point that made the curve change, right?
Icon
of
Graph
Jo
ur na
lP
re
-p
ro
4. Discussion and Conclusion 4.1 Discussion In this study, we conducted user-centered sessions to design a novel patient-facing application intended to promote patients' and their caregivers’ engagement in safety efforts. Based on the user input collected in these sessions, our application's design seeks to address three key needs: the need for a clear framing of the patients' role in patients safety efforts and the purpose and use of the tool, the need for action oriented design to help patients engage in safety efforts, and the need to use plain language and simple graphical display of how an individuals' risk has changed over time and by location. We believe the findings of this study may apply to the design of other efforts to engage patients and caregivers in care processes. The resulting design of the patient-facing dashboard of this project is in line with similar research in providing safety related information to patients. For example, O'Leary et al. found that patients in the hospital who were given access to medical data wanted the ability to record questions for their care team.[9] Similarly Prey et al. found that while patients perceived the information in their medical records as highly useful, they often did not understand the medical terms used and wanted additional information linked to medical information provided by the portal. [9,10] We hypothesize that by addressing these needs our design may facilitate increased understanding and improve patient- care team communication. This study has strengths; The integration of user-centered design sessions with content developed by others (e.g. questions and actions from safety guides , links to patient education) into the design of the dashboard is a strength. There are also limitations to this project: The fact that most participants were recruited from a single university hospital is a limitation; The use of patients that had been hospitalized in the last 2 years reflects a potential bias in our selection of patients, the small number of patients involved, and the missing demographic data also limit the generalizability of our findings. Finally, showing participants the existing PSAM before discussing the design of the patientfacing dashboard likely constrained this "ideation" phase of the design process. The rationale for this approach was practical; this project was conceived and funded as the design and evaluation of a patientfacing safety dashboard. Further research which elicits open-ended input from patients and caregivers on engaging them in safety efforts may elicit innovative concepts we did not address. The design developed in this project is undergoing pilot testing. We have integrated the patient safety dashboard with a patient portal to examine the effects of the system on risk perceptions, patient activation, and question asking. We will also be examining any associations between usage of the system and clinical outcomes. Finally we are collecting qualitative feedback that should provide further
insights on the design of the dashboard and actual use in context, a critical aspect of Design Thinking.[19] 4.2 Conclusion We present the user-centered design of a real-time patient safety dashboard. We believe our findings generalize to other efforts to engage patients in their care: framing for engagement and expectations, action-oriented design, and clarity of presented information.
of
4.3 Practice Implications Designing for patients to increase their engagement in safety efforts requires that designers elicit patients' perspective both on their current role and their ideal role. In this study we found that patients were initially unclear their role in safety, and once this was clear, they wanted context-specific actions to take and questions to ask in order to engage in safety.
ro
Funding: Robert Wood Johnson Foundation
-p
Acknowledgements We thank all the patient and caregiver participants in this project whose input was invaluable. This project was funded with the generous support of the Robert Wood Johnson Foundation
Jo
ur na
lP
re
Conflict of interest statements: Bryan Gibson, none declared, Jorie Butler none declared, etc. Kumiko Schnock non declared. David Bates consults for EarlySense, which makes patient safety monitoring systems. He receives cash compensation from CDI (Negev), Ltd, which is a not-for-profit incubator for health IT startups. He receives equity from ValeraHealth which makes software to help patients with chronic diseases. He receives equity from Clew which makes software to support clinical decisionmaking in intensive care. He receives equity from MDClone which takes clinical data and produces deidentified versions of it. His financial interests have been reviewed by Brigham and Women’s Hospital and Partners HealthCare in accordance with their institutional policies. David Classen is Chief medical Information Officer for Pascal metrics
Jo
ur na
lP
re
-p
ro
of
References 1. Institute of Medicine. To Err Is Human: Building a Safer Health System. Linda TK, Janet MC, Molla SD, editors. Washington, DC: The National Academies Press; 2000. 2. Makary MA, Daniel M. Medical error the third leading cause of death in the US. Brit Med J 2016;353.i2139 3. Adler L, Yi D, Li M, McBroom B, Hauck L, Sammer C, et al. Impact of Inpatient Harms on Hospital Finances and Patient Clinical Outcomes. J Patient Saf 2015; 14:67-73 4. Classen DC, Griffin FA, Berwick DM. Measuring patient safety in real time: An essential method for effectively improving the safety of care. Ann Intern Med 2017;167:882–3. 5. Sammer C, Jones C, Nelson A, Garrett P, Classen D, Stockwell D MS. Developing and evaluating an automated all cause harm trigger system. Jt Comm J Qual Patient Saf 2017;43:155–65. 6. AHQR. Guide to Patient and Family Engagement in Hospital Quality and Safety. Rockville, MD; 2017. 7. CampaignZero.Your Hospital Safety Checklists 2019 Available from: http://www.campaignzero.org/patient-safety-checklists/ 8. Louise H Batz Foundation. Batz Guide for Bedside Advocacy. 2013. Available from: http://www.louisebatz.org/patients/the-batz-guide/ 9. O’Leary KJ, Sharma RK, Killarney A, O’Hara LS, Lohman ME, Culver E, et al. Patients’ and healthcare providers’ perceptions of a mobile portal application for hospitalized patients. BMC Med Inform Decis Mak 2016 ;16:123-31. 10. Prey JE, Restaino S, Vawdrey DK. Providing hospital patients with access to their medical records. AMIA . Annu Symp proceedings AMIA Symp . American Medical Informatics Association; 2014:1884–93. 11. Greysen SR, Harrison JD, Rareshide C, Magan Y, Seghal N, Rosenthal J, et al. A randomized controlled trial to improve engagement of hospitalized patients with their patient portals. J Am Med Informatics Assoc [Internet]. 2018;25:1626–33. 12. Masterson Creber RM, Grossman L V, Ryan B, Qian M, Polubriaginof FCG, Restaino S, et al. Engaging hospitalized patients with personalized health information: a randomized trial of an inpatient portal. J Am Med Informatics Assoc 2019 ; 26:115-23 13. Classen D, Li M, Miller S, Ladner D. An Electronic Health Record–Based Real-Time Analytics Program For Patient Safety Surveillance And Improvement. Health Affairs 2018;37:1805-12. 14. Brown T. Design Thinking. Harvard Business Review 2008;86:84-92. 15. Patton M. Qualitative Research and Evaluation. Thousand Oaks, CA: Sage; 2002. 16. AHRQ. Questions to ask your doctor . Available from: https://www.ahrq.gov/patientsconsumers/patient-involvement/ask-your-doctor/index.html 17. Gibson B, Butler J, Classen D. User Centered Design of a Real-Time Patient Safety Dashboard. Annals of Behavioral Medicine 2017; 51:S777 18. Fagerlin A, BJ Z-F, PA U. Helping patients decide: ten steps to better risk communication. J Natl Cancer Inst. 2011;103:1436–43. 19. Brown T, Martin R. Design for Action. Harvard Business Review. 2015;93(9):56-1
re
-p
ro
of
Figure 1
Jo
ur na
lP
Figure 2
-p
ro
of
Figure 3
Jo
ur na
lP
re
Figure 4
of
ro
-p
re
lP
ur na
Jo Figure 5