Developing a mobile epilepsy management application integrated with an electronic health record for effective seizure management

Journal Pre-proof Developing a mobile epilepsy management application integrated with an electronic health record for effective seizure management Sooyoung Yoo, Kahyun Lim, Hyunyoung Baek, Sae-Kyun Jang, Ga-young Hwang, Hunmin Kim, Hee Hwang

PII:

S1386-5056(19)30745-2

DOI:

https://doi.org/10.1016/j.ijmedinf.2019.104051

Reference:

IJB 104051

To appear in:

International Journal of Medical Informatics

Received Date:

9 July 2019

Revised Date:

25 November 2019

Accepted Date:

3 December 2019

Please cite this article as: Yoo S, Lim K, Baek H, Jang S-Kyun, Ga-young H, Kim H, Hwang H, Developing a mobile epilepsy management application integrated with an electronic health record for effective seizure management, International Journal of Medical Informatics (2019), doi: https://doi.org/10.1016/j.ijmedinf.2019.104051

This is a PDF file of an article that has undergone enhancements after acceptance, such as the addition of a cover page and metadata, and formatting for readability, but it is not yet the definitive version of record. This version will undergo additional copyediting, typesetting and review before it is published in its final form, but we are providing this version to give early visibility of the article. Please note that, during the production process, errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain. © 2019 Published by Elsevier.

Developing a mobile epilepsy management application integrated with an electronic health record for effective seizure management

Sooyoung Yoo, PhDa*, Kahyun Lim, MSa*, Hyunyoung Baek, RN, MPHa, Sae-Kyun

a

ro of

Jang, MPHb, Ga-young, Hwang, MSb, Kim, MDc, Hee Hwang, MDc

Office of eHealth Research and Businesses, Seoul National University Bundang Hospital,

South Korea,

Department of Pediatrics, Seoul National University Bundang Hospital, South Korea

lP

re

c

Healthconnect Co., Ltd. South Korea,

-p

b

*These authors contributed equally to this work.

na

Corresponding and co-corresponding authors: Hee Hwang, [email protected]

ur

Hunmin Kim, [email protected]

Jo

Address for correspondence: Department of Pediatrics, Seoul National University Bundang Hospital, 82 Gumi-ro 173 beon-gil, Bundang-gu, Seongnam 13620, Korea. Tel: +82-031-787-7284, E-mail: [email protected] Department of Pediatrics, Seoul National University Bundang Hospital, 82 Gumi-ro 173 beon-gil, Bundang-gu, Seongnam 13620, Korea. Tel: +82-031-787-7297, E-mail: [email protected]

Highlights 

The study developed and evaluated an mHealth service for managing epilepsy compatible with the hospital’s EHR system.



Functions such as seizure Diary, Medication Reminder, Appointments, Outpatient



ro of

Survey, and My Epilepsy were developed to aid patients’ epilepsy management. The usability test showed that participants were highly satisfied with the service and there were differences between participants in scenario-based task success



-p

rate.

The degree of perceived importance of the functions differed by participants’

lP

re

characteristics.

na

Keywords: Epilepsy, Seizure, Medication, Mobile application, Electronic medical record

ur

Abstract1

Introduction: Epilepsy is a chronic neurological disorder characterized by recurrent

Jo

spontaneous seizures. Over 70% epilepsy patients can live normally if their seizures can be controlled. For this, many factors should be tracked and managed, but doing so is hard because of individual differences. There are mobile applications to help track these factors;

1

TF: task force; GUI: graphic user interface; UI: user interface; EHR: electronic health record; UX: user experience; FHIR: Fast Healthcare Interoperability Resources; SUS: System Usability Scale

however, no application covers crucial factors comprehensively, and they are complicated to use. Therefore, this study aimed to develop a mobile epilepsy management application covering crucial factors comprehensively in a user-friendly way. We evaluated the pilot version with a usability and satisfaction survey and an interview. Methods: We established a task force comprising professionals from various fields who participated in all processes of this research. Existing service analysis and professional

ro of

interviews were conducted to draw a function list. User interface and graphic user interface were designed under the supervision of the task force. After developing the application’s pilot version, usability and satisfaction of the application were evaluated with eight patients and

-p

caregivers through scenario-based usability test, satisfaction survey, and interview.

Results: All existing mobile epilepsy management applications provide seizure and

re

medication diary functions. We decided to provide six main functions: seizure diary,

lP

medication reminder, appointments, outpatient survey, education materials, and personal dashboard (My epilepsy). We also integrated the application with the hospital’s electronic

na

health record system. To simplify usability, frequently used and relatively important functions are located in the main page as “seizure recording” and “medication diary.” Additionally,

ur

when designing graphics, art therapy was used to enhance psychological stability. For evaluation, eight participants were recruited. In scenario-based tasks, among 10 tasks, all

Jo

participants completed six tasks. However, only 37.5% participants recorded seizures in detail. System Usability Scale score was 84.5 points, indicating system satisfaction. Conclusion: This study confirmed that patients’ satisfaction of this application were high. Additionally, it helped them record their seizures accurately, which is very useful for seizure trend analysis, discovering seizure trigger factors, and ensuring efficient management of epilepsy. Through integration with the electronic health record, patient medical information

could be utilized to guide physicians’ decision-making for future medical treatment and could contribute greatly to the overall management of epilepsy. Keywords: Epilepsy, Seizure, Medication, Mobile application, Electronic medical record

Introduction

ro of

Epilepsy is a chronic neurological disorder characterized by recurrent spontaneous seizures. The worldwide prevalence of epilepsy is estimated to be 0.4–1%[1]. Due to abnormal brain activity, certain seizures cause convulsion or loss of consciousness and result in injuries. In

-p

addition, such damage can result in traumatic brain injury, broken bones, concussions, and

re

breathing problems [2].

Over 70% epilepsy patients can live their lives without major problems if they have the

lP

appropriate means to control seizures [3]. Medication is the most common treatment among numerous interventions [4]. However, it is challenging to determine the appropriate treatment

na

to control seizures for each patient, as the disorder is highly complex, and each patient has different physiological status. To determine which medication regime is appropriate,

ur

physicians initially suggest some medications for trial. If the first trial is effective without serious side effects, patients continue the medications. If not, physicians suggest other

Jo

medications as an alternative until they find the active ones [5]. To reduce time and ensure efficiency, information concerning seizure status, medication side effects, dose history, and seizure trigger factors should be tracked continuously [6]. Therefore, patients or caregivers note these in their own way, such as on paper or on their mobile phones, to show physicians. This is very inconvenient for the patients to record and difficult for physicians to read and infer associations [7].

To address these issues, many mobile applications including EpApp [8], EpiWatch [9], and Seizure Sync [10] have been developed. These provide seizure diary and medication reminders in common. Additionally, depending on applications, various functions are included such as educational materials, emotion diary, and automatic seizure detection. There are many factors to be addressed for managing epilepsy, but there is no application covering all these factors comprehensively. Moreover, there is no application integrated with the

ro of

hospital’s electronic health record (EHR) system, which is crucial for usability. This not only makes it hard to communicate with the physician but also degrades usability. Puneet et al.

recently demonstrated that although the number of mobile epilepsy management applications is increasing, their functional coverage is still limited [11]. Another review of mobile epilepsy

-p

self-management apps also noted this and suggested the need to develop mobile epilepsy

re

management apps that provide broader domains of functionality to support self-management [12]. Usability is particularly important in the design and development of health applications

lP

to ensure their adoption and sustained use, as users tend to quickly abandon applications when facing inconvenience (e.g., problems in access and complex processes) [13]. Therefore,

na

a usability test is a useful evaluation method to detect fatal usability problems before releasing the application [14].

ur

Therefore, the primary objective of this study was to develop a mobile epilepsy management application that comprehensively covers crucial factors for managing epilepsy. As a

Jo

secondary goal, considering usability, we tried to enhance the application’s convenience through a simple user interface (UI), graphic design, and integration with the hospital’s EHR system. To draw a list of functions, interviews with professionals from various fields and existing service analysis were conducted. Moreover, integration with the hospital EHR system was processed to reduce user effort in typing data such as prescribed medicine and lab

results and improving communication with physicians. In addition, UI was designed as simple as possible. Art therapy was adopted in graphic design for helping users feel psychologically relieved. After developing the pilot version of the application, a scenariobased usability test, satisfaction survey, and interview were conducted. 1. Methods

ro of

1.1 Study site and task force team This study was conducted at Seoul National University Bundang Hospital (SNUBH), a public tertiary general hospital located in the Seoul metropolitan area. The EHR system of the

hospital was accredited by the Healthcare Information and Management System Society’s

-p

Electronic Medical Record Adoption Model Stage 7, which indicates that the hospital has an

lP

applications linked to the EHR system.

re

optimal information technology infrastructure that can develop and evaluate mobile

To design the mobile epilepsy management application, we established a multidisciplinary task force (TF) composed of experts from various fields including neurology, nursing,

na

medical informatics, user experience (UX), and software development. The TF held a meeting every two weeks from January 1 to December 31, 2018 and were involved in every

ur

process from service planning to development and evaluation.

Jo

1.2 Functions of mobile epilepsy management application We investigated the existing mobile epilepsy management applications to derive a list of functions to be adopted. Six applications were analyzed: EpiWatch [9], Alert App [15], SeizAlarm [16], Epilepsy Tool Kit [17], EpiDiary [18], and EpApp [8]. Other than these applications, there are other useful services such as My Seizure Diary [19] and Seizure Tracker [20]. The selected applications were in the top six positions of the total number of

considered downloads in App Store, a mobile application distribution platform. The application functions found were categorized and divided into seven main features: seizure diary, automatic seizure detection, emergency call, medication (reminder), education, image/video upload, and emotion management. With the exception of automatic seizure detection and emergency call, all features were adopted for the application. The automatic seizure detection function was excluded due to its low accuracy and emergency call was ruled

ro of

out due to the local regulatory issues with service linkage and provision. Educational materials were produced reflecting a common opinion of physicians that most patients and caregivers find it difficult to understand the various types of seizures. New features for

checking lab test results, physical growth, and self-surveying their psychological status were

-p

designed and provided through the integration with the hospital’s EHR system. The key

re

functions derived by the TF for patients are as follows.

 Seizure diary: recording and monitoring the seizure type, trigger/aggravation factors,

lP

seizure occurrence time, duration of the seizure, upload image/video, and memo

na

 Medication diary: checking prescribed medications, recording dose and adverse drug effects, and setting a medication reminder

ur

 Education: 3D-animation video explaining different seizure types, physicians’

Jo

answers to frequently asked questions  Emotion management: recording daily emotion changes using emoticons  Test results: Checking medical test results such as electrocardiogram and blood test  Survey: providing self-survey tools such as the Children’s Depression Inventory [21], Beck Anxiety Inventory [22], Attention Deficit Hyperactivity Disorder rating scale [23], physical development assessment [24], quality of life survey [25], and

outpatient examination survey Based on the derived function list, we designed the information architecture as depicted in Figure 1. Along with the patient mobile application, we developed a monitoring web application for physicians linked to the EHR system.

Log In

My Epilepsy

Survey & Test results

Message

Dashboard

Survey

Seizure Diary

Diary

Test Results

Medication

Report

Education

Setting

Education Materials

My Info.

Connect with health apps.

-p

Main

*Only first login user

ro of

Basic Health Survey

re

Calendar

Notification Setting

Seizure Diary

Privacy Policy

Survey & Test Results

Medication

Version Info.

lP

My Epilepsy

Education

Emotion

Log Out

Sleep

na

Appointment

ur

Figure 1. Information architecture of Brain4U for patients; Info: Information

Jo

1.3 System architecture

The system architecture of Brain4U is detailed in Figure 2. As the data within the Brain4U contains features of personal medical information, we located the system within the demilitarized zone of the hospital. Fast Healthcare Interoperability Resources (FHIR) standard was used for interoperability. Data in Brain4U were mapped to analogous FHIR resources [26]. The data elements included FHIR resources such as Encounter (associated

with a particular encounter), Appointment (information concerning a planned appointment), Medication Order (all kinds of medication orders for the patient), Observation (i.e. vital signs, laboratory data, and imaging test results), Patient (demographic information of patient), Practitioner (detailed description of the physician), Medication (identification and definition of a medication - i.e. manufacturing site or injection method). ExtraNet

DMZ

IntraNet

ro of

Push Alarm -Address: android.googleapis.com -Port: 443, 5228, 5229, 5230

Epilepsy Management System

EHR

AP P AP I

Monitoring Web Sleep

EHR I/F FCM I/F -Usage of Mobile Service -Survey, Seizure diary -Steps, Sleep -Address: any -P ort: 446(https)

MariaDB OS(Win2012)

lP

Steps, Sleep data (https, json)

Batch P rocess WAS(Tomcat 8.x)

Retrieve Patient Inf o. -Blood test result -Brain MRI result -EEG test result -P atient Info. -Medication

-p

Smartphone (Android, iOS)

re

HealthKit

Physician’s Web -Retrieve patient test results -Retrieve patient info from the mobile app

Physicians

Figure 2. System architecture of Brain4U linked to the Electronic Health Record (EHR)

na

system; DMZ: demilitarized zone; MRI: Magnetic Resonance Imaging; EEG: electrocardiogram; FHIR: Fast Healthcare Interoperability Resources; WAS: Web Application

ur

Server; FCM: Firebase Cloud Messaging; API: Application Programming Interface; Info:

Jo

Information

1.4 Usability test

Eight participants were recruited for the usability test. The number of participants was based on a guideline that 6–8 people is an appropriate sample size for measuring UX in a usability test [27]. Outpatients diagnosed with epilepsy for over 3 years and their respective caregivers were included as potential participants for this study. Patients or their parents who were

unavailable or unable to use a smart phone and refused to provide informed consent were excluded. The usability test was conducted in a quiet meeting room with the minimum objects essential for the test such as desk, cameras, and camera holders. It was conducted from November 29 to December 29, 2018. We designed 10 scenario-based tasks using main functions as described in Table 1. Three measures were evaluated: task success rate, time for completion, and satisfaction. To

ro of

calculate the task success rate, we defined a scoring policy. When a participant completed the task immediately, we gave him 1 point for the task. If not, the participant was assigned 0

point. After finishing tests, the average score was calculated for each task to find out which

-p

task had defects in usability. Efficiency refers to the time spent by an individual for

completing a task. Satisfaction evaluates user satisfaction, and we used the System Usability

re

Scale (SUS) [28] as it is a reliable tool for measuring the level of satisfaction.

lP

We educated participants in advance regarding the “Think Aloud” method, which involves voicing every emerging idea. In the test, one participant, one moderator, and one assistant

na

moderator attended. When the participant entered, first, the outline of the application and the usability test were explained, and we acquired a consent form representing the participant’s

ur

consent to record and collect data. Before starting the usability test, a short survey was conducted asking demographic information and pre-expectations for the application. Finally,

Jo

the scenario-based usability test was started. The entire process was performed under the guidance of a moderator, while the observer recorded what he/she observed. The moderators took the role of instructing the whole process to the participants and were instructed not to intervene while the participants were working on tasks. They could offer advice once the participant completely gave up on the task. The entire process was recorded in three ways: a smartphone screen recording, participant’s facial expression recording, and physical motion

recording. After completing all the tasks, a short interview inquiring their broad feelings and thoughts and a SUS were conducted. Table 1. Scenario-based tasks for evaluating usability of Brain4U mobile application No. Task

Contents

Log in

Log in with your hospital account.

2

Basic Survey

Complete the basic survey.

3

Main

Move to the main page.

4

Seizure Diary

Record seizure based on the scenario.

5

Seizure Diary (detail)

Record seizure in detail based on the scenario.

6

Medication

Record medication and adverse events based on the scenario.

7

Appointments Check your appointments.

My Epilepsy

-p

Record changes in your physical development. Check the overall trend of your disorder.

na

10

Complete the outpatient survey.

re

9

Outpatient Survey Physical Development

lP

8

ro of

1

We summarized the results and compiled a list of improvement requirements, such as adding

ur

a scroll bar on the right of vertically long menus, changing words that might confuse users, and rearranging information. The list was sent to the development team, and changes were

Jo

reflected in the released version of the application. This study was approved by the Institutional Review Board at SNUBH (IRB No. B-1804/460-306) and every participant voluntarily decided to take part in this research, and written consent was obtained.

2. Results 2.1 Mobile epilepsy management application The mobile based EHR-integrated epilepsy management application was launched on February 9, 2019. Figure 3 shows the main screenshots of the launched version. Art therapy was used to design the user interface (UI) for relieving stress and improving the psychological stability of users. Art therapy is a therapeutic method to positively influence

ro of

users’ emotion through design. We aimed to relieve stress and stabilize the mental status of users with friendly illustrations including calm pastel toned bright and soft colors. Graphic

design varies by season. Two functions, “Seizure Record” and “Medication,” are located on

-p

the main screen considering the frequency of their use and relative importance. Patients’

medical data such as prescription and test results are automatically linked with the mobile

Jo

ur

na

lP

re

application.

(a)

(b)

(c)

(d)

Figure 3. Brain4U mobile application for patients

*(a) main page, (b) seizure record, (c) my epilepsy, (d) 3D animation-based seizure type educational contents Besides the patient mobile application, we developed a monitoring web application for

physicians as shown in Figure 4. The application enabled physicians to check their patients’ daily disease management status such as medication adherence, frequency of seizures,

ur

na

lP

re

-p

ro of

accompanying diseases, and emotional changes.

Jo

Figure 4. Physicians’ monitoring page integrated with hospital Electronic Health Record system; ADHD: Attention Deficit Hyperactivity Disorder; OXC: oxcarbazepine; VPA: valproic acid; LTG: lamotrigine

2.2 End-user usability test result

3.2.1 Demographics of participants Of the eight participants, five were caregivers (mean age = 42.6 years) and three were patients (mean age = 20.7 years). Five participants were female and three were male. Four participants answered that they were familiar with the mobile application and the remaining

ro of

participants reported being unfamiliar with it.

3.2 Patients’ and caregivers’ perceptions about epilepsy management

Figure 5 (Left) indicates what participants think is important for managing epilepsy and what

-p

they actually care about in their daily lives. According to the results, participants considered “sleep” as the most important factor for managing epilepsy. Medication records, seizure

re

records, and stress control were implemented. Only few participants answered that they did

lP

not regulate the intensity of sunlight exposure in practice though many of them consider sunlight an important seizure trigger. Most participants, in reality, handled sleep time, medication, and stress carefully; however, seizure recording and regulating sunlight exposure

na

were rarely practiced. Moreover, no participant engaged in exercise as a means to manage epilepsy. Figure 5 (Right) refers to the intensity of interest. All participants reported that they

Jo

ur

were interested in managing epilepsy.

ro of -p

Figure 5. (Left) Response rate of perceived important factors and what participants actually

re

did for managing epilepsy; (right) response rate of the degree of interest in managing epilepsy

lP

and the status of recording seizures and medications

3.3 Patients’ and caregivers’ expectations of the epilepsy management service

na

Participants were then asked to complete questionnaires about their prior expectations of the application. For all questions, the average scores were found to be greater than 4.5 points out

ur

of 5 as shown in Table 2.

Jo

Table 2. Pre-expectation survey result

Questions

I think this application is very essential for managing epilepsy. I think this application is improving my understanding of epilepsy. I think this application is improving the efficacy of epilepsy management. I think I can manage seizure and medication better with this application.

Score (Mean) 4.8 4.5 4.8 4.8

I think this application is very helpful to communicate with physicians.

4.9

I think I will consistently use this application.

4.9

na

lP

re

-p

ro of

3.4 Task success rate

Figure 6. Task success rate; info: information

ur

In terms of the success rate, most participants completed the tasks well without significant difficulties as shown in Figure 6. However, in recording the seizure in detail (task 5), only

Jo

37.5% completed it on their first try. Among all participants, 25% had difficulties in checking appointments (task 7) and changing physical growth information (task 9). They could not find the right menu for achieving tasks. Additionally, 37.5% participants could not access the My Epilepsy page (task 10) at once, because it was difficult to figure out the meaning of the My Epilepsy button in the main screen. In terms of the completion time for each task, the basic survey (task 2) took the longest time as shown in Figure 7. However, this was not due to a

usability issue, because it took considerable time to answer all questions related to their status in task 2. None of the participants experienced problems in accessing the function (basic survey). However, participants appeared very confused when they tried to record the seizure in detail (task 5). They were unsure of where to navigate in order to record the seizure in detail. In the outpatient survey (task 8), except for one participant who spent some time to find “outpatient survey” in the survey list, all other participants finished without any

Jo

ur

na

lP

re

-p

ro of

hindrance.

Figure 7. Completion time for each task; info: information 3.5 Satisfaction survey The SUS and few more questions were used to investigate participants’ satisfaction and willingness to use the application after its launch [28]. The SUS result indicates scores of 84.5±12.336, which fall within the top 10%. Figure 8 shows participants’ responses to the

usefulness and necessity of the application and willingness to use it. My Epilepsy and the outpatient survey function received the highest scores, while the basic survey received the lowest score. Only patients gave high score to the medication function, and caregivers (parents) were more interested in the seizure diary than patients. How frequently are you going to use each function in reality?

lP

re

-p

ro of

How useful and necessary do you think the function is in managing epilepsy?

Figure 8. Usefulness and expected frequency of each function’s use by participant

ur

3.6 Interview

na

characteristics

Jo

As a final stage, an in-depth interview was conducted for identifying why they had any trouble completing tasks and their overall feelings regarding Brain4U. Table 3 depicts the issues raised by the participants collected through usability tests and interviews. Issues were found in Basic survey, Seizure record, and Medication record functions, and these functions were transferred to the development team to improve service usability.

Table 3. Issues and implications of Brain4U design of user interface User Interface

Issues Some participants were confused because they could not find their diagnosis within the selection box.

Implication All possible diagnoses were listed but participants could not find it because there was no scroll bar available, which prevented them from viewing the lists below. Additionally, every select box should be modified. We judged that the word used on the button may confuse users, so we decided to change the word “No” to “Cancel” and “Yes” to “Save” for clarity.

ro of

Task

re

-p

Basic Survey

na

lP

The “No” and “Yes” buttons actually meant “cancel” and “save” respectively. However, because it was located directly below the selection box asking, “Was there an aggravation factor?”, participants tended to not use the selection box but click the “Yes” button when they had aggravation factors. Participants were confused as to which button status – colored or not – was a mark to indicate taken.

Jo

ur

Seizure Record

Medication Record

We concluded that this could confuse users, so we decided to change the button to a toggle shape button, because it is more intuitive.

Aside from this, we compiled an overview of participants’ positive and negative feedback on Brain4U, as described in Table 4. Every participant indicated that the EHR-integrated service would be very useful. Three applications—patient mobile application, EHR system, and physician monitoring web—are connected and share useful information without any additional efforts from the end-users. Participants reported that it makes using the mobile application and managing the disorder very simple and convenient. In addition, based on

ro of

users’ opinion that information about the prescribed drug should be available, we added a function that allowed users to check this information by clicking on the prescription drug list. Table 4. Overview of the positive and negative opinions of Brain4U

● It is very useful; we can

Disadvantages of use

-p

Advantages of use confirm medical test results with the application.

re

● It seems like we are more

● Medication reminders are set automatically based on the prescription data saved in the EHR system. This is very convenient.

na

Mentioned 5 times or more

lP

closely connected with the physicians and it comforts me.

● It was not difficult to use

ur

compared to my expectations. ● I liked the design of this

Jo

application.

Mentioned 2 to 4 times

● (All patient participants) Self-

● Can I get more detailed

survey functions that manage my behavioral status would be very useful.

information, such as expected side effects and the appropriate dosage, about drugs I am taking?

● (All patient participants) An outpatient survey will be very useful, because usually I go to the hospital with my parents, and even

●I think the patient account should be shared with caregivers. I (parent) want to

if I have things to ask, I can’t ask because my parents would worry about me.

supervise whether my child inputs information properly.

● Seizure animations will be very beneficial in helping me understand my disorder. ● For some patients with serious conditions, managing these factors is not very practical.

ro of

Mentioned once

3. Discussion

Finding proper medication and monitoring seizure trigger factors is crucial in managing

-p

epilepsy. Therefore, patients write down or type details on their mobile phone to record them.

re

However, this is very troublesome for patients, and difficult for physicians to read and infer associations as well, making this method difficult to use practically in the clinical field. To

lP

support this, numerous mobile epilepsy management applications have been released. We analyzed six epilepsy management applications; all applications provide seizure diary

na

function and most of them provide medication reminder function. Other functions such as automatic seizure detection, education materials, photo/video upload, emergency call, and

ur

emotion management function are adopted depending on applications. Therefore, we decided to provide all these functions except automatic seizure detection. Automatic seizure detection

Jo

function was excluded because of issues in its accuracy. Educational materials and 3D animation videos explaining seizure types were also developed, reflecting the opinions of physicians, with the understanding that identifying different types of seizures is very challenging for most patients. Additionally, frequently asked questions and physician answers were added to the menu. To develop a practically helpful service,

accessibility and convenience of use are of utmost importance. Similarly, recent studies developing healthcare applications tend to place importance on the usability test for deriving unrecognizable issues and needs of the end-users. Therefore, unlike other applications, we integrated the application with the hospital EHR system that strongly contributes to the convenience of use by drawing patient data such as lab results and prescription automatically from the hospital EHR system. Moreover, FHIR standard was adopted for exchanging EHR.

ro of

Simultaneously, to overcome concerns that the application may become too complicated, we particularly focused on the UI. As a result of these efforts, participants completed tasks

without major difficulties in the usability test, and the satisfaction scores indicated high

-p

satisfaction.

In addition, we compiled the advantages and disadvantages of use from the interview. Every

re

participant stated that the EHR-integrated application would be very useful. Further, we realized that perceived important functions differ depending on the user characteristics.

lP

Parents normally place more emphasis on seizure occurrence, while patients tend to be concerned more with regular medicine dose. However, bedridden patients who have

na

difficulties in controlling seizures with medications were less willing to use this application because seizures occur very often among this group, and so their caregivers may not forget

ur

medication, and it is impossible to record every seizure’s occurrence. This response was completely unexpected because we anticipated that the seriousness of the disorder would

Jo

correspond to the demand for the application. In addition, although we could not quantify data in this paper, we found few issues unexpectedly through the usability test. Adolescent epilepsy patients tend not to communicate with their parents, especially regarding their disorder. This was a common distress for adolescents and their parents. Parents were pleased because they can monitor their children

without conflicts. Normally, caregivers (parents) of adolescent patients visit the hospital instead of patients themselves, or they visit together. Adolescent patients confessed feeling uncomfortable about asking questions to the physicians when they were with their caregivers. Similarly, they appreciated the outpatient survey because they could ask or notice something about their disease privately and directly to their physicians. Unfortunately, we could not address all detected issues in this research. We expect to handle them in the future for a more

ro of

considerate service. We would like to suggest a few guidelines for future studies. First, when collecting

information about epilepsy from mobile users, mHealth epilepsy data standard should be

-p

considered. Common Data Elements are currently available for mHealth in epilepsy devices [29]. Integration of our data with other international datasets for research purposes will

re

contribute to the development of epilepsy management protocols and improvement of interoperability, and patients will have great flexibility when managing their disorder. Second,

lP

in the aspects of utilizing data on seizure occurrence data, the gathered data can be used to evaluate temporal patterns of seizure events with statistical models such as zero‐ inflated

na

negative binomial mixed‐ effects model or incidence rate rations. Through this, the relationship of seizures with its triggers factors can be identified more accurately [30].

ur

Moreover, the general pattern analysis is possible, which provides valuable insight to the physicians [31, 32]. Lastly, when physicians or researchers refer to seizure data that patients

Jo

input, they should consider that patient reports are not accurate in terms of frequency and seizure type. Hence, research to complement this needs to be processed [33]. The test results were positive, but this should only be speculated upon while considering its limitations. First, we conducted the usability test only with a single hospital, so it is difficult to generalize the study results. Therefore, it is necessary to initiate research later with more

participants to draw insights for representing a wider range of users. Second, we could not verify the clinical effectiveness of the application with control and intervention groups. Nevertheless, we want to highlight that this study tried to overcome the limitations of existing epilepsy management mobile applications by improving UX, the wide coverage of trigger factors, practical educational materials, and the integration with the hospital EHR system.

ro of

4. Conclusions In this study, we designed, developed, and evaluated the usability of the EHR-integrated mobile application Brain4U. This investigation covered a wide range of factors that are

-p

important in the management of epilepsy. In addition, based on the concern that this

application could be complicated to use, we tried to design the UI and flow to be as simple as

re

possible. Our efforts paid off upon observing the usability test results, but there are still many limitations to be overcome in future research. We expect our application to be a foundational

patients and their caregivers.

na

5. Acknowledgments

lP

stone for improving not only epilepsy disorder management but also the quality of life of

ur

This work was supported by the Information Technology Research & Development program

Jo

of Seoul National University Bundang Hospital and SK Telecom.

6. Summary Points

What was already known

ro of

Controlling seizures to manage epilepsy is important, but it is not easy due to the diversity and complexity of seizures.

There are various epilepsy management mHealth services but none that provide functions targeting the management of seizure trigger factors comprehensively or that are compatible with physicians’ EHR systems.

-p

What this study added

re

The study developed and evaluated an mHealth service for managing epilepsy compatible with the hospital’s EHR system. Functions such as seizure Diary, Medication Reminder, Appointments, Outpatient Survey, and My Epilepsy were developed to aid patients’ epilepsy management.

lP

The usability test showed that participants were highly satisfied with the service and there were differences between participants in scenario-based task success rate.

Jo

ur

na

The degree of perceived importance of the functions differed by participants’ characteristics.

Legends Figure 1. Information architecture of Brain4U for patients

ro of

Figure 2. System architecture of Brain4U linked to the Electronic Health Record system Figure 3. Brain4U mobile application for patients

Figure 4. Physicians’ monitoring page integrated with hospital Electronic Health Record system

Figure 7. Completion time for each task

re

-p

Figure 5. (Left) Response rate of perceived important factors and what participants actually did for managing epilepsy; (right) response rate of the degree of interest in managing epilepsy and the status of recording seizures and medications

lP

Figure 8. Usefulness and expected frequency of each function’s use by participant characteristics

na

Table 1. Scenario-based tasks for evaluating usability of Brain4U mobile application Table 2. Pre-expectation survey result

ur

Table 3. Issues and implications of Brain4U design of user interface

Jo

Table 4. Overview of the positive and negative opinions of Brain4U

References [1] WHO. (2019). Epilepsy. Retrieved from https://www.who.int/news-room/factsheets/detail/epilepsy [2] S.M. Bowman, M.E. Aitken, G.B. Sharp, Disparities in injury death location for people with epilepsy/seizures, Epilepsy Behav, 17 (2010) 369-372. [3] H.B. Alexander, D.K. Broshek, M. Quigg, Quality of life in adults with epilepsy is

78 (2018) 96-99. [4] G. Mignot, Drug trials in epilepsy, BMJ, 313 (1996) 1158.

ro of

associated with anticonvulsant polypharmacy independent of seizure status, Epilepsy Behav,

-p

[5] WebMD. (unknown). Epilepsy Treatments: Find the Right Medication. Retrieved from https://www.webmd.com/epilepsy/guide/choosing-best-epilepsy-medication-for-you#1

re

[6] J.S. Mbwana, Z.M. Grinspan, R. Bailey, M. Berl, J. Buchhalter, A. Bumbut, Z. Danner, T. Glauser, A. Glotstein, H. Goodkin, B. Jacobs, L. Jones, B. Kroner, G. Lapham, T.

lP

Loddenkemper, D.M. Maraganore, D. Nordli, W.D. Gaillard, Using EHRs to advance epilepsy care, Neurol Clin Pract, 9 (2019) 83-88.

na

[7] R.S. Fisher, D.E. Blum, B. DiVentura, J. Vannest, J.D. Hixson, R. Moss, S.T. Herman, B.E. Fureman, J.A. French, Seizure diaries for clinical research and practice: limitations and

ur

future prospects, Epilepsy Behav, 24 (2012) 304-310.

Jo

[8] F.A. Le Marne, S. Butler, E. Beavis, D. Gill, A.M.E. Bye, EpApp: Development and evaluation of a smartphone/tablet app for adolescents with epilepsy, J Clin Neurosci, 50 (2018) 214-220.

[9] The Johns Hopkins University (unkown). EpiWatch. Retrieved from https://apps.apple.com/us/app/epiwatch/id1047757228EpiWatch.EpiWatch. [10] Neutun Labs inc (2017). Seizure Sync Epilepsy Log. Retrieved from

https://apps.apple.com/us/app/seizure-sync-epilepsy-log/id1084926036 Seizure Sync.Seizure Sync. [11] P.S. Pandher, K.K. Bhullar, Smartphone applications for seizure management, Health Informatics J, 22 (2016) 209-220. [12] C. Escoffery, R. McGee, J. Bidwell, C. Sims, E.K. Thropp, C. Frazier, E.D. Mynatt, A review of mobile apps for epilepsy self-management, Epilepsy Behav, 81 (2018) 62-69.

ro of

[13] S.N. Haron, M.Y. Hamida, A. Talib, Towards Healthcare Service Quality: An Understanding of the Usability Concept in Healthcare Design, Procedia - Social and Behavioral Sciences, 42 (2012) 63-73.

[14] I. Maramba, A. Chatterjee, C. Newman, Methods of usability testing in the development

-p

of eHealth applications: A scoping review, Int J Med Inform, 126 (2019) 95-104.

re

[15] Empatica. (2019, August 26). Alert for Embrace watch. Retrieved from https://play.google.com/store/apps/details?id=com.empatica.embrace.alert&hl=en.Alert App,

lP

Empatica Inc.,.Alert App, Empatica Inc.

[16] SeizAlarm, LLC. (2015). SeizAlarm. Retrieved from

na

https://apps.apple.com/us/app/seizalarm-seizure-detection/id978475280.SeizAlarm. [17] MCM Net Limited. (unknown). Epilepsy Tool Kit. Retrieved from

ur

http://myhealthapps.net/app/details/73/epilepsy-tool-kit.Epilepsy Tool Kit.Epilepsy Tool Kit. [18] Irody Inc.. (unknown). EpiDiary. Retrieved from

Jo

https://apps.apple.com/us/app/epidiary/id362460642.EpiDiary.EpiDiary. [19] E. Foundation. (unknown)., My Seizure Diary. Retrieved from https://diary.epilepsy.com/. [20] S.T. LLC. (unknown). , Seizure Tracker. Retrieved from https://www.seizuretracker.com/index.php. [21] M. Kovacs, The Children's Depression, Inventory (CDI), Psychopharmacol Bull, 21

(1985) 995-998. [22] A.T. Beck, N. Epstein, G. Brown, R.A. Steer, An inventory for measuring clinical anxiety: psychometric properties, J Consult Clin Psychol, 56 (1988) 893-897. [23] S. Zhang, D.E. Faries, M. Vowles, D. Michelson, ADHD Rating Scale IV: psychometric properties from a multinational study as a clinician-administered instrument, Int J Methods Psychiatr Res, 14 (2005) 186-201.

ro of

[24] B. Echenne, V. Humbertclaude, R. Cheminal, [Treatment of epilepsy in children], Pediatrie, 48 (1993) 883-887.

[25] S.W. Goodwin, A.I. Lambrinos, M.A. Ferro, M. Sabaz, K.N. Speechley, Development

(QOLCE-55), Epilepsia, 56 (2015) 864-872.

-p

and assessment of a shortened Quality of Life in Childhood Epilepsy Questionnaire

re

[26] J.C. Mandel, D.A. Kreda, K.D. Mandl, I.S. Kohane, R.B. Ramoni, SMART on FHIR: a standards-based, interoperable apps platform for electronic health records, J Am Med Inform

lP

Assoc, 23 (2016) 899-908.

[27] T. Tullis, W. Albert, Measuring the User Experience: Collecting, Analyzing, and

Kaufumann2013.

na

Presenting Usability Metrics (Interactive Technologies), 2nd edition, Morgan

ur

[28] System Usability Scale (SUS), U.S. Department of Health & Human Services

Jo

[29] D.M. Goldenholz, R. Moss, D.A. Jost, N.E. Crone, G. Krauss, R. Picard, C. Caborni, J.E. Cavazos, J. Hixson, T. Loddenkemper, T.D. Salazar, L. Lubbers, L.C. Harte-Hargrove, V. Whittemore, J. Duun-Henriksen, E. Dolan, N. Kasturia, M. Oberemk, M.J. Cook, M. Lehmkuhle, M.R. Sperling, P.O. Shafer, Common data elements for epilepsy mobile health systems, Epilepsia, 59 (2018) 1020-1026. [30] V. Ferastraoaru, D.M.G.S. Chiang, R. Moss, W.H. Theodore, S.R. Haut, Characteristics

of large patient-reported outcomes: Where can one million seizures get us?, Epilepsia Open, 4 (2018) 364-373. [31] M. Daniel Goldenholz, PhD, M. Kshitiz Rakesh, P. Kush Kapur, M. Gaínza-Lein, R. Hodgeman, R. Moss, M. William H Theodore, M. and Tobias Loddenkemper, Different as Night and Day: Patterns of Isolated Seizures, Clusters, and Status Epilepticus, Epilepsia, 59 (2018) e73–e77.

ro of

[32] L.D. Schneider, R.E. Moss, D.M. Goldenholz, Daylight saving time transitions are not associated with increased seizure incidence, Epilepsia, 60 (2019).

[33] P. Karoly, D. Goldenholz, M. Cook, Are the days of counting seizures numbered?,

Jo

ur

na

lP

re

-p

Current Opinion in Neurology, 31 (2018) 162–168.