A clinician survey of using speech recognition for clinical documentation in the electronic health record

Accepted Manuscript Title: A Clinician Survey of Using Speech Recognition for Clinical Documentation in the Electronic Health Record Authors: Foster R. Goss, Suzanne V. Blackley, Carlos A. Ortega, Leigh T. Kowalski, Adam B. Landman, Chen-Tan Lin, Marie Meteer, Samantha Bakes, Stephen C. Gradwohl, David W. Bates, Li Zhou PII: DOI: Reference:

S1386-5056(19)30473-3 https://doi.org/10.1016/j.ijmedinf.2019.07.017 IJB 3938

To appear in:

International Journal of Medical Informatics

Received date: Revised date: Accepted date:

30 April 2019 20 July 2019 30 July 2019

Please cite this article as: Goss FR, Blackley SV, Ortega CA, Kowalski LT, Landman AB, Lin C-Tan, Meteer M, Bakes S, Gradwohl SC, Bates DW, Li Z, A Clinician Survey of Using Speech Recognition for Clinical Documentation in the Electronic Health Record, International Journal of Medical Informatics (2019), https://doi.org/10.1016/j.ijmedinf.2019.07.017 This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

A Clinician Survey of Using Speech Recognition for Clinical Documentation in the Electronic Health Record

Foster R. Goss, DO, MMSc1,Suzanne V. Blackley, MA2,Carlos A. Ortega, BS3 Leigh T. Kowalski, MS3,Adam B. Landman, MD4,6,Chen-Tan Lin, MD5,Marie Meteer, PhD6 Samantha Bakes1,Stephen C. Gradwohl, MD, MSCI9,David W. Bates, MD, MSc2,3,6,Li Zhou,

ro

1

of

MD, PhD3,6

Department of Emergency Medicine, University of Colorado School of Medicine,

-p

Aurora, CO, USA

Clinical & Quality Analysis, Partners HealthCare System, Boston, MA, USA

3

Division of General Medicine and Primary Care, Brigham and Women’s Hospital, Boston, MA,

re

2

lP

USA

Department of Emergency Medicine, Brigham and Women’s Hospital, Boston, MA, USA

5

Department of Internal Medicine, University of Colorado School of Medicine, Aurora, CO,

USA

ur na

4

Harvard Medical School, Boston, MA, USA

7

Brandeis University, Waltham, MA, USA

Jo

6

8

Children’s Hospital Colorado, University of Colorado, Aurora, CO, USA

Corresponding Author: Foster Goss ([email protected])

Mail Stop B215, Leprino Building 12401 East 17th Ave Aurora, Colorado 80045

HIGHLIGHTS 78.8% of clinicians were satisfied with SR



77.2% agreed that SR improves efficiency



75.5% of respondents estimated seeing 10 or fewer errors per dictation



Satisfaction was positively associated with efficiency



Satisfaction was negatively associated with error prevalence and editing time

re

-p

ro

of



ABSTRACT:

lP

Objective: To assess the role of speech recognition (SR) technology in clinicians’ documentation workflows by examining use of, experience with and opinions about this

ur na

technology.

Materials and Methods: We distributed a survey in 2016-2017 to 1,731 clinician SR users at two large medical centers in Boston, Massachusetts and Aurora, Colorado. The survey asked about demographic and clinical characteristics, SR use and preferences, perceived accuracy,

Jo

efficiency, and usability of SR, and overall satisfaction. Associations between outcomes (e.g., satisfaction) and factors (e.g., error prevalence) were measured using ordinal logistic regression. Results: Most respondents (65.3%) had used their SR system for under one year. 75.5% of respondents estimated seeing 10 or fewer errors per dictation, but 19.6% estimated half or more of errors were clinically significant. Although 29.4% of respondents did not include SR among

their preferred documentation methods, 78.8% were satisfied with SR, and 77.2% agreed that SR improves efficiency. Satisfaction was associated positively with efficiency and negatively with error prevalence and editing time. Respondents were interested in further training about using SR effectively but expressed concerns regarding software reliability, editing and workflow. Discussion: Compared to other documentation methods (e.g., scribes, templates, typing, traditional dictation), SR has emerged as an effective solution, overcoming limitations inherent

of

in other options and potentially improving efficiency while preserving documentation quality.

ro

Conclusion: While concerns about SR usability and accuracy persist, clinicians expressed

positive opinions about its impact on workflow and efficiency. Faster and better approaches are

-p

needed for clinical documentation, and SR is likely to play an important role going forward.

re

Keywords: Clinical Documentation, Natural Language Processing, Efficiency, Quality of Care,

Jo

ur na

lP

Safety, Artificial Intelligence, Speech Recognition

1. INTRODUCTION Speech recognition (SR), a subfield of artificial intelligence involving automatic recognition and translation of voice into text, has been increasingly adopted by hospitals and clinicians for documentation in the electronic health record (EHR). First widely used by radiologists in the 1990s, SR can significantly reduce report turnaround time and costs compared to traditional dictation and transcription.[1-5] Clinicians across different clinical settings use diverse

of

documentation methods, such as typing, templates, scribes, traditional dictation and SR,[6]

ro

depending on available resources, cost, clinician preference and other factors. Trade-offs in quality, satisfaction, productivity and workflow exist with each method.[3, 6]

-p

In the United States, nearly 60% of physicians report experiencing burnout due to increasing

re

clerical burden and challenges with EHR usability.[7, 8] A 2016 study found nearly half of ambulatory physicians’ office time was spent on EHR and desk work, with 38.5% of that time

lP

spent on documentation-related tasks.[9] With recent technology advances, many institutions have adopted SR software to meet clinicians’ documentation needs.[6, 10] Reports estimate

12]

ur na

nearly half of all licensed physicians currently use some form of SR-assisted documentation.[11,

A recent systematic review of SR for clinical documentation found studies have primarily focused on the effects of SR on productivity and chart review-based quality measures in a

Jo

limited number of clinical settings, particularly radiology, which may have more scripted workflows than other settings.[11, 13-17] However, researchers are increasingly employing surveys and interviews to gather feedback from real clinician users.[18-24] Most studies to date have had relatively few participants (range: 2 to 186; median: 10),[18-24] with the largest

conducted in Denmark.[24] Further, most focused on a specific topic, such as expectations and experiences before and after using SR.[18, 19, 24] To examine multiple aspects of SR, we surveyed clinicians from two large integrated healthcare systems and assessed its role in documentation workflows, users’ satisfaction with their SR systems and perceived quality of SR-generated medical documents.

of

2. METHODS 2.1 Study Design, Setting and Participants

ro

The survey was conducted at Brigham and Women’s Hospital (BWH) in Boston,

Massachusetts between September 20 and October 14, 2016 and University of Colorado health

-p

system (UCHealth) in Aurora, Colorado between February 22 and March 16, 2017. Both

re

institutions use front-end SR medical dictation systems (Dragon® 10.1 and Dragon® Medical 360 made by Nuance (Burlington, MA)) integrated with electronic health record systems (Epic

lP

Systems; Verona, WI). The survey was distributed to 1,731 clinicians registered as SR users (808 from BWH, 923 from UCHealth). The Partners Healthcare Human Research Committee and

ur na

Colorado Multiple Institutional Review Board approved this study. 2.2 Survey Instrument Development and Data Collection The survey was developed by a multidisciplinary team of physicians, informaticians and computational linguists. It contained multiple-choice and free-text questions about

Jo

demographics, clinical characteristics, SR use and preferences, perceived accuracy, efficiency, usability and satisfaction. Responses were collected and managed using REDCap[25] electronic data capture tools hosted at each institution to ensure anonymity and data integrity. A copy of the survey is included in the appendix.

A participation invitation and link to the online survey form was distributed by email to eligible subjects, with scheduled reminders sent during each week of the study period. 2.3 Statistical Analysis We described and compared respondents’ demographic and clinical characteristics, SR use and training, perceived accuracy, efficiency and satisfaction between institutions. Chi-squared or Fisher’s Exact tests were used, depending on sample size, to compare differences among groups.

of

Ordinal logistic regression with the Laplace method was used to examine relationships between

ro

the primary outcome (i.e., satisfaction) and secondary outcomes (i.e., efficiency, errors per

document, clinically significant errors and editing time) and factors (e.g., age, language, training,

-p

role, specialty, patient volume and site). P-values below 0.05 were considered statistically

re

significant. Ordinal logistic regression was performed using SAS version 9.4.[26] Other analyses were performed using R Studio version 3.5.0.[27]

lP

3. RESULTS 3.1 Respondent Characteristics

ur na

We received responses from 348 users (20.1%), of whom 103 were excluded for using dictation with medical transcriptionists (n=72), not using dictation (n=12), not completing the survey (n=18) or listing their specialty as radiology (n=1). Radiologists were excluded because of the field’s unique workflow and early SR adoption (Figure 1). The remaining 245 responses

Jo

(14.2% of invited subjects) were included in analysis (Table 1); 95 (38.8%) were from BWH and 150 (61.2%) from UCHealth. Most respondents were between 35-54 years old (66.9%), white (84.4%), native English speakers (93.1%) and received medical education in English (96.7%). Most respondents were physicians (81.6%), although BWH had fewer non-physician respondents (e.g., Physician Assistant, Nurse Practitioner) than UCHealth (14.7% vs. 20.7%; p=0.23).

Nearly half (47.3%) of respondents specialized in general medicine, followed by emergency medicine (20.8%) and surgery (13.1%). Average patient load varied, with the majority (77.1%) seeing an average of 25-100 patients per week. Ambient noise level during dictation (rated from 1 [silence] to 4 [e.g., a large crowd talking]) was most often (44.1%) rated as 3 (e.g., a few people talking), with fewer users at BWH giving ratings of 3 or higher (50.5% vs 67.3%;

Figure 1: Enrollment of SR users from BWH and UCHealth.

-p

ro

1,731 clinicians registered as SR users assessed for eligibility (923 UCHealth/808 BWH)

of

p=0.029).

ur na

lP

re

348 (20.1%) responded to the survey

103 were excluded from the study • 72 used medical transcriptionists • 12 did not use dictation • 18 did not complete the survey • 1 listed their specialty as radiology

245 clinician SR users included (150 UCHealth/95 BWH)

Jo

176 (71.8%) users of Dragon v360

6 (2.4%) users of Dragon v10.1

63 (25.7%) unsure of SR type

Table 1. Demographic and Clinical Characteristics of Healthcare Provider Respondents

No. (%)a

Institution

All Providers

BWH

UCHealth

(n = 245)

(n = 95)

(n = 150)

25-34

24 (9.8)

12 (12.6)

12 (8.0)

35-44

82 (33.5)

35 (36.8)

47 (31.3)

45-54

82 (33.5)

31 (32.6)

51 (34.0)

55-64

44 (18.0)

10 (10.5)

34 (22.7)

65 and older

13 (5.3)

p-value Respondents’ Characteristics

ro 6 (4.0)

207 (84.5)

78 (82.1)

129 (86.0)

Black

2 (0.8)

2 (2.1)

0 (0.0)

21 (8.6)

13 (13.7)

8 (5.3)

3 (1.2)

0 (0.0)

3 (2.0)

12 (4.9)

2 (2.1)

10 (1.3)

English

228 (93.1)

86 (90.5)

142 (94.7)

Non-Englishb

14 (5.7)

8 (8.4)

6 (4.0)

English + Otherc

3 (1.2)

1 (1.1)

2 (1.3)

English

237 (96.7)

91 (95.8)

146 (97.3)

Otherd

8 (3.3)

4 (4.2)

4 (2.7)

Physician

200 (81.6)

81 (85.3)

119 (79.3)

Physician Assistant

31 (12.7)

11 (11.6)

20 (13.3)

Hispanic or Latino

ur na

Other or Declineda

lP

White

Asian or Pacific Islander

0.097

-p 7 (7.4)

re

Race or Ethnic Group

of

Age Group, y

0.011

Jo

Native Language

0.29

Primary Language of Medical Education

Provider Role

0.26

Nurse/Nurse Practitioner

12 (4.9)

2 (2.1)

10 (6.7)

Othere

2 (0.8)

1 (1.1)

1 (0.7)

General medicinef

116 (47.3)

47 (49.5)

69 (46.0)

Emergency medicine

50 (20.4)

14 (14.7)

36 (24.0)

Surgeryg

32 (13.1)

15 (15.8)

17 (11.3)

Othersh

47 (19.2)

19 (20.0)

28 (18.7)

0.39

Clinical characteristics Hospital Department

of

0.38

How many patients do you see on average over a clinical week? 43 (17.6)

19 (20.0)

24 (16.0)

25-50

75 (30.6)

38 (40.0)

37 (24.7)

50-75

63 (25.7)

21 (22.1)

42 (28.0)

75-100

51 (20.8)

16 (16.8)

35 (23.3)

More than 100

13 (5.4)

1 (1.1)

12 (8.0)

re

-p

ro

Less than 25

0.31

1-Silent 2-floor fan

ur na

3-a few people talking

lP

On a scale of 1-4, how would you describe the environment that you dictate in? 44 (18.0)

25 (26.3)

19 (12.7)

52 (21.2)

22 (23.2)

30 (20.0)

108 (44.1)

33 (34.7)

75 (50.0)

41 (16.7)

15 (15.8)

26 (17.3)

0.024

4-a large crowd talking or a busy clinical environment

Percentages may not sum to exactly 100.0% due to rounding

b

Non-English includes: Chinese (1), French (2), German (3), Korean (2) and Other (9)

c

Other includes: Arabic (1), Spanish (2), Spanish + German (1) and Other (2)

d

Other includes: Afrikaans (1), Dutch (1), Farsi (2), French (1), German, Korean (1), Spanish (3) and Turkish (1)

e

Other includes: Social Worker and Neuropsychologist

f

General Medicine includes: Family Medicine, Internal Medicine, Pediatrics, Primary Care and dual specialties with

Jo

a

Internal Medicine (e.g., General Medicine + Hematology)

g

Surgical specialties include: Cardiothoracic, General Surgery, Neurosurgery, Obstetrics and Gynecology, Ophthalmology, Orthopedics, Otolaryngology and Urology

h

Other specialties include: Anesthesia, Cardiac Electrophysiology, Cardiology, Dermatology, Endocrinology, Geriatrics, Gastroenterology, Hematology, Neurology, Occupational Medicine, Oncology, Other, Pain Management, Palliative Care, Pathology, Pharmacology, Physical Medicine and Rehabilitation, Psychiatry, Pulmonary Critical Care, Radiation Oncology, Rheumatology, Sleep Medicine, Social Work and Urgent Care.

of

3.2 SR Utilization At the time of the survey, most respondents (72.0%) used Dragon® Medical 360; only 2.0%

ro

used Dragon® Medical 10.1, and 26.0% did not know which version they used (Table 2). About

-p

one quarter (25.7%) of respondents had used their SR system for less than 3 months, and only 14.3% had used their system for over 2 years, though, to a lesser extent at BWH (5.3% vs.

re

20.0%; p=0.003). Besides the EHR, SR was also commonly used with Microsoft Outlook (24.9%) and Microsoft Word (22.0%), with more respondents from BWH reporting use of either

lP

program (36.8% vs. 27.3%; p=0.15).

Timing of dictation differed between sites. Respondents from BWH more often reported

ur na

dictating at the end of the day (71.6% vs. 46.0%; p<0.001) or the next day (31.6% vs. 16.7%; p=0.012), though dictating immediately after seeing a patient was less common (60.0% vs. 76.0%, p=0.009). Although most respondents dictated at their workplaces, 13.1% reported

Jo

dictating at home, and more than half (56.7%) of respondents agreed that they would like to be able to dictate at home or on a personal computer. While all respondents were registered SR users, only 70.6% included dictation among their

preferred documentation methods. Templates were preferred by 53.9%, SmartPhrases (i.e., EHR commands that insert pre-specified text into a note) by 46.5% and typing by 29.8%. SmartPhrases were preferred less often at BWH (38.9% vs. 51.3%; p=0.067), as were macros

(5.3%; vs. 17.3%; p=0.009). BWH respondents more often preferred templates (61.1% vs. 49.3%; p<0.001). Most respondents (58.0%) used dictation for over 75% of their patients. About one quarter (26.9%) used dictation for all sections of the chart. Specific sections where dictation was used most included history of present illness (66.9%) and assessment and plan (66.9%). Sections where dictation was used least were medications (3.3%) and allergies (1.2%). 3.3 Perception of SR Accuracy

of

Perceived error incidence was low, with 43.7% of respondents reporting 5 or fewer errors per

ro

document and only 7.3% reporting more than 20. More importantly, most respondents (69.0%) considered 25% or fewer errors to be clinically significant. Overall, 21.2% of respondents spent

-p

25% or more of their documentation time editing. Respondents had mixed opinions when asked

re

if they had noticed improved accuracy with the latest version of the SR system, respondents strongly agreed, agreed, felt neutral and disagreed in roughly equal numbers, while a few

lP

disagreed strongly. 3.4 Efficiency and Satisfaction Using SR

ur na

Most respondents (77.1%) agreed that using SR saved them time and improved their efficiency. When asked whether SR decreased administrative burden, 61.6% agreed that it did, but 35.9% disagreed or were neutral. Overall, 86.2% of respondents felt their SR system was easy to use, 78.8% were satisfied with their SR system, and only 5.7% were very unsatisfied.

Jo

3. 5 Training and Education

Most respondents (93.1%) had participated in training. Of those, 45.6% expressed a desire

for additional training, and most (82.0%) felt that they were able to apply the knowledge gained during training to their clinical practice and that the training should be offered as part of EHR Go-Live training. Among those who received no training, 29.4% were not interested in receiving

training, 23.5% were interested, and the remaining 47.1% were ambivalent. Frequently requested topics for additional training included dictation tips (32.9%), the correction process (32.5%) and entering of Epic SmartPhases (28.9%). Table 2. Provider use and type of SR, user reported accuracy, and training. No. (%)a

Institution

All Providers

BWH

UCHealth

(n = 245)

(n = 95)

(n = 150)

Dragon 10.1

6 (2.4)

0 (0.0)

6 (4.0)

Dragon 360

176 (71.8)

79 (83.2)

97 (64.7)

I don’t know

63 (25.7)

p-value

of

Use of SR

-p 16 (16.8)

re

How long have you been using your current SR system?

ro

Which version of Dragon do you use?

0-3 months

0.002

47 (31.3)

63 (25.7)

22 (23.2)

41 (27.3)

49 (20.0)

30 (31.6)

19 (12.7)

48 (19.6)

19 (20.0)

29 (19.3)

50 (20.4)

19 (20.0)

31 (20.7)

35 (14.3)

5 (5.3)

30 (20.0)

Epic/eCare

245 (100.0)

95 (100.0)

150 (100.0)

0.39

Microsoft Outlook

61 (24.9)

31 (32.6)

30 (20.0)

< 0.001

Microsoft Word

54 (22.0)

26 (27.4)

28 (18.7)

0.21

11 (4.5)

4 (4.2)

7 (4.7)

0.002

While in the room with the patient

4 (1.6)

4 (4.2)

0 (0.0)

0.022

Immediately after seeing the patient

171 (69.8)

57 (60.0)

114 (76.0)

0.009

At the end of the day

137 (55.9)

68 (71.6)

69 (46.0)

< 0.001

The next day

55 (22.4)

30 (31.6)

25 (16.7)

0.012

6-12 months 1-2 years

ur na

More than 2 years

lP

3-6 months

0.001

Jo

With which applications do you use SR?

Other

When do you do your dictations?

Other

21 (8.6)

11 (11.6)

10 (6.7)

0.18

Office

146 (59.6)

61 (64.2)

85 (56.7)

0.35

Hospital

80 (32.7)

31 (32.6)

49 (32.7)

1

Clinic

58 (23.7)

29 (30.5)

29 (19.3)

0.071

Home

32 (13.1)

17 (17.9)

15 (10.0)

0.12

Where do you primarily do your dictations?

I would like to use the SR system at home on a personal Windows laptop or Windows PC 98 (40.0)

38 (40.0)

60 (40.0)

Somewhat Agree

31 (12.7)

12 (12.6)

Neutral

37 (15.1)

19 (20.0)

Somewhat Disagree

16 (6.5)

4 (4.2)

12 (8.0)

Strongly Disagree

25 (10.2)

7 (7.4)

18 (12.0)

N/A

38 (15.5)

15 (15.8)

23 (15.3)

of

Strongly Agree

19 (12.7)

re

-p

ro

18 (12.0)

0.44

I would like to use the SR system at home on a personal Mac

49 (51.6)

54 (36.0)

38 (15.5)

12 (12.6)

26 (17.3)

31 (12.7)

20 (21.1)

11 (7.3)

8 (3.3)

2 (2.1)

6 (4.0)

Strongly Disagree

31 (12.7)

9 (9.5)

22 (14.7)

N/A

34 (13.9)

12 (12.6)

22 (14.7)

38 (15.5)

18 (18.9)

20 (13.3)

25-50%

26 (10.6)

8 (8.4)

18 (12.0)

50-75%

39 (15.9)

20 (21.1)

19 (12.7)

75-100%

142 (58.0)

49 (51.6)

93 (62.0)

Somewhat Agree Neutral

ur na

Somewhat Disagree

103 (42.0)

lP

Strongly Agree

0.26

What percentage of your patients do you use dictation for?

Jo

0-25%

0.15

What sections of the chart do you use dictations for? All

66 (26.9)

36 (37.9)

30 (20.0)

0.004

History of present illness

164 (66.9)

55 (57.9)

109 (72.7)

0.032

33 (13.5)

15 (15.8)

18 (12.0)

0.49

Past medical history

18 (7.3)

11 (11.6)

7 (4.7)

0.045

Past surgical history

15 (6.1)

8 (8.4)

7 (4.7)

0.23

Social history

25 (10.2)

16 (16.8)

9 (6.0)

0.007

Family history

17 (6.9)

10 (10.5)

7 (4.7)

0.081

Laboratories

24 (9.8)

6 (6.3)

18 (12.0)

0.33

Medications

8 (3.3)

3 (3.2)

5 (3.3)

0.74

Physical exam

61 (24.9)

19 (20.0)

42 (28.0)

0.22

Allergies

3 (1.2)

1 (1.1)

2 (1.3)

0.64

Assessment and plan

164 (66.9)

54 (56.8)

110 (73.3)

0.015

Other

13 (5.3)

2 (2.1)

11 (7.3)

0.26

112 (74.7)

0.12

173 (70.6)

Macros

Accuracy

ur na

Typing

ro

31 (12.7)

5 (5.3)

26 (17.3)

0.009

114 (46.5)

37 (38.9)

77 (51.3)

0.067

132 (53.9)

58 (61.1)

74 (49.3)

< 0.001

73 (29.8)

30 (31.6)

43 (28.7)

0.77

lP

Smart Phrases Templates

61 (64.2)

re

Dictation

-p

What are your preferred documentation methods?

of

Review of systems

What percentage of your clinical documentation time is spent editing? 77 (31.4)

25 (26.3)

52 (34.7)

10-25%

116 (47.3)

49 (51.6)

67 (44.7)

25-50%

38 (15.5)

14 (14.7)

24 (16.0)

14 (5.7)

7 (7.4)

7 (4.7)

Jo

0-10%

More than 50%

0.35

On average, how many errors are generated per document? 0-5

107 (43.7)

35 (36.8)

72 (48.0)

6-10

78 (31.8)

29 (30.5)

49 (32.7)

10-20

31 (12.7)

17 (17.9)

14 (9.3)

0.21

More than 20

18 (7.3)

10 (10.5)

8 (5.3)

Unsure

11 (4.5)

4 (4.2)

7 (4.7)

Of the errors made, what percentage is clinically significant? 169 (69.0)

64 (67.4)

105 (70.0)

25-50%

28 (11.4)

12 (12.6)

16 (10.7)

50-75%

21 (8.6)

8 (8.4)

13 (8.7)

75-100%

10 (4.1)

5 (5.3)

5 (3.3)

Unsure

17 (6.9)

6 (6.3)

11 (7.3)

I have noticed improved accuracy with the SR system 54 (22.0)

16 (16.8)

Somewhat Agree

59 (24.1)

22 (23.2)

37 (24.7)

Neutral

57 (23.3)

30 (31.6)

27 (18.0)

Somewhat Disagree

39 (15.9)

13 (13.7)

26 (17.3)

Strongly Disagree

20 (8.2)

8 (8.4)

12 (8.0)

6 (6.3)

10 (6.7)

94 (38.4)

36 (37.9)

58 (38.7)

Somewhat Agree

95 (38.8)

40 (42.1)

55 (36.7)

Neutral

24 (9.8)

9 (9.5)

15 (10.0)

Somewhat Disagree

14 (5.7)

4 (4.2)

10 (6.7)

Strongly Disagree

17 (6.9)

5 (5.3)

12 (8.0)

1 (0.4)

1 (1.1)

0 (0.0)

-p

re lP

16 (6.5)

Productivity and efficiency

38 (25.3)

ro

Strongly Agree

N/A

0.93

of

0-25%

0.17

My SR dictation system saves me time and makes me more efficient

Jo

ur na

Strongly Agree

N/A

0.69

The administrative burden associated with charting has been reduced because of the SR system and related training Strongly Agree

74 (30.2)

28 (29.5)

46 (30.7)

Somewhat Agree

77 (31.4)

32 (33.7)

45 (30.0)

Neutral

40 (16.3)

15 (15.8)

25 (16.7)

0.72

Somewhat Disagree

23 (9.4)

11 (11.6)

12 (8.0)

Strongly Disagree

25 (10.2)

7 (7.4)

18 (12.0)

N/A or blank

6 (2.4)

2 (2.1)

4 (2.7)

Very Easy

81 (33.1)

28 (29.5)

53 (35.3)

Somewhat Easy

130 (53.1)

56 (58.9)

74 (49.3)

Neutral

19 (7.8)

7 (7.4)

12 (8.0)

Somewhat Difficult

10 (4.1)

3 (3.2)

Very Difficult

5 (2.0)

1 (1.1)

Very Satisfied

73 (29.8)

26 (27.4)

47 (31.3)

Somewhat Satisfied

120 (49.0)

54 (56.8)

66 (44.0)

Neutral

18 (7.3)

4 (4.2)

14 (9.3)

5 (5.3)

15 (10.0)

14 (5.7)

6 (6.3)

8 (5.3)

228 (93.1)

89 (93.7)

139 (92.7)

17 (6.9)

6 (6.3)

11 (7.3)

Satisfaction How easy do you find your SR system to use?

Training and Education

of 7 (4.7) 4 (2.7)

ro

re 20 (8.2)

lP

Very Unsatisfied

-p

How satisfied are you with your SR system?

Somewhat Unsatisfied

0.62

0.18

Yes No

ur na

Have you participated in SR system training?

0.94

Based on my experience with the SR system, I feel the training should be offered as part of

Jo

general EHR training Strongly Agree

125 (51.0)

43 (45.3)

82 (54.7)

Somewhat Agree

71 (29.0)

36 (37.9)

35 (23.3)

Neutral

36 (14.7)

13 (13.7)

23 (15.3)

Somewhat Disagree

5 (2.0)

3 (3.2)

2 (1.3)

Strongly Disagree

3 (1.2)

0 (0.0)

3 (2.0)

N/A

5 (2.0)

0 (0.0)

5 (3.3)

0.042

If you have already received training, would you like additional training? Yes

104 (45.6)

47 (52.8)

57 (38.0)

No

124 (54.4)

42 (47.2)

82 (54.7)

Which topics would you want to review during follow-up training? 75 (72.1)

34 (72.3)

41 (71.9)

Entering Epic EHR SmartPhrases*

66 (63.5)

26 (55.3)

40 (70.2)

Orders using the SR system

48 (46.2)

24 (51.1)

24 (42.1)

Basic SR commands

44 (42.3)

17 (36.2)

27 (47.4)

Correction process

74 (71.2)

30 (63.8)

Vocabulary editor

57 (54.8)

25 (53.2)

Custom SR commands (text)

49 (47.1)

23 (48.9)

26 (45.6)

SR custom commands (step-by-step)

44 (42.3)

19 (40.4)

25 (43.9)

20 (42.6)

23 (40.4)

37 (35.6)

lP

Microphone (e.g., PowerMic) configuration Other

10 (9.6)

44 (77.2) 32 (56.1)

ro

re

43 (41.3) system

-p

Custom templates in order to better use the SR

of

Dictation tips

18 (38.3)

19 (33.3)

6 (12.8)

4 (7.0)

If you have not received training, are you interested in receiving training?

No Maybe *

ur na

Yes

4 (23.5)

1 (16.7)

3 (27.3)

5 (29.4)

2 (33.3)

3 (27.3)

8 (47.1)

3 (50.0)

5 (44.4)

SmartPhrase: An EHR command that inserts a pre-specified text into a clinical note.

Jo

3.6 Association Analysis of Selected Outcomes Satisfaction was significantly associated with efficiency (p<0.001), percentage of time spent

editing documentation (p=0.006) and number of errors (p<0.001, Table 3). Odds of satisfaction increased as user efficiency increased and as the number of errors and editing time decreased (Figure 2). Improved efficiency was significantly associated with fewer errors (p<0.001) and less editing time (p=0.02). Increased errors per document and increased rates of clinically relevant

errors were significantly associated with greater editing time (p<0.001). Native language, age, training, role, ambient noise level and SR software version were not significantly associated with estimated error prevalence. Users reporting more than 20 errors per document were 14 times more likely to be dissatisfied than those reporting 0-5. Native language was significantly associated with editing time, with native English speakers reporting less editing time compared to native speakers of

of

other languages, as was specialty, with respondents specializing in emergency or general

ro

medicine reporting less editing time compared to those specializing in surgery. Satisfaction did appear to increase until ages 33-44 but declined thereafter. Satisfaction was highest in providers

-p

who saw 55-70 per week and lowest in those with greater than 100 patients per week. Language,

re

age, training, role, specialty, patient volume and ambient noise level were not significant predictors of satisfaction or efficiency. No significant differences were found between

Jo

ur na

lP

organizations (Table 3).

Figure 2: Odds of Satisfaction as errors, time editing, efficiency, age and number of patients per week increase compared to reference [R].

Satisfaction as time editing increases

ro

of

Satisfaction as errors increase

ur na

Satisfaction as age increases

lP

Satisfaction as efficiency increases

Satisfaction as # of patients increases

Jo

re

-p

Satisfaction as clinically significant errors rise

OR (95% CI)

p-value

Increased errors

efficiency

per document

OR (95% CI)

p-value

OR (95% CI)

Efficiency (My SR dictation system saves me time and makes me more efficient) 1 [Reference]

------

Somewhat disagree

27.86 (5.89, 131.72)

------

Neutral

23.06 (5.47, 97.16)

Somewhat agree

136.73 (34.13, 547.81)

------

Strongly agree

699.70 (164.71, 2972.43)

------

1.01 (0.26, 3.98)

------

1.20 (0.36, 4.07)

p=0.05

Increased clinically

Increased time for

significant errors

editing

OR (95% CI)

p-value

OR (95% CI)

[1 Reference]

1 [Reference]

0.34 (0.09, 1.30)

0.58 (0.15, 2.20)

0.76 (0.24, 2.41)

p<0.001

0.36 (0.11, 1.20)

0.55 (0.20, 1.51)

0.19 (0.07, 0.50)

0.35 (0.13, 0.97)

0.39 (0.14, 1.09)

0.10 (0.04, 0.30)

0.26 (0.09, 0.71)

------

-----

1 [Reference]

------

-----

3.13 (1.75, 5.59)

------

-----

3.81 (1.71, 8.47)

------

-----

16.96 (6.18, 46.55)

------

------

1 [Reference]

------

------

3.28 (1.54, 6.99)

re

p<0.001

1 [Reference]

-p

Strongly disagree

p-value

ro

Increased satisfaction

Increased

of

Table 3: Association analysis of selected outcomes variables *.

p-value

P=0.09

0-5

1 [Reference]

6-10

0.38 (0.21, 0.68)

1 [Reference]

0.79 (0.45, 1.38)

p<0.001 0.17 (0.08, 0.37)

>20

0.07 (0.03, 0.18)

p<0.001

0.36 (0.17, 0.75)

ur na

10-20

lP

Errors per document (On average, how many errors are generated per document?)

0.13 (0.05, 0.32)

p<0.001

Clinically significant errors (Of the errors made, which percentage is clinically significant?) 0-25

1 [Reference]

25-50%

0.26 (0.12, 0.57)

1 [Reference] 0.19 (0.09, 0.40)

P<0.001

p<0.001

p<0.001

0.28 (0.12, 0.67)

0.21 (0.09, 0.49)

------

------

4.28 (1.71, 10.71)

75-100%

0.22 (0.07, 0.72)

0.57 (0.16, 2.08)

------

------

5.05 (1.22, 20.84)

1 [Reference]

1 [Reference]

------

Jo

50-75%

Time for editing (What percentage of your clinical documentation time is spent editing?) 0-10 10-25%

1 [Reference]

1 [Reference]

0.52 (0.30, 0.90)

0.53 (0.30, 0.91) p=0.006

2.36 (1.31, 4.26) p=0.02

1.70 (0.76, 3.79) p<0.001

-----p<0.001

25-50%

0.33 (0.16, 0.71)

0.65 (0.32, 1.35)

5.81 (2.64, 12.77)

4.13 (1.60, 10.64)

------

>50

0.20 (0.06, 0.65)

0.19 (0.06, 0.59)

18.17 (5.40, 61.16)

18.73 (5.31, 66.03)

------

Age 1.21 (0.32, 4.67)

35-44 years old

1.53 (0.46, 5.05)

0.88 (0.24, 3.16)

0.64 (0.17, 2.35)

1.22 (0.40, 3.78) p=0.88

0.94 (0.30, 2.95) p=0.43

0.82 (0.27, 2.54)

0.74 (0.23, 2.32)

55-64 years old

1.13 (0.33, 3.94)

0.65 (0.20, 2.11)

65 and older

1 [Reference]

1 [Reference]

Physician

0.48 (0.24, 0.98)

0.39 (0.19, 0.82)

Nurse/nurse practitioner

0.69 (0.19, 2.56)

0.81 (0.24, 2.71)

0.41 (0.11, 1.54)

2.27 (0.69, 7.50)

1 [Reference]

1 [Reference]

1 [Reference]

2.24 (0.74, 6.79)

0.93 (0.46, 1.89)

1.11 (0.53, 2.33) 0.31 (0.07, 1.38)

re

0.45 (0.12, 1.61)

p=0.71 1.72 (0.55, 5.35)

-p

Role

1.61 (0.52, 5.02) p=0.69

0.58 (0.17, 1.97)

ro

1.21 (0.37, 3.98)

1.91 (0.53, 6.93)

0.45 (0.13, 1.53)

p=0.87

45-54 years old

p=0.23

0.51 (0.12, 2.17)

of

25-34 years old

p=0.10

1.35 (0.21, 8.79) p=0.057

1.06 (0.31, 3.65) p=0.37

p=0.11

16.06 (1.07, 242.

0.43 (0.04, 4.55)

Physician assistant

1 [Reference]

0.30 (0.03, 2.88)

Clinical Specialty 1.12 (0.48, 2.61)

Medicine

0.68 (0.33, 1.42)

6.70 (0.41, 109.54)

22.71 (1.62, 318.56)

1 [Reference]

1 [Reference]

1 [Reference]

1 [Reference]

1.36 (0.57, 3.22)

1.93 (0.79, 4.74)

1.24 (0.43, 3.60)

0.49 (0.21, 1.15)

0.79 (0.37, 1.65)

ur na

Emergency

15)

lP

Other

p=0.41

1.91 (0.85, 4.30)

p=0.163

1.35 (0.53, 3.45)

p=0.88

0.51 (0.24, 1.11)

p=0.02

p=0.37

Other

1.0 (0.43, 2.33)

0.96 (0.41, 2.25)

2.87 (1.15, 7.15)

1.04 (0.35, 3.10)

1.28 (0.53, 3.08)

Surgery

1 [Reference]

1 [Reference]

1 [Reference]

1 [Reference]

1 [Reference]

Native Language

1.0 (0.34, 2.90)

English + Other

745274.19 (0, >999)

Non-English

Jo

English

p=0.99

1 [Reference]

1.09 (0.40, 2.98)

p=0.55

1.04 (0.39, 2.78)

p>0.99

1.07 (0.28, 4.03)

p=0.40

0.61 (0.20, 1.80)

3.95 (0.31, 49.88)

1.05 (0.12, 8.97)

4.02 (0.40, 40.11)

10.12 (1.11, 92.11)

1 [Reference]

1 [Reference]

1 [Reference]

1 [Reference]

p=0.02

Have you participated in SR system training? No Yes Patients per week

0.91 (0.35, 2.33) 1 [Reference]

0.73 (0.30, 1.75) p=0.84

0.50 (0.18, 1.38) p=0.47

1 [Reference]

0.92 (0.29, 2.93) p=0.18

1 [Reference]

0.58 (0.23, 1.46) p=0.88

1 [Reference]

p=0.25 1 [Reference]

1 [Reference]

1 [Reference]

1 [Reference]

1 [Reference]

1 [Reference]

25-50

1.39 (0.69, 2.79)

1.48 (0.75, 2.91)

1.38 (0.66, 2.91)

1.0 (0.44, 2.30)

0.99 (0.50, 1.99)

50-75

2.08 (1.0, 4.32)

1.83 (0.89, 3.75)

1.16 (0.54, 2.50) p=0.17

1.49 (0.70, 3.18)

2.04 (0.95, 4.38)

1.15 (0.51, 2.57)

More than 100

0.96 (0.27, 3.38)

0.62 (0.18, 2.14)

Silent

0.82 (0.36, 1.86)

0.57 (0.25, 1.31)

Floor fan

0.80 (0.37, 1.75)

1.53 (0.44, 5.26)

1.53 (0.38, 6.20)

2.85 (0.80, 10.09)

2.04 (0.76, 5.49)

0.97 (0.44, 2.17)

0.71 (0.35, 1.42)

Large crowd talking

1 [Reference]

1 [Reference]

UCHealth

1 [Reference]

*

1 [Reference]

1.13 (0.70, 1.81)

p=0.74

1.44 (0.88, 2.35)

p=0.62

1 [Reference]

1.21 (0.45, 3.30) p=0.96

0.84 (0.42, 1.68)

lP

Location 1.08 (0.67, 1.75)

0.94 (0.43, 2.03)

re

0.91 (0.46, 1.82)

BWH

0.95 (0.42, 2.16)

p=0.41

Few people talking

p=0.05 0.76 (0.36, 1.60)

Noise Level

0.54 (0.25, 1.17)

0.51 (0.25, 1.06) p=0.29

0.60 (0.23, 1.58)

ro

75-100

p=0.94

0.49 (0.19, 1.27)

p=0.91

-p

p=0.34

of

Less than 25

p=0.98

1.03 (0.41, 2.57)

0.92 (0.48, 1.80)

1 [Reference]

1 [Reference]

1.21 (0.67, 2.20) p=0.14

1 [Reference]

1.05 (0.50, 2.23) p=0.32

1.32 (0.82, 2.14) p=0.53

1 [Reference]

p=0.25 1 [Reference]

ur na

Analysis excludes comparisons when predictor and the outcome are similar or identical (e.g., time editing | time editing, errors per document | errors per document, or errors per

Jo

document | clinically relevant errors).

3.7 Participant Comments

Eighty-five respondents (34.7%) entered comments (n=100) in the free-text field at the end of the survey. Comments were classified into 6 themes/topics: usability, accuracy, availability, training, workflow and efficiency (Table 4). Only eleven (11.0%) comments were positive, applauding improved accuracy, mobile phone integration and increased time for patient care.

of

Most comments (89.0%) were negative, referencing usability and technical issues (37.1%), inadequate availability (20.2%) and dictation errors (18.0%). While most users still preferred

ro

dictation to typing, many desired improved SR accuracy, which they felt would increase

-p

usability. Some felt that front-end SR did not reduce documentation burden, as it requires clinicians to edit documents themselves, unlike workflows involving editing by professional

re

transcriptionists. Concerns about system availability were common, with users reporting limited

Jo

ur na

lP

capability to dictate in multiple clinical locations or at home.

Positive Comments

(n = 100)

(n = 11)

n (%)

Examples

Usability/

35 (35.0)

“SR worked well with cellphone application.”

Errors/

21 (21.0)

re

Technical

“Pleased with the improvements is SR technology

(n = 89)

n (%)

Examples

n (%)

2 (18.2)

“Issues with SR hardware decrease usability of SR

33 (37.1)

-p

Theme/Topic

Negative Comments

ro

Total

of

Table 4: Theme analysis of SR users’ comments with examples.

5 (45.5)

systems”; “SR software crashes, freezes, and loads slowly which decrease reliability of SR systems.” “Definitely faster than my typing but the back editing is

Accuracy/

over time”; “First used Dragon naturally speaking

Quality

about 15 years ago but stopped due to an

Example - substituting the word ‘Asian' when I said

unacceptable rate of dictation errors. The

'patient'”; “Particularly discouraging to receive charts

improvements in the system over that time are

with syntax errors from dragon that are over one to two

vast.”

years old with demands that these 'errors' be fixed

Education/ Training

a constant hassle or when I review notes in the future:

lP

ur na

18 (18.0)

Jo

Availability

15 (15.0)

N/A

16 (18.0)

sometime as simple as a 'he' for 'she'.” 0 (0.0)

“Desire to be able to dictate in different rooms,

18 (20.2)

computers (including personal), and at home”; “Wish to use SR systems outside of the EHR.”

“I have not had much trouble with the new dragon

1 (9.1)

“I do not have the time to do training. It is a very low

system and I feel my training was adequate.”; “I

priority for me right now”; “Follow-up training would

am happy with the IT response as well and have

be helpful for unlocking the full potential of SR.”

14 (15.7)

Workflow

8 (8.0)

“Would increase Dragon usage if scribe wasn’t

1 (9.1)

present.”

ro

timely and satisfactory manner.”

of

always been helped with any dragon issues in a

“When we had human transcriptionists, they would

7 (7.9)

make sense out of what we said and make syntax fit.

Efficiency

3 (3.0)

re

-p

Those people lost their jobs and I get an error prone

“Dragon has transformed my documentation

lP

experience. I am a slow typist and did not find

mediocre accuracy.” “Dragon is very useful but recent 'updates' have caused it to be slower than previously or not work”; “System

available shortcuts in EPIC offered efficiency”;

does not relieve increased time to review documentation

“Allowed me to spend more time with patients,

and correct system errors made by dragon.”

ur na

and more time thinking about the patient's issues (over the task of documentation).”

Jo

2 (18.2)

system that takes constant vigilance to get even

1 (1.1)

4. DISCUSSION We evaluated a wide array of registered users of SR dictation systems at two large medical centers and found that they believed it improved their clinical workflow and efficiency. Additionally, satisfaction was strongly correlated with efficiency, error prevalence and editing time. In general, SR has been rapidly adopted by hospitals and clinicians in recent years but has received relatively little evaluation. In this study, almost half of users had used SR for clinical

of

documentation for less than 6 months and over 85% for less than 2 years.

ro

Compared to previous studies,[18-24] our survey population was more diverse (e.g., in age, provider role and clinical disciplines/settings), larger, and multisite, all of which increase the

-p

generalizability of our findings. In addition, many prior studies were conducted before the

re

explosion of EHR adoption and have focused primarily on users’ expectations and experiences before and after SR adoption rather than its perceived impact on efficiency, workflow, error

lP

prevalence, time needed for editing, and satisfaction. Although most (78.8%) respondents were satisfied with the SR system they were using, users raised concerns regarding SR usability,

ur na

availability and accuracy, indicating many gaps and challenges still exist. Documentation burden and limitations with EHR usability have become a key driver of physician burnout, posing a threat to the American healthcare system.[7, 8] This burden is in part reflected in our findings. For example, 13.1% of respondents dictated at home and over half of

Jo

participants wanted to dictate on their personal computers, suggesting that clinicians increasingly bring work home. Compared to other documentation methods (e.g., scribes, templates, typing and traditional dictation), SR has emerged as an effective solution that may overcome some of the limitations inherent in other options, potentially reducing documentation burden and costs while preserving documentation quality.[11, 12]

In general, SR technology--despite rapid advances over the past three decades--has raised concerns regarding accuracy, efficiency and ability to reduce costs and improve clinical workflow.[5, 11, 20] While increased productivity is touted as a clear value of SR technology, concern about errors and their potential for patient harm is justified, with recent simulation data finding decreased efficiency and increased errors with SR compared to keyboard and mouse.[22] In this study, most users estimated seeing fewer than 10 errors per document, with less than 25%

of

of those being clinically significant. These estimates are lower than the error rates reported in

ro

many formal evaluations of SR accuracy.[28-30] In previous studies, SR error rates (i.e., number of errors divided by the total number of words in the document) have been found to be as low as

-p

0.3% with traditional dictation (i.e., with editing/revision by professional transcriptionists) but as

re

high as 23-52% when clinicians use front-end SR systems (i.e. with editing/revision by clinicians).[23, 24] Our previous study found that 7 in 100 words in SR-generated documents

lP

contain errors, but the error rate dropped down to 0.4% after transcriptionist review, demonstrating the importance of manual editing and review.[31] In this study, we found that

ur na

increased errors per document and increased rates of clinically relevant errors were significantly associated with greater editing time, which may impact efficiency. In surgical notes, we found clinicians spend more time editing, but the reason behind this difference remains unclear. Is it that surgical documentation is more consequential or that the SR system is less tuned for surgical

Jo

notes, requiring them spend more time editing? Or is it that in some specialties (e.g., surgery), there is a lower tolerance for errors, resulting in increased editing time? For other specialties, this is unsurprising, as users may not sufficiently review their notes and, as such, underestimate error prevalence. Future research will need to further evaluate the association of documentation errors with editing time, considering both note type (e.g., operative note, progress note) and length of

dictation (e.g., discharge summary vs. clinic note), as well as the impact of errors on communication between clinicians and their patients. With SR systems typically configured to the user’s specialty (e.g., surgery, emergency medicine), additional research would be valuable to understand if SR systems can be further trained to the granularity of the note type (e.g., procedural note vs. discharge summary) and its impact on accuracy. Another factor to consider is the recent movement toward making notes accessible to patients (e.g., OpenNotes[32]), as this

of

may influence clinicians’ editing time, knowing that their patients will be viewing their

ro

dictations and focusing on their notes’ general comprehensibility through the avoidance of obscure abbreviations.

-p

Furthermore, early adopters have provided cautionary tales demonstrating that dictation

re

environment, hardware integration and training/support must be carefully considered to avoid detrimental effects on efficiency and workflow.[33] Our analysis of users’ comments revealed

lP

shared sentiments among clinicians with respect to system availability (e.g., through Citrix®), technical issues (e.g. profile corruption) and usability (e.g., poor EHR integration). Users were

ur na

interested in learning best practices for key types, workarounds or helpful phrases. Those who had used traditional dictation highlighted its benefits with respect to accuracy, clarity and reduced need for additional clinician editing. As SR technology continues to mature, many of these challenges have been mitigated. Newer

Jo

SR solutions are now cloud-based, letting users dictate from any workstation, mobile applications, or even from home, easing clinicians’ availability concerns. SR accuracy is also improving, with less training needed for new users. For example, with newer cloud-based systems, users can begin dictating immediately with minimal loss in accuracy, rather than needing to repeat long, scripted dictations to train the system to their voice. Cloud-based systems

now eliminate the risk of users’ profiles becoming corrupted, a common problem that plagued older SR systems. Institutional training requirements are also being relaxed; previously, new SR users often needed to attend a 1-2 hour training session, but with newer, more user-friendly SR systems, some institutions have adopted shorter online training modules with no loss in user competency. Lastly, the shift to cloud-based SR has allowed the use of more advanced algorithms and acoustic models by virtue of increased computational power and the ability to add

of

additional servers on demand, rather than being limited to the power of the clinician’s

ro

workstation. For example, newer SR systems typically employ neural network-based language and/or acoustic models, which was not possible with older, locally-installed SR software. These

-p

advancements have resulted in significant increases in both the accuracy and reliability of SR

re

systems.

Future applications of SR are rapidly expanding. Many SR systems can now be launched

lP

from directly within the EHR or integrated into the exam room or a mobile device and used to capture the clinical encounter, essentially acting as a virtual scribe to automatically produce the

ur na

clinician’s note from the voice recording. Natural language processing is now being integrated with these systems to provide real-time clinical decision support, helping improve documentation and medical decision making and potentially reducing the need to click numerous checkboxes to satisfy billing requirements. Despite these advances, post-processing tools integrated with

Jo

existing SR technologies to identify and correct dictation errors and improve the accuracy and quality of dictated medical documents are needed. Until such systems are in place, clinicians’ careful proof-reading of their dictations remains vital.

4.1 Limitations The study population was limited to clinician users of SR systems at two large and integrated health systems. Both centers used the same SR vendor product, but results may vary with other products or vendors. Additionally, participants were mostly white and English-speaking, making our findings less generalizable to other sites. Future studies will be needed to examine the influence of other factors (e.g., accent) on SR usability and accuracy. We evaluated perceived

ro

low, which may be because many users were new and still in training.

of

impact on efficiency, and not actual efficiency, which may differ. Lastly, our response rate was

5. CONCLUSION

-p

In recent years, SR has been rapidly adopted by hospitals and clinicians for clinical

re

documentation. While concerns regarding usability and accuracy persist, clinicians believed it improved their clinical workflow and efficiency. Faster and better approaches are needed for

lP

clinical documentation, and SR is likely to play an important role going forward.

ur na

Author Contributions: Dr. Goss and Dr. Zhou had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Concept and design: Goss, Zhou, Blackley, Kowalski, Meteer, Bates. Acquisition, analysis, or interpretation of data: Goss, Zhou, Blackley, Kowalski, Landman,

Jo

Meteer, Gradwohl, Ortega.

Drafting of the manuscript: Goss, Zhou, Blackley, Gradwohl, Ortega. Critical revision of the manuscript for important intellectual content: Goss, Zhou, Blackley, Ortega, Kowalski, Landman, Meteer, Bates, Gradwohl, Bakes, Lin.

Statistical analysis: Goss, Zhou, Blackley, Gradwohl. Obtained funding: Goss, Zhou, Bates. Administrative, technical, or material support: Goss, Zhou, Kowalski, Ortega, Landman. Supervision: Goss, Zhou, Meteer, Bates.

of

Conflict of Interest Disclosures: Dr. Goss reported grants from the Agency for Healthcare

ro

Research and Quality during the conduct of the study. Dr. Goss provides consulting for

RxREVU, which develops web-based decision support for prescribing of medications and he

-p

receives cash compensation. Dr Zhou reported grants from the Agency for Healthcare Research

re

and Quality during the conduct of the study. Dr. Meteer reported grants from Agency for Healthcare Research and Quality during the conduct of the study. Dr. Bates reported grants from

lP

National Library of Medicine during the conduct of the study; in addition, Dr. Bates had a patent number 6029138 issued, licensed, and with royalties paid; and Dr. Bates is a coinventor on

ur na

Patent No. 6029138 held by Brigham and Women’s Hospital on the use of decision support software for medical management, licensed to the Medicalis Corporation. He holds a minority equity position in the privately held company Medicalis, which develops web-based decision support for radiology test ordering. He serves on the board for S.E.A. Medical Systems, which

Jo

makes intravenous pump technology. He consults for EarlySense, which makes patient safety monitoring systems. He receives cash compensation from CDI-Negev, which is a nonprofit incubator for health IT startups. He receives equity from Valera Health, which makes software to help patients with chronic diseases. He receives equity from Intensix, which makes software to support clinical decision making in intensive care. He receives equity from MDClone, which

takes clinical data and produces deidentified versions of it. Dr. Bates’s financial interests have been reviewed by Brigham and Women’s Hospital and Partners HealthCare in accordance with their institutional policies. No other disclosures were reported.

Acknowledgments

of

This study was funded through a grant by the Agency for Healthcare Research and Quality (grant

ro

R01HSO24264). SUMMARY TABLE

Despite rapid advances and adoption of SR technology, scant research exists on clinician

-p





re

use of, experience with and opinions about this technology.

Among respondents surveyed, most were satisfied with SR and agreed that using SR

lP

saved time and increased efficiency. Errors were perceived to be low, but 20% estimated half or more of all errors to be clinically significant. Satisfaction was positively associated with efficiency and negatively associated with

ur na



error prevalence and editing time. 

Compared to other documentation methods (e.g., scribes, templates, typing, traditional dictation), SR has emerged as an effective solution, overcoming limitations inherent in

Jo

other options and potentially improving efficiency while preserving documentation quality.

References

[1] D.J. Quint, Voice recognition: ready for prime time?, J Am Coll Radiol, 4 (2007) 667-669; discussion 670-661.

[2] A. Mehta, K.J. Dreyer, A. Schweitzer, J. Couris, D. Rosenthal, Voice recognition--an emerging necessity within radiology: experiences of the Massachusetts General Hospital, J Digit Imaging, 11 (1998) 20-23. [3] J. Weber, Tomorrow's transcription tools: what new technology means for healthcare, J AHIMA, 74 (2003) 39-43;45-36.

automatic speech recognition, J Rehabil Res Dev, 41 (2004) 739-754.

of

[4] H.H. Koester, Usage, performance, and satisfaction outcomes for experienced users of

ro

[5] Y.D. Derman, T. Arenovich, J. Strauss, Speech recognition software and electronic

psychiatric progress notes: physicians' ratings and preferences, BMC Med Inform Decis Mak, 10

-p

(2010) 44.

re

[6] S.E. Pollard, P.M. Neri, A.R. Wilcox, L.A. Volk, D.H. Williams, G.D. Schiff, H.Z. Ramelson, D.W. Bates, How physicians document outpatient visit notes in an electronic health

lP

record, Int J Med Inform, 82 (2013) 39-46.

[7] A.A. Wright, I.T. Katz, Beyond Burnout - Redesigning Care to Restore Meaning and Sanity

ur na

for Physicians, N Engl J Med, 378 (2018) 309-311.

[8] T.D. Shanafelt, O. Hasan, L.N. Dyrbye, C. Sinsky, D. Satele, J. Sloan, C.P. West, Changes in burnout and satisfaction with work-life balance in physicians and the general US working population between 2011 and 2014, Mayo Clinic Proceedings, 90 (2015) 1600-1613.

Jo

[9] C. Sinsky, L. Colligan, L. Li, M. Prgomet, S. Reynolds, L. Goeders, J. Westbrook, M. Tutty, G. Blike, Allocation of Physician Time in Ambulatory Practice: A Time and Motion Study in 4 Specialties, Ann Intern Med, 165 (2016) 753-760.

[10] J.S. Shapiro, S. Bakken, S. Hyun, G.B. Melton, C. Schlegel, S.B. Johnson, Document ontology: supporting narrative documents in electronic health records, AMIA Annu Symp Proc, (2005) 684-688. [11] M. Singh, T.R. Pal, Voice recognition technology implementation in surgical pathology: advantages and limitations, Arch Pathol Lab Med, 135 (2011) 1476-1481. [12] A. Young, H.J. Chaudhry, J. Rhyne, M. Dugan, A census of actively licensed physicians in

of

the United States, 2010, J of Med Reg, 96 (2011) 10-20.

ro

[13] E.C. Callaway, C.F. Sweet, E. Siegel, J.M. Reiser, D.P. Beall, Speech recognition interface to a hospital information system using a self-designed visual basic program: initial experience, J

-p

Digit Imaging, 15 (2002) 43-53.

re

[14] S.G. Langer, Impact of speech recognition on radiologist productivity, J Digit Imaging, 15 (2002) 203-209.

lP

[15] D.N. Mohr, D.W. Turner, G.R. Pond, J.S. Kamath, C.B. De Vos, P.C. Carpenter, Speech recognition as a transcription aid: a randomized comparison with standard transcription, J Am

ur na

Med Inform Assoc, 10 (2003) 85-93.

[16] J.Y. Irwin, S. Fernando, T. Schleyer, H. Spallek, Speech recognition in dental software systems: features and functionality, Medinfo 2007: Proceedings of the 12th World Congress on Health (Medical) Informatics; Building Sustainable Health Systems, (2007) 1127.

Jo

[17] S.V. Blackley, J. Huynh, L. Wang, Z. Korach, L. Zhou, Speech recognition for clinical documentation from 1990 to 2018: a systematic review, J Am Med Inform Assoc, 26 (2019) 324-338.

[18] T. Keskinen, A. Melto, J. Hakulinen, M. Turunen, S. Saarinen, T. Pallos, P. Kallioniemi, R. Danielsson-Ojala, S. Salanterä, Mobile dictation for healthcare professionals, Proceedings of the 12th International Conference on Mobile and Ubiquitous Multimedia, (2013) 41. [19] H. Suominen, M. Johnson, L. Zhou, P. Sanchez, R. Sirel, J. Basilakis, L. Hanlen, D. Estival, L. Dawson, B. Kelly, Capturing patient information at nursing shift changes: methodological evaluation of speech recognition and information extraction, J Am Med Inform Assoc, 22 (2014)

of

e48-e66.

ro

[20] C. Ahlgrim, O. Maenner, M.W. Baumstark, Introduction of digital speech recognition in a specialised outpatient department: a case study, BMC Med Inform Decis Mak, 16 (2016) 132.

-p

[21] Y.D. Derman, T. Arenovich, J. Strauss, Speech recognition software and electronic

re

psychiatric progress notes: physicians' ratings and preferences, BMC Med Inform Decis Mak, 10 (2010) 44.

lP

[22] T. Hodgson, F. Magrabi, E. Coiera, Evaluating the usability of speech recognition to create clinical documentation using a commercial electronic health record, Int J Med Inform, 113

ur na

(2018) 38-42.

[23] J.P. Lyons, S.A. Sanders, D. Fredrick Cesene, C. Palmer, V.L. Mihalik, T. Weigel, Speech recognition acceptance by physicians: A temporal replication of a survey of expectations and experiences, Health Inform J, 22 (2016) 768-778.

Jo

[24] A. Alapetite, H.B. Andersen, M. Hertzum, Acceptance of speech recognition by physicians: A survey of expectations, experiences, and social influence, Int J Hum Compt Stud, 67 (2009) 36-49.

[25] P.A. Harris, R. Taylor, R. Thielke, J. Payne, N. Gonzalez, J.G. Conde, Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support, J Biomed Inform, 42 (2009) 377-381. [26] SAS Institute Inc, SAS System for [Unix]. Copyright © 2018 Cary, NC, USA, [27] RStudio Team. RStudio: Integrated Development for R. RStudio, Inc., Boston, MA. 2015; http://www.rstudio.com/, 2015).

ro

physician charting in the ED, Am J Emerg Med, 19 (2001) 295-298.

of

[28] R.G. Zick, J. Olsen, Voice recognition software versus a traditional transcription service for

[29] E.G. Devine, S.A. Gaehde, A.C. Curtis, Comparative evaluation of three continuous speech

-p

recognition software packages in the generation of medical reports, J Am Med Inform Assoc, 7

re

(2000) 462-468.

[30] K.M. Kanal, N.J. Hangiandreou, A.M. Sykes, H.E. Eklund, P.A. Araoz, J.A. Leon, B.J.

Imaging, 14 (2001) 30-37.

lP

Erickson, Initial evaluation of a continuous speech recognition program for radiology, J Digit

ur na

[31] L. Zhou, S.V. Blackley, L. Kowalski, R. Doan, W.W. Acker, A.B. Landman, E. Kontrient, D. Mack, M. Meteer, D.W. Bates, F.R. Goss, Analysis of Errors in Dictated Clinical Documents Assisted by Speech Recognition Software and Professional Transcriptionists, JAMA Netw Open, 1 (2018).

Jo

[32] Open Notes. https://www.opennotes.org/ (Accessed April 1, 2019). [33] J.D. Houston, F.W. Rupp, Experience with implementation of a radiology speech recognition system, J Digit Imaging, 13 (2000) 124-128.