- Email: [email protected]
Personal Digital Assistant-Based Self-Work Sampling Study of Pediatric Interns Quantifies Workday and Educational Value
Accepted Manuscript PDA-based Self-Work Sampling Study of Pediatric Interns Quantifies Workday and Educational Value Joyce K. Campbell, BSN, MS, Michael V. Ortiz, MD, Mary C. Ottolini, MD MPH, Sarah Birch, DNP, APRN, CPNP-PC, AE-C, Dewesh Agrawal, MD PII:
S1876-2859(16)30531-9
DOI:
10.1016/j.acap.2016.12.001
Reference:
ACAP 948
To appear in:
Academic Pediatrics
Received Date: 6 April 2016 Revised Date:
23 November 2016
Accepted Date: 3 December 2016
Please cite this article as: Campbell JK, Ortiz MV, Ottolini MC, Birch S, Agrawal D, PDA-based SelfWork Sampling Study of Pediatric Interns Quantifies Workday and Educational Value, Academic Pediatrics (2017), doi: 10.1016/j.acap.2016.12.001. This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
ACCEPTED MANUSCRIPT
PDA-based Self-Work Sampling Study of Pediatric Interns Quantifies Workday and
RI PT
Educational Value Joyce K. Campbell, BSN, MS, Michael V. Ortiz, MDa, Mary C. Ottolini MD MPH, Sarah Birch
SC
DNP, APRN, CPNP-PC, AE-C, Dewesh Agrawal, MD
Affiliation: Children's National Medical Center, 111 Michigan Ave. NW, Washington D.C.
M AN U
20010, USA
Corresponding author: Joyce K. Campbell, BSN, MS, Medical Education, Children’s National Medical Center, WW 3.5 Suite 600, 111 Michigan Ave. N.W., Washington, D.C. 20010, [email protected], Telephone: 202-476-4564
TE D
Key Words: Education, intern, PDA, self-work sampling Running Title: PDA-based Self-Work Sampling Study of Pediatric Interns
EP
Funding Source: PDAs were purchased using funding provided by the Elda Arce Teaching
AC C
Excellence Award received by Mary Ottolini. Financial Disclosure: The authors have no financial relationships relevant to the article to disclose.
Conflict of Interest: The authors have no conflicts of interest to disclose. Word Count: Abstract 235; Manuscript 2,879
1
ACCEPTED MANUSCRIPT
What’s New: This is the first study to use real-time self-work sampling to evaluate pediatric interns’ perceived educational value of daily work activities. The distribution of daily activities
EP
TE D
M AN U
SC
Current Affiliation: Memorial Sloan Kettering Cancer Center
AC C
a
RI PT
is discordant with the perceived educational value of those activities.
2
ACCEPTED MANUSCRIPT
ABSTRACT: Objective: Optimizing clinical proficiency and education of residents has become more
RI PT
important with restricted residency duty hours. Our objective was to investigate how interns spend their time on inpatient rotations and the perceived educational value of workday activities. Methods: A descriptive, self-work sampling study using a personal digital assistant (PDA) to
educational value of that activity on a 4-point Likert scale.
SC
randomly query interns on inpatient rotations in real time as to their activity and the perceived
M AN U
Results: A total of 31 interns participated on 88 workdays over a 5 month period generating 2082 samples from which the average workday was modeled. Time spent using the electronic health record (EHR) accounted for 33% of intern time, communicating with the healthcare team 23%, educational activities 17%, and time with patients and families 12%. Time with patients
TE D
and families was perceived to be the most educational part of clinical service. Time spent using the EHR was perceived as the least educational. Interns perceived clinical service as excellent or
81% of the time.
EP
good 37% of the time, while planned educational activities were perceived as excellent or good
AC C
Conclusions: Interns spend the majority of their time using the EHR and communicating with the healthcare team. Interns perceive time spent in planned educational activities has more educational value than time spent in clinical service. The distribution of daily activities is discordant with the perceived educational value of those activities.
3
ACCEPTED MANUSCRIPT
INTRODUCTION: A core challenge in graduate medical education (GME) is finding the balance between providing excellent education and meeting service requirements while delivering safe, high quality patient
RI PT
care. The work compression resulting from the 2011 ACGME duty hour regulations restricting interns to 16-hour workdays has been associated with concerns about both the quality of training and patient care. Residents feel less satisfied with their education and have a sense of
SC
diminished clinical preparedness.1,2 In a national survey of all ACGME-accredited residency programs in the United States, a majority of residents believed preparation for more senior roles
M AN U
and responsibilities has worsened, and about 40% reported a perception of lower quality education following implementation of duty hour restrictions. Second year residents believed a chief goal of training – preparedness for transitioning from intern to senior resident – was being delayed.3 The optimization of clinical proficiency and the quality of education are very
TE D
important in the current educational environment.
In order to accurately assess the current balance between service and education, we felt it was prudent to quantify both the time spent and the perceived educational value of different activities
EP
in the typical intern’s workday. We utilized self-work sampling methodology because it is an
AC C
established and effective job analysis approach for estimating the percentage of time workers are engaged in predefined work activities. This methodology is based on probability sampling using multi-level categories to examine work activities in a detailed manner.4 Self-work sampling has the advantage of recording all activities in real time, even those activities which might be perceived as minor or insignificant.4
4
ACCEPTED MANUSCRIPT
The primary aim of this study was to quantify how interns spend their time on a typical inpatient workday. The secondary study aim was to assess intern perceptions of the educational value of the workday activities.
RI PT
METHODS: Setting and Participants
SC
The study was conducted at a free-standing, 303-bed, university-affiliated children’s hospital in Washington, DC. This institution has an ACGME-approved pediatric residency program of 118
M AN U
pediatric residents. Pediatric interns (n=40) were eligible to participate if they were rotating on the five inpatient intern services between February and June 2012. Two services were subspecialty-based (gastroenterology and neurology), and the other three services were general pediatric hospitalist services. Intern responsibilities and time allocations are distinct on
TE D
weekdays, weekends, and nights. In order to limit variability associated with these different time periods, this study focused on weekdays, where there is the largest focus on structured learning. As a result, weekend shifts and interns on the night float team were excluded. The daytime team The
EP
is separate from the nighttime team (there is no "long call" system or post-call interns).
intern self-selected three days during the week to be surveyed. Continuity clinic assignments
AC C
occurred one-half day each week and were included to allow the intern to collect data on their clinic day and model the typical weekday work day (Table 1). Initially, a general information and training session was held for all interns. Coding information and the meaning of the query labels were reviewed in the interns’ training session for the research study and again in the individual orientation where the intern was able to practice using the PDA (Figure 1). We met with a group of interns after the pilot phase to validate their understanding of coding.
5
ACCEPTED MANUSCRIPT
Participation was voluntary and identifying information was not collected. Interns choosing not to participate could either not pick-up the PDA or simply not collect data. Attending physicians, fellows, and senior residents were informed that interns would be participating in a research
RI PT
study. They were not informed of time or identity of specific participants. At the start of their participation week, 2-3 interns picked up a PDA, received an individual orientation to the study and a detailed manual of operations to maximize inter-rater reliability of self-sampling data. A
SC
support number was also posted in resident team rooms. Interns were informed that participation in this study was voluntary. Responses were not linked to the responder identity. The IRB
M AN U
exempted this study from the need for consent. Activity Definitions
In order to develop a user friendly PDA that allowed the interns to accurately describe what they were doing, the different possible resident tasks were collated into sequential activities on 3
TE D
levels (Figure 1). The activities were arrived at by consensus utilizing the mini-Delphi process,5 an iterative process involving multiple meetings of 4-5 residents and graduate medical education leaders. This process arrived at three levels of activities. All levels were designed to be
EP
exhaustive, mutually exclusive, clear and adaptable to the PDA sampling methodology. For
AC C
example, Level 1 activities were pre-rounding, rounding, post-rounding, continuity clinic, signout, planned education, and personal time. Levels 2 and 3 further elucidated these activities by asking questions such as “what”, “why”, “how”, or “with whom” in relation to the prior activity. The final screen queried the interns to rate the perceived educational value of the work activity on a 4-point Likert scale (none, fair, good, or excellent). The 4-point Likert scale was chosen to sufficiently describe educational value as it avoids a central neutral response.6 After the intern chose the activity on one screen, the PDA automatically forwarded to the next level’s screen with 6
ACCEPTED MANUSCRIPT
the corresponding activity. To assure construct validity and to test both the ease of participation and the completeness of responses to the multi-level task categories that modeled the typical workday, a pilot study was conducted during development. During this pilot the interns reported
to do, and it did not interfere with their workday activities.
SC
Data Collection
RI PT
it typically took 10-20 seconds in total to complete the 4-levels for each sample entry, it was easy
The interns entered their team assignment and the size of their panel of patients at the beginning
M AN U
and end of a workday. All data used in the study started when the intern arrived at the hospital and ended when the intern left the hospital at the end of the work day. The alarm vibrates randomly with a range between 10 minutes and 54 minutes and an average time interval of 26 minutes. The alarm interval was consistent for all interns. This sampling frequency was
TE D
chosen based on similar work sampling studies with other healthcare professionals.7 This alarm frequency is not too often to interfere with the interns’ work and not too infrequent as to require an extended study period for adequate data collection. When it was time to query the intern to
EP
collect a sample, the PDA vibrated, displaying a pick-list of level 1 activities. If the intern did not respond within 20 seconds, the PDA continued to vibrate every 20 seconds until the intern
AC C
either responded to the query or until the next random query occurred. At the end of their workday, interns manually terminated study collection on the PDA. Data from each PDA were downloaded in an automated fashion onto the UMTPlus software by docking the PDA to specified computers in the physician team rooms. Data Analysis
7
ACCEPTED MANUSCRIPT
A sample size of at least 2,000 queries was chosen to achieve an absolute error of 2% with a 95% confidence interval. Data were downloaded from UMTPlus into Excel and descriptive statistics were used to identify the distribution of work shift activities and the perceived educational
RI PT
ratings for each activity. Personal time was included for descriptions of the entire workday but excluded for analyses of clinical activities and perceived educational value. Some categories were combined during analysis. For example, to determine the time spent using the electronic
SC
health record (EHR), level 2 and 3 activities of documenting, reading the EHR, writing orders and preparing sign-out were combined. To determine the total time spent with patients and
M AN U
families, level 2 and 3 activities of bedside work, procedures and communication with patient and/or family were combined. Data were aggregated from all participants and used to generate descriptive statistics for both time of activities and perceived educational value.4 Data are expressed as mean with standard deviation (SD) as appropriate. Perceived educational value for
TE D
various workday activities was calculated utilizing the 4-point Likert scale, with values assigned
RESULTS Daily Activities
EP
as none=1, fair=2, good=3, and excellent=4.
AC C
A total 78% (31/40) of the intern class participated for 88 workdays, generating a total of 2082 work-sampling queries from which to model the average workday. Data entry for these workdays commenced between 4:50am and 8:30am and ended between 4:59pm and 7:18pm. Five interns missed one day each of data collection during their assigned week. The mean intern workday was 11.0 (SD 1.1) hours with 24.7 (2.8) work queries per day. This is the expected number of queries given a random query on average every 26 minute.
8
ACCEPTED MANUSCRIPT
The distribution of level 1 activities for the average intern workday is shown in Table 2. Eighty percent of the workday involved clinical service activities (pre-rounds, rounds, post-rounds,
RI PT
continuity clinic and sign-out). The major level 2 and 3 activities included time spent with the EHR (33%), professional
communication (23%), planned and unplanned education (17%) and direct contact with the
SC
patient and family (12%). Only 3% of time was spent in administrative tasks, and a majority of these were perceived by the interns as appropriate activities for a resident. Interns spent 5% of
M AN U
their time in non-productive activities (Table 3). Perceived Educational Value of Activities
The perceived educational value was rated in 99% (1914 / 1928) of the activity queries excluding personal time. Overall, 15% of level 1 activities were perceived as educationally excellent,
TE D
while 29% were perceived as good, 31% were perceived as fair, and 26% were perceived as having no educational value. The activity with the highest perceived educational activity was planned education while the activity with the lowest perceived educational activity was sign-out
EP
(Figure 2). The level 2 and 3 activities having the highest perceived educational values were planned and unplanned education and time spent with the patient and family, while the activities
AC C
with the lowest perceived educational value were activities involving the EHR, administrative tasks and nonproductive time (Figure 3). Activities involving the EHR were perceived as having no or fair educational value 78% of the time, and interns spent 3.4 hours/day in the EHR. Of activities involving the EHR, the discharge note was perceived as having the least educational value. Interns spent 41 minutes/day writing discharge summaries which at our institution
9
ACCEPTED MANUSCRIPT
includes a review of the hospital course, patient education, discharge instructions and the medication reconciliation.
RI PT
All planned educational activities had perceived values of excellent or good a majority (81%) of the time. Simulation and inpatient attending teaching rounds had the highest perceived
educational value, while institutional grand rounds had the lowest (Figure 4). Average patient
SC
load per intern was consistent across educational ratings of level 1 activities. DISCUSSION:
M AN U
Self-work sampling methodology was effective in gathering data to assess the activities and educational values of interns’ workdays. This is the first study to use real-time self-work sampling to evaluate pediatric interns’ perceived educational value of their daily work activities. Interns perceive time spent in planned educational activities is more educationally valuable than
TE D
time spent in patient care related tasks and the EHR, yet their most common tasks involve communicating with the healthcare team and using the EHR. Activities perceived to be nonvaluable are more frequent than activities perceived to be valuable. Perceived valuable activities
EP
are less frequent than those perceived to be of little value. Utilizing PDAs is an effective way to conduct a self-work sampling study and avoids the need
AC C
for an observer. Self-work sampling has several advantages over observational studies. One-toone observation can influence the behavior of the participant, increase the cost of the research and provide more subjective, less pragmatic results. One-to-one observation may also limit the number of participants and the length of time the study can continue.4,8,9 10 With selfassessment, important perceptions and other cognitive processes can be obtained from the participant rather than an observer. 11
Studies in nursing and laboratory medicine demonstrate
the validity of the PDA in self-work sampling to provide reliable insight into the roles and 10
ACCEPTED MANUSCRIPT
responsibilities of health care providers. 7,12,13 The samples may include what, how, why, and with whom the activity is being done, as well as document perceptions and other cognitive
RI PT
processes. It is noteworthy that our findings are relatively similar to those of observational time-motion studies. Mamykina et al observed 7 general medicine residents each for one full workday and found they spend 50.6 % of their time on a computer, 40.6% of the day in communication, and
SC
9.4% of the day interacting with patients.14 Starmer et al conducted a multicenter study
M AN U
observing pediatric interns 24 hours a day and 7 days a week and found that they spent 31% of time in intraprofessional communication, 22% of time on the computer, and 14% of time with patients-families.15 Block showed internal medicine interns spend 40% of time at the computer and 12% of time in direct patient care.16 A self-work sampling study in 2011 showed internal medicine residents spending 43% of the day at workstations and 20% of time in patient rooms.17
TE D
Interns spend more time documenting than they spend with their patients, and they have reported being frustrated and dissatisfied with the demands of documentation as they appear to detract
EP
from their education.18,19
In our study, interns perceived greater educational value from planned small group sessions such
AC C
as simulation and case-based seminars such as attending teaching rounds that employ active learning techniques. This is consistent with adult learning theory 20 which postulates that learners value educational activities in which they are actively engaged and perceive less value from traditional, passive, lecture-based formats. The most highly rated educational sessions in our study were designed to address intern knowledge and skills gaps for commonly encountered clinical problems on the rotation. Trainees often chose topics and case discussions specifically
11
ACCEPTED MANUSCRIPT
tailored to address their concerns. In contrast, institutional grand rounds are more theoretical, lecture-based, passive, and not based on topics commonly encountered in practice.
RI PT
Interns perceive a higher educational value associated with planned educational activities than clinical activities involving patients. This perception is counter to Osler’s view of learning in the presence of patients: “He who studies medicine without books sails an uncharted sea, but he who
SC
studies medicine without patients does not go to sea at all.”21 Our study is consistent with a
concerning trend towards an uncoupling of education from direct patient care. Interestingly,
M AN U
residents have rated classroom-based instruction as superior to bedside teaching. Pediatric residents surveyed on a hematology-oncology inpatient service perceived classroom teaching as more effective because they were “concerned about patients’ comfort and misunderstanding… and becoming ashamed when they had wrong answers or comments in the presence of the
TE D
patients.”22
Several studies found that communication and physical exam skills can be best taught and assessed at the bedside, 23-27 but the over-emphasis on teaching and learning “standard” medical
EP
knowledge diminished the value of bedside learning. Gonzalo interviewed 34 experienced clinical teachers at 10 institutions to determine strategies to overcome bedside teaching barriers.
AC C
One of the barriers was “time compression” to finish rounds quickly so that trainees could attend scheduled didactic sessions that transmit knowledge to pass standardized board exams. We perceive similar barriers to patient-based learning at our institution. There are several limitations to this study. Despite a sample size of 2082 queries, the reliability of activities that occur at or below 20 minutes/day may not be accurately assessed. For example, time spent on administrative tasks and non-productive activity might have been better assessed
12
ACCEPTED MANUSCRIPT
with a larger data set. Second, our sample consisted only of interns working weekdays on inpatient rotations. The activity distribution and perceived educational values of these activities could change for higher level residents, those working nights and weekend call shifts, or those on
RI PT
outpatient rotations. Third, this study was conducted during the second half of the intern year, February through June to ensure expertise with the electronic medical records. The time of the year might alter the results. As is the case with many other children’s hospitals, the medical
SC
patient census at our institution is generally higher during the winter months. Our finding that average patient load was consistent across educational ratings suggests a lack of relationship
M AN U
between this measure of work load and perceived educational value of the typical inpatient intern day. However, a longer study encompassing the entire year, would be better suited to identify seasonal variations in roles and educational perceptions. Fourth, this study was done within a single institution and pediatric residency program, which impacts generalizability. Fifth,
TE D
measures other than average patient load (e.g. patient turnover which factors in admissions and discharges, both EHR-intensive activities) may be better predictors of intern work load and service needs. Sixth, utilizing a voluntary sample of interns may bias results. However, 78% of
EP
interns participated. Finally, there may not be a correlation between perceived educational value
AC C
and actual educational value. Assessing this was beyond the scope of this study. CONCLUSIONS:
In conclusion, utilizing a self-work sampling technique via a PDA we found that pediatric interns spend a large proportion of their day in the EHR and less time with patients, and that the distribution of daily activities of interns does not parallel the perceived educational value of these activities. These findings inform program improvement and faculty development in an effort to
13
ACCEPTED MANUSCRIPT
better align educational value with service requirements while maximizing educational experiences through effective patient care delivery.
RI PT
ACKNOWLEDGEMENTS: The authors wish to acknowledge the contributions of the following individuals: Nelson Lee (Rapid Modeling Corporation, Cincinnati, OH) and Jonathan Davies (former Pediatric Chief
SC
Resident, Children's National Medical Center, Washington, DC). We also wish to acknowledge the Academic Pediatric Association for presenting this project the Ray E. Helfer Award for
M AN U
Innovation in Medical Education, which was awarded at the Pediatric Academic Societies 2013
AC C
EP
TE D
annual meeting.
14
ACCEPTED MANUSCRIPT
REFERENCES: 1.
Desai SV, Feldman L, Brown L, et al. Effect of the 2011 vs 2003 duty hour regulation-
RI PT
compliant models on sleep duration, trainee education, and continuity of patient care among internal medicine house staff: a randomized trial. JAMA Intern Med 2013;173:649-55. 2.
Fitzgibbons SC, Chen J, Jagsi R, Weinstein D. Long-term follow-up on the educational
SC
impact of ACGME duty hour limits: a pre-post survey study. Annals of Surgery 2012;256:110812.
Drolet BC, Christopher DA, Fischer SA. Residents' response to duty-hour regulations--a
M AN U
3.
follow-up national survey. The New England Journal of Medicine 2012;366:e35. 4.
Robinson MA. Work sampling: Methodological advances and new applications. Human
Factors and Ergonomics in Manufacturing & Service Industries 2010;20:42-60. Hanafin S. Review of literature on the Delphi Technique. Dublin: National Children’s
Office 2004. 6.
TE D
5.
Brown S, Agree S, Disagree S, et al. Likert Scale Examples for Surveys. ANR Program
Evaluation Iowa State University Extension 2010. Hendrich A, Chow MP, Skierczynski BA, Lu Z. A 36-hospital time and motion study:
EP
7.
8.
AC C
how do medical-surgical nurses spend their time? The Permanente Journal 2008;12:25. Sittig DF. Work-sampling: a statistical approach to evaluation of the effect of computers
on work patterns in healthcare. Methods Inf Med 1993;32:167-74. 9.
Oddone E, Guarisco S, Simel D. Comparison of housestaff's estimates of their workday
activities with results of a random work-sampling study. Academic Medicine: Journal of the Association of American Medical Colleges 1993;68:859-61.
15
ACCEPTED MANUSCRIPT
10.
Pelletier D, Duffield C. Work sampling: valuable methodology to define nursing practice
patterns. Nursing & Health Sciences 2003;5:31-8. 11.
MacGregor DL, Tallett S, MacMillan S, Gerber R, O'Brodovich H. Clinical and
RI PT
education workload measurements using personal digital assistant-based software. Pediatrics 2006;118:e985-91. 12.
Watkins N, Kennedy M, Lee N, et al. Destination bedside: using research findings to
SC
visualize optimal unit layouts and health information technology in support of bedside care. J Nurs Adm 2012;42:256-65.
Trotter MJ, Larsen ET, Tait N, Wright JR, Jr. Time study of clinical and nonclinical
M AN U
13.
workload in pathology and laboratory medicine. Am J Clin Pathol 2009;131:759-67. 14.
Mamykina L, Vawdrey DK, Hripcsak G. How Do Residents Spend Their Shift Time? A
Time and Motion Study With a Particular Focus on the Use of Computers. Academic Medicine:
15.
TE D
Journal of the Association of American Medical Colleges 2016;91:827-32. Starmer AJ, Destino L, Yoon CS, Landrigan CP. Intern and Resident Workflow Patterns
on Pediatric Inpatient Units: A Multicenter Time-Motion Study. JAMA Pediatrics
16.
EP
2015;169:1175-7.
Block L, Habicht R, Wu AW, et al. In the wake of the 2003 and 2011 duty hours
AC C
regulations, how do internal medicine interns spend their time? Journal of General Internal Medicine 2013;28:1042-7. 17.
Siegler JE, Patel NN, Dine CJ. Prioritizing Paperwork Over Patient Care: Why Can't We
Do Both? Journal of Graduate Medical Education 2015;7:16-8.
16
ACCEPTED MANUSCRIPT
18.
Westbrook JI, Ampt A, Kearney L, Rob MI. All in a day's work: an observational study
to quantify how and with whom doctors on hospital wards spend their time. Med J Aust 2008;188:506-9. Christino MA, Matson AP, Fischer SA, Reinert SE, Digiovanni CW, Fadale PD.
RI PT
19.
Paperwork versus patient care: a nationwide survey of residents' perceptions of clinical documentation requirements and patient care. Journal of Graduate Medical Education
20.
SC
2013;5:600-4.
Knowles MS, Holton EF, Swanson RA. The adult learner : the definitive classic in adult
21.
Osler W. On the need of a radical reform in our teaching methods; senior students.
Medical News. 1903: 82: 49-53. 22.
M AN U
education and human resource development. 5th ed. Houston, Tex.: Gulf Pub. Co.; 1998.
Shahriari M. Case based teaching at the bed side versus in classroom for undergraduates
2014;2:135-6.
TE D
and residents of pediatrics. Journal of Advances in Medical Education & Professionalism
LaCombe MA. On bedside teaching. Annals of Internal Medicine 1997;126:217-20.
24.
Mangione S, Nieman LZ, Gracely E, Kaye D. The teaching and practice of cardiac
EP
23.
auscultation during internal medicine and cardiology training: a nationwide survey. Annals of
25.
AC C
Internal Medicine 1993;119:47-54.
Mattern WD, Weinholtz D, Friedman CP. The attending physician as teacher. New
England Journal of Medicine 1983;308:1129-32. 26.
Nair B, Coughlan J, Hensley M. Student and patient perspectives on bedside teaching.
Medical Education 1997;31:341-6.
17
ACCEPTED MANUSCRIPT
27.
Gonzalo JD, Heist BS, Duffy BL, et al. Identifying and overcoming the barriers to
AC C
EP
TE D
M AN U
SC
RI PT
bedside rounds: a multicenter qualitative study. Academic Medicine 2014;89:326-34.
18
ACCEPTED MANUSCRIPT
Table 1. An Intern's Typical Day
Grand Rounds
Attending Teaching Rounds
Work Rounds
Work Rounds
Work Rounds Patient Care
Friday Arrival, PreRounds
RI PT
Work Rounds
Arrival, Pre-Rounds
Thursday Arrival, PreRounds Morning Report Attending Teaching Rounds
SC
Work Rounds
Noon Conference
Patient Care (Continuity Clinic 1 day/week)
AC C
8:30 9:00 9:30 10:00 10:30 11:00 11:30 12:00 12:30 13:00 13:30 14:00 14:30 15:00 15:30 16:00 16:30 17:00 17:30 18:00
Attending Teaching Rounds
Arrival, PreRounds
Wednesday
M AN U
8:00
Tuesday Arrival, PreRounds Morning Report Attending Teaching Rounds
TE D
7:30
Monday
Sign Out, Finish work and leave hospital
EP
Time 6:00 6:30 7:00
ACCEPTED MANUSCRIPT
Table 2. Proportion of workday spent in Level 1 Activities (n=2082).
250
12
1.3
Rounds
340
16
1.8
Post-Rounds
779
37
4.1
Continuity Clinic
127
6
0.7
Sign-out
158
8
0.9
Planned Education 274
13
1.4
Personal Time
154
7
0.8
Totals
2082
100
11
AC C
EP
TE D
M AN U
Pre-Rounds
RI PT
Mean % of Workday Hours
N
SC
Activity
ACCEPTED MANUSCRIPT
Table 3. Breakdown of Level 1 activities into Level 2 and 3 activity categories (n=2080). Level 2 data was missing on 2 pre-rounds queries.
Contact with Patient or Family Unplanned Education Administrative Tasks
Planned Education Personal Time
77 130
Preparing for Sign-out Communicate with Healthcare Team Listen-Present on Rounds Give/Receive Sign-out At the Bedside Procedures
59
33%
3.4 (1.2)
192
206 89 167 20
Communicate with Patient/Family
63
Teach, learn, reflect, read
75
Appropriate for resident
39
Better done by support staff
27
Travel, wait, technology problem or supply problem
Mean Hours (SD)
RI PT
Read Medical Record Write Orders
AC C
Total Day
414
EP
Non-Productive Time
Document (admission, discharge, transfer, other)
% of Workday
SC
Professional Communication
N
TE D
EHR
Activity Breakdown
M AN U
Activity Categories
23%
2.4 (1.3)
12%
1.3 (0.7)
4%
0.8 (0.5)
3%
0.8 (0.5) 0.9 (0.6)
101
5%
267
13%
1.5 (0.7)
154
7%
1.1 (0.7)
2080
100
12.2
ACCEPTED MANUSCRIPT
Figure 1. Summary of activity definitions included in the manual of operations. Level 1Activities
RI PT
Pre-Rounds: The time between starting the day’s work and starting rounds. Pre-rounds include gathering patient information and physical exam. Pre-rounds can begin on arrival to the inpatient unit or when opening the EHR from alternate location. Rounds: Rounding starts when attending physician and/or senior resident begins patient care rounds with the team and ends when last patient has been seen or when leaving rounds to start another activity.
M AN U
SC
Post-Rounds: Usually starts after rounds, but can start during rounds if required to leave rounds early for patient care related duties. Includes bedside work, procedures, reading, writing orders and documenting in the healthcare record and communications with patients, families and the healthcare team. Ends when a continuity clinic assignment planned educational activity or signout begins. Continuity Clinic: Clinic starts upon leaving an activity for the outpatient assignment and ends when leaving the outpatient area. Travel time for clinic assignments should be included as nonproductive time.
TE D
Planned Education: All regularly planned educational sessions occurring in the inpatient unit, auditorium, simulation center or classroom. Sign-out: Sign-out begins when ready to start resident-to-resident handoff communication. Signout will include preparation in EHR and actual handoff. Waiting for sign-out should be included in sign-out activity as non-productive time.
EP
Level 2 and 3 Activities
Bedside Work: Done with team on rounds or independently; can include the history and physical.
AC C
Procedure: Other procedures include sutures, sedation, G-tube management, urinary catheterization, splinting, fracture reduction, LP and CPR resuscitation. Phlebotomy and IV insertions are documented separately. Read Patient Record: Gathering information from EHR. Write Order: Any order entry completed in EHR. Documentation: Any patient related documentation in EHR separated into categories (1) admission, (2) transfers, (3) discharge and (4) others which include progress notes, event notes and family meetings and clinic documentation.
ACCEPTED MANUSCRIPT
RI PT
Communication: Information exchange which includes talking, texting and emailing separated into exchange between (1) staff physicians includes the attending, consultant, fellow, resident, (2) primary care physicians, (3) medical students, (4) non-physician staff includes nurses, social workers, case managers, NP, PA, technicians, chaplains, dieticians, interpreters, OT, PT and units clerks, (5) patients and their families and (6) other. Present-Listen: Time spent presenting and listening during patient care rounds (i.e. not when you are completing other duties).
SC
Administrative Tasks: Defined as a duty the residents perceive to be administrative. Requested to document the staff member most appropriate to complete the task including the (1) case manager, (2) clerk, (3) social worker, (4) Resident Assistant, (5) nursing, or (6) the resident themselves.
M AN U
Unplanned Education: Knowledge transfer through either: (1) teaching, (2) learning, (3) reflecting or (4) reading that occurs during any activity outside planned educational didactics (planned education includes morning report, noon conference, attending rounds, grand rounds, professorial rounds and simulation center).
EP
TE D
Non-Productive Time: Time is divided into the following categories: (1) traveling, (2) supplies, (3) technology, (4) waiting, and (5) other. 1. Time spent traveling includes walking between patients’ rooms and/or units, driving to offsite locations and walking to other buildings (East Wing, Main Hospital, and West Wing). 2. Time related to supplies is time spent looking or waiting for necessary equipment or supplies. 3. Time related to technology is wasted time due to problems with web, hardware, or software malfunctions, computers, printers, fax or phones. 4. Time related to waiting is time when you cannot proceed until something happens. 5. Other non-productive time is used when one of the previous categories is not appropriate.
AC C
Sign-out Preparation: Time spent organizing, summarizing, documenting in EHR in preparation for handoff. Sign-out Handoff: Time spent communicating patient information to another health care provider. Personal time: Includes meals, rest periods, and personal activities such as phone calls and emails.
ACCEPTED MANUSCRIPT
Figure 2. Perceived Educational Value of Level 1 Activities. Calculated mean perceived educational values for workday activities are shown on top of each column. The overall mean value for Level 1 activities was 2.3. 100% 2.1
2.6
2.1
Pre-Rounds
Rounds
Post Rounds
2.0
2.2
3.2
RI PT
90% 80% 70% 60% 50%
SC
40% 30%
10% 0%
Good
AC C
EP
TE D
Excellent
M AN U
20%
Continuity Clinic
Fair
None
Sign-Out
Planned Education
ACCEPTED MANUSCRIPT
Figure 3. Perceived Educational Value of Level 2 & 3 Activities. Calculated mean perceived educational values for Level 2 and Level 3 activities are shown on top of each column. Personal time was excluded from the calculations for educational activities.
90%
1.9
2.8
2.4
3.0
3.2
1.5
1.0
RI PT
100% 80% 70% 60% 50% 40%
SC
30% 20% 10%
M AN U
0%
Good
AC C
EP
TE D
Excellent
Fair
None
ACCEPTED MANUSCRIPT
Figure 4. Perceived Educational Value (EV) of Planned Education Activities. Calculated mean perceived educational value for educational activities are shown on top of each column. The overall mean value for all educational activities was 3.2.
3.6
3.5
Simulation
Attending Rounds
3.4
3.1
3.2
2.6
RI PT
100% 90% 80% 70%
SC
60% 50%
30% 20% 10% 0%
M AN U
40%
Prof Rounds Morning Report good
AC C
EP
TE D
excellent
fair
none
Noon Conf
Grand Rounds
S1876-2859(16)30531-9
DOI:
10.1016/j.acap.2016.12.001
Reference:
ACAP 948
To appear in:
Academic Pediatrics
Received Date: 6 April 2016 Revised Date:
23 November 2016
Accepted Date: 3 December 2016
Please cite this article as: Campbell JK, Ortiz MV, Ottolini MC, Birch S, Agrawal D, PDA-based SelfWork Sampling Study of Pediatric Interns Quantifies Workday and Educational Value, Academic Pediatrics (2017), doi: 10.1016/j.acap.2016.12.001. This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
ACCEPTED MANUSCRIPT
PDA-based Self-Work Sampling Study of Pediatric Interns Quantifies Workday and
RI PT
Educational Value Joyce K. Campbell, BSN, MS, Michael V. Ortiz, MDa, Mary C. Ottolini MD MPH, Sarah Birch
SC
DNP, APRN, CPNP-PC, AE-C, Dewesh Agrawal, MD
Affiliation: Children's National Medical Center, 111 Michigan Ave. NW, Washington D.C.
M AN U
20010, USA
Corresponding author: Joyce K. Campbell, BSN, MS, Medical Education, Children’s National Medical Center, WW 3.5 Suite 600, 111 Michigan Ave. N.W., Washington, D.C. 20010, [email protected], Telephone: 202-476-4564
TE D
Key Words: Education, intern, PDA, self-work sampling Running Title: PDA-based Self-Work Sampling Study of Pediatric Interns
EP
Funding Source: PDAs were purchased using funding provided by the Elda Arce Teaching
AC C
Excellence Award received by Mary Ottolini. Financial Disclosure: The authors have no financial relationships relevant to the article to disclose.
Conflict of Interest: The authors have no conflicts of interest to disclose. Word Count: Abstract 235; Manuscript 2,879
1
ACCEPTED MANUSCRIPT
What’s New: This is the first study to use real-time self-work sampling to evaluate pediatric interns’ perceived educational value of daily work activities. The distribution of daily activities
EP
TE D
M AN U
SC
Current Affiliation: Memorial Sloan Kettering Cancer Center
AC C
a
RI PT
is discordant with the perceived educational value of those activities.
2
ACCEPTED MANUSCRIPT
ABSTRACT: Objective: Optimizing clinical proficiency and education of residents has become more
RI PT
important with restricted residency duty hours. Our objective was to investigate how interns spend their time on inpatient rotations and the perceived educational value of workday activities. Methods: A descriptive, self-work sampling study using a personal digital assistant (PDA) to
educational value of that activity on a 4-point Likert scale.
SC
randomly query interns on inpatient rotations in real time as to their activity and the perceived
M AN U
Results: A total of 31 interns participated on 88 workdays over a 5 month period generating 2082 samples from which the average workday was modeled. Time spent using the electronic health record (EHR) accounted for 33% of intern time, communicating with the healthcare team 23%, educational activities 17%, and time with patients and families 12%. Time with patients
TE D
and families was perceived to be the most educational part of clinical service. Time spent using the EHR was perceived as the least educational. Interns perceived clinical service as excellent or
81% of the time.
EP
good 37% of the time, while planned educational activities were perceived as excellent or good
AC C
Conclusions: Interns spend the majority of their time using the EHR and communicating with the healthcare team. Interns perceive time spent in planned educational activities has more educational value than time spent in clinical service. The distribution of daily activities is discordant with the perceived educational value of those activities.
3
ACCEPTED MANUSCRIPT
INTRODUCTION: A core challenge in graduate medical education (GME) is finding the balance between providing excellent education and meeting service requirements while delivering safe, high quality patient
RI PT
care. The work compression resulting from the 2011 ACGME duty hour regulations restricting interns to 16-hour workdays has been associated with concerns about both the quality of training and patient care. Residents feel less satisfied with their education and have a sense of
SC
diminished clinical preparedness.1,2 In a national survey of all ACGME-accredited residency programs in the United States, a majority of residents believed preparation for more senior roles
M AN U
and responsibilities has worsened, and about 40% reported a perception of lower quality education following implementation of duty hour restrictions. Second year residents believed a chief goal of training – preparedness for transitioning from intern to senior resident – was being delayed.3 The optimization of clinical proficiency and the quality of education are very
TE D
important in the current educational environment.
In order to accurately assess the current balance between service and education, we felt it was prudent to quantify both the time spent and the perceived educational value of different activities
EP
in the typical intern’s workday. We utilized self-work sampling methodology because it is an
AC C
established and effective job analysis approach for estimating the percentage of time workers are engaged in predefined work activities. This methodology is based on probability sampling using multi-level categories to examine work activities in a detailed manner.4 Self-work sampling has the advantage of recording all activities in real time, even those activities which might be perceived as minor or insignificant.4
4
ACCEPTED MANUSCRIPT
The primary aim of this study was to quantify how interns spend their time on a typical inpatient workday. The secondary study aim was to assess intern perceptions of the educational value of the workday activities.
RI PT
METHODS: Setting and Participants
SC
The study was conducted at a free-standing, 303-bed, university-affiliated children’s hospital in Washington, DC. This institution has an ACGME-approved pediatric residency program of 118
M AN U
pediatric residents. Pediatric interns (n=40) were eligible to participate if they were rotating on the five inpatient intern services between February and June 2012. Two services were subspecialty-based (gastroenterology and neurology), and the other three services were general pediatric hospitalist services. Intern responsibilities and time allocations are distinct on
TE D
weekdays, weekends, and nights. In order to limit variability associated with these different time periods, this study focused on weekdays, where there is the largest focus on structured learning. As a result, weekend shifts and interns on the night float team were excluded. The daytime team The
EP
is separate from the nighttime team (there is no "long call" system or post-call interns).
intern self-selected three days during the week to be surveyed. Continuity clinic assignments
AC C
occurred one-half day each week and were included to allow the intern to collect data on their clinic day and model the typical weekday work day (Table 1). Initially, a general information and training session was held for all interns. Coding information and the meaning of the query labels were reviewed in the interns’ training session for the research study and again in the individual orientation where the intern was able to practice using the PDA (Figure 1). We met with a group of interns after the pilot phase to validate their understanding of coding.
5
ACCEPTED MANUSCRIPT
Participation was voluntary and identifying information was not collected. Interns choosing not to participate could either not pick-up the PDA or simply not collect data. Attending physicians, fellows, and senior residents were informed that interns would be participating in a research
RI PT
study. They were not informed of time or identity of specific participants. At the start of their participation week, 2-3 interns picked up a PDA, received an individual orientation to the study and a detailed manual of operations to maximize inter-rater reliability of self-sampling data. A
SC
support number was also posted in resident team rooms. Interns were informed that participation in this study was voluntary. Responses were not linked to the responder identity. The IRB
M AN U
exempted this study from the need for consent. Activity Definitions
In order to develop a user friendly PDA that allowed the interns to accurately describe what they were doing, the different possible resident tasks were collated into sequential activities on 3
TE D
levels (Figure 1). The activities were arrived at by consensus utilizing the mini-Delphi process,5 an iterative process involving multiple meetings of 4-5 residents and graduate medical education leaders. This process arrived at three levels of activities. All levels were designed to be
EP
exhaustive, mutually exclusive, clear and adaptable to the PDA sampling methodology. For
AC C
example, Level 1 activities were pre-rounding, rounding, post-rounding, continuity clinic, signout, planned education, and personal time. Levels 2 and 3 further elucidated these activities by asking questions such as “what”, “why”, “how”, or “with whom” in relation to the prior activity. The final screen queried the interns to rate the perceived educational value of the work activity on a 4-point Likert scale (none, fair, good, or excellent). The 4-point Likert scale was chosen to sufficiently describe educational value as it avoids a central neutral response.6 After the intern chose the activity on one screen, the PDA automatically forwarded to the next level’s screen with 6
ACCEPTED MANUSCRIPT
the corresponding activity. To assure construct validity and to test both the ease of participation and the completeness of responses to the multi-level task categories that modeled the typical workday, a pilot study was conducted during development. During this pilot the interns reported
to do, and it did not interfere with their workday activities.
SC
Data Collection
RI PT
it typically took 10-20 seconds in total to complete the 4-levels for each sample entry, it was easy
The interns entered their team assignment and the size of their panel of patients at the beginning
M AN U
and end of a workday. All data used in the study started when the intern arrived at the hospital and ended when the intern left the hospital at the end of the work day. The alarm vibrates randomly with a range between 10 minutes and 54 minutes and an average time interval of 26 minutes. The alarm interval was consistent for all interns. This sampling frequency was
TE D
chosen based on similar work sampling studies with other healthcare professionals.7 This alarm frequency is not too often to interfere with the interns’ work and not too infrequent as to require an extended study period for adequate data collection. When it was time to query the intern to
EP
collect a sample, the PDA vibrated, displaying a pick-list of level 1 activities. If the intern did not respond within 20 seconds, the PDA continued to vibrate every 20 seconds until the intern
AC C
either responded to the query or until the next random query occurred. At the end of their workday, interns manually terminated study collection on the PDA. Data from each PDA were downloaded in an automated fashion onto the UMTPlus software by docking the PDA to specified computers in the physician team rooms. Data Analysis
7
ACCEPTED MANUSCRIPT
A sample size of at least 2,000 queries was chosen to achieve an absolute error of 2% with a 95% confidence interval. Data were downloaded from UMTPlus into Excel and descriptive statistics were used to identify the distribution of work shift activities and the perceived educational
RI PT
ratings for each activity. Personal time was included for descriptions of the entire workday but excluded for analyses of clinical activities and perceived educational value. Some categories were combined during analysis. For example, to determine the time spent using the electronic
SC
health record (EHR), level 2 and 3 activities of documenting, reading the EHR, writing orders and preparing sign-out were combined. To determine the total time spent with patients and
M AN U
families, level 2 and 3 activities of bedside work, procedures and communication with patient and/or family were combined. Data were aggregated from all participants and used to generate descriptive statistics for both time of activities and perceived educational value.4 Data are expressed as mean with standard deviation (SD) as appropriate. Perceived educational value for
TE D
various workday activities was calculated utilizing the 4-point Likert scale, with values assigned
RESULTS Daily Activities
EP
as none=1, fair=2, good=3, and excellent=4.
AC C
A total 78% (31/40) of the intern class participated for 88 workdays, generating a total of 2082 work-sampling queries from which to model the average workday. Data entry for these workdays commenced between 4:50am and 8:30am and ended between 4:59pm and 7:18pm. Five interns missed one day each of data collection during their assigned week. The mean intern workday was 11.0 (SD 1.1) hours with 24.7 (2.8) work queries per day. This is the expected number of queries given a random query on average every 26 minute.
8
ACCEPTED MANUSCRIPT
The distribution of level 1 activities for the average intern workday is shown in Table 2. Eighty percent of the workday involved clinical service activities (pre-rounds, rounds, post-rounds,
RI PT
continuity clinic and sign-out). The major level 2 and 3 activities included time spent with the EHR (33%), professional
communication (23%), planned and unplanned education (17%) and direct contact with the
SC
patient and family (12%). Only 3% of time was spent in administrative tasks, and a majority of these were perceived by the interns as appropriate activities for a resident. Interns spent 5% of
M AN U
their time in non-productive activities (Table 3). Perceived Educational Value of Activities
The perceived educational value was rated in 99% (1914 / 1928) of the activity queries excluding personal time. Overall, 15% of level 1 activities were perceived as educationally excellent,
TE D
while 29% were perceived as good, 31% were perceived as fair, and 26% were perceived as having no educational value. The activity with the highest perceived educational activity was planned education while the activity with the lowest perceived educational activity was sign-out
EP
(Figure 2). The level 2 and 3 activities having the highest perceived educational values were planned and unplanned education and time spent with the patient and family, while the activities
AC C
with the lowest perceived educational value were activities involving the EHR, administrative tasks and nonproductive time (Figure 3). Activities involving the EHR were perceived as having no or fair educational value 78% of the time, and interns spent 3.4 hours/day in the EHR. Of activities involving the EHR, the discharge note was perceived as having the least educational value. Interns spent 41 minutes/day writing discharge summaries which at our institution
9
ACCEPTED MANUSCRIPT
includes a review of the hospital course, patient education, discharge instructions and the medication reconciliation.
RI PT
All planned educational activities had perceived values of excellent or good a majority (81%) of the time. Simulation and inpatient attending teaching rounds had the highest perceived
educational value, while institutional grand rounds had the lowest (Figure 4). Average patient
SC
load per intern was consistent across educational ratings of level 1 activities. DISCUSSION:
M AN U
Self-work sampling methodology was effective in gathering data to assess the activities and educational values of interns’ workdays. This is the first study to use real-time self-work sampling to evaluate pediatric interns’ perceived educational value of their daily work activities. Interns perceive time spent in planned educational activities is more educationally valuable than
TE D
time spent in patient care related tasks and the EHR, yet their most common tasks involve communicating with the healthcare team and using the EHR. Activities perceived to be nonvaluable are more frequent than activities perceived to be valuable. Perceived valuable activities
EP
are less frequent than those perceived to be of little value. Utilizing PDAs is an effective way to conduct a self-work sampling study and avoids the need
AC C
for an observer. Self-work sampling has several advantages over observational studies. One-toone observation can influence the behavior of the participant, increase the cost of the research and provide more subjective, less pragmatic results. One-to-one observation may also limit the number of participants and the length of time the study can continue.4,8,9 10 With selfassessment, important perceptions and other cognitive processes can be obtained from the participant rather than an observer. 11
Studies in nursing and laboratory medicine demonstrate
the validity of the PDA in self-work sampling to provide reliable insight into the roles and 10
ACCEPTED MANUSCRIPT
responsibilities of health care providers. 7,12,13 The samples may include what, how, why, and with whom the activity is being done, as well as document perceptions and other cognitive
RI PT
processes. It is noteworthy that our findings are relatively similar to those of observational time-motion studies. Mamykina et al observed 7 general medicine residents each for one full workday and found they spend 50.6 % of their time on a computer, 40.6% of the day in communication, and
SC
9.4% of the day interacting with patients.14 Starmer et al conducted a multicenter study
M AN U
observing pediatric interns 24 hours a day and 7 days a week and found that they spent 31% of time in intraprofessional communication, 22% of time on the computer, and 14% of time with patients-families.15 Block showed internal medicine interns spend 40% of time at the computer and 12% of time in direct patient care.16 A self-work sampling study in 2011 showed internal medicine residents spending 43% of the day at workstations and 20% of time in patient rooms.17
TE D
Interns spend more time documenting than they spend with their patients, and they have reported being frustrated and dissatisfied with the demands of documentation as they appear to detract
EP
from their education.18,19
In our study, interns perceived greater educational value from planned small group sessions such
AC C
as simulation and case-based seminars such as attending teaching rounds that employ active learning techniques. This is consistent with adult learning theory 20 which postulates that learners value educational activities in which they are actively engaged and perceive less value from traditional, passive, lecture-based formats. The most highly rated educational sessions in our study were designed to address intern knowledge and skills gaps for commonly encountered clinical problems on the rotation. Trainees often chose topics and case discussions specifically
11
ACCEPTED MANUSCRIPT
tailored to address their concerns. In contrast, institutional grand rounds are more theoretical, lecture-based, passive, and not based on topics commonly encountered in practice.
RI PT
Interns perceive a higher educational value associated with planned educational activities than clinical activities involving patients. This perception is counter to Osler’s view of learning in the presence of patients: “He who studies medicine without books sails an uncharted sea, but he who
SC
studies medicine without patients does not go to sea at all.”21 Our study is consistent with a
concerning trend towards an uncoupling of education from direct patient care. Interestingly,
M AN U
residents have rated classroom-based instruction as superior to bedside teaching. Pediatric residents surveyed on a hematology-oncology inpatient service perceived classroom teaching as more effective because they were “concerned about patients’ comfort and misunderstanding… and becoming ashamed when they had wrong answers or comments in the presence of the
TE D
patients.”22
Several studies found that communication and physical exam skills can be best taught and assessed at the bedside, 23-27 but the over-emphasis on teaching and learning “standard” medical
EP
knowledge diminished the value of bedside learning. Gonzalo interviewed 34 experienced clinical teachers at 10 institutions to determine strategies to overcome bedside teaching barriers.
AC C
One of the barriers was “time compression” to finish rounds quickly so that trainees could attend scheduled didactic sessions that transmit knowledge to pass standardized board exams. We perceive similar barriers to patient-based learning at our institution. There are several limitations to this study. Despite a sample size of 2082 queries, the reliability of activities that occur at or below 20 minutes/day may not be accurately assessed. For example, time spent on administrative tasks and non-productive activity might have been better assessed
12
ACCEPTED MANUSCRIPT
with a larger data set. Second, our sample consisted only of interns working weekdays on inpatient rotations. The activity distribution and perceived educational values of these activities could change for higher level residents, those working nights and weekend call shifts, or those on
RI PT
outpatient rotations. Third, this study was conducted during the second half of the intern year, February through June to ensure expertise with the electronic medical records. The time of the year might alter the results. As is the case with many other children’s hospitals, the medical
SC
patient census at our institution is generally higher during the winter months. Our finding that average patient load was consistent across educational ratings suggests a lack of relationship
M AN U
between this measure of work load and perceived educational value of the typical inpatient intern day. However, a longer study encompassing the entire year, would be better suited to identify seasonal variations in roles and educational perceptions. Fourth, this study was done within a single institution and pediatric residency program, which impacts generalizability. Fifth,
TE D
measures other than average patient load (e.g. patient turnover which factors in admissions and discharges, both EHR-intensive activities) may be better predictors of intern work load and service needs. Sixth, utilizing a voluntary sample of interns may bias results. However, 78% of
EP
interns participated. Finally, there may not be a correlation between perceived educational value
AC C
and actual educational value. Assessing this was beyond the scope of this study. CONCLUSIONS:
In conclusion, utilizing a self-work sampling technique via a PDA we found that pediatric interns spend a large proportion of their day in the EHR and less time with patients, and that the distribution of daily activities of interns does not parallel the perceived educational value of these activities. These findings inform program improvement and faculty development in an effort to
13
ACCEPTED MANUSCRIPT
better align educational value with service requirements while maximizing educational experiences through effective patient care delivery.
RI PT
ACKNOWLEDGEMENTS: The authors wish to acknowledge the contributions of the following individuals: Nelson Lee (Rapid Modeling Corporation, Cincinnati, OH) and Jonathan Davies (former Pediatric Chief
SC
Resident, Children's National Medical Center, Washington, DC). We also wish to acknowledge the Academic Pediatric Association for presenting this project the Ray E. Helfer Award for
M AN U
Innovation in Medical Education, which was awarded at the Pediatric Academic Societies 2013
AC C
EP
TE D
annual meeting.
14
ACCEPTED MANUSCRIPT
REFERENCES: 1.
Desai SV, Feldman L, Brown L, et al. Effect of the 2011 vs 2003 duty hour regulation-
RI PT
compliant models on sleep duration, trainee education, and continuity of patient care among internal medicine house staff: a randomized trial. JAMA Intern Med 2013;173:649-55. 2.
Fitzgibbons SC, Chen J, Jagsi R, Weinstein D. Long-term follow-up on the educational
SC
impact of ACGME duty hour limits: a pre-post survey study. Annals of Surgery 2012;256:110812.
Drolet BC, Christopher DA, Fischer SA. Residents' response to duty-hour regulations--a
M AN U
3.
follow-up national survey. The New England Journal of Medicine 2012;366:e35. 4.
Robinson MA. Work sampling: Methodological advances and new applications. Human
Factors and Ergonomics in Manufacturing & Service Industries 2010;20:42-60. Hanafin S. Review of literature on the Delphi Technique. Dublin: National Children’s
Office 2004. 6.
TE D
5.
Brown S, Agree S, Disagree S, et al. Likert Scale Examples for Surveys. ANR Program
Evaluation Iowa State University Extension 2010. Hendrich A, Chow MP, Skierczynski BA, Lu Z. A 36-hospital time and motion study:
EP
7.
8.
AC C
how do medical-surgical nurses spend their time? The Permanente Journal 2008;12:25. Sittig DF. Work-sampling: a statistical approach to evaluation of the effect of computers
on work patterns in healthcare. Methods Inf Med 1993;32:167-74. 9.
Oddone E, Guarisco S, Simel D. Comparison of housestaff's estimates of their workday
activities with results of a random work-sampling study. Academic Medicine: Journal of the Association of American Medical Colleges 1993;68:859-61.
15
ACCEPTED MANUSCRIPT
10.
Pelletier D, Duffield C. Work sampling: valuable methodology to define nursing practice
patterns. Nursing & Health Sciences 2003;5:31-8. 11.
MacGregor DL, Tallett S, MacMillan S, Gerber R, O'Brodovich H. Clinical and
RI PT
education workload measurements using personal digital assistant-based software. Pediatrics 2006;118:e985-91. 12.
Watkins N, Kennedy M, Lee N, et al. Destination bedside: using research findings to
SC
visualize optimal unit layouts and health information technology in support of bedside care. J Nurs Adm 2012;42:256-65.
Trotter MJ, Larsen ET, Tait N, Wright JR, Jr. Time study of clinical and nonclinical
M AN U
13.
workload in pathology and laboratory medicine. Am J Clin Pathol 2009;131:759-67. 14.
Mamykina L, Vawdrey DK, Hripcsak G. How Do Residents Spend Their Shift Time? A
Time and Motion Study With a Particular Focus on the Use of Computers. Academic Medicine:
15.
TE D
Journal of the Association of American Medical Colleges 2016;91:827-32. Starmer AJ, Destino L, Yoon CS, Landrigan CP. Intern and Resident Workflow Patterns
on Pediatric Inpatient Units: A Multicenter Time-Motion Study. JAMA Pediatrics
16.
EP
2015;169:1175-7.
Block L, Habicht R, Wu AW, et al. In the wake of the 2003 and 2011 duty hours
AC C
regulations, how do internal medicine interns spend their time? Journal of General Internal Medicine 2013;28:1042-7. 17.
Siegler JE, Patel NN, Dine CJ. Prioritizing Paperwork Over Patient Care: Why Can't We
Do Both? Journal of Graduate Medical Education 2015;7:16-8.
16
ACCEPTED MANUSCRIPT
18.
Westbrook JI, Ampt A, Kearney L, Rob MI. All in a day's work: an observational study
to quantify how and with whom doctors on hospital wards spend their time. Med J Aust 2008;188:506-9. Christino MA, Matson AP, Fischer SA, Reinert SE, Digiovanni CW, Fadale PD.
RI PT
19.
Paperwork versus patient care: a nationwide survey of residents' perceptions of clinical documentation requirements and patient care. Journal of Graduate Medical Education
20.
SC
2013;5:600-4.
Knowles MS, Holton EF, Swanson RA. The adult learner : the definitive classic in adult
21.
Osler W. On the need of a radical reform in our teaching methods; senior students.
Medical News. 1903: 82: 49-53. 22.
M AN U
education and human resource development. 5th ed. Houston, Tex.: Gulf Pub. Co.; 1998.
Shahriari M. Case based teaching at the bed side versus in classroom for undergraduates
2014;2:135-6.
TE D
and residents of pediatrics. Journal of Advances in Medical Education & Professionalism
LaCombe MA. On bedside teaching. Annals of Internal Medicine 1997;126:217-20.
24.
Mangione S, Nieman LZ, Gracely E, Kaye D. The teaching and practice of cardiac
EP
23.
auscultation during internal medicine and cardiology training: a nationwide survey. Annals of
25.
AC C
Internal Medicine 1993;119:47-54.
Mattern WD, Weinholtz D, Friedman CP. The attending physician as teacher. New
England Journal of Medicine 1983;308:1129-32. 26.
Nair B, Coughlan J, Hensley M. Student and patient perspectives on bedside teaching.
Medical Education 1997;31:341-6.
17
ACCEPTED MANUSCRIPT
27.
Gonzalo JD, Heist BS, Duffy BL, et al. Identifying and overcoming the barriers to
AC C
EP
TE D
M AN U
SC
RI PT
bedside rounds: a multicenter qualitative study. Academic Medicine 2014;89:326-34.
18
ACCEPTED MANUSCRIPT
Table 1. An Intern's Typical Day
Grand Rounds
Attending Teaching Rounds
Work Rounds
Work Rounds
Work Rounds Patient Care
Friday Arrival, PreRounds
RI PT
Work Rounds
Arrival, Pre-Rounds
Thursday Arrival, PreRounds Morning Report Attending Teaching Rounds
SC
Work Rounds
Noon Conference
Patient Care (Continuity Clinic 1 day/week)
AC C
8:30 9:00 9:30 10:00 10:30 11:00 11:30 12:00 12:30 13:00 13:30 14:00 14:30 15:00 15:30 16:00 16:30 17:00 17:30 18:00
Attending Teaching Rounds
Arrival, PreRounds
Wednesday
M AN U
8:00
Tuesday Arrival, PreRounds Morning Report Attending Teaching Rounds
TE D
7:30
Monday
Sign Out, Finish work and leave hospital
EP
Time 6:00 6:30 7:00
ACCEPTED MANUSCRIPT
Table 2. Proportion of workday spent in Level 1 Activities (n=2082).
250
12
1.3
Rounds
340
16
1.8
Post-Rounds
779
37
4.1
Continuity Clinic
127
6
0.7
Sign-out
158
8
0.9
Planned Education 274
13
1.4
Personal Time
154
7
0.8
Totals
2082
100
11
AC C
EP
TE D
M AN U
Pre-Rounds
RI PT
Mean % of Workday Hours
N
SC
Activity
ACCEPTED MANUSCRIPT
Table 3. Breakdown of Level 1 activities into Level 2 and 3 activity categories (n=2080). Level 2 data was missing on 2 pre-rounds queries.
Contact with Patient or Family Unplanned Education Administrative Tasks
Planned Education Personal Time
77 130
Preparing for Sign-out Communicate with Healthcare Team Listen-Present on Rounds Give/Receive Sign-out At the Bedside Procedures
59
33%
3.4 (1.2)
192
206 89 167 20
Communicate with Patient/Family
63
Teach, learn, reflect, read
75
Appropriate for resident
39
Better done by support staff
27
Travel, wait, technology problem or supply problem
Mean Hours (SD)
RI PT
Read Medical Record Write Orders
AC C
Total Day
414
EP
Non-Productive Time
Document (admission, discharge, transfer, other)
% of Workday
SC
Professional Communication
N
TE D
EHR
Activity Breakdown
M AN U
Activity Categories
23%
2.4 (1.3)
12%
1.3 (0.7)
4%
0.8 (0.5)
3%
0.8 (0.5) 0.9 (0.6)
101
5%
267
13%
1.5 (0.7)
154
7%
1.1 (0.7)
2080
100
12.2
ACCEPTED MANUSCRIPT
Figure 1. Summary of activity definitions included in the manual of operations. Level 1Activities
RI PT
Pre-Rounds: The time between starting the day’s work and starting rounds. Pre-rounds include gathering patient information and physical exam. Pre-rounds can begin on arrival to the inpatient unit or when opening the EHR from alternate location. Rounds: Rounding starts when attending physician and/or senior resident begins patient care rounds with the team and ends when last patient has been seen or when leaving rounds to start another activity.
M AN U
SC
Post-Rounds: Usually starts after rounds, but can start during rounds if required to leave rounds early for patient care related duties. Includes bedside work, procedures, reading, writing orders and documenting in the healthcare record and communications with patients, families and the healthcare team. Ends when a continuity clinic assignment planned educational activity or signout begins. Continuity Clinic: Clinic starts upon leaving an activity for the outpatient assignment and ends when leaving the outpatient area. Travel time for clinic assignments should be included as nonproductive time.
TE D
Planned Education: All regularly planned educational sessions occurring in the inpatient unit, auditorium, simulation center or classroom. Sign-out: Sign-out begins when ready to start resident-to-resident handoff communication. Signout will include preparation in EHR and actual handoff. Waiting for sign-out should be included in sign-out activity as non-productive time.
EP
Level 2 and 3 Activities
Bedside Work: Done with team on rounds or independently; can include the history and physical.
AC C
Procedure: Other procedures include sutures, sedation, G-tube management, urinary catheterization, splinting, fracture reduction, LP and CPR resuscitation. Phlebotomy and IV insertions are documented separately. Read Patient Record: Gathering information from EHR. Write Order: Any order entry completed in EHR. Documentation: Any patient related documentation in EHR separated into categories (1) admission, (2) transfers, (3) discharge and (4) others which include progress notes, event notes and family meetings and clinic documentation.
ACCEPTED MANUSCRIPT
RI PT
Communication: Information exchange which includes talking, texting and emailing separated into exchange between (1) staff physicians includes the attending, consultant, fellow, resident, (2) primary care physicians, (3) medical students, (4) non-physician staff includes nurses, social workers, case managers, NP, PA, technicians, chaplains, dieticians, interpreters, OT, PT and units clerks, (5) patients and their families and (6) other. Present-Listen: Time spent presenting and listening during patient care rounds (i.e. not when you are completing other duties).
SC
Administrative Tasks: Defined as a duty the residents perceive to be administrative. Requested to document the staff member most appropriate to complete the task including the (1) case manager, (2) clerk, (3) social worker, (4) Resident Assistant, (5) nursing, or (6) the resident themselves.
M AN U
Unplanned Education: Knowledge transfer through either: (1) teaching, (2) learning, (3) reflecting or (4) reading that occurs during any activity outside planned educational didactics (planned education includes morning report, noon conference, attending rounds, grand rounds, professorial rounds and simulation center).
EP
TE D
Non-Productive Time: Time is divided into the following categories: (1) traveling, (2) supplies, (3) technology, (4) waiting, and (5) other. 1. Time spent traveling includes walking between patients’ rooms and/or units, driving to offsite locations and walking to other buildings (East Wing, Main Hospital, and West Wing). 2. Time related to supplies is time spent looking or waiting for necessary equipment or supplies. 3. Time related to technology is wasted time due to problems with web, hardware, or software malfunctions, computers, printers, fax or phones. 4. Time related to waiting is time when you cannot proceed until something happens. 5. Other non-productive time is used when one of the previous categories is not appropriate.
AC C
Sign-out Preparation: Time spent organizing, summarizing, documenting in EHR in preparation for handoff. Sign-out Handoff: Time spent communicating patient information to another health care provider. Personal time: Includes meals, rest periods, and personal activities such as phone calls and emails.
ACCEPTED MANUSCRIPT
Figure 2. Perceived Educational Value of Level 1 Activities. Calculated mean perceived educational values for workday activities are shown on top of each column. The overall mean value for Level 1 activities was 2.3. 100% 2.1
2.6
2.1
Pre-Rounds
Rounds
Post Rounds
2.0
2.2
3.2
RI PT
90% 80% 70% 60% 50%
SC
40% 30%
10% 0%
Good
AC C
EP
TE D
Excellent
M AN U
20%
Continuity Clinic
Fair
None
Sign-Out
Planned Education
ACCEPTED MANUSCRIPT
Figure 3. Perceived Educational Value of Level 2 & 3 Activities. Calculated mean perceived educational values for Level 2 and Level 3 activities are shown on top of each column. Personal time was excluded from the calculations for educational activities.
90%
1.9
2.8
2.4
3.0
3.2
1.5
1.0
RI PT
100% 80% 70% 60% 50% 40%
SC
30% 20% 10%
M AN U
0%
Good
AC C
EP
TE D
Excellent
Fair
None
ACCEPTED MANUSCRIPT
Figure 4. Perceived Educational Value (EV) of Planned Education Activities. Calculated mean perceived educational value for educational activities are shown on top of each column. The overall mean value for all educational activities was 3.2.
3.6
3.5
Simulation
Attending Rounds
3.4
3.1
3.2
2.6
RI PT
100% 90% 80% 70%
SC
60% 50%
30% 20% 10% 0%
M AN U
40%
Prof Rounds Morning Report good
AC C
EP
TE D
excellent
fair
none
Noon Conf
Grand Rounds