Differences in Resident Perceptions by Postgraduate Year of Duty Hour Policies: An Analysis from the Flexibility in Duty Hour Requirements for Surgical Trainees (FIRST) Trial

Accepted Manuscript Differences in Resident Perceptions by Postgraduate Year of Duty Hour Policies: An Analysis from the Flexibility in Duty hour Requirement for Surgical Trainees (FIRST) Trial Anthony D. Yang, MD, MS, FACS, Jeanette W. Chung, PhD, Allison R. Dahlke, MPH, Thomas Biester, MS, Christopher M. Quinn, MS, Richard S. Matulewicz, MD, MS, David D. Odell, MD, MMSc, Rachel R. Kelz, MD, MSCE, FACS, Judy Shea, PhD, Frank Lewis, MD, FACS, Karl Y. Bilimoria, MD, MS, FACS PII:

S1072-7515(16)31591-5

DOI:

10.1016/j.jamcollsurg.2016.10.045

Reference:

ACS 8528

To appear in:

Journal of the American College of Surgeons

Received Date: 5 October 2016 Accepted Date: 10 October 2016

Please cite this article as: Yang AD, Chung JW, Dahlke AR, Biester T, Quinn CM, Matulewicz RS, Odell DD, Kelz RR, Shea J, Lewis F, Bilimoria KY, Differences in Resident Perceptions by Postgraduate Year of Duty Hour Policies: An Analysis from the Flexibility in Duty hour Requirement for Surgical Trainees (FIRST) Trial, Journal of the American College of Surgeons (2016), doi: 10.1016/ j.jamcollsurg.2016.10.045. This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

ACCEPTED MANUSCRIPT

Differences in Resident Perceptions by Postgraduate Year of Duty Hour Policies: An Analysis from the Flexibility in Duty hour Requirement for Surgical Trainees (FIRST)

RI PT

Trial

Anthony D Yang, MD, MS, FACS1, Jeanette W Chung, PhD1, Allison R Dahlke, MPH1, Thomas Biester, MS2, Christopher M Quinn,1 MS, Richard S Matulewicz, MD, MS1, David D Odell,

SC

MD, MMSc1, Rachel R Kelz, MD, MSCE, FACS3, Judy Shea, PhD4, Frank Lewis, MD, FACS2,

M AN U

Karl Y Bilimoria, MD, MS, FACS1,5

1. Surgical Outcomes and Quality Improvement Center, Department of Surgery and Center for Healthcare Studies, Feinberg School of Medicine and Northwestern Medicine, Northwestern University, Chicago, IL

TE D

2. American Board of Surgery, Philadelphia, PA

3. Department of Surgery and Center for Surgery and Health Economics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA

EP

4. Division of General Internal Medicine, Department of Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA

AC C

5. American College of Surgeons, Chicago, IL

Disclosure Information: Nothing to disclose. The Flexibility In duty hour Requirements for Surgical Trainees trial was funded by the American Board of Surgery, the American College of Surgeons, and the Accreditation Council for Graduate Medical Education; and ClinicalTrials.gov number, NCT02050789.

1

SC

Correspondence address: Karl Y. Bilimoria, MD, MS Surgical Outcomes and Quality Improvement Center (SOQIC) Department of Surgery Feinberg School of Medicine, Northwestern Medicine Northwestern University 633 N. St. Clair Street, 20th Floor Chicago, IL 60611 [email protected] Office: (312) 695-4853

RI PT

ACCEPTED MANUSCRIPT

Running head: Differences in Resident Perceptions by Postgraduate Year

AC C

EP

TE D

Surgical Education, FIRST Trial, PGY

M AN U

Keywords: Surgery, Resident, Duty hours, Outcomes, Policy, Randomized controlled trial,

2

ACCEPTED MANUSCRIPT

ABSTRACT BACKGROUND: In the Flexibility in Duty Hour Requirements for Surgical Trainees (FIRST) Trial, there were several differences in residents’ perceptions of aspects of their education, well-

RI PT

being, and patient care that differed between standard and flexible duty hour policies. Our objective was to assess whether these perceptions differed by level of training.

STUDY DESIGN: A survey assessed residents participating in the FIRST Trial’s perceptions of

SC

the effect of duty hour policies on aspects of patient safety, continuity of care, resident education, clinical training, and resident well-being. Hierarchical logistic regression models were used to

Junior Residents, and Senior Residents).

M AN U

examine the association between residents’ perceptions, study arm, and level of training (Interns,

RESULTS: In the Standard Policy arm, as the PGY level increased, residents more frequently reported that duty hour policies negatively affected patient safety, professionalism, morale, and

TE D

career choice (all interactions p<0.001). However, in the Flexible Policy arm, as the PGY level increased, residents less frequently perceived negative effects of duty hour policies on resident health, rest, and time for family/friends and extracurricular activities (all interactions p<0.001).

EP

Overall, there was an increase by PGY level in the proportion of residents expressing a preference for training in programs with flexible duty hour policies, and this preference for

AC C

flexible duty hour policies was even more apparent among residents who were in the Flexible Policy arm (p<0.001).

CONCLUSIONS: As PGY level increased, residents had increasing concerns about patient care and resident education/training under standard duty hour policies, but they had decreasing concerns about well-being under flexible policies. When given the choice between training under

3

ACCEPTED MANUSCRIPT

standard or flexible duty hour policies, only 14% of residents expressed a preference for standard

AC C

EP

TE D

M AN U

SC

RI PT

policies.

4

ACCEPTED MANUSCRIPT

INTRODUCTION The Flexibility In duty hour Requirements for Surgical Trainees (FIRST) Trial was a prospective, national, cluster-randomized, pragmatic trial designed to study the impact of duty

RI PT

hour flexibility on patient outcomes, resident education, and resident well-being.1-3 This was the first large trial to compare standard surgical resident duty hour requirements (Standard Policy) with more flexible policies (Flexible Policy). The major findings of the FIRST Trial were that

SC

flexible duty hour policies for surgical residents did not lead to worse patient outcomes

compared to standard Accreditation Council for Graduate Medical Education (ACGME) duty

well-being and education quality.4

M AN U

hour policies and resulted in no difference in resident self-reported satisfaction with their overall

As a part of the FIRST Trial, residents in participating programs were surveyed to assess their

TE D

perceptions of the effect of the particular duty hour policies in place at their respective institutions on aspects of their education, their well-being, and patient care. Residents in Flexible Policy programs were less likely to perceive negative effects of duty hour policies on patient

EP

safety, continuity of care, and resident education and clinical training, but were more likely to perceive negative effects on certain aspects of their well-being compared to residents training

AC C

under Standard Policy.

Prior studies of resident perceptions of the effects of the ACGME duty hour reforms have been mixed, but many have noted that there are different effects on well-being observed for interns compared to more senior residents.5-10 The initial analyses of the FIRST Trial did not examine whether there were differences by level of training in resident perceptions of patient safety,

5

ACCEPTED MANUSCRIPT

continuity of care, resident education and clinical training, and resident well-being, especially within each arm of the trial. Furthermore, no prospective study, to our knowledge, has expressly examined differences in how residents at different levels of training perceive the effects of duty

RI PT

hour policies. Understanding differences in how residents at different stages of training perceive the effects of duty hour policies may provide important insights into the FIRST Trial findings and guide duty hour policy changes. Thus, the objectives of this study were to utilize the

SC

resident-reported survey results from the FIRST Trial to assess resident perceptions of the effects of duty hour policies by study arm, and level of training, on measures of patient safety,

METHODS Data Source and Study Subjects

M AN U

continuity of care, resident education and clinical training, and resident well-being.

TE D

Data for this study was collected as a part of the FIRST Trial. Complete details of the FIRST Trial study protocol have been previously described elsewhere.1,3 A total of 117 ACGMEaccredited General Surgery residency programs in the U.S. were randomized either to the

EP

Standard Policy arm, in which programs were required to adhere to existing ACGME duty hour policies instituted in 2003 and 2011, or to the Flexible Policy arm in which programs could

AC C

pursue more flexible duty hour policies that waived rules on maximum shift lengths (e.g., 16 hour maximum for interns and 24 hour maximum for PGY 2-5 residents) and time off between shifts (e.g., minimum of 8 hours off between shifts and minimum of 14 hours off after an overnight call).11,12 As described previously, randomization also demonstrated balance for a broad range of program characteristics between study arms.4

6

ACCEPTED MANUSCRIPT

The present study draws on resident survey data that were collected during the FIRST Trial in collaboration with the American Board of Surgery, which administered the survey to all residents who took the American Board of Surgery In-Training Examination (ABSITE) in January 2015.13

RI PT

The closed-ended survey contained over 60 items designed to assess residents’ perceptions of the effect of the 2014-2015 academic year duty hour policies at their respective institutions on

aspects of patient safety, continuity of care, resident education and clinical training, and resident

SC

well-being. Additional survey items asked residents to report the frequency over the last month at which they experienced various interruptions in their clinical training due to duty hour

M AN U

restrictions. Survey items were adapted from previously published surveys, pre-tested with residents using cognitive interviews, and iteratively revised.3 Data were obtained for a total of 4,330 General Surgery residents training in FIRST Trial programs (2220 Standard Policy, 2110 Flexible Policy). Response rates varied across survey items, ranging from 95.5% to 98.5% for

TE D

outcomes examined. The initial trial protocol was reviewed by the Northwestern University Institutional Review Board office, which determined the trial to be non–human-subjects

EP

research.1,3,4,14

Resident Postgraduate Year

AC C

Resident postgraduate year (PGY) was determined based on data from the ABS. We grouped postgraduate year as follows: PGY1 (“Interns”), PGY2-3 (“Junior Residents”), and PGY4-5 (“Senior Residents”) to reflect the broadly recognized categories of surgical residents by their experience level and level of training. For comparability, “preliminary” residents (those residents who are part of the general surgery residency program for only the first 1-3 years) were excluded

7

ACCEPTED MANUSCRIPT

from this study (i.e., only data from “categorical” general surgery residents were retained for

Outcome Measures

RI PT

analysis).

Four “domains” of resident outcomes were examined: patient safety, continuity of care, resident

SC

education and clinical training, and resident well-being.

Perceived Effect of Institutional Duty Hours. The ABSITE Survey contained a series of

M AN U

questions asking residents whether duty hours at their institution had a positive effect, negative effect, or no effect on aspects of patient care, continuity of care, resident education and clinical training, and resident well-being. For all domains, the outcomes were dichotomized to enhance the clarity of our analyses and exposition without any demonstrated loss of generality, as was

TE D

previously described in the FIRST Trial main analysis.4 In general, the responses were dichotomized to contrast negative responses against combined neutral and positive responses,

EP

unless otherwise described.

Perceptions of Continuity of Care. Residents were also asked how often in the last month they

AC C

had to hand off an active patient care issue, leave during the middle of an operation, and miss an operation altogether (0 times, 1-2 times, 3-4 times, 5+ times). These outcomes were dichotomized to contrast those who reported zero interruptions in the last month vs. those reporting one or more interruptions.

8

ACCEPTED MANUSCRIPT

Resident Satisfaction. A number of items in the ABSITE Survey asked residents to rate their level of satisfaction with various aspects of patient care, clinical training, and resident wellbeing. These outcomes were dichotomized to contrast Very Satisfied/Satisfied and Neutral vs.

RI PT

Dissatisfied and Very Dissatisfied.

Preference for Working in a Flexible Policy Program. We also sought to determine whether

SC

residents preferred flexible duty hour programs over standard policy programs. Interns were asked whether they agreed or disagreed with the opportunity to be in the Flexible Policy Arm of

M AN U

the FIRST Trial as an appealing feature of a residency program when formulating their rank lists for the residency match. Similarly, Junior and Senior Residents were asked whether they agreed or disagreed with the statement that, given their personal experience in residency, they would choose to train in a program which offered increased flexibility in duty hour regulations over

TE D

current standard duty hour regulations. These responses were obtained from the ABSITE Survey of January 2016, as the question was not included in the 2015 survey and provides important insights into PGY-based duty hour issues. Responses were measured using a 5-point Likert scale,

EP

which were then consolidated into three groups: Strongly Agree/Agree, Neutral, and Disagree /Strongly Disagree. Interns were given the additional option of answering that they did not know

AC C

about the trial, which was classified as a neutral preference.

Statistical Analysis

To investigate subgroup effects of assignment to Flexible Policy by postgraduate year of training, we first examined unadjusted (“raw”) frequencies of each resident outcome variable by postgraduate year group (Interns, Junior Residents, Senior Residents) and study arm (Standard

9

ACCEPTED MANUSCRIPT

Policy vs. Flexible Policy). We then estimated hierarchical logistic regression models that regressed each dichotomous outcome on study arm (Standard Policy [reference]) vs. Flexible Policy, PGY level (Interns [reference], Junior Residents, Senior Residents), and an interaction

RI PT

term between study arm and PGY level. Models also controlled for program-level tertile of 30day postoperative death or serious morbidity (stratifying variable in randomization of residency programs in the FIRST Trial)3 and included program-level random intercepts. A test for overall

SC

significance of the interaction terms was conducted to evaluate the presence of PGY subgroup effects for each outcome. P-values reported herein have not been corrected for multiple

M AN U

comparisons using the Bonferroni method because, as described in the FIRST Trial main study,4 in the context of hypothesizing no difference in outcomes across study arms, correction for multiple comparisons is not a conservative approach for reducing the false discovery rate. The level of significance was set at 0.05. Analyses were conducted using Stata, Release 13.15 Details

RESULTS

TE D

of our methods have been described previously.1,3,4

EP

Our study sample included 4,330 surgical residents training in 117 ACGME-accredited General Surgery residency programs and 151 affiliated hospitals (Table 1). Of these, 2,220 residents were

AC C

in programs randomized to Standard Policy, and 2,110 were randomized to Flexible Policy. Of the total, 621 preliminary residents and 475 non-clinically active residents (e.g., those spending dedicated time performing research) were excluded. There were no differences between study

arms by gender, postgraduate year, or resident type, demonstrating that the randomization resulted in a balance of resident characteristics.

10

ACCEPTED MANUSCRIPT

The frequency of resident-reported perceptions of and satisfaction with the effect of their program’s specific duty hour policy during the FIRST Trial (Standard Policy or Flexible Policy) by resident PGY group (Interns, Junior Residents, Senior Residents) is reported in Table 2. The

RI PT

interaction significance shows the impact of study arm by resident PGY level on the same

resident-reported outcomes utilizing interaction coefficients (IC), which measure the odds of

SC

perceiving a negative effect of assignment to the Flexible Policy arm (Table 3).

Patient Safety

M AN U

Overall, residents in Standard Policy arm programs more frequently perceived a negative effect of duty hour policies on patient safety compared to Flexible Policy arm residents. In the Standard Policy arm, as PGY level increased, the frequency of residents reporting a negative perception of duty hours on patient safety increased (Percent perceiving a negative effect: Interns 22.75%,

TE D

Junior Residents 26.64%, Senior Residents 34.79%; Table 2); whereas, the percent perceiving a negative effect was relatively similar across PGY groups in the Flexible Policy arm. Consistent with these patterns, we found evidence of study arm assignment effect modification by PGY

AC C

Continuity of Care

EP

level (p<0.001; Table 3).

Overall, residents in Standard Policy arm programs more frequently perceived that duty hour policies had a negative effect on continuity of care compared to Flexible Policy arm residents (p<0.001; Table 2). In the Flexible Policy arm, negative perceptions regarding the effect of duty hours on continuity of care increased with increasing PGY level (Interns: 10.54%, Junior Residents: 21.07%, Senior Residents: 25.61%; Table 2), but stayed relatively stable in the

11

ACCEPTED MANUSCRIPT

Standard Policy arm. However, even in the Flexible arm Senior Residents, the frequency of perceiving a negative effect was less than half of the frequency in the Standard Policy arm. Consistent with these unadjusted frequencies, the interaction between PGY level and study arm

RI PT

was significant (p<0.001; Table 3).

Residents reported that they more frequently had to leave an operation in the Standard Policy

SC

arm when compared to the Flexible Policy arm. In the Standard Policy arm, the frequency of reporting having to leave an operation decreased with increasing PGY level, but remained

M AN U

relatively similar in the Flexible Policy arm across PGY levels. The differences were most evident in Interns, who noted that they had to leave during an operation much more frequently under Standard Policy than under Flexible Policy (Standard Policy: 17.9% vs. Flexible Policy: 6.9%; Table 2). Standard Policy Junior Residents and Senior Residents also reported having to

TE D

leave an operation at a frequency approximately two-fold greater than their counterparts in the Flexible Policy arm (Table 2). Again, we found a significant differences across study arms within each PGY level, as well as evidence of a significant effect of PGY level on leaving an

EP

operation (p<0.001, Table 3).

AC C

Resident Education and Clinical Training With respect to one of the FIRST Trial’s primary resident outcomes, there was decreasing dissatisfaction with overall resident education quality in the Flexible Policy arm with increasing PGY levels (Interns: 14.94%, Junior Residents: 9.64%, Senior Residents: 8.57%; Table 2). In contrast, dissatisfaction with education quality was fairly similar across PGY levels in the

12

ACCEPTED MANUSCRIPT

Standard Policy arm. There was not a significant interaction for between PGY level and study arm for overall education quality (p=0.066; Table 3).

RI PT

Resident Well-being

Compared to Standard Policy residents, Flexible Policy residents across all PGY levels more frequently responded that duty hour policies have a negative effect on activities outside of work

SC

such as rest, time for extracurricular activities, and time with family and friends (Table 2). These negative perceptions under Flexible Policy were most pronounced in the Intern group but,

M AN U

notably, the differences became less prominent with increasing PGY level. For example, Flexible Policy residents reported perceiving a negative effect of duty hour policies on time for rest less frequently by increasing PGY level (Interns: 38.34%, Junior Residents: 24.37%; Senior Residents: 15.85%; Table 2); whereas, there were not significant differences by PGY level in the

TE D

Standard Policy arm. Similarly, Flexible Policy residents perceived fewer negative effects of duty hour policies on other measures of well-being with increasing PGY levels. Accordingly, we found a significant difference across PGY levels in the effect of duty hour policies with respect

EP

to activities outside of work such as rest, time for extracurricular activities, time with family and

AC C

friends, and own health (p<0.001 for all; Table 3).

While there were no overall differences between study arms for resident morale, job satisfaction and the decision to become a surgeon, there were differences by PGY level. The frequency of residents reporting the perception of a negative effect of duty hour policies in these areas increased with increasing PGY level in the Standard Policy arm, but under Flexible Policy, this negative perception decreased with increasing PGY level (Table 2). Consistent with this finding,

13

ACCEPTED MANUSCRIPT

we found a significant interaction between PGY level and study arm for these resident-reported outcomes (p<0.001 for all; Table 3).

RI PT

Standard Policy residents also frequently perceived a negative effect of current duty hour

policies on professionalism, and this effect increased with increasing PGY level (Interns: 4.2%, Junior Residents: 13.4%, Senior Residents: 21.6%; Table 2). In the Flexible Policy arm, the

SC

perception of a negative effect of duty hours on professionalism was much less frequent and was relatively similar across PGY levels (Table 2). Again, we found significant PGY level

M AN U

differences in the effect of study arm assignment for professionalism (P<0.001; Table 3).

Dissatisfaction with overall well-being decreased with increasing PGY level in both study arms. Thus, there was not a significant interaction between PGY level and study arm (p=0.355; Table

TE D

3).

Resident Duty Hour Policy Preference

EP

In the Flexible Policy arm, residents noted decreasing dissatisfaction with increasing PGY level with respect to the work hours/scheduling and the work hour regulations at their institution

AC C

(Table 2). In contrast, the frequency of dissatisfaction was similar across PGY levels in the Standard Policy arm. However, the interaction between PGY level and study arm did not reach significance (p=0.92 for work hour scheduling and p=0.061 work hour regulations; Table 3).

In 2016, when residents in both of arms of the trial were asked if they would prefer to train in a surgical residency with flexible duty hour policies, an increasing proportion of residents

14

ACCEPTED MANUSCRIPT

preferred flexible policies as the PGY level increased (Figure 1). Overall, 54.5% of residents preferred flexible duty hour policies, 31.8% were neutral, and 13.6% preferred current standard duty hour policies. When Standard Policy residents were examined by PGY group, there was a

RI PT

progressive increase in the proportion of residents who expressed a positive preference (i.e., excluding neutral responses) for training under flexible duty hour policies as PGY level

increased (Interns: 20.12%; Junior Residents: 46.07%; Senior Residents: 56.88%; p<0.001;

SC

Figure 1). This trend was even more striking among Flexible Policy residents as they showed a progressive increase with level of training in their preference for training in a flexible duty policy

M AN U

program (Interns: 41.88%; Junior Residents: 64.64%; Senior Residents: 76.90%; p<0.001; Figure 1). Overall, when the neutral responses were included, most residents at all PGY levels either expressed a preference for flexible training under flexible duty hour policies or no preference (Interns: 86.01%; Junior Residents: 84.28%; Senior Residents: 88.93%; p<0.001;

DISCUSSION

TE D

Figure 1).

EP

The FIRST Trial was the first national randomized trial to compare standard surgical resident duty hour requirements with more flexible policies. The national ABSITE Resident Duty Hour

AC C

Survey, administered during the FIRST Trial, provided a unique opportunity for an in-depth analysis of the effect of differing duty hour policies on specific resident-reported outcomes. Residents in the two study arms of the FIRST Trial had differing perceptions of the effect of duty hour policies on patient safety, continuity of care, resident education and clinical training, and resident well-being. These differences were not only attributable to study arm assignment (i.e., standard ACGME duty hour restrictions vs. more flexible duty hour policies) but also frequently

15

ACCEPTED MANUSCRIPT

differed considerably by the level of training of the residents. As PGY level increased, residents had increasing concerns about patient care and resident education/training under standard duty hour policies, but they had decreasing concerns about well-being under flexible policies.

RI PT

Interestingly, only a minority of residents indicated that they would prefer to train in a program with standard duty hour policies; most residents expressed a preference or neutral attitudes

SC

toward training under flexible duty hour policies.

Residents at all levels working under current ACGME duty hour policies (Standard Policy arm)

M AN U

were more likely to perceive a negative effect of duty hour policies on patient safety and continuity of care. The negative perception of duty hour regulations on patient safety was most pronounced in Senior Residents training in Standard Policy arm programs, and may reflect differing levels of experience or responsibilities in patient care. However, Standard Policy arm

TE D

Interns were the most likely to perceive a negative effect of duty hour policies on continuity of care, possibly reflecting their increased level of involvement in transitions of care compared to their more senior peers. Notably, Standard Policy arm Interns reported having to leave during an

AC C

6.9%).

EP

operation at about three times the frequency of their Flexibly Policy arm counterparts (17.9% vs.

Overall, Standard Policy residents were more likely to report perceived negative effects of duty hour policies on their education, and this was similar across all PGY levels. There was decreasing dissatisfaction with overall resident education quality in the Flexible Policy arm as PGY level increased. While the reasons for this cannot be elucidated from the survey, the additional years of training experienced by Junior and Senior Residents may allow them to better

16

ACCEPTED MANUSCRIPT

recognize the benefits of more flexible duty hour policies on their education. Future qualitative studies will investigate this further.

RI PT

Overall, Flexible Policy residents more frequently perceived that duty hour policies have a

negative effect on activities outside of the hospital such as rest, extracurricular activity time, and time with family and friends. However, we found that this finding is significantly modified by

SC

PGY level. Flexible Policy Senior Residents reported perceived negative effects at

approximately half the frequency of Interns. Though there was no overall difference in well-

M AN U

being between study arms, Flexible Policy Interns were most likely to perceive a negative effect of duty hours policies on their own well-being, but this negative perception again decreased as PGY level increased. This study cannot identify the underlying factors that impact resident wellbeing, but we hypothesize that the findings could be related to Interns’ job responsibilities and

TE D

their level of experience. For instance, the findings could at least be partially explained if Interns are mostly utilizing their flexible duty time to perform mundane service duties with little educational benefit (e.g., writing notes, paperwork) while the Junior and Senior Residents are

EP

spending the flexible time in the operating room, directly caring for patients, or taking advantage of valuable educational opportunities. It may also take residents longer than one year to

AC C

appreciate how flexible duty hours can directly benefit patient care and/or their own education. We are continuing to administer the ABSITE Survey on an annual basis in addition to currently conducting a qualitative study of residents in order to better understand if residents’ perceptions change over time and why residents report that well-being improves with increasing PGY level.

17

ACCEPTED MANUSCRIPT

It is notable that when they were asked about their preferences for training under standard or flexible duty hour policies, most residents expressed a preference for flexible duty hour policies, particularly those who experienced flexibility. When we included those with no preference for

RI PT

either duty hour policy, the vast majority would be at least open to, if not outright prefer, training under flexible duty hour policies. This preference for flexible duty hours in both arms of the trial progressively increased as the PGY level increased, and it is clear that the majority of Senior

SC

Residents in both arms of the trial prefer training under flexible duty hours. On the other hand, only 14% of residents overall expressed a preference for training under standard duty hour

M AN U

policies. Thus, despite their expressed concerns regarding their well-being, senior residents, in particular are the more experienced residents and those who have experience with flexible duty hour policies, seem to recognize value in training and caring for patients under flexible duty

TE D

hours.

In aggregate, the findings of this study may be explained by the idea that as residents gain more experience (i.e., increasing PGY level), their views of how duty hour restrictions affect their

EP

activities evolve. There are a fixed number of hours in a week and residents often have to make exceptionally difficult choices between using their time for activities that take place in the

AC C

hospital related to patient care or their training, or spending that time on activities outside of the hospital that may benefit their well-being. From the findings of this study, it could be hypothesized that the most senior (i.e., experienced) residents may be able to better recognize the tradeoffs and consequences resulting from duty hour restrictions, and thus may better appreciate the benefits of having the flexibility to make choices regarding how they utilize their time. This is supported by the finding that the proportion of residents’ who prefer training in a program with

18

ACCEPTED MANUSCRIPT

flexible duty hour policies increased with their training level, particularly in those residents in the Flexible Policy arm of the trial who had experienced flexible duty hours. More specifically,

of the hospital in order for them to fulfill their calling as surgeons.

RI PT

residents may understand that some sacrifices must be made in regard to their activities outside

However, the findings that Interns, in particular, perceive that flexible duty hour policies have a

SC

negative impact on multiple measures of their well-being is particularly troubling. It may be that Interns are less able to experience the beneficial aspects of flexible duty hours than their more

M AN U

senior peers, especially if much of the flexible time is used to complete mundane tasks such as writing notes, entering orders, or clerical work. If flexible duty hour policies are to be instituted, then systems must be instituted to ensure that all residents, and particularly Interns, are protected from pressures to work excessive hours if it compromises patient safety, their personal safety, or

TE D

if it is not beneficial to their education. Further research needs to focus on how current and flexible duty hour policies differentially impact resident well-being, and on achieving the correct balance between safely caring for patients while maintaining the well-being of the physicians

EP

who are providing that care.

AC C

This study has several limitations. First, only data from surgical residents from the 117 programs enrolled in the FIRST Trial were utilized for this study, out of a total of 252 ACGME-accredited general surgery training programs in the United States. Programs participating in the FIRST Trial were larger, more frequently large academic programs, and had fewer international medical graduates. Thus the findings may not reflect resident perceptions at non-participating programs. Second, the FIRST Trial focused only on general surgery training programs, and while our

19

ACCEPTED MANUSCRIPT

results may be relevant to other surgical disciplines, they may not be generalizable to nonsurgical specialties. Third, we excluded preliminary residents from this analysis since the temporary nature of their positions and their underlying commitment to move on to other

RI PT

specialties may have introduced bias in their responses. Including preliminary residents may bias the Intern responses, in particular, since preliminary residents comprise a substantial proportion of this resident population. Fourth, this survey was conducted halfway through the trial period

SC

during standard ABSITE test administration dates in order to optimize data collection logistics and response rates. Measures of resident perceptions and experiences could vary and shift over a

M AN U

longer exposure period prior to survey administration. Additional ABSITE Resident Surveys are to be administered annually, and we expect the results of these additional surveys will allow us to discover whether resident perceptions vary with increasing exposure time to flexible duty hour

CONCLUSION

TE D

policies.

Residents participating in the FIRST Trial had differing perceptions of measures of patient

EP

safety, continuity of care, their own education and clinical training, and their well-being by both study arm and level of training. Senior Residents had more concerns about standard policies

AC C

negatively affecting patient safety and continuity of care. In contrast, Interns were more likely to perceive that flexible duty hour policies negatively affect their well-being, morale, and job satisfaction. More experienced residents may have a more informed understanding of the tradeoffs between caring for patients, their education, and their lives outside of the hospital, while Interns may be utilizing duty hour flexibility for mundane tasks. Nonetheless, only a minority of residents overall would prefer to train under standard duty hour policies. Thus, it

20

ACCEPTED MANUSCRIPT

would seem that residents recognize value in training and caring for patients under flexible duty hours. The medical community needs to continue to gain a better understanding of how we can best create systems and duty hour policies that help residents manage the tradeoffs in these areas

RI PT

in order to better inform decision-making regarding duty hour restrictions for trainees.

ACKNOWLEDGEMENT

SC

The authors would like to thank all those who contributed to the administration and execution of the trial: Jonathan Fryer, MD, Anne Grace, PhD, Julie K Johnson, PhD, Lindsey J Kreutzer,

M AN U

MPH, Shari Meyerson, MD, Emily S Pavey, MA, Sean Perry, JD, Alfred Rademaker, PhD, and Ravi Rajaram, MD, Larry V Hedges, PhD, Remi Love, BS, David M Mahvi, MD, David D Odell, MD, Jonah J Stulberg, MD, PhD, MPH; Sameera Ali, MPH, Amy Hart, BS, Emma Malloy, BA, Brian Matel, BA, Craig Miller, BSEE, Lynn Modla, MS, Ajit Sachdeva, MD, and

TE D

Lynn Zhou, PhD, Mark E Cohen, PhD, Clifford Y Ko., MD, David B Hoyt, MD (American College of Surgeons); James Hebert, MD (University of Vermont); Michael Englesbe, MD, MPH, and Paul Gauger, MD (University of Michigan); Christine V Kinnier, MD (Massachusetts

EP

General Hospital); Joseph Cofer, MD (University of Tennessee, Chattanooga); John D Mellinger, MD, (Southern Illinois University) Mitchell Posner, MD (University of Chicago);

AC C

Eugene Foley, MD (University of Wisconsin); Thomas Louis, PhD (Johns Hopkins), and Andrew Jones, PhD (American Board of Surgery); John L Tarpley, MD (Vanderbilt), Rebecca Miller, MS, Thomas Nasca, MD, and John Potts, MD (ACGME); Margaret M Class (Defense Health Agency); all of the surgeon champions and surgical clinical reviewers at the 151 participating American College of Surgeons NSQIP hospitals; and all of the program directors and program coordinators at the 117 participating general surgery residency programs.

21

ACCEPTED MANUSCRIPT

REFERENCES 1.

FIRST Trial Study Protcol and Statistical Analysis Plan. http://www.thefirsttrial.org/Documents/FIRSTTRIAL_StatisticalAnalysisPlan_Updated0

2.

RI PT

3NOV2015.pdf Last accessed 12/1/2015.

Flexibility in Duty Hour Requirements for Surgical Trainees Trial: The FIRST Trial. www.thefirsttrial.org. Accessed 1/15/2015.

Bilimoria KY, Chung JW, Hedges LV, et al. Development of the Flexibility in Duty Hour

SC

3.

Requirements for Surgical Trainees (FIRST) trial protocol: a national cluster-randomized

4.

M AN U

trial of resident duty hour policies. JAMA Surg 2016;151:273-281.

Bilimoria KY, Chung JW, Hedges LV, et al. National cluster-randomized trial of dutyhour flexibility in surgical training. N Engl J Med 2016.

5.

Drolet BC, Christopher DA, Fischer SA. Residents' response to duty-hour regulations--a

6.

TE D

follow-up national survey. N Engl J Med 2012;366:e35.

Drolet BC, Sangisetty S, Tracy TF, Cioffi WG. Surgical residents' perceptions of 2011 Accreditation Council for Graduate Medical Education duty hour regulations. JAMA

7.

EP

Surg 2013;148:427-433.

Drolet BC, Soh IY, Shultz PA, Fischer SA. A thematic review of resident commentary on

8.

AC C

duty hours and supervision regulations. J Grad Med Educ 2012;4:454-459.

Drolet BC, Spalluto LB, Fischer SA. Residents' perspectives on ACGME regulation of

supervision and duty hours--a national survey. N Engl J Med 2010;363:e34.

9.

Drolet BC, Spalluto LB, Zuckerman M, McDonnell M. New ACGME rules for

supervision and duty hours: resident commentary. Med Health RI 2011;94:167, 171-162.

22

ACCEPTED MANUSCRIPT

10.

Fargen KM, Chakraborty A, Friedman WA. Results of a national neurosurgery resident survey on duty hour regulations. Neurosurgery 2011;69:1162-1170.

11.

Bilimoria KY, Hoyt DB, Lewis F. Making the Case for Investigating Flexibility in Duty

12.

RI PT

Hour Limits for Surgical Residents. JAMA Surg 2015;150:503-504.

ACGME Redlined Duty Hour Requirements for FIRST Trial Intervention Hospitals. http://www.thefirsttrial.org/Documents/Redlined%20Duty%20Hour%20Requirements%2

13.

SC

0for%20Intervention%20Arm%20Hospitals.pdf Last accessed 12/1/2015. The American Board of Surgery In Training Exam (ABSITE).

14.

M AN U

http://www.absurgery.org/default.jsp?examoffered_g. .Assessed 12.1.2015. Minami CA, Odell, D.D., . Eithical Considerations in the Development of the Flexibility in Duty Hour Requirements for Surgical Trainees Trial. JAMA Surg 2016; Published online October 12, 2016.

EP

LP.

TE D

StataCorp. 2015. Stata Statistical Software: Release 13. College Station, TX: StataCorp

AC C

15.

23

ACCEPTED MANUSCRIPT

TABLE 1. Comparison of Resident Characteristics by Study Arm

1,737 (40.12) 2,593 (59.88)

866 (39.01) 1,354 (60.99)

871 (41.28) 1,239 (58.72)

0.23*

1,156 (26.70) 1,081 (24.97) 872 (20.14) 628 (14.50) 593 (13.70)

616 (27.75) 554 (24.95) 438 (19.73) 313 (14.10) 299 (13.47)

540 (25.59) 527 (24.98) 434 (20.57) 315 (14.93) 294 (13.93)

0.57*

3,699 (85.43) 621 (14.34) 10 (0.23)

1,874 (84.41) 340 (15.32) 6 (0.27)

1,825 (86.49) 281 (13.32) 4 (0.18)

0.73*

M AN U

Residents, n Gender, n (%) Female Male Postgraduate year, n (%) PGY 1 PGY-2 PGY-3 PGY-4 PGY-5 Resident type, n (%) Categorical Preliminary Other

RI PT

Flexible policy arm 2,110

p Value

4,330

Standard policy arm 2,220

All programs

SC

Resident characteristics

AC C

EP

TE D

*Two-tailed Chi-square test.

24

ACCEPTED MANUSCRIPT

TABLE 2. Resident Perceptions of Duty Hour Policies by Study Arm and Postgraduate Year Level

Resident outcomes Standard policy

RI PT

Frequency, n (%) Flexible policy

AC C

EP

TE D

M AN U

SC

Percent of residents perceiving a negative effect of institutional duty PGY1 PGY2-3 PGY4-5 PGY1 PGY2-3 hours on* Patient safety 76 (22.75) 179 (26.64) 199 (34.79) 34 (10.86) 81 (12.76) Continuity of care 195 (58.56) 386 (57.36) 351 (61.26) 33 (10.54) 134 (21.07) Time with family and friends 22 (6.61) 65 (9.70) 39 (6.81) 105 (33.65) 145 (22.80) Extracurricular activity time 27 (8.13) 70 (10.42) 37 (6.48) 110 (35.14) 148 (23.31) Own health 14 (4.20) 50 (7.49) 32 (5.60) 67 (21.41) 106 (16.75) Rest 27 (8.11) 72 (10.73) 38 (6.64) 120 (38.34) 155 (24.37) Morale 34 (10.21) 118 (17.53) 123 (21.43) 61 (19.55) 99 (15.57) Job satisfaction 38 (11.41) 98 (14.61) 95 (16.61) 44 (14.01) 76 (11.95) Career satisfaction 13 (3.92) 72 (10.71) 64 (11.19) 31 (9.90) 52 (8.20) Professionalism 14 (4.20) 90 (13.39) 124 (21.64) 23 (7.40) 44 (6.94) Residents who reported being † dissatisfied or very dissatisfied with Education quality 32 (9.73) 81 (12.11) 58 (10.18) 46 (14.94) 61 (9.64) Well-being 40 (12.16) 82 (12.26) 53 (9.28) 50 (16.18) 93 (14.67) Work hour regulations 24 (7.29) 58 (8.67) 51 (8.93) 32 (10.36) 42 (6.64) Work hours/scheduling 36 (10.98) 92 (13.75) 60 (10.53) 47 (15.26) 74 (11.67) Residents who experienced the following at least once in the last ‡ month Leave during operation 62 (17.87) 86 (12.52) 59 (10.03) 22 (6.92) 46 (7.14) Number of residents varies across outcome variable. Preliminary residents have been excluded from these analyses. *Outcomes were dichotomized to contrast perceived negative effect vs. perceived positive effect or no effect. †

Outcomes were dichotomized to contrast dissatisfied/very dissatisfied vs. satisfied/very satisfied or neutral.

25

PGY4-5 73 (12.70) 148 (25.61) 86 (14.98) 93 (16.20) 72 (12.54) 91 (15.85) 71 (12.31) 47 (8.19) 35 (6.12) 53 (9.17) 49 (8.57) 62 (10.86) 31 (5.43) 43 (7.53)

35 (5.91)

ACCEPTED MANUSCRIPT

‡

Frequency of clinical disruptions were dichotomized to contrast 1 or more times vs. 0 times.

AC C

EP

TE D

M AN U

SC

RI PT

PGY, postgraduate year.

26

ACCEPTED MANUSCRIPT

Table 3. Postgraduate Year Subgroup Effects of Assignment to Flexible Duty Hours on Resident-Reported

RI PT

Perceptions of the Effect of Institutional Duty Hour Policies on Measures of Patient Safety, Continuity of Care, Education, and Well-Being

SC

PGY x flexible interaction overall significance, p Value

M AN U

TE D

-0.870 (-1.341- -0.400) -2.689 (-3.181- -2.198) 2.101 (1.541-2.661)

<0.001 <0.001 <0.001

-0.902 (-1.235 - -0.570) -1.776 (-2.116- -1.436) 1.090 (0.683-1.496)

<0.001 <0.001 <0.001

-1.307 (-1.649- -0.965) -1.674 (-2.019- -1.329) 0.954 (0.479-1.428)

<0.001 <0.001 <0.001

<0.001 <0.001 <0.001

1.948 (1.425-2.471)

<0.001

1.053 (0.660-1.446)

<0.001

1.118 (0.648-1.588)

<0.001

<0.001

1.940 (1.278-2.602) 2.131 (1.607-2.655) 0.854 (0.359-1.348) 0.291 (-0.208-0.789) 1.030 (0.345-1.714) 0.687 (-0.032-1.407)

<0.001 <0.001 0.001 0.253 0.003 0.061

0.981 (0.535-1.427) 1.100 (0.705-1.490) -0.082 (-0.431-0.267) -0.184 (-0.550-0.181) -0.275 (-0.677-0.126) -0.684 (-1.119- -0.249)

<0.001 <0.001 0.645 0.323 0.179 0.002

0.945 (0.433-1.456) 1.061 (0.591-1.531) -0.615 (-0.986- -0.243) -0.769 (-1.179- -0.359) -0.640 (-1.093- -0.187) -0.963 (-1.370- -0.557)

<0.001 <0.001 0.001 <0.001 0.006 <0.001

<0.001 <0.001 <0.001 0.001 <0.001 <0.001

EP

Effect of assignment to flexible policy (vs standard policy) on log odds of perceiving a negative effect of institutional duty hours on* Patient safety Continuity of care Time with family and friends Extracurricular activity time Own health Rest Morale Job satisfaction Career satisfaction Professionalism Effect of assignment to flexible policy (vs standard policy) on log odds of being dissatisfied † or very dissatisfied with

AC C

Resident outcomes

Treatment effects within PGY subgroups PGY 1, flexible vs standard PGY2-3, flexible vs standard PGY4-5, flexible vs standard p p p Unexponentiated Unexponentiated Unexponentiated Value Value Value coefficient coefficient coefficient (95% confidence (95% confidence (95% Confidence interval) interval) interval)

27

ACCEPTED MANUSCRIPT

Work hours/scheduling Effect of assignment to flexible policy (vs standard policy) on log odds the following events having occurred at least ‡ once in the last month Leave during operation

0.409 (-0.110-0.928)

0.123

0.478 (-0.128-1.084)

0.122

0.430 (-0.099-0.959)

0.111

-0.213 (-0.6710.245) 0.257 (-0.1490.663) -0.223 (-0.6980.252) -0.161 (-0.5680.246)

0.362 0.215 0.358

RI PT

Work hour regulations

0.045

0.438

-0.156 (-0.6530.342) 0.222 (-0.2410.684) -0.473 (-0.9920.045) -0.347 (-0.8240.131)

0.540

0.066

0.348

0.355

0.074

0.061

0.155

0.092

SC

Well-being

-0.583 (-0.014-1.152)

-1.136 (-1.726- -0.546)

<0.001

M AN U

Education quality

-0.694 (-1.165- -0.224)

0.004

-0.634 -1.154- -0.113)

0.017

Number varies across resident outcomes. Estimates are unexponentiated coefficients from 2-level hierarchical logistic regression models that

TE D

regress outcomes on study arm assignment (flexible vs standard [reference]) and PGY level (PGY1 [reference], PGY2-3, PGY4-5). All models controlled for program-level tertile of 2013 30-d postoperative death/serious morbidity (stratifying variable) and included program-level random

subgroup and study arm assignment ≠ 0.

EP

intercepts. Entries in each cell represent the treatment effect within each PGY subgroup. Overall F-test tests that all interactions between

*Outcome variables were dichotomized to contrast perceived negative effect vs. perceived positive effect or no effect. Outcome variables were dichotomized to contrast dissatisfied/very dissatisfied vs. satisfied/very satisfied or neutral.

‡

Frequency of clinical disruptions were were dichotomized to contrast 1 or more times vs. 0 times.

AC C

†

28

<0.001

ACCEPTED MANUSCRIPT

FIGURE LEGENDS

Figure 1. Comparison of the proportion of residents who would prefer or are neutral to

AC C

EP

TE D

M AN U

SC

RI PT

training in a program with flexible duty hours by study arm and postgraduate year group.

29

ACCEPTED MANUSCRIPT

PRECIS Duty hour flexibility in the FIRST trial was associated with striking differences in residents’ perceptions of patient safety, continuity of care, resident education, clinical training, and resident

RI PT

well-being that differed considerably by level of training and type of duty hour policies (standard

AC C

EP

TE D

M AN U

SC

or flexible) under which they trained.

30

AC C

EP

TE D

M AN U

SC

RI PT

ACCEPTED MANUSCRIPT