- Email: [email protected]
Informed consent training improves surgery resident performance in simulated encounters with standardized patients
The American Journal of Surgery (2015) -, -–-
Informed consent training improves surgery resident performance in simulated encounters with standardized patients Britta M. Thompson, Ph.D., M.S.a,b,*, Rhonda A. Sparks, M.D.c, Jonathan Seavey, M.D.d,e, Michelle D. Wallace, B.S.c, Jeremy Irvan, M.D.d, Alexander R. Raines, M.D.d, Heather McClure, J.D.f, Mikio A. Nihira, M.D.g, Jason S. Lees, M.D.d a
University of Oklahoma College of Medicine, Office of Medical Education and Clinical Skills Education and Testing Center, 941 Stanton L. Young Boulevard, BSEB 115A, Oklahoma City, OK 73104, USA; bPennState Hershey College of Medicine, Hershey, PA, USA; cUniversity of Oklahoma College of Medicine, Clinical Skills Education and Testing Center, Oklahoma City, OK, USA; d Department of Surgery, University of Oklahoma College of Medicine, Oklahoma City, OK, USA; e National Naval Medical Center, Bethesda, MD, USA; fOU Physicians, Professional Liability, Patient Safety & Risk Management, Oklahoma City, OK, USA; gDepartment of Obstetrics and Gynecology, University of Oklahoma College of Medicine, Oklahoma City, OK, USA
KEYWORDS: Informed consent; Resident education; Standardized patients; Communication
Abstract BACKGROUND: Although informed consent is vital to patient–physician communication, little training is provided to surgical trainees. We hypothesized that highlighting critical aspects of informed consent would improve resident performance. METHODS: Eighty (out of 88) surgical postgraduate year 1 surgical residents were randomly assigned to one of the 2 cases (laparoscopic cholecystectomy or ventral herniorrhaphy) and instructed to obtain and document informed consent with a standardized patient (SP) followed by a didactic training session. The residents then obtained and documented informed consent with the other case with the other SP. SPs graded encounters (‘‘Checklist’’); trained raters graded notes. Repeated measures multivariate analysis of variance (MANOVA) was used to determine differences between pre- and post-training and Checklist versus ‘‘Note’’ scores. RESULTS: Statistically significant pre- to post differences for Note (P , .01) and Checklist (P , .01) along with significant differences between Note and Checklist (P , .01) were noted. CONCLUSIONS: Training improved surgery residents’ ability to discuss and document informed consent. Despite this improvement, significant differences between discussion and documentation persisted. Documentation training is a future area for improvement. Ó 2015 Elsevier Inc. All rights reserved.
This project was funded, in part, through OU Physicians. No other support was provided. * Corresponding author. Tel.: 717-531-0003 x281249; fax: 717-531-3925. E-mail address: [email protected] Manuscript received September 12, 2014; revised manuscript November 17, 2014 0002-9610/$ - see front matter Ó 2015 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.amjsurg.2014.12.044
2
The American Journal of Surgery, Vol -, No -, - 2015
Informed consent is the communication between a patient and a physician resulting in a patient’s agreement for a medical intervention. According to the American College of Surgeons, informed consent is a standard of ethical surgical practice that serves to enhance the patient/surgeon relationship, which may result in improved patient care and outcomes.1 During the informed consent process, the physician should explain the illness itself and the natural course of the illness, the proposed procedure, benefits and common risks (including death) of the procedure, alternatives to the procedure (including no surgery), and who will comprise the surgery team.1 In a systematic review, researchers have documented that many physicians do not meet the minimum standards when they conduct informed consent with patients.2 Physicians tend to discuss the procedure but rarely discuss alternatives, risks, and benefits of the procedure. Even when they deliver complete informed consent, they tend to overestimate patient’s comprehension of the information.2,3 Properly conducted and documented informed consent decreases litigation risks. Approximately 40% of all physicians and almost 60% of surgical subspecialists have been sued.4 Root cause analysis of the lawsuits indicates that communication failures between the physician and the patient rather than treatment failures are the provoking factors.5 Analysis of malpractice claims indicates that properly conducted and documented informed consent is associated with decreased indemnity risk.6 A recent review of gastroenterologists found that inadequate documentation was one of the primary reasons that lawsuits were awarded to patients.7 This review showed that limited documentation in the medical record was critical to provide evidence that the informed consent process had occurred.7 Surprisingly, there have been few published studies regarding interventions that improve residents’ skills in communicating and documenting informed consent. Leclercq et al5 reported that informed consent training interventions with faculty and residents were time consuming, costly, and of questionable success. A small pilot study almost a decade ago with only 8 surgical residents showed significant improvements from pre- to post-test in communication skills with standardized patients (SP) (establishing rapport, discussing the surgical diagnosis and procedure, explaining the risks and complications of the procedure, and providing appropriate feedback to the patient).8 Grossman et al9 reported that case-based instruction of ethics (including informed consent) resulted in increased confidence of residents to discuss options with patients. A more recent educational intervention included an online module, a small group discussion with faculty, and 2 SP cases. Data collected from the training included resident skill self-assessment, which significantly increased pre to post.10 However, most of the reported trainings have included only a few participants; none of them evaluated improvements in informed consent communication and documentation skills. We hypothesized that a short intervention for residents highlighting critical aspects of informed consent would improve resident performance in both communication and documentation skills.
Patients and Methods Participants Eighty-eight postgraduate year (PGY) 1 surgical residents (general, orthopedics, otorhinolaryngology, neurosurgery, urology) who matriculated at the University of Oklahoma College of Medicine during the 2009 to 2013 years were eligible to participate in the study. Approval was obtained prior to this study from the University of Oklahoma Health Sciences Center Institutional Review Board.
Standardized patient cases and training intervention Two SP cases were developed by board-certified surgeons (J.S.L. and M.A.N.), a board-certified family medicine practitioner (R.A.S.), and an attorney specializing in professional liability and healthcare risk management (H.M.). The SP cases (laparoscopic cholecystectomy and ventral herniorrhaphy) were chosen based on commonly performed surgical procedures and risks of complications for the surgery. Both cases were 15 minutes in length for the encounter followed by 10 minutes for postencounter documentation. At pretraining, approximately half of the residents were randomly assigned to the ventral herniorrhaphy case, while the other half were assigned the cholecystectomy case. The case was followed by a didactic training session (approximately 1.5 hours) with an attorney specializing in professional liability and healthcare risk management (H.M.) and the surgery residency program director (J.S.L.). This didactic training with targeted discussion covered basic tenants of malpractice with emphasis on informed consent. After the training (post-training), residents completed the second SP case. We determined direct costs for the training and the 2 cases. We also estimated the amount of time needed for the resident training. To determine the costs associated with the training, we used publically available mean estimates of a mid-career general attorney ($116/hour) (www.salary.com) and mean compensation rates for a general surgeon ($142/ hour).11 To determine the cost of the cases, we used publically available mean estimates for SPs.12 These costs were not inclusive of benefits. We also reported the amount of time required for residents to complete the training.
Instrument development and scoring After an extensive review of the literature, we noted no existing informed consent cases or checklists. Therefore, we created each SP performance checklist based on the American College of Surgeons (2008) guidelines1 for obtaining informed consent, along with experts in the field (J.S.L. and R.A.S.). Each of the items was dichotomously scored (0 5 did not discuss, 1 5 discussed) in accordance with published recommendations for creation of performance tests.13 The checklist covered areas such as nature of the illness and the natural
B.M. Thompson et al.
Informed consent training improves performance
consequences of no treatment, the proposed operation, risks and benefits, and alternative forms of treatment including nonoperative techniques. Representative items included ‘‘The clinician explained the problem with my gallbladder.’’ The post ‘‘Notes’’ were scored in the same manner using the same items (0 5 not included in note, 1 5 included in note). For this study, we were interested in whether the resident had documented each item, not whether they covered an exhaustive list of benefits, risks, alternatives, and so on. Criterion was determined in advance regarding the specific items that were given credit. To ensure consistency, SPs were trained by first having them read through the case. Then, the SPs were extensively trained using role play. We used the same SPs for the cases to ensure consistency, and used SPs who had over 1,000 hours of experience. To ensure consistency between Note ratings, a surgical educator (J.S.L.) and a medical student (J.S.) first independently rated 6 notes from the 2009 to 2010 matriculants. The raters met and reviewed all note grading. Where there were discrepancies in ratings, they came to consensus. The rater agreement for Note grading on each of the 6 post notes was 90.9%, and Kappa was .82, or almost perfect agreement according to published standards.14 A note rating guide was created by the 2 raters and was used by another investigator (B.M.T.) to grade the notes from the 2011 to 2013 matriculants.
and inferential statistics (repeated measures multivariate analysis of variance) to determine differences pre to post and between Note and Checklist scores and between matriculant cohort years using IBM SPSS 22.0 (IBM Corp, Armonk, NY). We set alpha at .05 and determined effect size (educational significance) using h2; we used recommendations of h2 5 .01 as a small effect, h2 5 .06 as a medium effect size, and h2 5 .16 and above as a large effect.15
Results Eighty of the 88 surgery residents at the University of Oklahoma College of Medicine completed both pre- and post-training for this study, for a completion rate of 90.91%. All residents were PGY1. Analysis of the Checklist summed scores from pre- to post-training indicated that at pretraining, residents discussed an average of 6.85 (standard deviation [SD] 51.16) of the 8 items and documented an average of 4.80 (SD 5 1.52) items. At post-training, they discussed an average of 7.40 (SD 5 .89) items and documented an average of 5.89 (SD 5 1.56) items (Fig. 1). Multivariate analysis of variance indicated statistically significant differences overall (P , .001). Checklist (F 5 14.73, P , .01, h2 5 .16) and Note documentation (F 5 33.31, P , .01, h2 5 .30) statistically significantly increased from pre- to post-training. We noted no differences between the 5 matriculant cohorts at pretraining for either Checklist or Note (P . .05). We examined differences between what residents discussed with the SPs as indicated by the Checklist and what they documented in the Note. At pretraining, residents discussed significantly more items (Checklist) than they
Statistics To determine differences between pre- and posttraining, we analyzed each checklist item separately and then combined items to create a summed score for the checklist rated by the SPs (‘‘Checklist’’) and the written post-note (Note). We analyzed our data using descriptive
8
7.4 6.85
7 5.89
Number of Items
6 5
3
4.8
4
Pre Training Encounter
3
Post Training Encounter
2 1 0 Note (p<0.01)
Checklist (p<0.01)
Figure 1 Mean pre- and post-training results for surgical residents performing informed consent with a standardized patient. Numbers represent mean number of items discussed during the encounter (Checklist) and documented in the note (Note) out of 8 items.
4
The American Journal of Surgery, Vol -, No -, - 2015
documented in the Note (P , .001, h2 5 .63); the same trend continued at post-training (P , .001, h2 5 .48). Table 1 indicates the percentage of residents who discussed and documented each of the informed consent items. The training required approximately 1.5 hours of a malpractice defense attorney’s time along with 1.5 hours of a surgical faculty member. Each resident spent approximately 2.5 hours in training (1.5 hours in training, 50 minutes in SP cases). Analysis indicated SP costs for 18 residents per session of $13.33. We also estimated the total cost of a general surgeon and mid-career attorney at $174.15 and $212.73 for the didactic training and discussion, respectively. For 18 residents per session, the cost of SPs, an attorney, and a general surgeon totaled $34.83 per resident (not inclusive of benefits).
approximately $34.83 per resident. Although faculty, attorney, and SP costs may vary, this training is cost effective if it can improve communication with patients regarding informed consent as well as potentially prevent lawsuits. In addition, our training required less than 2.5 hours of resident time, suggesting that this intervention was feasible to implement, maintain, and score. Our study indicated that residents documented significantly fewer items than they discussed with patients, both before and after the training. Most or all residents (.75%) discussed the disease itself, the risks and benefits of the surgical procedure, specifics of the procedure, and asked the patient if they had any questions. Far fewer discussed items such as the alternatives, natural course of the disease, and the surgical team at pretraining. Most residents (.75%) documented discussing the risks, benefits, and alternatives of the procedure; however, we noted an almost 20% to 50% difference between discussion and notation of the pathology of the disease, specifics of the proposed procedure, course of the disease, the surgical team involved in the surgery, and patient questions. While our results at pretraining appear better than those reported approximately a decade ago,3 the results still suggest the need for training. Our results showed no significant differences between the 5 years of matriculants. This finding suggests that as a training program, we did not improve during the 5 years of the study on our informal education regarding informed consent to the PGY1s. Importantly, few, if any, publications have reported documentation of informed consent. Our pretraining results clearly showed a need for training, especially in the area of documentation. In our study, residents’ documentation of the informed consent process improved from pre- to post-training. Although our training touched on the importance of documentation, it did not specifically address how to document the informed consent process, which may explain why far fewer residents documented informed consent. Our study has several limitations. First, our study was conducted at only one institution with first year residents over a 5-year period. In the future, this study should be replicated by other programs to determine if our results are
Comments The purpose of this study was to determine whether a training program highlighting critical aspects of the informed consent process could improve resident performance in obtaining informed consent. Our study is one of the first to evaluate resident performance for both discussing and documenting informed consent with patients. We noted statistically significant changes in the number of informed consent items documented in the Note and discussed with the SP (Checklist) when comparing pre- to post-training gains. Very little has been published to guide educators on how to train physicians to obtain informed consent from patients. A pilot study about a decade ago suggested that the use of SPs could result in positive gains for residents when talking with patients about informed consent.8 A more recent study showed that SPs improved residents’ self-assessment ratings of their informed consent skills. However, the authors of this study reported the intervention to be costly in terms of faculty time and direct SP costs.10 They indicated that the small group discussions were time intensive for faculty. In addition, SP costs of $7,000 for 138 residents (or over $50/resident) were incurred (not inclusive of faculty costs). We calculated direct SP costs of
Table 1 Percent residents (n 5 80) who discussed specific informed consent items with a standardized patient (as recorded by the standardized patient) and who documented discussing the items in the note Note % Documenting Explained pathology Discussed procedure Discussed benefits (including resolution of symptoms) Discussed alternatives (including no surgery) Discussed risk of procedure (including death) Discussed course of disease without surgery Discussed surgery team Asked patients for questions
Checklist % Discussing
Pre
Post
Pre
Post
42.50 68.75 80.00 81.25 96.25 43.75 20.00 47.50
50.00 82.50 87.50 92.50 100.00 68.75 32.50 75.00
90.00 95.00 91.25 78.75 100.00 67.50 65.00 97.50
91.25 98.75 100.00 91.25 98.75 93.75 66.25 100.00
B.M. Thompson et al.
Informed consent training improves performance
generalizable. It would also be informative to follow-up with the residents at a later date to determine retention of the skills learned and if the skills were used in actual patient care practice. It would be informative to determine how frequently the training should be repeated for full benefit.
Conclusions In conclusion, our data suggests that informed consent training using SPs and a structured training lead to improved resident ability to discuss and document the informed consent process. Despite the training, a discrepancy persisted between what was discussed and what was documented in the note. We feel that this discrepancy revealed an area of training, which needs to be improved and will be addressed in future informed consent training sessions. Because our protocol had moderate costs associated with it and required limited faculty and resident time, we feel this could be replicated in other schools throughout the country.
References 1. American College of Surgeons. Statement on Principles Available at: http:// www.facs.org/fellows.info/statements/stoneprin.html#anchor171960; 2008. Accessed June 26, 2014. 2. Hall DE, Prochazka AV, Fink AS. Informed consent for clinical treatment. Can Med Assoc J 2012;184:533–40. 3. Braddock III CH, Edwards KA, Hasenberg NM, et al. Informed decision making in outpatient practice: time to get back to basics. JAMA 1999;282:2313–20.
5
4. Kane CK. Medical Liability Claim Frequency: A 2007-2008 Snapshot of Physicians. American Medical Association; August 2010. Available at: www.ama-assn.org/ama1/pub/upload/mm/363/prp-201001-claim-freq.pdf. Accessed June 1, 2015. 5. Leclercq WK, Keulers BJ, Scheltinga MRM, et al. A review of surgical informed consent: past, present, and future. A quest to help patients make better decisions. World J Surg 2010;34:1406–15. 6. Bhattacharya T, Yeon H, Harris MB. The medical-legal aspects of informed consent in orthopaedic surgery. J Bone Joint Surg 2005;87: 2395–400. 7. Karim Z, Alkhowaiter SS, Bressler B, et al. Informed consent and its documentation: implications for medical malpractice liability. Br Col Med J 2013;55:376–9. 8. Leeper-Majors K, Veale JR, Westbrook TS, et al. The effect of standardized patient feedback in teaching surgical residents informed consent: results of a pilot study. Curr Surg 2003;60:615–22. 9. Grossman E, Posner MC, Angelos P. Ethics education in surgical residency: past, present, and future. Surgery 2010;147:114–9. 10. Joyce BL, Scher E, Steenbergh T, et al. Development of an institutional resident curriculum in communication skills. J Grad Med Educ 2011;3:524–8. 11. Kane L, Peckham C. Medscape Physician Compensation Report. Available at: http://www.medscape.com/features/slideshow/compensation/ 2014/public/overview; 2014. Accessed June 1, 2015. 12. Howley LD, Gliva-McConvey G, Thorton J. Standardized patient practices: initial report on the survey of US and Canadian medical schools. Med Educ Online 2009;14:7. 13. Yudkowsky R. Performance tests. In: Downing SM, Yudkowsk R, editors. Assessment in Health Professions Education. New York: Routledge Taylor and Francis Group; 2009. p. 217–43. 14. Landix JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33:159–74. 15. Kline RB. Beyond Significance Testing: Reforming Data Analysis Methods in Behavioral Sciences. Washington, DC: American Psychological Association; 2013.
The American Journal of Surgery, Vol -, No -, - 2015
6
Appendix A Standardized patient checklists for cases
Case: Hernia in Abdomen During the history, the resident explained the problem with my abdominal wall weakening in the muscles/wall of abdomen. During the history, the resident discussed the procedure repairing the defect in the abdominal wall using ‘‘mesh’’ or material to close the defect. During the history, the resident discussed the benefits of the surgery- (select all that apply) During the history, the resident discussed the alternatives(select all that apply) During the history, the resident discussed the following risks of surgery- (select all that apply)
During the history, the resident explained what would likely happen if you do not have the surgery- (select all that apply) During the history, the resident told me who would be involved in performing the operation discussed teaching hospital environment. During the history, the resident asked if I had any questions about the procedure.
[ ] Yes
[ ] No
[ ] Yes
[ ] No
[ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [
[ ] No
] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ]
resolution of symptoms other (note benefit in comments) DID NOT DISCUSS no surgery weight loss other (note alternatives in comments) DID NOT DISCUSS seroma wound infection injury to intra-abdominal structures entero-cutaneous fistula intestinal blockage recurrent hernia mesh infection blood clots death other (note risks in comments) DID NOT DISCUSS risk of enlargement risk of incarceration (‘‘stuck bowel’’) other (note in comments) DID NOT DISCUSS Yes
[ ] Yes
[ ] No
[ ] Yes
[ ] No
[ ] Yes
[ ] No
Case: Pain in Right Side of Stomach During the history, the resident explained the problem with my gallbladder. During the history, the resident discussed the procedure removal of the gallbladderby laparoscopy.
B.M. Thompson et al.
Informed consent training improves performance
7
(continued )
Case: Pain in Right Side of Stomach During the history, the resident discussed the benefits of the surgery- (select all that apply)
During the history, the resident discussed the alternatives(select all that apply) During the history, the resident discussed the following risks of surgery- (select all that apply)
During the history, the resident explained what would likely happen if you do not have the surgery- (select all that apply) During the history, the resident told me who would be involved in performing the operation discussed teaching hospital environment. During the history, the resident asked if I had any questions about the procedure.
[ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [
] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ]
resolution of symptoms no future risk of gallbladder infection no future risk of gallstone decreased risk of pancreatitis other (note benefit in comments) DID NOT DISCUSS no surgery diet change other (note alternatives in comments) DID NOT DISCUSS excessive bleeding need for blood transfusion abdominal wall hernia damage to liver injury to the common bile duct blood clots injury to the intestines death other (note risks in comments) DID NOT DISCUSS pain persists infection of gallbladder other (note in comments) DID NOT DISCUSS Yes
[ ] Yes
[ ] No
[ ] No
Informed consent training improves surgery resident performance in simulated encounters with standardized patients Britta M. Thompson, Ph.D., M.S.a,b,*, Rhonda A. Sparks, M.D.c, Jonathan Seavey, M.D.d,e, Michelle D. Wallace, B.S.c, Jeremy Irvan, M.D.d, Alexander R. Raines, M.D.d, Heather McClure, J.D.f, Mikio A. Nihira, M.D.g, Jason S. Lees, M.D.d a
University of Oklahoma College of Medicine, Office of Medical Education and Clinical Skills Education and Testing Center, 941 Stanton L. Young Boulevard, BSEB 115A, Oklahoma City, OK 73104, USA; bPennState Hershey College of Medicine, Hershey, PA, USA; cUniversity of Oklahoma College of Medicine, Clinical Skills Education and Testing Center, Oklahoma City, OK, USA; d Department of Surgery, University of Oklahoma College of Medicine, Oklahoma City, OK, USA; e National Naval Medical Center, Bethesda, MD, USA; fOU Physicians, Professional Liability, Patient Safety & Risk Management, Oklahoma City, OK, USA; gDepartment of Obstetrics and Gynecology, University of Oklahoma College of Medicine, Oklahoma City, OK, USA
KEYWORDS: Informed consent; Resident education; Standardized patients; Communication
Abstract BACKGROUND: Although informed consent is vital to patient–physician communication, little training is provided to surgical trainees. We hypothesized that highlighting critical aspects of informed consent would improve resident performance. METHODS: Eighty (out of 88) surgical postgraduate year 1 surgical residents were randomly assigned to one of the 2 cases (laparoscopic cholecystectomy or ventral herniorrhaphy) and instructed to obtain and document informed consent with a standardized patient (SP) followed by a didactic training session. The residents then obtained and documented informed consent with the other case with the other SP. SPs graded encounters (‘‘Checklist’’); trained raters graded notes. Repeated measures multivariate analysis of variance (MANOVA) was used to determine differences between pre- and post-training and Checklist versus ‘‘Note’’ scores. RESULTS: Statistically significant pre- to post differences for Note (P , .01) and Checklist (P , .01) along with significant differences between Note and Checklist (P , .01) were noted. CONCLUSIONS: Training improved surgery residents’ ability to discuss and document informed consent. Despite this improvement, significant differences between discussion and documentation persisted. Documentation training is a future area for improvement. Ó 2015 Elsevier Inc. All rights reserved.
This project was funded, in part, through OU Physicians. No other support was provided. * Corresponding author. Tel.: 717-531-0003 x281249; fax: 717-531-3925. E-mail address: [email protected] Manuscript received September 12, 2014; revised manuscript November 17, 2014 0002-9610/$ - see front matter Ó 2015 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.amjsurg.2014.12.044
2
The American Journal of Surgery, Vol -, No -, - 2015
Informed consent is the communication between a patient and a physician resulting in a patient’s agreement for a medical intervention. According to the American College of Surgeons, informed consent is a standard of ethical surgical practice that serves to enhance the patient/surgeon relationship, which may result in improved patient care and outcomes.1 During the informed consent process, the physician should explain the illness itself and the natural course of the illness, the proposed procedure, benefits and common risks (including death) of the procedure, alternatives to the procedure (including no surgery), and who will comprise the surgery team.1 In a systematic review, researchers have documented that many physicians do not meet the minimum standards when they conduct informed consent with patients.2 Physicians tend to discuss the procedure but rarely discuss alternatives, risks, and benefits of the procedure. Even when they deliver complete informed consent, they tend to overestimate patient’s comprehension of the information.2,3 Properly conducted and documented informed consent decreases litigation risks. Approximately 40% of all physicians and almost 60% of surgical subspecialists have been sued.4 Root cause analysis of the lawsuits indicates that communication failures between the physician and the patient rather than treatment failures are the provoking factors.5 Analysis of malpractice claims indicates that properly conducted and documented informed consent is associated with decreased indemnity risk.6 A recent review of gastroenterologists found that inadequate documentation was one of the primary reasons that lawsuits were awarded to patients.7 This review showed that limited documentation in the medical record was critical to provide evidence that the informed consent process had occurred.7 Surprisingly, there have been few published studies regarding interventions that improve residents’ skills in communicating and documenting informed consent. Leclercq et al5 reported that informed consent training interventions with faculty and residents were time consuming, costly, and of questionable success. A small pilot study almost a decade ago with only 8 surgical residents showed significant improvements from pre- to post-test in communication skills with standardized patients (SP) (establishing rapport, discussing the surgical diagnosis and procedure, explaining the risks and complications of the procedure, and providing appropriate feedback to the patient).8 Grossman et al9 reported that case-based instruction of ethics (including informed consent) resulted in increased confidence of residents to discuss options with patients. A more recent educational intervention included an online module, a small group discussion with faculty, and 2 SP cases. Data collected from the training included resident skill self-assessment, which significantly increased pre to post.10 However, most of the reported trainings have included only a few participants; none of them evaluated improvements in informed consent communication and documentation skills. We hypothesized that a short intervention for residents highlighting critical aspects of informed consent would improve resident performance in both communication and documentation skills.
Patients and Methods Participants Eighty-eight postgraduate year (PGY) 1 surgical residents (general, orthopedics, otorhinolaryngology, neurosurgery, urology) who matriculated at the University of Oklahoma College of Medicine during the 2009 to 2013 years were eligible to participate in the study. Approval was obtained prior to this study from the University of Oklahoma Health Sciences Center Institutional Review Board.
Standardized patient cases and training intervention Two SP cases were developed by board-certified surgeons (J.S.L. and M.A.N.), a board-certified family medicine practitioner (R.A.S.), and an attorney specializing in professional liability and healthcare risk management (H.M.). The SP cases (laparoscopic cholecystectomy and ventral herniorrhaphy) were chosen based on commonly performed surgical procedures and risks of complications for the surgery. Both cases were 15 minutes in length for the encounter followed by 10 minutes for postencounter documentation. At pretraining, approximately half of the residents were randomly assigned to the ventral herniorrhaphy case, while the other half were assigned the cholecystectomy case. The case was followed by a didactic training session (approximately 1.5 hours) with an attorney specializing in professional liability and healthcare risk management (H.M.) and the surgery residency program director (J.S.L.). This didactic training with targeted discussion covered basic tenants of malpractice with emphasis on informed consent. After the training (post-training), residents completed the second SP case. We determined direct costs for the training and the 2 cases. We also estimated the amount of time needed for the resident training. To determine the costs associated with the training, we used publically available mean estimates of a mid-career general attorney ($116/hour) (www.salary.com) and mean compensation rates for a general surgeon ($142/ hour).11 To determine the cost of the cases, we used publically available mean estimates for SPs.12 These costs were not inclusive of benefits. We also reported the amount of time required for residents to complete the training.
Instrument development and scoring After an extensive review of the literature, we noted no existing informed consent cases or checklists. Therefore, we created each SP performance checklist based on the American College of Surgeons (2008) guidelines1 for obtaining informed consent, along with experts in the field (J.S.L. and R.A.S.). Each of the items was dichotomously scored (0 5 did not discuss, 1 5 discussed) in accordance with published recommendations for creation of performance tests.13 The checklist covered areas such as nature of the illness and the natural
B.M. Thompson et al.
Informed consent training improves performance
consequences of no treatment, the proposed operation, risks and benefits, and alternative forms of treatment including nonoperative techniques. Representative items included ‘‘The clinician explained the problem with my gallbladder.’’ The post ‘‘Notes’’ were scored in the same manner using the same items (0 5 not included in note, 1 5 included in note). For this study, we were interested in whether the resident had documented each item, not whether they covered an exhaustive list of benefits, risks, alternatives, and so on. Criterion was determined in advance regarding the specific items that were given credit. To ensure consistency, SPs were trained by first having them read through the case. Then, the SPs were extensively trained using role play. We used the same SPs for the cases to ensure consistency, and used SPs who had over 1,000 hours of experience. To ensure consistency between Note ratings, a surgical educator (J.S.L.) and a medical student (J.S.) first independently rated 6 notes from the 2009 to 2010 matriculants. The raters met and reviewed all note grading. Where there were discrepancies in ratings, they came to consensus. The rater agreement for Note grading on each of the 6 post notes was 90.9%, and Kappa was .82, or almost perfect agreement according to published standards.14 A note rating guide was created by the 2 raters and was used by another investigator (B.M.T.) to grade the notes from the 2011 to 2013 matriculants.
and inferential statistics (repeated measures multivariate analysis of variance) to determine differences pre to post and between Note and Checklist scores and between matriculant cohort years using IBM SPSS 22.0 (IBM Corp, Armonk, NY). We set alpha at .05 and determined effect size (educational significance) using h2; we used recommendations of h2 5 .01 as a small effect, h2 5 .06 as a medium effect size, and h2 5 .16 and above as a large effect.15
Results Eighty of the 88 surgery residents at the University of Oklahoma College of Medicine completed both pre- and post-training for this study, for a completion rate of 90.91%. All residents were PGY1. Analysis of the Checklist summed scores from pre- to post-training indicated that at pretraining, residents discussed an average of 6.85 (standard deviation [SD] 51.16) of the 8 items and documented an average of 4.80 (SD 5 1.52) items. At post-training, they discussed an average of 7.40 (SD 5 .89) items and documented an average of 5.89 (SD 5 1.56) items (Fig. 1). Multivariate analysis of variance indicated statistically significant differences overall (P , .001). Checklist (F 5 14.73, P , .01, h2 5 .16) and Note documentation (F 5 33.31, P , .01, h2 5 .30) statistically significantly increased from pre- to post-training. We noted no differences between the 5 matriculant cohorts at pretraining for either Checklist or Note (P . .05). We examined differences between what residents discussed with the SPs as indicated by the Checklist and what they documented in the Note. At pretraining, residents discussed significantly more items (Checklist) than they
Statistics To determine differences between pre- and posttraining, we analyzed each checklist item separately and then combined items to create a summed score for the checklist rated by the SPs (‘‘Checklist’’) and the written post-note (Note). We analyzed our data using descriptive
8
7.4 6.85
7 5.89
Number of Items
6 5
3
4.8
4
Pre Training Encounter
3
Post Training Encounter
2 1 0 Note (p<0.01)
Checklist (p<0.01)
Figure 1 Mean pre- and post-training results for surgical residents performing informed consent with a standardized patient. Numbers represent mean number of items discussed during the encounter (Checklist) and documented in the note (Note) out of 8 items.
4
The American Journal of Surgery, Vol -, No -, - 2015
documented in the Note (P , .001, h2 5 .63); the same trend continued at post-training (P , .001, h2 5 .48). Table 1 indicates the percentage of residents who discussed and documented each of the informed consent items. The training required approximately 1.5 hours of a malpractice defense attorney’s time along with 1.5 hours of a surgical faculty member. Each resident spent approximately 2.5 hours in training (1.5 hours in training, 50 minutes in SP cases). Analysis indicated SP costs for 18 residents per session of $13.33. We also estimated the total cost of a general surgeon and mid-career attorney at $174.15 and $212.73 for the didactic training and discussion, respectively. For 18 residents per session, the cost of SPs, an attorney, and a general surgeon totaled $34.83 per resident (not inclusive of benefits).
approximately $34.83 per resident. Although faculty, attorney, and SP costs may vary, this training is cost effective if it can improve communication with patients regarding informed consent as well as potentially prevent lawsuits. In addition, our training required less than 2.5 hours of resident time, suggesting that this intervention was feasible to implement, maintain, and score. Our study indicated that residents documented significantly fewer items than they discussed with patients, both before and after the training. Most or all residents (.75%) discussed the disease itself, the risks and benefits of the surgical procedure, specifics of the procedure, and asked the patient if they had any questions. Far fewer discussed items such as the alternatives, natural course of the disease, and the surgical team at pretraining. Most residents (.75%) documented discussing the risks, benefits, and alternatives of the procedure; however, we noted an almost 20% to 50% difference between discussion and notation of the pathology of the disease, specifics of the proposed procedure, course of the disease, the surgical team involved in the surgery, and patient questions. While our results at pretraining appear better than those reported approximately a decade ago,3 the results still suggest the need for training. Our results showed no significant differences between the 5 years of matriculants. This finding suggests that as a training program, we did not improve during the 5 years of the study on our informal education regarding informed consent to the PGY1s. Importantly, few, if any, publications have reported documentation of informed consent. Our pretraining results clearly showed a need for training, especially in the area of documentation. In our study, residents’ documentation of the informed consent process improved from pre- to post-training. Although our training touched on the importance of documentation, it did not specifically address how to document the informed consent process, which may explain why far fewer residents documented informed consent. Our study has several limitations. First, our study was conducted at only one institution with first year residents over a 5-year period. In the future, this study should be replicated by other programs to determine if our results are
Comments The purpose of this study was to determine whether a training program highlighting critical aspects of the informed consent process could improve resident performance in obtaining informed consent. Our study is one of the first to evaluate resident performance for both discussing and documenting informed consent with patients. We noted statistically significant changes in the number of informed consent items documented in the Note and discussed with the SP (Checklist) when comparing pre- to post-training gains. Very little has been published to guide educators on how to train physicians to obtain informed consent from patients. A pilot study about a decade ago suggested that the use of SPs could result in positive gains for residents when talking with patients about informed consent.8 A more recent study showed that SPs improved residents’ self-assessment ratings of their informed consent skills. However, the authors of this study reported the intervention to be costly in terms of faculty time and direct SP costs.10 They indicated that the small group discussions were time intensive for faculty. In addition, SP costs of $7,000 for 138 residents (or over $50/resident) were incurred (not inclusive of faculty costs). We calculated direct SP costs of
Table 1 Percent residents (n 5 80) who discussed specific informed consent items with a standardized patient (as recorded by the standardized patient) and who documented discussing the items in the note Note % Documenting Explained pathology Discussed procedure Discussed benefits (including resolution of symptoms) Discussed alternatives (including no surgery) Discussed risk of procedure (including death) Discussed course of disease without surgery Discussed surgery team Asked patients for questions
Checklist % Discussing
Pre
Post
Pre
Post
42.50 68.75 80.00 81.25 96.25 43.75 20.00 47.50
50.00 82.50 87.50 92.50 100.00 68.75 32.50 75.00
90.00 95.00 91.25 78.75 100.00 67.50 65.00 97.50
91.25 98.75 100.00 91.25 98.75 93.75 66.25 100.00
B.M. Thompson et al.
Informed consent training improves performance
generalizable. It would also be informative to follow-up with the residents at a later date to determine retention of the skills learned and if the skills were used in actual patient care practice. It would be informative to determine how frequently the training should be repeated for full benefit.
Conclusions In conclusion, our data suggests that informed consent training using SPs and a structured training lead to improved resident ability to discuss and document the informed consent process. Despite the training, a discrepancy persisted between what was discussed and what was documented in the note. We feel that this discrepancy revealed an area of training, which needs to be improved and will be addressed in future informed consent training sessions. Because our protocol had moderate costs associated with it and required limited faculty and resident time, we feel this could be replicated in other schools throughout the country.
References 1. American College of Surgeons. Statement on Principles Available at: http:// www.facs.org/fellows.info/statements/stoneprin.html#anchor171960; 2008. Accessed June 26, 2014. 2. Hall DE, Prochazka AV, Fink AS. Informed consent for clinical treatment. Can Med Assoc J 2012;184:533–40. 3. Braddock III CH, Edwards KA, Hasenberg NM, et al. Informed decision making in outpatient practice: time to get back to basics. JAMA 1999;282:2313–20.
5
4. Kane CK. Medical Liability Claim Frequency: A 2007-2008 Snapshot of Physicians. American Medical Association; August 2010. Available at: www.ama-assn.org/ama1/pub/upload/mm/363/prp-201001-claim-freq.pdf. Accessed June 1, 2015. 5. Leclercq WK, Keulers BJ, Scheltinga MRM, et al. A review of surgical informed consent: past, present, and future. A quest to help patients make better decisions. World J Surg 2010;34:1406–15. 6. Bhattacharya T, Yeon H, Harris MB. The medical-legal aspects of informed consent in orthopaedic surgery. J Bone Joint Surg 2005;87: 2395–400. 7. Karim Z, Alkhowaiter SS, Bressler B, et al. Informed consent and its documentation: implications for medical malpractice liability. Br Col Med J 2013;55:376–9. 8. Leeper-Majors K, Veale JR, Westbrook TS, et al. The effect of standardized patient feedback in teaching surgical residents informed consent: results of a pilot study. Curr Surg 2003;60:615–22. 9. Grossman E, Posner MC, Angelos P. Ethics education in surgical residency: past, present, and future. Surgery 2010;147:114–9. 10. Joyce BL, Scher E, Steenbergh T, et al. Development of an institutional resident curriculum in communication skills. J Grad Med Educ 2011;3:524–8. 11. Kane L, Peckham C. Medscape Physician Compensation Report. Available at: http://www.medscape.com/features/slideshow/compensation/ 2014/public/overview; 2014. Accessed June 1, 2015. 12. Howley LD, Gliva-McConvey G, Thorton J. Standardized patient practices: initial report on the survey of US and Canadian medical schools. Med Educ Online 2009;14:7. 13. Yudkowsky R. Performance tests. In: Downing SM, Yudkowsk R, editors. Assessment in Health Professions Education. New York: Routledge Taylor and Francis Group; 2009. p. 217–43. 14. Landix JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33:159–74. 15. Kline RB. Beyond Significance Testing: Reforming Data Analysis Methods in Behavioral Sciences. Washington, DC: American Psychological Association; 2013.
The American Journal of Surgery, Vol -, No -, - 2015
6
Appendix A Standardized patient checklists for cases
Case: Hernia in Abdomen During the history, the resident explained the problem with my abdominal wall weakening in the muscles/wall of abdomen. During the history, the resident discussed the procedure repairing the defect in the abdominal wall using ‘‘mesh’’ or material to close the defect. During the history, the resident discussed the benefits of the surgery- (select all that apply) During the history, the resident discussed the alternatives(select all that apply) During the history, the resident discussed the following risks of surgery- (select all that apply)
During the history, the resident explained what would likely happen if you do not have the surgery- (select all that apply) During the history, the resident told me who would be involved in performing the operation discussed teaching hospital environment. During the history, the resident asked if I had any questions about the procedure.
[ ] Yes
[ ] No
[ ] Yes
[ ] No
[ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [
[ ] No
] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ]
resolution of symptoms other (note benefit in comments) DID NOT DISCUSS no surgery weight loss other (note alternatives in comments) DID NOT DISCUSS seroma wound infection injury to intra-abdominal structures entero-cutaneous fistula intestinal blockage recurrent hernia mesh infection blood clots death other (note risks in comments) DID NOT DISCUSS risk of enlargement risk of incarceration (‘‘stuck bowel’’) other (note in comments) DID NOT DISCUSS Yes
[ ] Yes
[ ] No
[ ] Yes
[ ] No
[ ] Yes
[ ] No
Case: Pain in Right Side of Stomach During the history, the resident explained the problem with my gallbladder. During the history, the resident discussed the procedure removal of the gallbladderby laparoscopy.
B.M. Thompson et al.
Informed consent training improves performance
7
(continued )
Case: Pain in Right Side of Stomach During the history, the resident discussed the benefits of the surgery- (select all that apply)
During the history, the resident discussed the alternatives(select all that apply) During the history, the resident discussed the following risks of surgery- (select all that apply)
During the history, the resident explained what would likely happen if you do not have the surgery- (select all that apply) During the history, the resident told me who would be involved in performing the operation discussed teaching hospital environment. During the history, the resident asked if I had any questions about the procedure.
[ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [
] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ]
resolution of symptoms no future risk of gallbladder infection no future risk of gallstone decreased risk of pancreatitis other (note benefit in comments) DID NOT DISCUSS no surgery diet change other (note alternatives in comments) DID NOT DISCUSS excessive bleeding need for blood transfusion abdominal wall hernia damage to liver injury to the common bile duct blood clots injury to the intestines death other (note risks in comments) DID NOT DISCUSS pain persists infection of gallbladder other (note in comments) DID NOT DISCUSS Yes
[ ] Yes
[ ] No
[ ] No