Resident Perspectives of Virtual Simulation in an Online Complex Care Curriculum

ACADEMIC PEDIATRICS BACKGROUND: The AAP released its first policy statement on community pediatrics in 1999. There has since been increased emphasis on community pediatrics and advocacy (CP&A) training in residency. The most recent study of CP &A teaching was published in 2005, prior to a program requirement for CP&A teaching. OBJECTIVE: Determine how CP&A is being taught and evaluated in US Pediatric Residency programs and if there is variability by program location or size. METHODS: A web-based, APPD-approved survey sent through the Program Directors listserv in September 2014 asked about teaching and evaluation of 10 community pediatrics topics. RESULTS: Of 85 programs (43% response rate), 30% offer a separate training track and/or 6-Block Individualized Curriculum in community pediatrics or advocacy. More than 75% require all residents to learn about 7/10 CP&A topics queried, with provision of culturally effective care being the most common topic required for all residents (92.94%). Public speaking on behalf of children’s health, collection of population level data and community-based research are required of all residents in <1/3 of programs and are not taught at all in w20% of programs. Respondents in urban settings were significantly more likely to teach care of special populations (p¼0.018) and public speaking (p¼0.002). Larger programs were more likely to teach (p¼0.048) and evaluate (p¼0.041) community-based research. Experiential learning and classroom-based didactics were the most common teaching methodologies reported. Many programs used multiple teaching methodologies for all topics. Observation was the most common evaluation technique used; portfolio review and written reflection were also commonly reported. CONCLUSION: There is a continued strong emphasis on CP&A training among pediatric residency programs in the US, with >75% of programs requiring all their residents to learn about 7/10 community pediatrics topics. Although respondents report a variety of teaching and evaluation methods, there are few statistically significant differences between programs. 29. RELIABILITY OF A TOOL TO MEASURE CULTURE SHOCK IN MEDICAL TRAINEES - PROJECT PRIME (PSYCHOSOCIAL RESPONSE TO INTERNATIONAL MEDICAL ELECTIVES) PILOT STUDY Vanessa C. McFadden, MD, PhD, Sabrina Butteris, MD, Tifany Frazer, MPH, Ashley Hines, Zahra Ismail, Jacquelyn Kuzminski, MD, Melodee Nugent, MA, Pippa Simpson, PhD, Samantha Wilson, PhD, Nicole E. St Clair, MD, Medical College of Wisconsin Affiliated Hospitals, Milwaukee, WI BACKGROUND: Trainees with global health (GH) interests often participate in a GH elective, a clinical experience in an international setting. Such electives occur within markedly different cultural, ethical, economic and clinical paradigms, resulting in intense immersion experiences. There is minimal data pertaining to how medical trainees experience culture shock during GH electives; however, GH educators have anecdotally noted culture shock as a frequent experience for medical trainees, with varying degrees of negative impact on the training experience. Improved understanding of trainee culture shock would inform pre-travel preparation and on-site support, would maximize the benefit of trainee participation during the elective and, ideally, would enhance their desire to address health disparities throughout their career through a positive global training experience. Our objective was to assess the reliability of the Culture Shock Profile

ABSTRACTS

e13

(CSP) (Zapf, Social Work, 1993) to quantify the degree of culture shock experienced by medical trainees during a one month GH elective. METHODS: Three residents and 9 medical students participated in a pilot study. Participants completed a pre-travel survey including demographic information and Resilience Assessment (CD-RISC 10). Participants were then prompted to complete the CSP Questionnaire every 4 days during their GH elective. RESULTS: The Resilience Assessments and CSP Scores demonstrated good-to-excellent reliability as assessment tools. Pearson correlations showed a significant inverse relationship between resilience and culture shock for days 6 (R¼0.77) and 22 (R¼0.83). CONCLUSIONS: The pilot study demonstrated good reliability and feasibility of the assessment tools and supported an inverse relationship between trainee resilience and culture shock severity. A larger study using this assessment tool is warranted and will allow the delineation of culture shock patterns, along with identification of factors that influence the severity of culture shock experienced by medical trainees during a GH elective. 30. RESIDENT PERSPECTIVES OF VIRTUAL SIMULATION IN AN ONLINE COMPLEX CARE CURRICULUM Keri Toner, MD, Dewesh Agrawal, MD, Priti Bhansali, MD, Neha Shah, MD, Children’s National Medical Center, Washington, DC BACKGROUND: There are a growing number of children with medical complexity (CMC). These children may be dependent on devices such as G tubes, VP shunts, or tracheostomies and have unique needs. Resident physicians are involved in the care of these patients but may have less exposure to these devices. CMC serve as a special population for which virtual simulation (VS) may serve a beneficial role in pediatric residency medical education. While simulation is a widely recognized training tool in graduate medical education, VS is relatively unique in this context. OBJECTIVE: To ascertain resident perceptions toward the use of VS for learning to care for CMC with a medical device-related emergency DESIGN/METHODS: This was a cross-sectional survey of pediatric residents enrolled in a national randomized study evaluating an online curriculum on CMC. Pediatric residents were asked to complete the curriculum and associated assessments, which included VS. Each participant was asked to complete one of three VS scenarios of an emergency related to medical device malfunction. In the VS scenario, participants were provided with scenes where information was presented with audio and video inputs, prompting actions to address the acute problem. Survey outcomes included assessment of resident experience and perceptions toward VS, and confidence in addressing medical device malfunction after participating in the VS. RESULTS: 86 pediatric or medicine-pediatric residents from 20 pediatric residency programs across the nation completed the survey. Of the respondents, 77% have participated in live simulations in residency. However, only 42% have participated in VS. 93% agree or strongly agree that simulation enhances learning methods in residency, and 94% agree or strongly agree that VS would be a useful supplement to live simulations. Regarding CMC, 88% agree or strongly agree that VS would be helpful to learn about children with medical devices. CONCLUSIONS: While live simulations are an integrated part of pediatric residency education, less than half of resident participants have utilized VS. VS provides a reality-based, safe learning

e14

ABSTRACTS

ACADEMIC PEDIATRICS

environment, without the resources and personnel required of live simulation. The majority of participants in this study believe that VS is particularly helpful in learning about the growing population of CMC with medical devices. VS is a resource that from the learners’ perspective would be a beneficial adjunct to current methods of residency education. 31. NATIONAL TRENDS IN PROCEDURAL TRAINING IN PEDIATRIC RESIDENCY Allison M. Whalen, Catherine D. Michelson, MD, Children’s Hospital/Boston Medical Center, Boston, MA BACKGROUND: The Accreditation Council for Graduate Medical Education mandates procedural training in pediatric residency. Over the years, there have been decreases in the number of procedures performed by residents. Simulation has augmented residents exposure to procedures, but resident experience, even within the same program, remains variable. There is also little known about methods being used to deliver and assess procedural training across residency programs. OBJECTIVE: Describe national trends in procedural training across pediatric residency programs. METHODS: Pediatric program directors completed anonymous surveys about procedure curricula and assessment methods used in their programs, their own perceptions about the importance of specific procedures, and the level of supervision residents from their program attained prior to graduation. Descriptive statistics were used to analyze results. RESULTS: 128 pediatric program directors completed the survey; 22% represented small programs (<30 residents) and 27% represented large programs (>60 residents). 98% of respondents reported using computerized logs to track resident procedures. Assessment by real-time verbal feedback was used by 85% of programs while only 25% used a written assessment tool. 72% of programs discussed progress towards procedural training in milestone meetings. The perceived importance of training was procedure-specific, with 87% of respondents believing bag mask ventilation was important for all pediatric residents and only 1.3% believing that arterial line placement was important for all residents. There was also variability in level of supervision attained by graduating residents with fewer than 1% of graduating residents able to perform thoracentesis and 90% able to perform lumbar puncture without supervision. CONCLUSIONS: While the majority of pediatric residency programs are consistently tracking residents procedural experience, few programs have established methods for assessing competency. Furthermore, there exist procedure-based differences in

program directors perceptions of importance and level of supervision attained at the time of graduation. 32. DO FELLOWSHIP PROGRAM DIRECTORS (FPD) AND CLINICAL COMPETENCY COMMITTEES (CCC) AGREE IN FELLOW ENTRUSTMENT DECISIONS? Richard B. Mink, MD, MACM, Los Angeles County-Harbor UCLA Medical Center, Torrance, CA, Carol L. Carraccio, MD, MA, American Board of Pediatrics, Chapel Hill, NC, Bruce E. Herman, MD, University of Utah, Salt Lake City, UT, Tandy Aye, MD, Stanford University, Stanford, CA, Jeanne M. Baffa, MD, Jefferson Medical College/duPont Hospital for Children, Wilmington, DE, Patricia R. Chess, MD, University of Rochester, Rochester, NY, Jill J. Fussell, MD, University of Arkansas for Medical Sciences, Little Rock, AZ, Cary G. Sauer, MD, MSc, Emory University, Atlanta, GA, Diane E. Stafford, MD, Boston Children’s Hospital, Boston, MA, Pnina Weiss, MD, Yale-New Haven Medical Center, New Haven, CT, Alan Schwartz, PhD, University of Illinois College of Medicine at Chicago, For the Subspecialty Pediatrics Investigators Network, Chicago, IL INTRODUCTION: Throughout fellowship, FPDs assess fellow performance, including their required level of supervision. However, CCCs now also evaluate fellow progress. We examined the association of the entrustment levels determined by the FPD with that of the CCC for 6 common pediatric subspecialty Entrustable Professional Activities (EPAs). Since both groups evaluate fellows longitudinally, we hypothesized that there would be a strong correlation and minimal bias between their judgments. METHODS: The Subspecialty Pediatrics Investigators Network (SPIN) conducted a multi-subspecialty study in which FPDs and CCCs were asked to separately assign a level of supervision to each of their fellows for 6 common pediatric subspecialty EPAs. A supervision scale specific to these EPAs was created and assessments were made in 2014 (fall) and 2015 (spring). FPDs were asked to complete the evaluations 1 week before the CCC meeting and to indicate if they were a CCC member. For each EPA, the correlation between FPD and CCC assessments was analyzed with Spearman rho and bias was calculated as FPD-CCC values. RESULTS: 209 programs from 14 pediatric subspecialties participated. For the fall and spring, there were 598 and 513 FPDs who were CCC members and 433 and 399 FPDs who were not, respectively. In both periods and for each EPA, there was a strong correlation between the FPD entrustment level and

FPD CCC Member

FPD Not CCC Member

Period

EPA

rho

bias

rho

bias

Fall Fall Fall Fall Fall Fall Fall Spring Spring Spring Spring Spring Spring Spring

Public health Consultation Management Handovers Lead team Lead profession Mean(95%CI) Public health Consultation Management Handovers Lead team Lead profession Mean(95%CI)

0.70 0.77 0.74 0.80 0.79 0.74 0.76(0.72-0.79) 0.79 0.81 0.77 0.80 0.80 0.73 0.78(0.75-0.82)

0.11 -0.01 0.12 0.02 0.03 0.01 0.05(-0.01-0.10) 0.07 0.05 0.07 -0.01 0.07 0.02 0.05(0.01-0.08)

0.63 0.70 0.63 0.64 0.62 0.61 0.64(0.58-0.69) 0.64 0.74 0.58 0.67 0.73 0.53 0.65(0.60-0.71)

0.08 0.02 0.18 0.13 0.16 0.18 0.12(0.06-0.19) 0.05 -0.02 0.01 0.07 0.10 0.03 0.04(0.00-0.08)