- Email: [email protected]
Core Curricular Priorities in the Care of Children with Medical Complexity: A North American Modified Delphi Study
Journal Pre-proof
Core Curricular Priorities in the Care of Children with Medical Complexity: A North American Modified Delphi Study Kathleen Huth MD, MMSc , Lori Newman MEd , Laurie Glader MD PII: DOI: Reference:
S1876-2859(20)30054-1 https://doi.org/10.1016/j.acap.2020.01.014 ACAP 1477
To appear in:
Academic Pediatrics
Received date: Accepted date:
22 July 2019 31 January 2020
Please cite this article as: Kathleen Huth MD, MMSc , Lori Newman MEd , Laurie Glader MD , Core Curricular Priorities in the Care of Children with Medical Complexity: A North American Modified Delphi Study, Academic Pediatrics (2020), doi: https://doi.org/10.1016/j.acap.2020.01.014
This is a PDF file of an article that has undergone enhancements after acceptance, such as the addition of a cover page and metadata, and formatting for readability, but it is not yet the definitive version of record. This version will undergo additional copyediting, typesetting and review before it is published in its final form, but we are providing this version to give early visibility of the article. Please note that, during the production process, errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain. © 2020 Published by Elsevier Inc. on behalf of Academic Pediatric Association
Title: Core Curricular Priorities in the Care of Children with Medical Complexity: A North American Modified Delphi Study Authors and affiliations: Kathleen Huth, MD, MMSc,1 Lori Newman, MEd,2 Laurie Glader, MD1 1
Department of Pediatrics, 2Department of Medical Education, Boston Children’s Hospital, Boston MA. Corresponding author Kathleen Huth, MD, MMSc Boston Children’s Hospital 300 Longwood Ave, Boston MA, 02115 Tel: 617-355-6162 Email: [email protected] Keywords Complex care, residency education, postgraduate medical education, pediatrics, curriculum development, children with medical complexity Running title Curricular priorities in complex care Word counts Abstract 249/250 Manuscript 3033/3500 Funding sources N/A Conflicts of interest The authors have no conflicts of interest to disclose.
Abstract Background: Previous studies of pediatric residents have identified educational gaps in caring for children with medical complexity (CMC). Training opportunities in complex care vary across residency programs. Defining core curricular topics in complex care is a priority in medical education.
Objective: To identify core topics to include in a standard complex care curriculum for pediatric residents.
Methods: An initial topic list was generated through literature review and proceedings of national meetings. Expert panelists were identified based on experience in complex care and residency education. A modified Delphi method was used to determine group consensus by asking participants to rate the importance of complex care curricular topics for pediatric residents. Consensus was defined as >70% of experts identifying a topic as essential. There was a predetermined maximum of three iterative, electronic survey rounds, with feedback provided to participants between each round.
Results: Sixteen experts participated. Response rate was 100% for all rounds. Experts were from the United States (44%) and Canada (56%); most were affiliated with an academic medical center (96%) involved in both inpatient and outpatient care (69%). Eleven topics were identified as essential across 3 rounds: feeding difficulties, pain/irritability, transition, feeding tube management, difficult discussions, team management/care coordination, dysmotility, aspiration, safety/emergency planning, neuromuscular/skeletal issues and advocacy. Essential topics were
organized according to the International Classification of Functioning, Disability and Health (ICF).
Conclusion: Eleven curricular priorities in complex care were identified across multiple domains of the ICF framework, serving as a guide for standardized curriculum development for future pediatricians.
What’s New We conducted a modified Delphi study to achieve consensus on essential topics in complex care education for pediatric residents. Our findings inform curriculum development in the care of children with medical complexity.
Introduction Pediatricians have a central role in caring for children with medical complexity (CMC), a population characterized by chronic conditions, medical fragility and, often, technology dependence.1 Due to their high healthcare utilization, CMC are cared for in all settings: on the inpatient ward where pediatricians evaluate and manage acute-on-chronic exacerbations of multisystem disease, and in the outpatient setting where pediatricians manage health maintenance, care planning, and coordination, often as part of a larger interprofessional care team.
Currently, pediatric residency programs offer variable educational opportunities in complex care to their trainees. Elective or mandatory clinical rotations can vary in length, setting, and dedicated teaching. Numerous publications have highlighted the necessity for formal training during pediatric residency to support the delivery of safe, evidence-based care for CMC.2–4 Inadequate training in complex care contributes to inconsistent, poor-quality and fragmented care, and means that many families are managing complex medical needs outside of the hospital with minimal support.5,6 Unfortunately, no clear educational priorities in caring for CMC have been established around which to structure learning opportunities and competency assessment. Given the Accreditation Council for Graduate Medical Education’s focus on ensuring graduating pediatrician competency,7 setting standards for curriculum development in the care of this vulnerable patient population is critical. Clinicians with experience caring for CMC and working with pediatric residents would have valuable insight into priorities for formal training in this field.
The purpose of this study was to obtain consensus on essential topics to be included in a standard complex care education for pediatric residents and to inform curriculum development efforts for complex care rotations in pediatric training programs.
Methods Study design We used a modified Delphi method to develop expert group consensus.8,9 This method was chosen as a systematic means of eliciting collective expert opinion in an area where there is limited evidence for effective clinical and educational approaches, and where experts are geographically dispersed.10 The Delphi method has been used to define essential teaching competencies,8,11,12 learning objectives,13 and content areas for training programs.9,14,15 We used previously established methodologic criteria for Delphi studies to ensure quality.16
Participants We selected expert panelists using the following criteria: 1) recognized clinical experience as a pediatrician caring for CMC, and 2) experience working with pediatric residents in a clinical setting. Participants were identified via their active involvement in complex care special interest groups, national conference meetings and/or published scholarship in complex care education. We used purposeful sampling to ensure the expert panel represented different institutions across the United States and Canada, different care settings, and varying years of experience. We anticipated that senior clinicians as well as junior faculty members, having recently completed residency training, would have valuable perspectives on essential curricular topics. We used a snowball technique whereby clinicians identified other potential participants who they
anticipated would have insight into the research question. We recruited 16 experts, consistent with a typical panel size, parametric determination, and study reliability.10,17 Participants indicated their interest in completing all rounds of the process via email prior to receiving the first survey, with the goal of optimizing response rate.
Data collection An initial topic list was generated through an in-depth literature review and proceedings of complex care special interest groups at national conference meetings (110 topics). The list was narrowed through input of key stakeholders including clinicians with experience in complex care and refined to ensure clarity and comprehensiveness of the topic list (40 topics, see Appendix 1). The topics were organized using the International Classification of Functioning, Disability and Health (ICF) domains,18 as this framework has previously been used to describe team-based competencies in complex care, and has been proposed as a guide for training program development.19 We aimed to achieve consensus on 10-15 essential curricular topics, as this was thought to be a feasible target to incorporate in a typical clinical rotation based on common patient care training models.20
We developed an electronic survey using REDCap,18 which allowed for study participant anonymity. Individual responses also remained anonymous, with results from each round reported to the expert panel in aggregate. The first iteration of the survey asked participants to consider a list of 40 curricular topics. To confirm participant eligibility, we also collected demographic data about practice location, setting, and years of clinical and educational experience. We specified our ultimate goal was to identify 10-15 core curricular topics. We
asked participants to indicate whether each topic was ―essential,‖ ―important,‖ or ―not important‖ to include in a standard complex care curriculum for pediatric residents prior to the completion of residency training. We also asked participants to indicate topics that were missing, redundant, or unclear, and offered opportunity to provide qualitative feedback.
Data analysis We used an iterative approach to inform participants of each topic’s consensus rating and qualitative feedback. The principal author collated responses after each round, which all members of the study team reviewed in-depth. We calculated the percentage of respondents who indicated a topic was ―essential,‖ ―important,‖ or ―not important‖ for each topic. Based on a systematic review of Delphi studies, determination of consensus is most commonly 75% (range: 50-94%).8,9,12,16 In our case, the research team agreed that topics that >70% of participants indicated as essential achieved consensus and would be removed from further survey iterations. Topics that only 50-70% of participants indicated were essential did not meet the consensus threshold. Participant comments were reviewed and discussed by all members of the study team to identify common and important recommendations, and to reach agreement on appropriate revisions and additions to the topics. These topics were sent back to participants with a synthesis of responses and feedback. Topics that <50% of participants indicated were essential were not included in future rounds.
Data collection and analysis continued until consensus was achieved for 10-15 topics, with a predetermined maximum of three rounds. No financial incentives were provided for participation. The study was deemed exempt by the Boston Children’s Hospital institutional review board.
Results Sixteen experts were approached and agreed to participate. Response rate was 100% (n=16) for all 3 rounds. Experts were located across the United States (n=7, 44%) and Canada (n=9, 56%). Most participants were affiliated with an academic center (n=15, 96%) and practiced in both inpatient and outpatient settings (n=11, 69%). All participants met our study criteria (Table 1).
A flowchart of the Delphi rounds is shown in Figure 1. Consensus was achieved for 6 topics in the first round (Table 2): feeding difficulties (100% essential), pain and irritability (94% essential), difficult discussions (94% essential), team management and care coordination (81% essential), feeding tube management (75% essential) and transition (75% essential).
Topics identified by 50-70% of participants as being essential were included in the second round, with revisions based on participant comments. A theme that emerged for multiple topics was the need to reframe topic areas with an emphasis on management by a general pediatrician (i.e. knowing when to refer). Participants also indicated topics typically addressed in other subspecialty rotations held less priority, with higher priority ascribed to topics requiring a generalist to ―put the pieces together.‖ After the authors synthesized the revisions, 9 refined topic areas were sent back to participants in the second round, as well as an additional topic proposed by 4 participants: advocacy for patients/families.
Consensus was achieved for 3 topics in the second round: aspiration (100% essential), dysmotility (88% essential) and safety/emergency planning (88% essential). Topics that did not
meet consensus in the second round were included in a third and final round with additional revisions based on participant comments. A theme that emerged was adjusting the scope and combining related topic areas. ―Altered tone‖ and ―osteoporosis‖ were combined into ―common neuromuscular and skeletal issues‖; ―family coping and structure‖ was incorporated in the topic of ―advocacy.‖
Five refined topic areas were sent back to participants in the third round. Consensus was achieved for 2 topics in the third round: common neuromuscular and skeletal issues (81% essential) and advocacy (75% essential).
A summary of the qualitative feedback across all Delphi rounds is provided as supplemental material (Appendix 2). The 11 topics for which consensus was obtained were organized using the ICF framework and sent to the participants in a final communication.19
Discussion Using a modified Delphi method, we identified 11 essential topics to include in standardized complex care training for graduating pediatricians. Consensus was achieved using a multinational expert panel to refine and prioritize topics that are essential to formally teach and assess in pediatric residency training. Our results have the potential to inform standardization of curricular content in an evolving field of clinical and educational practice for CMC where evidence and expert consensus has been lacking.
Many of the curricular priorities identified in our study have been shown to be areas of critical educational gaps in complex care. Prior research has described continuing education needs in complex care, including skills in the areas of team management and care coordination, and difficult discussions such as eliciting patient/family goals.21 These topics were all identified as essential for a standardized complex care curriculum by our expert panel. Residents have reported relatively low comfort in topics relating to the topic of advocacy, particularly accessing community resources and liaising with schools,4 as well as management of medical technologies including feeding tube care.2 There is limited literature specific to pediatric complex care on training needs in safety/emergency planning and facilitating transition, yet our expert panel identified these skills as central to care planning for CMC in pediatric practice, warranting dedicated training prior to the end of residency.
Consensus was achieved on several clinical topics related to body functions and structure— including feeding difficulties, pain and irritability, and aspiration. These issues represent multisystem conditions with a constellation of signs and symptoms for which there are no clear evidence-based guidelines in this population. Themes that emerged from participant comments include the important role of pediatricians in ―front-line‖ care, and ensuring trainees understand indications for involving subspecialists and members of the interprofessional team. For instance, aspiration may involve oromotor dysfunction and sialorrhea, and pulmonary complications necessitating airway clearance therapies. Occupational therapy or speech language pathology is often involved in cases where videofluoroscopic swallow study is indicated or for feeding therapy. Additionally, gastroenterology/nutrition referrals may be indicated for clinical care management when associated with gastroesophageal reflux disease or for patients requiring a
modified dietary regimen or enteral feeds. Feedback from our expert panel suggests an important scope of practice for the general pediatrician in the care of CMC: pediatricians provide comprehensive clinical assessment and initial management of multisystem issues using a holistic approach, involving triage and collaboration with an interprofessional team as appropriate.
The World Health Organization’s ICF framework has been used to explicate areas of expertise for teams caring for CMC, with an emphasis on medical technology, family system, and team management as critical environmental factors in the health and well-being of this population.19 The curricular topics identified by our expert panel span multiple domains of care, including body functions and structure, and personal and environmental factors. Almost half of the identified topics relate to body systems and structures, which may be due to the original topic list being heavily weighted toward this domain based on existing literature. Also, our expert panelists were all physicians, and it may be that clinicians of other disciplines would prioritize other domains in team-based care of CMC.
It is interesting that none of the identified topics directly relate to the domains of activities and participation. There were only four topics in the initial list that aligned specifically with these domains, which may have influenced this result. It is also possible that complex care clinicians recognize activities and participation as being central to the approach to every curricular topic in complex care. For example, pain and irritability should be evaluated and managed considering the impact on a child’s sleep, family life, participation in school, and therapies. Aspiration risk must be evaluated in terms of the impact of repeated hospitalizations for respiratory illnesses on missed school or caregiver employment. If gastrostomy tube feeds are initiated, management
includes maintaining participation in family mealtime and socialization consistent with patient/family goals.22 A standardized complex care curriculum must teach a holistic approach to each essential topic in order to truly build capacity for pediatricians to optimize quality of life and care for CMC and their families. The focus on activities and participation is thus pervasive across all curricular topics.
Topics for which consensus were not reached at the end of the third round still reflect important areas of focus in residency training. For example, children with complex chronic conditions and tracheostomies are an important subset of CMC, with frequent emergency department visits and high readmission rates,23–25 and for whom pediatric residents may provide care on-call or in an outpatient clinic. Tracheostomy management was identified as essential by 69% of participants, just shy of meeting our pre-determined threshold for consensus. This and other topics were not selected as essential by some participants as they anticipated these topics would be addressed in other subspecialty rotations (in the case of tracheostomy care, otorhinolaryngology or pulmonary), and participants prioritized other topics more specific to complex care, consistent with the research aim. Yet complex technology management, including troubleshooting tracheostomy-related issues, has been previously identified as a challenge in caring for CMC by pediatric residents.2
We hope that clinicians and educators take a pragmatic approach to using these study results, incorporating important topics in complex care into training programs across clinical rotations. We aimed to identify 10-15 topics, anticipating that this would be a feasible standard for complex care training during residency. A complex care curriculum addressing these topics may
involve varying instructional methods, within a dedicated clinical rotation and/or longitudinally across rotations. For example, in first year a trainee may complete a self-directed online module or attend a didactic session on enteral feeding tube care; in second year a trainee may participate in a difficult discussion with a family about gastrostomy tube placement in a real or simulated clinical encounter; and in third year a longitudinal outpatient experience with a CMC with a gastrostomy tube may involve anticipatory guidance and action plan development, with faculty observation and feedback. To the extent that programs can teach and assess skills relating to other important topics in the care of CMC, we encourage curriculum development in these areas as well.
We anticipate that these results will serve as a driver for developing, implementing, and evaluating effective educational approaches to address these essential topics in complex care training. For example, a recent study of web-based modules relating to topics such as neuromuscular and skeletal issues (specifically spasticity) and feeding tube management found improved learner satisfaction, knowledge acquisition, and behavior change.26 On MedEdPORTAL there are a growing number of educational resources available specific to pediatric complex care with accompanying evaluation data.27,28 In addition to clinical rotations in complex care, which may not be feasible at institutions without a dedicated complex care service, offering asynchronous or online learning opportunities, and longitudinal experiences in the care of CMC across other clinical rotations can provide alternative exposure to a comprehensive curriculum.
The list of essential topics may serve as the basis for defining entrustable professional activities (EPAs) in complex care.29 EPAs are concrete clinical activities that are important and measurable, requiring integration of multiple specific competencies,30 which characterizes most clinical activities in complex care. Elucidating complex care EPAs would recognize these skills as being within the scope of training and clinical practice in pediatrics. We hope that this expert consensus becomes a platform for advocacy in building the capacity of pediatricians to care for CMC, calling for multi-stakeholder involvement including residency program leadership, trainees, and family partners to define a developmental framework for fostering and assessing competency in complex care.
This modified Delphi study has a number of important limitations. We defined experts as being physicians in complex care involved in residency education. These selection criteria may have introduced bias as the vast majority of our participants are located at tertiary academic centers with dedicated complex care clinical services. Clinicians in community-based practice (i.e. with varying access to subspecialists) and primary care providers may have identified different curricular priorities. Additionally, our expert panel did not include members of the interprofessional healthcare team, who may have identified different essential topics—for example, therapists may have explicitly referenced topics in the domains of activities and participation. We felt that academic physicians would have an important familiarity with pediatric residency training. In future studies it would be illuminating to compare competency priorities identified by experts of different disciplines, which has been modelled in other studies.31 Our selection criteria meant that we did not include patient/family partners in our expert panel, who are key stakeholders in complex care training. Delphi studies have been
performed with multi-stakeholder panels including families,32 and we would recommend emulating this approach as a next step.
Responses in the second and third rounds may have been influenced by the feedback we provided after each round and our interpretation of the free-text comments.10 To mitigate this effect we ensured close review of all data, ensured agreement of all members of the study team, and explained rationales for revisions and additions to the participants. We included a relatively small number of participants, which did not allow us to account for the perspectives of experts in additional institutions and care settings; however our method is consistent with recommendations for typical panel size, and we were able to maintain an excellent response rate with this smaller size panel.9
Strengths of our study methods were the anonymity of electronic administration, limiting the bias of face-to-face study where dominant members can influence the group,8,10 and the representation of regions across the United States and Canada. In addition, junior and senior clinicians offered diverse and valuable perspectives on priorities in complex care education.
Conclusion Eleven curricular priorities for standardized complex care training were identified across multiple domains of care for CMC. These topics can be used to guide curriculum development for pediatric residents. Our findings suggest that effective educational interventions focused on these topics may build capacity for pediatricians to provide high-quality comprehensive care for
CMC. The modified Delphi method can be replicated to establish educational priorities for other health professions who care for CMC.
Acknowledgements We thank our expert panel for their valuable contribution and commitment to the study: Rishi Agrawal, Burak Alsan, Ryan Coller, Norah Emara, Emily Goodwin, Esther Lee, Sara LongGagne, Nathalie Major, Lisa McLeod, Julia Orkin, Hema Patel, Jim Plews-Ogan, Gina Rempel, Matthew Sadof, Neha Shah, and Gemma Vomiero.
References 1.
2.
3.
4.
5. 6. 7.
8.
9.
Cohen E, Kuo DZ, Agrawal R, et al. Children with medical complexity: an emerging population for clinical and research initiatives. Pediatrics. 2011;127(3):529-538. doi:10.1542/peds.2010-0910. Bogetz JF, Bogetz AL, Rassbach CE, Gabhart JM, Blankenburg RL. Caring for children with medical complexity: challenges and educational opportunities identified by pediatric residents. Acad Pediatr. 2015;15(6):621-625. doi:10.1016/j.acap.2015.08.004. Bogetz JF, Bogetz AL, Bergman D, Turner T, Blankenburg R, Ballantine A. Challenges and potential solutions to educating learners about pediatric complex care. Acad Pediatr. 2014;14(6):603-609. doi:10.1016/j.acap.2014.06.004. Nazarian BL, Glader L, Choueiri R, Shipman DL, Sadof M. Identifying what pediatric residents are taught about children and youth with special health care needs and the medical home. Pediatrics. 2010;126 Suppl(December):S183-9. doi:http://dx.doi.org/10.1542/peds.2010-1466O. Allshouse C, Comeau M, Rodgers R, Wells N. Families of children with medical complexity: A view from the front lines. Pediatrics. 2018;141(s3):s195-s201. Hewitt-Taylor J. Caring for children with complex needs: Staff education and training. J Child Heal Care. 2005;9(1):72-86. doi:10.1177/1367493505050506. The Accreditation Council for Graduate Medical Education and The American Board of Pediatrics. The Pediatrics Milestones Project. http://www.acgme.org/Specialties/Milestones/pfcatid/16/Pediatrics. Published 2017. Accessed August 10, 2017. Humphrey-Murto S, Varpio L, Gonsalves C, Wood TJ. Using consensus group methods such as Delphi and Nominal Group in medical education research. Med Teach. 2017;39(1):14-19. doi:10.1080/0142159X.2017.1245856. De Villiers M, De Villiers P, Kent A. The Delphi technique in health sciences education research. Med Teach. 2005;27(7):639-643. doi:10.1080/13611260500069947.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
Waggoner J, Carline JD, Durning SJ. Is There a Consensus on Consensus Methodology? Descriptions and Recommendations for Future Consensus Research. Acad Med. 2016;91(5):663-668. doi:10.1097/ACM.0000000000001092. Hordijk R, Hendrickx K, Lanting K, Macfarlane A, Suurmond J. Defining a framework for medical teachers ’ competencies to teach ethnic and cultural diversity : Results of a European Delphi study. Med Teach. 2018:1-7. doi:10.1080/0142159X.2018.1439160. Penciner R, Langhan T, Lee R, Mcewen J, Woods RA, Bandiera G. Using a Delphi process to establish consensus on emergency medicine clerkship competencies. Med Teach. 2011;33:e333-339. doi:10.3109/0142159X.2011.575903. Kuo A, Ciccarelli MR, Sharma N, Lotstein D. A Health Care Transition Curriculum for Primary Care Residents : Identifying Goals and Objectives. Pediatrics. 2018;141(April):S346-S354. Wright CJ, Katcher ML, Blatt SD, et al. Toward the Development of Advocacy Training Curricula for Pediatric Residents: A National Delphi Study. Ambul Pediatr. 2005;5:165171. Rana J, Sullivan A, Brett M, Weinstein A, Atkins K. Defining curricular priorities for student-as- teacher programs : A National Delphi Study. Med Teach. 2018;40(3):259-266. doi:10.1080/0142159X.2017.1401216. Diamond IR, Grant RC, Feldman BM, et al. Defining consensus : A systematic review recommends methodologic criteria for reporting of Delphi studies. J Clin Epidemiol. 2014;67:401-409. doi:10.1016/j.jclinepi.2013.12.002. Nair R, Aggarwal R, Khanna D. Methods of Formal Consensus in Classification/Diagnostic Criteria and Guideline Development. Semin Arthritis Rheum. 2011;41(2):95-105. doi:10.1016/j.semarthrit.2010.12.001.Methods. World Health Organization. International Classification of Functioning, Disability and Health (ICF). https://www.who.int/classifications/icf/en/. Published 2019. Accessed May 31, 2019. Glader L, Plews-Ogan J, Agrawal R. Children with medical complexity: creating a framework for care based on the International Classification of Functioning, Disability and Health. Dev Med Child Neurol. 2016;58(11):1116-1123. doi:10.1111/dmcn.13201. Accreditation Council for Graduate Medical Education. ACGME Program Requirements and FAQs. https://www.acgme.org/Specialties/Program-Requirements-and-FAQs-andApplications/pfcatid/16/Pediatrics. Published 2019. Accessed July 7, 2019. Bogetz JF, Bogetz AL, Gabhart JM, Bergman DA, Blankenburg RL, Rassbach CE. Continuing education needs of pediatricians across diverse specialties caring for children with medical complexity. Clin Pediatr (Phila). 2015;54(3):222-227. doi:10.1177/0009922814564049. Mahant S, Jovcevska V, Cohen E. Decision-Making Around Gastrostomy-Feeding in Children With Neurologic Disabilities. Pediatrics. 2011;127(6):e1471-e1481. doi:10.1542/peds.2010-3007. Spratling R. Understanding the health care utilization of children who require medical technology: A descriptive study of children who require tracheostomies. Appl Nurs Res. 2017;34:62-65. doi:10.1016/j.apnr.2017.02.017. Cohen E, Berry JG, Camacho X, Anderson G, Wodchis W, Guttmann A. Patterns and costs of health care use of children with medical complexity. Pediatrics. 2012;130(6):e1463-70. doi:10.1542/peds.2012-0175.
25.
26.
27. 28. 29.
30. 31.
32.
Berry JG, Hall DE, Kuo DZ, Hall M, Kueser J, Kaplan W. Hospital utilization and characteristics of patients experiencing recurrent readmissions within children ’s hospitals. JAMA. 2011;305(7):682-690. Shah NH, Bhansali P, Barber A, et al. Children With Medical Complexity: A Web-Based Multimedia Curriculum Assessing Pediatric Residents Across North America. Acad Pediatr. 2018;18(1):79-85. doi:10.1016/j.acap.2017.08.008. Huth K, Long-Gagne S, Mader J, Sbrocchi A. Understanding the needs of children with medical complexity. MedEdPORTAL. 2018;14(10709). Huth K, Long-Gagne S, Mader J, Sbrocchi A. Approach to clinical assessment of children with medical complexity. MedEdPORTAL. 2018;14(10765). American Board of Pediatrics. Entrustable professional activities for general pediatrics. https://www.abp.org/entrustable-professional-activities-epas. Published 2019. Accessed May 30, 2019. Cate O ten. Nuts and Bolts of Entrustable Professional Activities. J Grad Med Educ. 2013;5(1):157-158. Lalloo D, Demou E, Stevenson M, Gaffney M, Macdonald EB. Comparison of competency priorities between UK occupational physicians and occupational health nurses. Occup Env Med. 2017;74:384-386. doi:10.1136/oemed-2016-104049. Leyenaar JK, Rizzo PA, Khodyakov D, Leslie LK, Lindenauer PK, Mangione-Smith R. Importance and feasibility of transitional care for children with medical complexity: results of a multi-stakeholder Delphi process. Acad Pediatr. 2017. doi:10.1016/j.acap.2017.07.005.
Table 1: Characteristics of expert panelists. Characteristic
Number (Percentage)
Location
Canada United States
7 (44) 9 (56)
Practice type
Inpatient Outpatient Both inpatient and outpatient
2 (12) 3 (19) 11 (69)
Setting
Academic center Community-based practice
15 (96) 1 (4)
Clinical experience caring for children with medical complexity
0-4 years 5-9 years 10-14 years 15+ years 0-4 years 5-9 years 10-14 years 15+ years
4 (25) 3 (19) 4 (25) 5 (31) 4 (25) 4 (25) 4 (25) 4 (25)
Experience working with pediatric residents
Table 2: Topics identified as essential by >70% of participants in all rounds and alignment with the International Classification of Functioning, Disability and Health (ICF) domains. Domain
Curricular Topic
Body functions and structures
1.
Environment
Personal factors Technology Family system
Team management
Feeding difficulties and nutritional concerns (including decisionmaking for tube placement, poor weight gain and obesity) 2. Pain and irritability (including evaluation and management) 3. Dysmotility (including constipation, GERD, slow gastric emptying, feeding intolerance, and indications for subspecialty referral) 4. Aspiration (including evaluation and management of dysphagia, sialorrhea, chronic lung disease, and when to refer) 5. Common neuromuscular and skeletal issues (including basics about spasticity management, hip and spine surveillance, bone health) 6. Safety/emergency planning (including development of sick plans and emergency letters/summaries, recognizing special circumstances that require individualized management) 7. Transition (including access to services, medical team transition of care, employment, independent living) 8. Feeding tube management/troubleshooting (including gastrostomy, gastrojejunostomy) 9. Difficult discussions (including palliative care, DNR, developing shared goals of care) 10. Advocacy for patients/families (including developing partnerships, identifying stressors and risk factors for caregiver burnout, and liaising with the interdisciplinary team, community agencies and school) 11. Team management and care coordination
Figure 1: Flowchart of the Delphi process and topics that achieved >70% consensus. *Topics that did not achieve >50% consensus were not included in subsequent rounds
Core Curricular Priorities in the Care of Children with Medical Complexity: A North American Modified Delphi Study Kathleen Huth MD, MMSc , Lori Newman MEd , Laurie Glader MD PII: DOI: Reference:
S1876-2859(20)30054-1 https://doi.org/10.1016/j.acap.2020.01.014 ACAP 1477
To appear in:
Academic Pediatrics
Received date: Accepted date:
22 July 2019 31 January 2020
Please cite this article as: Kathleen Huth MD, MMSc , Lori Newman MEd , Laurie Glader MD , Core Curricular Priorities in the Care of Children with Medical Complexity: A North American Modified Delphi Study, Academic Pediatrics (2020), doi: https://doi.org/10.1016/j.acap.2020.01.014
This is a PDF file of an article that has undergone enhancements after acceptance, such as the addition of a cover page and metadata, and formatting for readability, but it is not yet the definitive version of record. This version will undergo additional copyediting, typesetting and review before it is published in its final form, but we are providing this version to give early visibility of the article. Please note that, during the production process, errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain. © 2020 Published by Elsevier Inc. on behalf of Academic Pediatric Association
Title: Core Curricular Priorities in the Care of Children with Medical Complexity: A North American Modified Delphi Study Authors and affiliations: Kathleen Huth, MD, MMSc,1 Lori Newman, MEd,2 Laurie Glader, MD1 1
Department of Pediatrics, 2Department of Medical Education, Boston Children’s Hospital, Boston MA. Corresponding author Kathleen Huth, MD, MMSc Boston Children’s Hospital 300 Longwood Ave, Boston MA, 02115 Tel: 617-355-6162 Email: [email protected] Keywords Complex care, residency education, postgraduate medical education, pediatrics, curriculum development, children with medical complexity Running title Curricular priorities in complex care Word counts Abstract 249/250 Manuscript 3033/3500 Funding sources N/A Conflicts of interest The authors have no conflicts of interest to disclose.
Abstract Background: Previous studies of pediatric residents have identified educational gaps in caring for children with medical complexity (CMC). Training opportunities in complex care vary across residency programs. Defining core curricular topics in complex care is a priority in medical education.
Objective: To identify core topics to include in a standard complex care curriculum for pediatric residents.
Methods: An initial topic list was generated through literature review and proceedings of national meetings. Expert panelists were identified based on experience in complex care and residency education. A modified Delphi method was used to determine group consensus by asking participants to rate the importance of complex care curricular topics for pediatric residents. Consensus was defined as >70% of experts identifying a topic as essential. There was a predetermined maximum of three iterative, electronic survey rounds, with feedback provided to participants between each round.
Results: Sixteen experts participated. Response rate was 100% for all rounds. Experts were from the United States (44%) and Canada (56%); most were affiliated with an academic medical center (96%) involved in both inpatient and outpatient care (69%). Eleven topics were identified as essential across 3 rounds: feeding difficulties, pain/irritability, transition, feeding tube management, difficult discussions, team management/care coordination, dysmotility, aspiration, safety/emergency planning, neuromuscular/skeletal issues and advocacy. Essential topics were
organized according to the International Classification of Functioning, Disability and Health (ICF).
Conclusion: Eleven curricular priorities in complex care were identified across multiple domains of the ICF framework, serving as a guide for standardized curriculum development for future pediatricians.
What’s New We conducted a modified Delphi study to achieve consensus on essential topics in complex care education for pediatric residents. Our findings inform curriculum development in the care of children with medical complexity.
Introduction Pediatricians have a central role in caring for children with medical complexity (CMC), a population characterized by chronic conditions, medical fragility and, often, technology dependence.1 Due to their high healthcare utilization, CMC are cared for in all settings: on the inpatient ward where pediatricians evaluate and manage acute-on-chronic exacerbations of multisystem disease, and in the outpatient setting where pediatricians manage health maintenance, care planning, and coordination, often as part of a larger interprofessional care team.
Currently, pediatric residency programs offer variable educational opportunities in complex care to their trainees. Elective or mandatory clinical rotations can vary in length, setting, and dedicated teaching. Numerous publications have highlighted the necessity for formal training during pediatric residency to support the delivery of safe, evidence-based care for CMC.2–4 Inadequate training in complex care contributes to inconsistent, poor-quality and fragmented care, and means that many families are managing complex medical needs outside of the hospital with minimal support.5,6 Unfortunately, no clear educational priorities in caring for CMC have been established around which to structure learning opportunities and competency assessment. Given the Accreditation Council for Graduate Medical Education’s focus on ensuring graduating pediatrician competency,7 setting standards for curriculum development in the care of this vulnerable patient population is critical. Clinicians with experience caring for CMC and working with pediatric residents would have valuable insight into priorities for formal training in this field.
The purpose of this study was to obtain consensus on essential topics to be included in a standard complex care education for pediatric residents and to inform curriculum development efforts for complex care rotations in pediatric training programs.
Methods Study design We used a modified Delphi method to develop expert group consensus.8,9 This method was chosen as a systematic means of eliciting collective expert opinion in an area where there is limited evidence for effective clinical and educational approaches, and where experts are geographically dispersed.10 The Delphi method has been used to define essential teaching competencies,8,11,12 learning objectives,13 and content areas for training programs.9,14,15 We used previously established methodologic criteria for Delphi studies to ensure quality.16
Participants We selected expert panelists using the following criteria: 1) recognized clinical experience as a pediatrician caring for CMC, and 2) experience working with pediatric residents in a clinical setting. Participants were identified via their active involvement in complex care special interest groups, national conference meetings and/or published scholarship in complex care education. We used purposeful sampling to ensure the expert panel represented different institutions across the United States and Canada, different care settings, and varying years of experience. We anticipated that senior clinicians as well as junior faculty members, having recently completed residency training, would have valuable perspectives on essential curricular topics. We used a snowball technique whereby clinicians identified other potential participants who they
anticipated would have insight into the research question. We recruited 16 experts, consistent with a typical panel size, parametric determination, and study reliability.10,17 Participants indicated their interest in completing all rounds of the process via email prior to receiving the first survey, with the goal of optimizing response rate.
Data collection An initial topic list was generated through an in-depth literature review and proceedings of complex care special interest groups at national conference meetings (110 topics). The list was narrowed through input of key stakeholders including clinicians with experience in complex care and refined to ensure clarity and comprehensiveness of the topic list (40 topics, see Appendix 1). The topics were organized using the International Classification of Functioning, Disability and Health (ICF) domains,18 as this framework has previously been used to describe team-based competencies in complex care, and has been proposed as a guide for training program development.19 We aimed to achieve consensus on 10-15 essential curricular topics, as this was thought to be a feasible target to incorporate in a typical clinical rotation based on common patient care training models.20
We developed an electronic survey using REDCap,18 which allowed for study participant anonymity. Individual responses also remained anonymous, with results from each round reported to the expert panel in aggregate. The first iteration of the survey asked participants to consider a list of 40 curricular topics. To confirm participant eligibility, we also collected demographic data about practice location, setting, and years of clinical and educational experience. We specified our ultimate goal was to identify 10-15 core curricular topics. We
asked participants to indicate whether each topic was ―essential,‖ ―important,‖ or ―not important‖ to include in a standard complex care curriculum for pediatric residents prior to the completion of residency training. We also asked participants to indicate topics that were missing, redundant, or unclear, and offered opportunity to provide qualitative feedback.
Data analysis We used an iterative approach to inform participants of each topic’s consensus rating and qualitative feedback. The principal author collated responses after each round, which all members of the study team reviewed in-depth. We calculated the percentage of respondents who indicated a topic was ―essential,‖ ―important,‖ or ―not important‖ for each topic. Based on a systematic review of Delphi studies, determination of consensus is most commonly 75% (range: 50-94%).8,9,12,16 In our case, the research team agreed that topics that >70% of participants indicated as essential achieved consensus and would be removed from further survey iterations. Topics that only 50-70% of participants indicated were essential did not meet the consensus threshold. Participant comments were reviewed and discussed by all members of the study team to identify common and important recommendations, and to reach agreement on appropriate revisions and additions to the topics. These topics were sent back to participants with a synthesis of responses and feedback. Topics that <50% of participants indicated were essential were not included in future rounds.
Data collection and analysis continued until consensus was achieved for 10-15 topics, with a predetermined maximum of three rounds. No financial incentives were provided for participation. The study was deemed exempt by the Boston Children’s Hospital institutional review board.
Results Sixteen experts were approached and agreed to participate. Response rate was 100% (n=16) for all 3 rounds. Experts were located across the United States (n=7, 44%) and Canada (n=9, 56%). Most participants were affiliated with an academic center (n=15, 96%) and practiced in both inpatient and outpatient settings (n=11, 69%). All participants met our study criteria (Table 1).
A flowchart of the Delphi rounds is shown in Figure 1. Consensus was achieved for 6 topics in the first round (Table 2): feeding difficulties (100% essential), pain and irritability (94% essential), difficult discussions (94% essential), team management and care coordination (81% essential), feeding tube management (75% essential) and transition (75% essential).
Topics identified by 50-70% of participants as being essential were included in the second round, with revisions based on participant comments. A theme that emerged for multiple topics was the need to reframe topic areas with an emphasis on management by a general pediatrician (i.e. knowing when to refer). Participants also indicated topics typically addressed in other subspecialty rotations held less priority, with higher priority ascribed to topics requiring a generalist to ―put the pieces together.‖ After the authors synthesized the revisions, 9 refined topic areas were sent back to participants in the second round, as well as an additional topic proposed by 4 participants: advocacy for patients/families.
Consensus was achieved for 3 topics in the second round: aspiration (100% essential), dysmotility (88% essential) and safety/emergency planning (88% essential). Topics that did not
meet consensus in the second round were included in a third and final round with additional revisions based on participant comments. A theme that emerged was adjusting the scope and combining related topic areas. ―Altered tone‖ and ―osteoporosis‖ were combined into ―common neuromuscular and skeletal issues‖; ―family coping and structure‖ was incorporated in the topic of ―advocacy.‖
Five refined topic areas were sent back to participants in the third round. Consensus was achieved for 2 topics in the third round: common neuromuscular and skeletal issues (81% essential) and advocacy (75% essential).
A summary of the qualitative feedback across all Delphi rounds is provided as supplemental material (Appendix 2). The 11 topics for which consensus was obtained were organized using the ICF framework and sent to the participants in a final communication.19
Discussion Using a modified Delphi method, we identified 11 essential topics to include in standardized complex care training for graduating pediatricians. Consensus was achieved using a multinational expert panel to refine and prioritize topics that are essential to formally teach and assess in pediatric residency training. Our results have the potential to inform standardization of curricular content in an evolving field of clinical and educational practice for CMC where evidence and expert consensus has been lacking.
Many of the curricular priorities identified in our study have been shown to be areas of critical educational gaps in complex care. Prior research has described continuing education needs in complex care, including skills in the areas of team management and care coordination, and difficult discussions such as eliciting patient/family goals.21 These topics were all identified as essential for a standardized complex care curriculum by our expert panel. Residents have reported relatively low comfort in topics relating to the topic of advocacy, particularly accessing community resources and liaising with schools,4 as well as management of medical technologies including feeding tube care.2 There is limited literature specific to pediatric complex care on training needs in safety/emergency planning and facilitating transition, yet our expert panel identified these skills as central to care planning for CMC in pediatric practice, warranting dedicated training prior to the end of residency.
Consensus was achieved on several clinical topics related to body functions and structure— including feeding difficulties, pain and irritability, and aspiration. These issues represent multisystem conditions with a constellation of signs and symptoms for which there are no clear evidence-based guidelines in this population. Themes that emerged from participant comments include the important role of pediatricians in ―front-line‖ care, and ensuring trainees understand indications for involving subspecialists and members of the interprofessional team. For instance, aspiration may involve oromotor dysfunction and sialorrhea, and pulmonary complications necessitating airway clearance therapies. Occupational therapy or speech language pathology is often involved in cases where videofluoroscopic swallow study is indicated or for feeding therapy. Additionally, gastroenterology/nutrition referrals may be indicated for clinical care management when associated with gastroesophageal reflux disease or for patients requiring a
modified dietary regimen or enteral feeds. Feedback from our expert panel suggests an important scope of practice for the general pediatrician in the care of CMC: pediatricians provide comprehensive clinical assessment and initial management of multisystem issues using a holistic approach, involving triage and collaboration with an interprofessional team as appropriate.
The World Health Organization’s ICF framework has been used to explicate areas of expertise for teams caring for CMC, with an emphasis on medical technology, family system, and team management as critical environmental factors in the health and well-being of this population.19 The curricular topics identified by our expert panel span multiple domains of care, including body functions and structure, and personal and environmental factors. Almost half of the identified topics relate to body systems and structures, which may be due to the original topic list being heavily weighted toward this domain based on existing literature. Also, our expert panelists were all physicians, and it may be that clinicians of other disciplines would prioritize other domains in team-based care of CMC.
It is interesting that none of the identified topics directly relate to the domains of activities and participation. There were only four topics in the initial list that aligned specifically with these domains, which may have influenced this result. It is also possible that complex care clinicians recognize activities and participation as being central to the approach to every curricular topic in complex care. For example, pain and irritability should be evaluated and managed considering the impact on a child’s sleep, family life, participation in school, and therapies. Aspiration risk must be evaluated in terms of the impact of repeated hospitalizations for respiratory illnesses on missed school or caregiver employment. If gastrostomy tube feeds are initiated, management
includes maintaining participation in family mealtime and socialization consistent with patient/family goals.22 A standardized complex care curriculum must teach a holistic approach to each essential topic in order to truly build capacity for pediatricians to optimize quality of life and care for CMC and their families. The focus on activities and participation is thus pervasive across all curricular topics.
Topics for which consensus were not reached at the end of the third round still reflect important areas of focus in residency training. For example, children with complex chronic conditions and tracheostomies are an important subset of CMC, with frequent emergency department visits and high readmission rates,23–25 and for whom pediatric residents may provide care on-call or in an outpatient clinic. Tracheostomy management was identified as essential by 69% of participants, just shy of meeting our pre-determined threshold for consensus. This and other topics were not selected as essential by some participants as they anticipated these topics would be addressed in other subspecialty rotations (in the case of tracheostomy care, otorhinolaryngology or pulmonary), and participants prioritized other topics more specific to complex care, consistent with the research aim. Yet complex technology management, including troubleshooting tracheostomy-related issues, has been previously identified as a challenge in caring for CMC by pediatric residents.2
We hope that clinicians and educators take a pragmatic approach to using these study results, incorporating important topics in complex care into training programs across clinical rotations. We aimed to identify 10-15 topics, anticipating that this would be a feasible standard for complex care training during residency. A complex care curriculum addressing these topics may
involve varying instructional methods, within a dedicated clinical rotation and/or longitudinally across rotations. For example, in first year a trainee may complete a self-directed online module or attend a didactic session on enteral feeding tube care; in second year a trainee may participate in a difficult discussion with a family about gastrostomy tube placement in a real or simulated clinical encounter; and in third year a longitudinal outpatient experience with a CMC with a gastrostomy tube may involve anticipatory guidance and action plan development, with faculty observation and feedback. To the extent that programs can teach and assess skills relating to other important topics in the care of CMC, we encourage curriculum development in these areas as well.
We anticipate that these results will serve as a driver for developing, implementing, and evaluating effective educational approaches to address these essential topics in complex care training. For example, a recent study of web-based modules relating to topics such as neuromuscular and skeletal issues (specifically spasticity) and feeding tube management found improved learner satisfaction, knowledge acquisition, and behavior change.26 On MedEdPORTAL there are a growing number of educational resources available specific to pediatric complex care with accompanying evaluation data.27,28 In addition to clinical rotations in complex care, which may not be feasible at institutions without a dedicated complex care service, offering asynchronous or online learning opportunities, and longitudinal experiences in the care of CMC across other clinical rotations can provide alternative exposure to a comprehensive curriculum.
The list of essential topics may serve as the basis for defining entrustable professional activities (EPAs) in complex care.29 EPAs are concrete clinical activities that are important and measurable, requiring integration of multiple specific competencies,30 which characterizes most clinical activities in complex care. Elucidating complex care EPAs would recognize these skills as being within the scope of training and clinical practice in pediatrics. We hope that this expert consensus becomes a platform for advocacy in building the capacity of pediatricians to care for CMC, calling for multi-stakeholder involvement including residency program leadership, trainees, and family partners to define a developmental framework for fostering and assessing competency in complex care.
This modified Delphi study has a number of important limitations. We defined experts as being physicians in complex care involved in residency education. These selection criteria may have introduced bias as the vast majority of our participants are located at tertiary academic centers with dedicated complex care clinical services. Clinicians in community-based practice (i.e. with varying access to subspecialists) and primary care providers may have identified different curricular priorities. Additionally, our expert panel did not include members of the interprofessional healthcare team, who may have identified different essential topics—for example, therapists may have explicitly referenced topics in the domains of activities and participation. We felt that academic physicians would have an important familiarity with pediatric residency training. In future studies it would be illuminating to compare competency priorities identified by experts of different disciplines, which has been modelled in other studies.31 Our selection criteria meant that we did not include patient/family partners in our expert panel, who are key stakeholders in complex care training. Delphi studies have been
performed with multi-stakeholder panels including families,32 and we would recommend emulating this approach as a next step.
Responses in the second and third rounds may have been influenced by the feedback we provided after each round and our interpretation of the free-text comments.10 To mitigate this effect we ensured close review of all data, ensured agreement of all members of the study team, and explained rationales for revisions and additions to the participants. We included a relatively small number of participants, which did not allow us to account for the perspectives of experts in additional institutions and care settings; however our method is consistent with recommendations for typical panel size, and we were able to maintain an excellent response rate with this smaller size panel.9
Strengths of our study methods were the anonymity of electronic administration, limiting the bias of face-to-face study where dominant members can influence the group,8,10 and the representation of regions across the United States and Canada. In addition, junior and senior clinicians offered diverse and valuable perspectives on priorities in complex care education.
Conclusion Eleven curricular priorities for standardized complex care training were identified across multiple domains of care for CMC. These topics can be used to guide curriculum development for pediatric residents. Our findings suggest that effective educational interventions focused on these topics may build capacity for pediatricians to provide high-quality comprehensive care for
CMC. The modified Delphi method can be replicated to establish educational priorities for other health professions who care for CMC.
Acknowledgements We thank our expert panel for their valuable contribution and commitment to the study: Rishi Agrawal, Burak Alsan, Ryan Coller, Norah Emara, Emily Goodwin, Esther Lee, Sara LongGagne, Nathalie Major, Lisa McLeod, Julia Orkin, Hema Patel, Jim Plews-Ogan, Gina Rempel, Matthew Sadof, Neha Shah, and Gemma Vomiero.
References 1.
2.
3.
4.
5. 6. 7.
8.
9.
Cohen E, Kuo DZ, Agrawal R, et al. Children with medical complexity: an emerging population for clinical and research initiatives. Pediatrics. 2011;127(3):529-538. doi:10.1542/peds.2010-0910. Bogetz JF, Bogetz AL, Rassbach CE, Gabhart JM, Blankenburg RL. Caring for children with medical complexity: challenges and educational opportunities identified by pediatric residents. Acad Pediatr. 2015;15(6):621-625. doi:10.1016/j.acap.2015.08.004. Bogetz JF, Bogetz AL, Bergman D, Turner T, Blankenburg R, Ballantine A. Challenges and potential solutions to educating learners about pediatric complex care. Acad Pediatr. 2014;14(6):603-609. doi:10.1016/j.acap.2014.06.004. Nazarian BL, Glader L, Choueiri R, Shipman DL, Sadof M. Identifying what pediatric residents are taught about children and youth with special health care needs and the medical home. Pediatrics. 2010;126 Suppl(December):S183-9. doi:http://dx.doi.org/10.1542/peds.2010-1466O. Allshouse C, Comeau M, Rodgers R, Wells N. Families of children with medical complexity: A view from the front lines. Pediatrics. 2018;141(s3):s195-s201. Hewitt-Taylor J. Caring for children with complex needs: Staff education and training. J Child Heal Care. 2005;9(1):72-86. doi:10.1177/1367493505050506. The Accreditation Council for Graduate Medical Education and The American Board of Pediatrics. The Pediatrics Milestones Project. http://www.acgme.org/Specialties/Milestones/pfcatid/16/Pediatrics. Published 2017. Accessed August 10, 2017. Humphrey-Murto S, Varpio L, Gonsalves C, Wood TJ. Using consensus group methods such as Delphi and Nominal Group in medical education research. Med Teach. 2017;39(1):14-19. doi:10.1080/0142159X.2017.1245856. De Villiers M, De Villiers P, Kent A. The Delphi technique in health sciences education research. Med Teach. 2005;27(7):639-643. doi:10.1080/13611260500069947.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
Waggoner J, Carline JD, Durning SJ. Is There a Consensus on Consensus Methodology? Descriptions and Recommendations for Future Consensus Research. Acad Med. 2016;91(5):663-668. doi:10.1097/ACM.0000000000001092. Hordijk R, Hendrickx K, Lanting K, Macfarlane A, Suurmond J. Defining a framework for medical teachers ’ competencies to teach ethnic and cultural diversity : Results of a European Delphi study. Med Teach. 2018:1-7. doi:10.1080/0142159X.2018.1439160. Penciner R, Langhan T, Lee R, Mcewen J, Woods RA, Bandiera G. Using a Delphi process to establish consensus on emergency medicine clerkship competencies. Med Teach. 2011;33:e333-339. doi:10.3109/0142159X.2011.575903. Kuo A, Ciccarelli MR, Sharma N, Lotstein D. A Health Care Transition Curriculum for Primary Care Residents : Identifying Goals and Objectives. Pediatrics. 2018;141(April):S346-S354. Wright CJ, Katcher ML, Blatt SD, et al. Toward the Development of Advocacy Training Curricula for Pediatric Residents: A National Delphi Study. Ambul Pediatr. 2005;5:165171. Rana J, Sullivan A, Brett M, Weinstein A, Atkins K. Defining curricular priorities for student-as- teacher programs : A National Delphi Study. Med Teach. 2018;40(3):259-266. doi:10.1080/0142159X.2017.1401216. Diamond IR, Grant RC, Feldman BM, et al. Defining consensus : A systematic review recommends methodologic criteria for reporting of Delphi studies. J Clin Epidemiol. 2014;67:401-409. doi:10.1016/j.jclinepi.2013.12.002. Nair R, Aggarwal R, Khanna D. Methods of Formal Consensus in Classification/Diagnostic Criteria and Guideline Development. Semin Arthritis Rheum. 2011;41(2):95-105. doi:10.1016/j.semarthrit.2010.12.001.Methods. World Health Organization. International Classification of Functioning, Disability and Health (ICF). https://www.who.int/classifications/icf/en/. Published 2019. Accessed May 31, 2019. Glader L, Plews-Ogan J, Agrawal R. Children with medical complexity: creating a framework for care based on the International Classification of Functioning, Disability and Health. Dev Med Child Neurol. 2016;58(11):1116-1123. doi:10.1111/dmcn.13201. Accreditation Council for Graduate Medical Education. ACGME Program Requirements and FAQs. https://www.acgme.org/Specialties/Program-Requirements-and-FAQs-andApplications/pfcatid/16/Pediatrics. Published 2019. Accessed July 7, 2019. Bogetz JF, Bogetz AL, Gabhart JM, Bergman DA, Blankenburg RL, Rassbach CE. Continuing education needs of pediatricians across diverse specialties caring for children with medical complexity. Clin Pediatr (Phila). 2015;54(3):222-227. doi:10.1177/0009922814564049. Mahant S, Jovcevska V, Cohen E. Decision-Making Around Gastrostomy-Feeding in Children With Neurologic Disabilities. Pediatrics. 2011;127(6):e1471-e1481. doi:10.1542/peds.2010-3007. Spratling R. Understanding the health care utilization of children who require medical technology: A descriptive study of children who require tracheostomies. Appl Nurs Res. 2017;34:62-65. doi:10.1016/j.apnr.2017.02.017. Cohen E, Berry JG, Camacho X, Anderson G, Wodchis W, Guttmann A. Patterns and costs of health care use of children with medical complexity. Pediatrics. 2012;130(6):e1463-70. doi:10.1542/peds.2012-0175.
25.
26.
27. 28. 29.
30. 31.
32.
Berry JG, Hall DE, Kuo DZ, Hall M, Kueser J, Kaplan W. Hospital utilization and characteristics of patients experiencing recurrent readmissions within children ’s hospitals. JAMA. 2011;305(7):682-690. Shah NH, Bhansali P, Barber A, et al. Children With Medical Complexity: A Web-Based Multimedia Curriculum Assessing Pediatric Residents Across North America. Acad Pediatr. 2018;18(1):79-85. doi:10.1016/j.acap.2017.08.008. Huth K, Long-Gagne S, Mader J, Sbrocchi A. Understanding the needs of children with medical complexity. MedEdPORTAL. 2018;14(10709). Huth K, Long-Gagne S, Mader J, Sbrocchi A. Approach to clinical assessment of children with medical complexity. MedEdPORTAL. 2018;14(10765). American Board of Pediatrics. Entrustable professional activities for general pediatrics. https://www.abp.org/entrustable-professional-activities-epas. Published 2019. Accessed May 30, 2019. Cate O ten. Nuts and Bolts of Entrustable Professional Activities. J Grad Med Educ. 2013;5(1):157-158. Lalloo D, Demou E, Stevenson M, Gaffney M, Macdonald EB. Comparison of competency priorities between UK occupational physicians and occupational health nurses. Occup Env Med. 2017;74:384-386. doi:10.1136/oemed-2016-104049. Leyenaar JK, Rizzo PA, Khodyakov D, Leslie LK, Lindenauer PK, Mangione-Smith R. Importance and feasibility of transitional care for children with medical complexity: results of a multi-stakeholder Delphi process. Acad Pediatr. 2017. doi:10.1016/j.acap.2017.07.005.
Table 1: Characteristics of expert panelists. Characteristic
Number (Percentage)
Location
Canada United States
7 (44) 9 (56)
Practice type
Inpatient Outpatient Both inpatient and outpatient
2 (12) 3 (19) 11 (69)
Setting
Academic center Community-based practice
15 (96) 1 (4)
Clinical experience caring for children with medical complexity
0-4 years 5-9 years 10-14 years 15+ years 0-4 years 5-9 years 10-14 years 15+ years
4 (25) 3 (19) 4 (25) 5 (31) 4 (25) 4 (25) 4 (25) 4 (25)
Experience working with pediatric residents
Table 2: Topics identified as essential by >70% of participants in all rounds and alignment with the International Classification of Functioning, Disability and Health (ICF) domains. Domain
Curricular Topic
Body functions and structures
1.
Environment
Personal factors Technology Family system
Team management
Feeding difficulties and nutritional concerns (including decisionmaking for tube placement, poor weight gain and obesity) 2. Pain and irritability (including evaluation and management) 3. Dysmotility (including constipation, GERD, slow gastric emptying, feeding intolerance, and indications for subspecialty referral) 4. Aspiration (including evaluation and management of dysphagia, sialorrhea, chronic lung disease, and when to refer) 5. Common neuromuscular and skeletal issues (including basics about spasticity management, hip and spine surveillance, bone health) 6. Safety/emergency planning (including development of sick plans and emergency letters/summaries, recognizing special circumstances that require individualized management) 7. Transition (including access to services, medical team transition of care, employment, independent living) 8. Feeding tube management/troubleshooting (including gastrostomy, gastrojejunostomy) 9. Difficult discussions (including palliative care, DNR, developing shared goals of care) 10. Advocacy for patients/families (including developing partnerships, identifying stressors and risk factors for caregiver burnout, and liaising with the interdisciplinary team, community agencies and school) 11. Team management and care coordination
Figure 1: Flowchart of the Delphi process and topics that achieved >70% consensus. *Topics that did not achieve >50% consensus were not included in subsequent rounds