- Email: [email protected]
Are graduated intensivists prepared for practice? A case study from The Netherlands
Journal of Critical Care 42 (2017) 47–53
Contents lists available at ScienceDirect
Journal of Critical Care journal homepage: www.jccjournal.org
Are graduated intensivists prepared for practice? A case study from The Netherlands Ids S. Dijkstra a,⁎, Paul L.P. Brand a,b, Jan Pols c, Hans Delwig d, Debbie A.D.C. Jaarsma c, Jaap E. Tulleken d a
Wenckebach Institute, University of Groningen, University Medical Center Groningen, Groningen, The Netherlands Princess Amalia Children's Centre, Isala Hospital, Zwolle, The Netherlands Center for Educational Development and Research in Health Professions, University of Groningen, University Medical Center Groningen, Groningen, The Netherlands d Department of Critical Care, University of Groningen, University Medical Center Groningen, Groningen, The Netherlands b c
a r t i c l e
i n f o
Keywords: CoBaTrICE Preparedness for practice Competency-based education ICM- training Evaluation
a b s t r a c t Purpose: An evaluation of the alignment between intensive care medicine (ICM) training and practice provides valuable information for the development of ICM training. Therefore this study examines how well recently licensed intensivists feel prepared for practice and whether intensivists from different background specialties attain comparable preparedness rates. Methods: An inventory was developed to cover the tasks that constitute ICM practice. Two hundred five recently licensed Dutch intensivists received a questionnaire in which they could indicate how well their ICM training programme prepared them for these tasks on a 5-point Likert scale. Results: Ninety-one respondents returned the questionnaire (response 45%). Respondents felt excellently prepared for 67 tasks, well prepared for 16 tasks, marginally sufficiently prepared for 6 tasks and insufficiently prepared for 15 tasks. Intensivists from anaesthesiology felt better prepared for IC specific activities (mean 4.25, SD 0.38) than those from internal medicine (mean 4.01, SD 0.40, P = .02).Average scores on tasks related to medical expertise were relatively high while tasks relating to management and leadership, science and professional development scored lower. Conclusions: Although recently licensed intensivists are well prepared for most tasks in ICM, lower preparedness scores on tasks related to leadership and management, science, and professional development call for reevaluation of the current curriculum. © 2017 Elsevier Inc. All rights reserved.
1. Introduction Competency-based education (CBE) has been adopted to better respond to future challenges in health-care and to meet societal requirements [1,2]. In intensive care medicine (ICM), 43 national societies have collaborated to develop a competency framework and common standards for postgraduate training within the “Competency Based Training in Intensive Care medicine for Europe collaboration” (CoBaTrICE) [3,4]. These standards should harmonize training in ICM without interfering with specific national regulations, allow for free movement of intensivists across Europe, and ensure high quality education in ICM. Most research on the development of CBE in ICM has focused on international standards, structures, processes and assessment, or on the patients' views on what makes a good intensivist [5-7]. In contrast, only few studies have evaluated the content of ICM training programmes. According to Bion & Rothen [8], the development of a competency framework is “no more than a product specification”. They posit that the next ⁎ Corresponding author at: FC 11, Hanzeplein 1, 9700 RB, Groningen, The Netherlands. Tel.: +31 503612734. E-mail address: [email protected] (I.S. Dijkstra).
http://dx.doi.org/10.1016/j.jcrc.2017.01.018 0883-9441/© 2017 Elsevier Inc. All rights reserved.
phase in research on CBE in ICM should aim at examining whether better training will result in better specialists delivering better care. Quality of training and medical specialist performance are difficult to measure, predominantly due to the diverse or conflicting conceptions of what constitutes a good doctor [9]. Each phase in medical training should however strive to prepare trainees optimally for the next phase. Therefore, a useful approach to study the quality of ICM training is to evaluate whether newly graduated intensivists feel sufficiently prepared by their training program for the tasks they have to perform in independent practice. This approach may reveal specific demands from practice for which graduate intensivists feel insufficiently prepared and hence provide valuable information for further improvements in ICM training and its alignment with independent practice. Although the introduction of CBE in medical education was mainly driven by an intention to adhere to societal requirements and to implement generic competencies (e.g. communication, collaboration and leadership) more strongly [1], research within other disciplines has shown that graduates do not feel optimally prepared for tasks which require generic competencies like leadership and professionalism [10]. There is however no data available which shows how well competency-based ICM training programmes prepare their trainees for practice.
48
I.S. Dijkstra et al. / Journal of Critical Care 42 (2017) 47–53
The Netherlands was one of the first countries to adopt the CoBaTrICE framework in ICM training. Under responsibility of the Joint Intensivists Committee (GIC) a syllabus with a translated list of all ICM competencies was developed which is now being used by all nine ICM training institutes in the Netherlands for many years. To further harmonize training programmes, the GIC organizes monthly educational seminars for ICM fellows and performs regular audits to evaluate the quality and content (implementation of CoBaTrICE) of all ICM training programmes. As such all ICM fellows in the Netherlands receive a comparable training program. Given the extensive experience with CoBaTrICE and the harmonization of training programmes, the Netherlands provides a unique opportunity to evaluate CBE in ICM [11]. As a specific feature of ICM training, a supraspecialty model (in which completion of several postgraduate training programmes provides access to ICM training) applies to 60% of the European countries [6]. While anaesthesiology and internal medicine are the base specialties in most countries, other base specialties include pulmonary medicine, cardiology, surgery and neurology. Due to this multidisciplinary background, a specific challenge for ICM training is to provide every trainee with a program that fits his or her background while ensuring a certain quality standard at the end of training. Accordingly our study evaluates how well ICM trainees feel prepared for practice and whether trainees from different base specialties feel prepared for practice differently.
7 interviews were conducted (ID) with recently licensed intensivists. Because the aim of our study was to examine the alignment between ICM training and independent practice, we used these interviews to ensure inclusion of those tasks which novice intensivists regarded as the most challenging. In this way the completeness of the provisional inventory was checked and adapted. Results were discussed within the research team until consensus was reached. The resulting pilot version of the inventory was assessed by several ICM specialists in the field. Remarks and suggestions were discussed within the research team and the inventory was adapted accordingly. The final inventory consisted of 104 tasks divided among 10 clusters of similar content. For each task, respondents were asked to rate their agreement to the statement: “my ICM training program prepared me well for… [task] e.g. Airway management” on a 5-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree). Respondents could indicate nonrelevance when they did not encounter a specific task in their practice. Respondents were also asked to provide demographic characteristics such as gender, age, base specialty, date of ICM license, work experience in years, percentage of full time contract employed as intensivist, type of contract (employed or private practice) and type of hospital (university or general), size of ICU and to rate the quality of their ICM training on a 10-point scale ranging from 0 (poor) to 10 (excellent).
2. Methods
For all 104 tasks and the 10 clusters, mean scores and SD were calculated. Mean scores between 4 and 5 were considered as excellently prepared, between 3.5 and 4 as well prepared, between 3 and 3.5 as marginally sufficiently prepared and a mean score below 3 as insufficiently prepared [10]. Independent t-tests were performed to assess differences in mean preparedness scores between subgroups, such as different base specialties, gender, type of contract, or type of hospital. The reliability of the cluster scales was assessed with Cronbach's alpha, with a score above.70 defined as sufficient [12].
2.1. Context and setting This study was performed in the Netherlands where ICM training programmes are being offered by eight university medical centers and one large general teaching hospital. Access from several primary specialties (internal medicine, pulmonary medicine, anaesthesiology, surgery and neurology) can be gained into a common national 2-year training programme.
2.4. Data analysis
2.2. Participants
2.5. Ethical statement
The program leaders of the 9 ICM training programmes in the Netherlands were approached to provide contact information of intensivists who had completed their training between 1 and 5 years prior to the administration of the questionnaire. Respondents initially received an e-mail in which the study was announced and the purpose, background, procedure, anonymity of participation and further ethical issues of the study were explained. One week after this initial e-mail all respondents received an e-mail with a link to a web-based questionnaire. Reminders were sent after 2, 4 and 8 weeks.
The need for ethics approval was waived by the medical ethical committee of the University Medical Center Groningen (M14.156539). Study participation was voluntary and anonymously. Respondents could withdraw their participation at any moment during the study. All collected data was stored and processed anonymously. Our study complied with the declaration of Helsinki and current ethical standards [13].
2.3. Measures
3.1. Descriptives
To study preparedness for practice among ICM medical specialists we developed an inventory of tasks for ICM, which was applied as a questionnaire. The specific ICM task inventory was based on a validated generic inventory of tasks which has been used to study preparedness for practice across all medical specialties [9]. Because our study aimed at analyzing preparedness for practice among young intensivists specifically, customization of the generic inventory was performed by an incremental procedure. As a first step, two experienced intensivists (HD and JT) selected items of the generic inventory which were regarded inapplicable for ICM and could therefore be deleted (e.g. “performing surgery in an operating theatre”). Remaining items were rephrased to reflect ICM practice if necessary (e.g. “admitting a patient was rephrased to admitting a patient to IC”). New items were defined to cover the full scope of tasks of ICM practice after comparing the core competencies of the Dutch ICM training curriculum with the task inventory (e.g. “airway management”). In the next round,
Of the 205 intensivists invited, 91 completed the questionnaire (response rate 45%). There were 38 male (42%) and 53 female respondents (58%). Mean age was 39 (SD 3.4) years, and respondents were on average licensed as an intensivist for 3 years (SD 1.4). Most respondents had their base specialty in anaesthesiology (43%) and internal medicine (33%), with few respondents from other base specialties: cardiology (2%), surgery (3%), pulmonology (3%) and neurology (1%). 46 respondents (51%) worked fulltime as intensivist, whilst 29 respondents (32%) worked more than 0.5 FTE as a specialist in their base specialty in addition to their work as intensivist. The remaining 16 (17%) had a minor appointment as a specialist in their base specialty. Sixty-seven respondents (74%) were employed by a hospital and 24 (25%) worked in private practice. Twenty-eight (28%) worked in an academic medical center, 19 (21%) worked in a general hospital with a small ICU, 21 (23%) in a general hospital with a medium sized ICU and 25 (28%) in a general hospital with a large ICU.
3. Results
I.S. Dijkstra et al. / Journal of Critical Care 42 (2017) 47–53
49
Table 1 Graduated intensivists' preparedness for practice
Transfer, admission and discharge Admitting a patient to the ICU Transferring a patient from the ICU to the ward Transferring a patient to the ICU from another hospital Refusal of patient ICU admission Transferring a patient to another health care institution (e.g. nursing home or rehabilitation center) Discharging a patient Communication with patients and their family Providing information about medical situation and treatment plan to a patient's family members or a legal representative Explaining diagnosis, treatment plan and alternatives to patients Breaking bad news to patients and/or their family members Agreement on resuscitation policies Discussing palliative care and treatment restrictions Speaking on the phone with a patient”s family members Supporting a patient”s family Being attentive to a patients comfort (physically and mentally) Taking the lead in end-of-life care Discussing consent (for diagnostic procedures, donation, medication or surgery) Discussing medical errors, incidents of complications with patients or their family members Dealing with inappropriate (e.g. demanding or aggressive) behavior by patients or their family members Intraprofessional collaboration Consulting a colleague intensivist Participating in multidisciplinary consultation (e.g. radiologist, microbiologist) Consulting a colleague from a different medical specialty (e.g. radiologist) about additional diagnostic procedures Presenting a patient at a clinical conference (e.g. pre-operative of high care) Offering advice to a colleague who asked for consultation Asking another medical specialist to perform a treatment as expert consultant Coordinating different aspects of care for an individual patient Consulting with a general practitioner about a patient Performing a treatment at the request of a fellow physician Being directly responsible for ICU care and call on the multidisciplinary expertise of various consultants. Leading a multidisciplinary patient care-team (intensivists and ICU nurses) Providing second opinion about a patient in another hospital IC-specific tasks Setting the mechanical ventilation in the uncomplicated (postoperative) patient Insertion of a central venous line Recognition of serious and life-threatening conditions Initial care of the ICU patient Postoperative care of the ICU patient Preparing a patient for technical or surgical procedures (sterility. Safety) Setting of treatment limits Withdrawing ICU treatment Leading a resuscitation procedure (BLS/ALS) Setting the indication for a patient ICU admission Preparing for the daily ICU rounds Making a treatment plan for the ICU patient Interpreting laboratory results Leading a team in the initial care of the unstable ICU patients or resuscitation (leader hands off) Setting the indication and the use of sedation/pain management protocols Making a diagnostic plan Setting the mechanical ventilation in a complicated patient (eg, ARDS) Doing services in the hospital or on call Performing airway management Supervising physician assistants and nurses in technical procedures Setting the indication for surgery in critically ill patients Recognize complications of organ supportive interventions Using of drugs in organ dysfunction Ultrasound guided insertion of central venous lines Prevention and treatment of infections in the ICU patient Providing ICU support during procedures in a non-ICU environment (eg, PCI or ERCP) Setting the indications and adjustment of technical equipment in organ dysfunction Setting the ventilation in a patient with severe COPD Interpreting imaging Preoperative optimization of IC patients Supervising a MICU transport Performing echocardiography Assessing patients on the post-ICU clinic Performing abdominal ultrasound Interprofessional collaboration Collaborating with nurses Collaborating with support staff (e.g. dieticians and physical therapists) Collaborating with administrative support staff Collaborating with spiritual or religious counselors Collaborating with psychosocial professionals
Mean
SD
4.25 4.55 4.45 4.36 4.10 4.06 3.91 4.22 4.42 4.38 4.37 4.35 4.35 4.32 4.30 4.28 4.20 4.19 3.86 3.65 4.19 4.44 4.35 4.34 4.29 4.29 4.28 4.27 4.26 4.19 4.13 4.06 3.02 4.16 4.67 4.63 4.60 4.58 4.57 4.47 4.44 4.44 4.43 4.42 4.41 4.38 4.38 4.35 4.30 4.28 4.26 4.24 4.24 4.24 4.20 4.18 4.16 4.14 4.10 4.06 4.06 4.05 4.05 3.93 3.73 2.53 2.35 2.03 4.14 4.51 4.24 4.14 3.96 3.96
.55 .50 .50 .56 .95 .84 .96 .54 .55 .58 .64 .60 .62 .63 .67 .58 .82 .64 .92 .91 .43 .50 .51 .50 .51 .53 .50 .50 .55 .56 .70 .70 1.15 .43 .52 .49 .52 .52 .50 .53 .57 .63 .66 .64 .60 .53 .51 .74 .72 .57 .82 .68 .94 .65 .61 .69 .57 1.08 .76 .98 .70 1.00 .79 .87 1.27 1.17 1.23 .95 .61 .53 .79 .71 .73 .76 (continued on next page)
50
I.S. Dijkstra et al. / Journal of Critical Care 42 (2017) 47–53
Table 1 (continued)
Collaborating with nurse practitioners Education Supervising medical students Supervising residents or fellows IC Teaching medical students. Residents or fellows IC Teaching nurses. Laboratory personnel or nurse practitioners Contributing to changes in medical education Contributing to continuing professional development of colleagues (e.g. medical specialists or family physicians) Providing medical information to groups (e.g. patient support groups or parents) Professional development Identify gaps in own knowledge and skills Staying up to date in the field by attending continuing professional development courses Keeping up with the literature in the field Dealing with feedback from colleague medical specialists Discussing incidents. Medical errors or complaints in a structured fashion Improving competence in medical education by attending faculty development courses Contributing to the development of the field in own specialty Providing feedback to colleague medical specialists Improving competence in organization and management by attending continuing professional development activities Patient-related administration Writing or dictating letters about patients Keeping a patients” chart up-to-date Using a patient data management systems (PDMS) Filling out patient registration forms (e.g. NICE) Filling out diagnosis forms Research Giving presentations at symposia and conferences Writing a research paper Performing scientific research Preparing and submitting grant applications Management and leadership Monitoring the quality and safety of patient care in the ICU Participating in hospital committees (e.g. infection or M&M committee) Taking decisions about deployment of ICU personnel Participating in regional or national committees (e.g. scientific society committees) Spending available resources for patient care effectively and responsibly Contributing to organizational changes of the unit or department Performing management and organization activities for the partnership. Ward or unit Taking decisions about the acquisition of materials and equipment for the ICU Negotiating with health care insurance providers about compensation for a treatment
Mean
SD
3.90 4.00 4.26 4.19 4.18 4.18 3.78 3.73 3.54 3.73 4.10 4.03 3.90 3.79 3.78 3.57 3.50 3.39 3.08 3.48 4.31 4.25 4.05 3.53 1.89 2.81 3.51 3.11 2.51 2.00 2.66 3.16 2.64 2.64 2.61 2.61 2.58 2.36 2.26 1.38
1.06 .55 .57 .59 .60 .62 .89 .85 1.07 .53 .55 .56 .68 .66 .87 1.00 .90 .91 1.04 .66 .67 .71 1.07 1.08 .98 .83 .90 1.09 1.07 .94 .90 1.03 1.09 1.04 1.01 1.07 1.05 1.05 .99 .49
Mean score between 4 and 5 = excellently prepared. Mean score between 3.5 and 4 = well prepared. Mean score between 3 and 3.5 = marginally sufficiently prepared. Mean score below 3 = insufficiently prepared.
3.2. Preparedness for practice
4. Discussion
Overall, intensivists felt well prepared for practice, with an aggregate mean score on all tasks of 3.95 (SD = 0.43). They rated their ICM training with a mean score of 8.4 (SD = 1.2) on a scale ranging from 0 (poor) to 10 (excellent). Respondents felt excellently prepared for 67 tasks (64%), well prepared for 16 tasks (15%), marginally sufficiently prepared for 6 tasks (6%) and insufficiently prepared for 15 tasks (14%) (Table 1). On cluster level, respondents felt insufficiently prepared for tasks concerning management & leadership (Mean = 2.66, SD = 0.9) and science (Mean = 2.81, SD = 0.83). They felt marginally sufficiently prepared for tasks concerning patient related administration (Mean = 3.48, SD = 0.63).
The results of this study show that recently licensed intensivists feel well prepared for the majority of the tasks that constitute independent practice. However there are numerous tasks for which they feel less well equipped by their ICM training programmes. In line with comparable studies in other disciplines [10,14,15], intensivists feel well prepared for tasks directly related to patient care, but preparedness scores are considerably lower for tasks that require more general competencies like management and leadership, science and professionalism. Because the explanations for and implications of these findings may differ between topics, they will be discussed separately. One of the most striking findings of our study is the low preparedness scores of all tasks of the management and leadership cluster. This finding is consistent with previous studies showing that new consultants generally feel not well prepared by postgraduate medical education for tasks concerning management and leadership [10,16]. A logical explanation for this common mismatch between postgraduate training programmes and independent practice is that trainees generally do not have such higher order leadership tasks, roles and responsibilities [17]. This may be problematic as intensivists can be confronted with such tasks soon after starting independent practice. The confrontation with new roles, tasks and responsibilities, together with their perceived incompetence in these general management and leadership competencies put recently licensed intensivists at risk of stress and
3.3. Differences between subgroups Due to the small sample sizes from other base specialties, only a comparison between intensivists from internal medicine and anaesthesiology could be calculated. Intensivists from anaesthesiology felt better prepared for IC specific activities (Mean = 4.25, SD = 0.38) than those from internal medicine (Mean = 4.01, SD = 0.40), t(66) = 2.43, P = .02, 95% CI [0.04–0. 76]. Anaesthesiology-based intensivists had higher scores on 17 tasks, and intensivists from internal medicine on 5 tasks (Table 2). Other subgroup analyses revealed no significant differences.
I.S. Dijkstra et al. / Journal of Critical Care 42 (2017) 47–53
51
Table 2 Statistical significant differences in preparedness for practice between intensivists with background in anaesthesiology or internal medicine (P b .05) Anesthesiology
Internal medicine
PValue
Transfer. admission and discharge Admitting a patient to the ICU
4.66
4.40
0.03
Communication with patients and their family Speaking on the phone with a patient”s family members Supporting a patient”s family
4.11 4.14
4.47 4.47
0.02 0.04
IC-specific tasks Insertion of a central venous line Initial care of the ICU patient Postoperative care of the ICU patient Leading a resuscitation procedure (BLS/ALS) Leading a team in the initial care of the unstable ICU patients or resuscitation Setting the indication and the use of sedation/pain management protocols Setting the mechanical ventilation in a complicated patient (e.g. ARDS) Performing airway management Ultrasound guided insertion of central venous lines Prevention and treatment of infections in the ICU patient Providing ICU support during procedures in a non-ICU environment (eg. PCI or ERCP) Setting the indications and adjustment of technical equipment in organ dysfunction Setting the ventilation in a patient with severe COPD Interpreting imaging Preoperative optimization of IC patients Supervising a MICU transport Performing echocardiography Performing abdominal ultrasound
4.78 4.76 4.66 4.59 4.53 4.57 4.47 4.76 4.70 4.00 4.43 3.89 4.32 3.87 4.37 4.15 2.92 2.28
4.40 4.30 4.40 4.17 4.03 3.97 4.03 3.57 3.80 4.33 3.59 4.30 3.83 4.27 3.42 3.35 2.03 1.61
b0.01 b0.01 0.03 b0.01 b0.01 b0.01 0.02 b0.01 b0.01 0.04 b0.01 0.02 0.05 0.01 b0.01 0.03 b0.01 b0.01
Professional Development Providing feedback to colleague medical specialists
3.58
3.13
0.04
burnout [15,18]. As the leadership role for physicians is becoming increasingly important, initiatives are being developed internationally to improve residents' training in management and leadership skills. [19] The results of our study call for a similar increased attention to and training in management and leadership skills in ICM training programmes. Respondents in our study reported low preparedness scores for research related tasks. Although roughly a quarter of our respondents worked in academic centers in which research is an integral part of medical practice, the majority of respondents worked in general hospitals. Because performing research generally forms a smaller part of daily practice in general hospitals than in academic centers, poor preparedness for research related tasks may not be a major concern in all cases. However, because of the increasingly rapidly expanding body of knowledge in medical science in general and in ICM in particular, the demands for applying evidence based medical practice have risen substantially and will probably continue to do so. The ability to perform research may positively influence the ability to judge the quality and implication of research, and to apply possible findings in daily practice. From this perspective low preparedness scores on research related tasks are undesirable, and systematic training and practice of evidence based medicine skills in ICM training is strongly recommended. Within the professional development cluster, three tasks received insufficiently preparedness scores, and four tasks were graded as marginally sufficiently prepared. The relatively low scores of providing feedback to colleagues, dealing with feedback from colleagues and discussing critical incidents, medical errors or complaints during a conference in a structured fashion are of interest. Open communication within health care teams are of key importance in team-based patient safety behavior and hence related to patient-outcomes [20]. In addition, allowing and encouraging team members to express thoughts and to provide feedback are important aspects of a safe and inspiring learning climate [21]. Hence, we feel that it is mandatory to prepare trainees optimally for such tasks. As the learning climate is deeply embedded in the working climate of a specific department or ICU [22], it is unlikely that the acquisition and sufficient training of these professional
development skills can be obtained through specific courses. Because the working climate is dependent on the values and beliefs of all team members, particularly those of the most influential team members [23,24], improvements in the graduate training of ICM trainees require an integral approach in which faculty, trainees and nursing personnel work together to achieve a safe and inspiring working climate. Examples of such integral interprofessional approaches moreover have proven to be beneficial for patient-outcomes [25]. Respondents of our study felt well prepared for the majority of the tasks within the Intensive Care specific task cluster. The low scores on performing ultrasound examinations can be explained by the fact that these techniques were not incorporated in the curriculum during the period that our respondents were trained. Because these techniques have now generally been added to the training programme, and are practiced regularly by ICM residents in their daily work, we expect graduate intensivists to be better prepared for ultrasound examination tasks in the coming years. Similarly, respondents felt insufficiently prepared for dealing with patients in intensive care follow-up clinics. The awareness of post intensive care syndrome [26], and the need for follow up on patients with physical and mental health problems after ICU treatment has only recently led to the development of post-ICU clinics in some hospitals in the Netherlands [27]. The comparisons between respondents with a background specialty in anaesthesiology and internal medicine showed several differences, particularly within the Intensive Care specific tasks cluster. The increased exposure to ICM specific skills such as airway management and vital function instability during base specialty training of anaesthesists as compared to internal medicine specialists likely accounts for most of these differences. It is of concern, however, that these differences are still significant at completion of ICM training. Therefore, to attain more uniform outcomes, individual training programmes need to be customized to each trainee's specific needs more extensively. This requires accurate assessment of knowledge, skills and behavior prior to admission and individualized learning trajectories, adequate supervision and assessment during training. Defining ICM specific skills as entrustable professional activities (EPA), [28] may help to ensure adequate training, supervision and assessment of
52
I.S. Dijkstra et al. / Journal of Critical Care 42 (2017) 47–53
competence to reduce these significant differences between ICM specialists from different backgrounds [11]. Although there is debate whether competency-based education provides better training than traditional time-based models, [29,30] ICM training may be specifically suitable for adopting EPA's due to its technical and well defined characteristics.
6. Take home message
4.1. Limitations
Funding
In absence of suitable objective data, we relied on self-report measures to measure the alignment between ICM training en independent practice. Therefore the results of our study do not necessarily reflect competence or performance. Feedback from former trainees is however valuable as they can reflect on the content of their training in relation to their experiences in independent practice. Our study hence provides valuable information about how well ICM training programmes prepare trainees for practice. The sample size was relatively small and nonresponse may therefore have affected the results of our study. Respondents were equally distributed across the different training programmes in the Netherlands and our sample seems representative for the population in terms of biographic parameters. Moreover the results of our study were in line with comparable studies [10]. Therefore we believe that non response has not distorted the results of our study meaningfully. During the administration of our questionnaire and onwards, there has been a continuing debate about the distribution and allocation of ICU's in the Netherlands and obligations for smaller ICU's to discuss or transfer certain patients to larger ICU's on a regular basis. An ICU is of key importance for hospitals and this debate may therefore have created feelings of insecurity about future employment of respondents in smaller hospitals particularly. Therefore this debate may have influenced the response rate.
This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.
5. Conclusions In line with findings in others specialties [10], graduate intensivists generally feel well prepared for independent ICM practice, but they feel insufficiently prepared for tasks related to leadership and management, science, and professional development. These results call for a reevaluation and update of the current (Dutch) CoBaTrICE framework. Although the medical expert role should evidently remain the core of ICM training, ICM training frameworks should expand its focus to the generic roles of independent practice more explicitly. It is however questionable whether a two-year training programme provides enough time and opportunities to train fellow intensivists better for these generic roles without compromising the development of ICM specific medical expertise. Additionally, our finding that trainees from different background specialties attain different preparedness rates for ICM specific tasks especially, may imply that the current two-year training programme is too short to prepare trainees optimally for the full scope of independent practice. Alternatively it could be argued that training programmes do not have to prepare trainees for every task to the same extend, provided that they receive sufficient support to develop expertise when confronted with these tasks in independent practice through continuing medical education or on the job support from senior co-workers. Although in the last decade much has been changed in undergraduate and postgraduate medical education to pay more attention to the generic roles of medical practice, new consultants apparently still report deficiencies in their training. As such our findings may guide discussions about the alignment and integration of the different phases of the medical education continuum, the required length of ICM training, or the desirability for ICM to become an independent specialty with corresponding training length.
Although graduated intensivist feel well prepared for the majority of the tasks of independent practice, they report deficiencies regarding the preparation for tasks related to leadership, management, science and professional development. Therefore ICM training programmes should address these generic roles of independent practice more explicitly.
Conflicts of interest None. Acknowledgements We would like to acknowledge all programme leaders of ICM training programmes in the Netherlands for their participation to our study. References [1] Frank J. The CanMEDS 2005 physician competency framework. Better standards. Better physicians. Better care; 2005. [2] Frank JR, Danoff D. The CanMEDS initiative: implementing an outcomes-based framework of physician competencies. Med Teach 2007;29:642–7. [3] Bion JF, Barrett H. Development of core competencies for an international training programme in intensive care medicine. Intensive Care Med 2006;32:1371–83. [4] Bion JF, Libreau G, Barrett H. CoBaTrICE competency based training in intensive care medicine in Europe: creating the European intensivist. Care Crit Ill 2004;20:57–8 [2 pp.]. [5] International standards for programmes of training in intensive care medicine in EuropeIntensive Care Med 2011;37:385–93. [6] The educational environment for training in intensive care medicine: structures, processes, outcomes and challenges in the European regionIntensive Care Med 2009;35:1575–83. [7] The views of patients and relatives of what makes a good intensivist: a European surveyIntensive Care Med 2007;33:1913–20. [8] Bion J, Rothen HU. Competency-based training in intensive care medicine, 12; 2012. [9] Dijkstra IS, Pols J, Remmelts P, Bakker B, Mooij JJ, Borleffs JC, et al. What are we preparing them for? Development of an inventory of tasks for medical, surgical and supportive specialties. Med Teach 2013;35:e1068–77. [10] Dijkstra IS, Pols J, Remmelts P, Brand PL. Preparedness for practice: a systematic cross-specialty evaluation of the alignment between postgraduate medical education and independent practice. Med Teach 2015;37:153–61. [11] Tulleken JE. Intensive care medicine curricula in Europe. Intensive Care Med 2015; 41:2241. [12] Bland JM, Altman DG. Cronbach's alpha. BMJ 1997;314:572. [13] Ten Cate O. Why the ethics of medical education research differs from that of medical research. Med Educ 2009;43:608–10. [14] Morrow G, Illing J, Redfern N, Burford B, Kergon C, Briel R. Are specialist registrars fully prepared for the role of consultant? Clin Teach 2009;6:87–90. [15] Westerman M, Teunissen PW, Fokkema JPI, van der Vleuten CP, Scherpbier AJ, Siegert CE, et al. The transition to hospital consultant and the influence of preparedness, social support, and perception: a structural equation modelling approach. Med Teach 2013;35:320–7. [16] Berkenbosch L, Brouns JW, Heyligers I, Busari JO. How Dutch medical residents perceive their competency as manager in the revised postgraduate medical curriculum. Postgrad Med J 2011;87:680–7. [17] Howieson B, Thiagarajah T. What is clinical leadership? A journal-based metareview. Int J Clin Leadersh 2011;17:7–18. [18] Westerman M, Teunissen PW, van der Vleuten CP, Scherpbier AJ, Siegert CE, van der Lee N, et al. Understanding the transition from resident to attending physician: a transdisciplinary, qualitative study. Acad Med 2010;85:1914–9. [19] Blumenthal DM, Bernard K, Bohnen J, Bohmer R. Addressing the leadership gap in medicine: Residents' need for systematic leadership development training. Acad Med 2012;87:513–22. [20] Brock D, Abu-Rish E, Chiu CR, Hammer D, Wilson S, Vorvick L, et al. Interprofessional education in team communication: working together to improve patient safety. BMJ Qual Saf 2013;22:414–23. [21] Roff S, McAleer S, Skinner A. Development and validation of an instrument to measure the postgraduate clinical learning and teaching educational environment for hospital-based junior doctors in the UK. Med Teach 2005;27:326–31. [22] Genn JM. AMEE medical education guide no. 23 (part 1): curriculum, environment, climate, quality and change in medical education—a unifying perspective. Med Teach 2001;23:337–44.
I.S. Dijkstra et al. / Journal of Critical Care 42 (2017) 47–53 [23] Paice E, Heard S, Moss F. How important are role models in making good doctors? BMJ 2002;325:707–10. [24] Passi V, Johnson S, Peile E, Wright S, Hafferty F, Johnson N. Doctor role modelling in medical education: BEME guide no. 27. Med Teach 2013;35:e1422–36. [25] Zwarenstein M, Goldman J, Reeves S. Interprofessional collaboration: effects of practice-based interventions on professional practice and healthcare outcomes. Cochrane Database Syst Rev 2009:CD000072. [26] Griffiths J, Fortune G, Barber V, Young JD. The prevalence of post-traumatic stress disorder in survivors of ICU treatment: a systematic review. Intensive Care Med 2007;33:1506–18.
53
[27] Van Der Schaaf M, Bakhshi-Raiez F, Van DS, Dongelmans DA, De Keizer NF. Recommendations for intensive care follow-up clinics; report from a survey and conference of Dutch intensive cares. Minerva Anestesiol 2015;81:135–44. [28] Ten Cate O, Chen HC, Hoff RG, Peters H, Bok H, van der Schaaf M. Curriculum development for the workplace using entrustable professional activities (EPAs): AMEE guide no. 99. Med Teach 2015;37:983–1002. [29] Hodges BD. A tea-steeping or i-doc model for medical education? Acad Med 2010; 85:S34–44. [30] Touchie C, ten Cate O. The promise, perils, problems and progress of competencybased medical education. Med Educ 2016;50:93–100.
Contents lists available at ScienceDirect
Journal of Critical Care journal homepage: www.jccjournal.org
Are graduated intensivists prepared for practice? A case study from The Netherlands Ids S. Dijkstra a,⁎, Paul L.P. Brand a,b, Jan Pols c, Hans Delwig d, Debbie A.D.C. Jaarsma c, Jaap E. Tulleken d a
Wenckebach Institute, University of Groningen, University Medical Center Groningen, Groningen, The Netherlands Princess Amalia Children's Centre, Isala Hospital, Zwolle, The Netherlands Center for Educational Development and Research in Health Professions, University of Groningen, University Medical Center Groningen, Groningen, The Netherlands d Department of Critical Care, University of Groningen, University Medical Center Groningen, Groningen, The Netherlands b c
a r t i c l e
i n f o
Keywords: CoBaTrICE Preparedness for practice Competency-based education ICM- training Evaluation
a b s t r a c t Purpose: An evaluation of the alignment between intensive care medicine (ICM) training and practice provides valuable information for the development of ICM training. Therefore this study examines how well recently licensed intensivists feel prepared for practice and whether intensivists from different background specialties attain comparable preparedness rates. Methods: An inventory was developed to cover the tasks that constitute ICM practice. Two hundred five recently licensed Dutch intensivists received a questionnaire in which they could indicate how well their ICM training programme prepared them for these tasks on a 5-point Likert scale. Results: Ninety-one respondents returned the questionnaire (response 45%). Respondents felt excellently prepared for 67 tasks, well prepared for 16 tasks, marginally sufficiently prepared for 6 tasks and insufficiently prepared for 15 tasks. Intensivists from anaesthesiology felt better prepared for IC specific activities (mean 4.25, SD 0.38) than those from internal medicine (mean 4.01, SD 0.40, P = .02).Average scores on tasks related to medical expertise were relatively high while tasks relating to management and leadership, science and professional development scored lower. Conclusions: Although recently licensed intensivists are well prepared for most tasks in ICM, lower preparedness scores on tasks related to leadership and management, science, and professional development call for reevaluation of the current curriculum. © 2017 Elsevier Inc. All rights reserved.
1. Introduction Competency-based education (CBE) has been adopted to better respond to future challenges in health-care and to meet societal requirements [1,2]. In intensive care medicine (ICM), 43 national societies have collaborated to develop a competency framework and common standards for postgraduate training within the “Competency Based Training in Intensive Care medicine for Europe collaboration” (CoBaTrICE) [3,4]. These standards should harmonize training in ICM without interfering with specific national regulations, allow for free movement of intensivists across Europe, and ensure high quality education in ICM. Most research on the development of CBE in ICM has focused on international standards, structures, processes and assessment, or on the patients' views on what makes a good intensivist [5-7]. In contrast, only few studies have evaluated the content of ICM training programmes. According to Bion & Rothen [8], the development of a competency framework is “no more than a product specification”. They posit that the next ⁎ Corresponding author at: FC 11, Hanzeplein 1, 9700 RB, Groningen, The Netherlands. Tel.: +31 503612734. E-mail address: [email protected] (I.S. Dijkstra).
http://dx.doi.org/10.1016/j.jcrc.2017.01.018 0883-9441/© 2017 Elsevier Inc. All rights reserved.
phase in research on CBE in ICM should aim at examining whether better training will result in better specialists delivering better care. Quality of training and medical specialist performance are difficult to measure, predominantly due to the diverse or conflicting conceptions of what constitutes a good doctor [9]. Each phase in medical training should however strive to prepare trainees optimally for the next phase. Therefore, a useful approach to study the quality of ICM training is to evaluate whether newly graduated intensivists feel sufficiently prepared by their training program for the tasks they have to perform in independent practice. This approach may reveal specific demands from practice for which graduate intensivists feel insufficiently prepared and hence provide valuable information for further improvements in ICM training and its alignment with independent practice. Although the introduction of CBE in medical education was mainly driven by an intention to adhere to societal requirements and to implement generic competencies (e.g. communication, collaboration and leadership) more strongly [1], research within other disciplines has shown that graduates do not feel optimally prepared for tasks which require generic competencies like leadership and professionalism [10]. There is however no data available which shows how well competency-based ICM training programmes prepare their trainees for practice.
48
I.S. Dijkstra et al. / Journal of Critical Care 42 (2017) 47–53
The Netherlands was one of the first countries to adopt the CoBaTrICE framework in ICM training. Under responsibility of the Joint Intensivists Committee (GIC) a syllabus with a translated list of all ICM competencies was developed which is now being used by all nine ICM training institutes in the Netherlands for many years. To further harmonize training programmes, the GIC organizes monthly educational seminars for ICM fellows and performs regular audits to evaluate the quality and content (implementation of CoBaTrICE) of all ICM training programmes. As such all ICM fellows in the Netherlands receive a comparable training program. Given the extensive experience with CoBaTrICE and the harmonization of training programmes, the Netherlands provides a unique opportunity to evaluate CBE in ICM [11]. As a specific feature of ICM training, a supraspecialty model (in which completion of several postgraduate training programmes provides access to ICM training) applies to 60% of the European countries [6]. While anaesthesiology and internal medicine are the base specialties in most countries, other base specialties include pulmonary medicine, cardiology, surgery and neurology. Due to this multidisciplinary background, a specific challenge for ICM training is to provide every trainee with a program that fits his or her background while ensuring a certain quality standard at the end of training. Accordingly our study evaluates how well ICM trainees feel prepared for practice and whether trainees from different base specialties feel prepared for practice differently.
7 interviews were conducted (ID) with recently licensed intensivists. Because the aim of our study was to examine the alignment between ICM training and independent practice, we used these interviews to ensure inclusion of those tasks which novice intensivists regarded as the most challenging. In this way the completeness of the provisional inventory was checked and adapted. Results were discussed within the research team until consensus was reached. The resulting pilot version of the inventory was assessed by several ICM specialists in the field. Remarks and suggestions were discussed within the research team and the inventory was adapted accordingly. The final inventory consisted of 104 tasks divided among 10 clusters of similar content. For each task, respondents were asked to rate their agreement to the statement: “my ICM training program prepared me well for… [task] e.g. Airway management” on a 5-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree). Respondents could indicate nonrelevance when they did not encounter a specific task in their practice. Respondents were also asked to provide demographic characteristics such as gender, age, base specialty, date of ICM license, work experience in years, percentage of full time contract employed as intensivist, type of contract (employed or private practice) and type of hospital (university or general), size of ICU and to rate the quality of their ICM training on a 10-point scale ranging from 0 (poor) to 10 (excellent).
2. Methods
For all 104 tasks and the 10 clusters, mean scores and SD were calculated. Mean scores between 4 and 5 were considered as excellently prepared, between 3.5 and 4 as well prepared, between 3 and 3.5 as marginally sufficiently prepared and a mean score below 3 as insufficiently prepared [10]. Independent t-tests were performed to assess differences in mean preparedness scores between subgroups, such as different base specialties, gender, type of contract, or type of hospital. The reliability of the cluster scales was assessed with Cronbach's alpha, with a score above.70 defined as sufficient [12].
2.1. Context and setting This study was performed in the Netherlands where ICM training programmes are being offered by eight university medical centers and one large general teaching hospital. Access from several primary specialties (internal medicine, pulmonary medicine, anaesthesiology, surgery and neurology) can be gained into a common national 2-year training programme.
2.4. Data analysis
2.2. Participants
2.5. Ethical statement
The program leaders of the 9 ICM training programmes in the Netherlands were approached to provide contact information of intensivists who had completed their training between 1 and 5 years prior to the administration of the questionnaire. Respondents initially received an e-mail in which the study was announced and the purpose, background, procedure, anonymity of participation and further ethical issues of the study were explained. One week after this initial e-mail all respondents received an e-mail with a link to a web-based questionnaire. Reminders were sent after 2, 4 and 8 weeks.
The need for ethics approval was waived by the medical ethical committee of the University Medical Center Groningen (M14.156539). Study participation was voluntary and anonymously. Respondents could withdraw their participation at any moment during the study. All collected data was stored and processed anonymously. Our study complied with the declaration of Helsinki and current ethical standards [13].
2.3. Measures
3.1. Descriptives
To study preparedness for practice among ICM medical specialists we developed an inventory of tasks for ICM, which was applied as a questionnaire. The specific ICM task inventory was based on a validated generic inventory of tasks which has been used to study preparedness for practice across all medical specialties [9]. Because our study aimed at analyzing preparedness for practice among young intensivists specifically, customization of the generic inventory was performed by an incremental procedure. As a first step, two experienced intensivists (HD and JT) selected items of the generic inventory which were regarded inapplicable for ICM and could therefore be deleted (e.g. “performing surgery in an operating theatre”). Remaining items were rephrased to reflect ICM practice if necessary (e.g. “admitting a patient was rephrased to admitting a patient to IC”). New items were defined to cover the full scope of tasks of ICM practice after comparing the core competencies of the Dutch ICM training curriculum with the task inventory (e.g. “airway management”). In the next round,
Of the 205 intensivists invited, 91 completed the questionnaire (response rate 45%). There were 38 male (42%) and 53 female respondents (58%). Mean age was 39 (SD 3.4) years, and respondents were on average licensed as an intensivist for 3 years (SD 1.4). Most respondents had their base specialty in anaesthesiology (43%) and internal medicine (33%), with few respondents from other base specialties: cardiology (2%), surgery (3%), pulmonology (3%) and neurology (1%). 46 respondents (51%) worked fulltime as intensivist, whilst 29 respondents (32%) worked more than 0.5 FTE as a specialist in their base specialty in addition to their work as intensivist. The remaining 16 (17%) had a minor appointment as a specialist in their base specialty. Sixty-seven respondents (74%) were employed by a hospital and 24 (25%) worked in private practice. Twenty-eight (28%) worked in an academic medical center, 19 (21%) worked in a general hospital with a small ICU, 21 (23%) in a general hospital with a medium sized ICU and 25 (28%) in a general hospital with a large ICU.
3. Results
I.S. Dijkstra et al. / Journal of Critical Care 42 (2017) 47–53
49
Table 1 Graduated intensivists' preparedness for practice
Transfer, admission and discharge Admitting a patient to the ICU Transferring a patient from the ICU to the ward Transferring a patient to the ICU from another hospital Refusal of patient ICU admission Transferring a patient to another health care institution (e.g. nursing home or rehabilitation center) Discharging a patient Communication with patients and their family Providing information about medical situation and treatment plan to a patient's family members or a legal representative Explaining diagnosis, treatment plan and alternatives to patients Breaking bad news to patients and/or their family members Agreement on resuscitation policies Discussing palliative care and treatment restrictions Speaking on the phone with a patient”s family members Supporting a patient”s family Being attentive to a patients comfort (physically and mentally) Taking the lead in end-of-life care Discussing consent (for diagnostic procedures, donation, medication or surgery) Discussing medical errors, incidents of complications with patients or their family members Dealing with inappropriate (e.g. demanding or aggressive) behavior by patients or their family members Intraprofessional collaboration Consulting a colleague intensivist Participating in multidisciplinary consultation (e.g. radiologist, microbiologist) Consulting a colleague from a different medical specialty (e.g. radiologist) about additional diagnostic procedures Presenting a patient at a clinical conference (e.g. pre-operative of high care) Offering advice to a colleague who asked for consultation Asking another medical specialist to perform a treatment as expert consultant Coordinating different aspects of care for an individual patient Consulting with a general practitioner about a patient Performing a treatment at the request of a fellow physician Being directly responsible for ICU care and call on the multidisciplinary expertise of various consultants. Leading a multidisciplinary patient care-team (intensivists and ICU nurses) Providing second opinion about a patient in another hospital IC-specific tasks Setting the mechanical ventilation in the uncomplicated (postoperative) patient Insertion of a central venous line Recognition of serious and life-threatening conditions Initial care of the ICU patient Postoperative care of the ICU patient Preparing a patient for technical or surgical procedures (sterility. Safety) Setting of treatment limits Withdrawing ICU treatment Leading a resuscitation procedure (BLS/ALS) Setting the indication for a patient ICU admission Preparing for the daily ICU rounds Making a treatment plan for the ICU patient Interpreting laboratory results Leading a team in the initial care of the unstable ICU patients or resuscitation (leader hands off) Setting the indication and the use of sedation/pain management protocols Making a diagnostic plan Setting the mechanical ventilation in a complicated patient (eg, ARDS) Doing services in the hospital or on call Performing airway management Supervising physician assistants and nurses in technical procedures Setting the indication for surgery in critically ill patients Recognize complications of organ supportive interventions Using of drugs in organ dysfunction Ultrasound guided insertion of central venous lines Prevention and treatment of infections in the ICU patient Providing ICU support during procedures in a non-ICU environment (eg, PCI or ERCP) Setting the indications and adjustment of technical equipment in organ dysfunction Setting the ventilation in a patient with severe COPD Interpreting imaging Preoperative optimization of IC patients Supervising a MICU transport Performing echocardiography Assessing patients on the post-ICU clinic Performing abdominal ultrasound Interprofessional collaboration Collaborating with nurses Collaborating with support staff (e.g. dieticians and physical therapists) Collaborating with administrative support staff Collaborating with spiritual or religious counselors Collaborating with psychosocial professionals
Mean
SD
4.25 4.55 4.45 4.36 4.10 4.06 3.91 4.22 4.42 4.38 4.37 4.35 4.35 4.32 4.30 4.28 4.20 4.19 3.86 3.65 4.19 4.44 4.35 4.34 4.29 4.29 4.28 4.27 4.26 4.19 4.13 4.06 3.02 4.16 4.67 4.63 4.60 4.58 4.57 4.47 4.44 4.44 4.43 4.42 4.41 4.38 4.38 4.35 4.30 4.28 4.26 4.24 4.24 4.24 4.20 4.18 4.16 4.14 4.10 4.06 4.06 4.05 4.05 3.93 3.73 2.53 2.35 2.03 4.14 4.51 4.24 4.14 3.96 3.96
.55 .50 .50 .56 .95 .84 .96 .54 .55 .58 .64 .60 .62 .63 .67 .58 .82 .64 .92 .91 .43 .50 .51 .50 .51 .53 .50 .50 .55 .56 .70 .70 1.15 .43 .52 .49 .52 .52 .50 .53 .57 .63 .66 .64 .60 .53 .51 .74 .72 .57 .82 .68 .94 .65 .61 .69 .57 1.08 .76 .98 .70 1.00 .79 .87 1.27 1.17 1.23 .95 .61 .53 .79 .71 .73 .76 (continued on next page)
50
I.S. Dijkstra et al. / Journal of Critical Care 42 (2017) 47–53
Table 1 (continued)
Collaborating with nurse practitioners Education Supervising medical students Supervising residents or fellows IC Teaching medical students. Residents or fellows IC Teaching nurses. Laboratory personnel or nurse practitioners Contributing to changes in medical education Contributing to continuing professional development of colleagues (e.g. medical specialists or family physicians) Providing medical information to groups (e.g. patient support groups or parents) Professional development Identify gaps in own knowledge and skills Staying up to date in the field by attending continuing professional development courses Keeping up with the literature in the field Dealing with feedback from colleague medical specialists Discussing incidents. Medical errors or complaints in a structured fashion Improving competence in medical education by attending faculty development courses Contributing to the development of the field in own specialty Providing feedback to colleague medical specialists Improving competence in organization and management by attending continuing professional development activities Patient-related administration Writing or dictating letters about patients Keeping a patients” chart up-to-date Using a patient data management systems (PDMS) Filling out patient registration forms (e.g. NICE) Filling out diagnosis forms Research Giving presentations at symposia and conferences Writing a research paper Performing scientific research Preparing and submitting grant applications Management and leadership Monitoring the quality and safety of patient care in the ICU Participating in hospital committees (e.g. infection or M&M committee) Taking decisions about deployment of ICU personnel Participating in regional or national committees (e.g. scientific society committees) Spending available resources for patient care effectively and responsibly Contributing to organizational changes of the unit or department Performing management and organization activities for the partnership. Ward or unit Taking decisions about the acquisition of materials and equipment for the ICU Negotiating with health care insurance providers about compensation for a treatment
Mean
SD
3.90 4.00 4.26 4.19 4.18 4.18 3.78 3.73 3.54 3.73 4.10 4.03 3.90 3.79 3.78 3.57 3.50 3.39 3.08 3.48 4.31 4.25 4.05 3.53 1.89 2.81 3.51 3.11 2.51 2.00 2.66 3.16 2.64 2.64 2.61 2.61 2.58 2.36 2.26 1.38
1.06 .55 .57 .59 .60 .62 .89 .85 1.07 .53 .55 .56 .68 .66 .87 1.00 .90 .91 1.04 .66 .67 .71 1.07 1.08 .98 .83 .90 1.09 1.07 .94 .90 1.03 1.09 1.04 1.01 1.07 1.05 1.05 .99 .49
Mean score between 4 and 5 = excellently prepared. Mean score between 3.5 and 4 = well prepared. Mean score between 3 and 3.5 = marginally sufficiently prepared. Mean score below 3 = insufficiently prepared.
3.2. Preparedness for practice
4. Discussion
Overall, intensivists felt well prepared for practice, with an aggregate mean score on all tasks of 3.95 (SD = 0.43). They rated their ICM training with a mean score of 8.4 (SD = 1.2) on a scale ranging from 0 (poor) to 10 (excellent). Respondents felt excellently prepared for 67 tasks (64%), well prepared for 16 tasks (15%), marginally sufficiently prepared for 6 tasks (6%) and insufficiently prepared for 15 tasks (14%) (Table 1). On cluster level, respondents felt insufficiently prepared for tasks concerning management & leadership (Mean = 2.66, SD = 0.9) and science (Mean = 2.81, SD = 0.83). They felt marginally sufficiently prepared for tasks concerning patient related administration (Mean = 3.48, SD = 0.63).
The results of this study show that recently licensed intensivists feel well prepared for the majority of the tasks that constitute independent practice. However there are numerous tasks for which they feel less well equipped by their ICM training programmes. In line with comparable studies in other disciplines [10,14,15], intensivists feel well prepared for tasks directly related to patient care, but preparedness scores are considerably lower for tasks that require more general competencies like management and leadership, science and professionalism. Because the explanations for and implications of these findings may differ between topics, they will be discussed separately. One of the most striking findings of our study is the low preparedness scores of all tasks of the management and leadership cluster. This finding is consistent with previous studies showing that new consultants generally feel not well prepared by postgraduate medical education for tasks concerning management and leadership [10,16]. A logical explanation for this common mismatch between postgraduate training programmes and independent practice is that trainees generally do not have such higher order leadership tasks, roles and responsibilities [17]. This may be problematic as intensivists can be confronted with such tasks soon after starting independent practice. The confrontation with new roles, tasks and responsibilities, together with their perceived incompetence in these general management and leadership competencies put recently licensed intensivists at risk of stress and
3.3. Differences between subgroups Due to the small sample sizes from other base specialties, only a comparison between intensivists from internal medicine and anaesthesiology could be calculated. Intensivists from anaesthesiology felt better prepared for IC specific activities (Mean = 4.25, SD = 0.38) than those from internal medicine (Mean = 4.01, SD = 0.40), t(66) = 2.43, P = .02, 95% CI [0.04–0. 76]. Anaesthesiology-based intensivists had higher scores on 17 tasks, and intensivists from internal medicine on 5 tasks (Table 2). Other subgroup analyses revealed no significant differences.
I.S. Dijkstra et al. / Journal of Critical Care 42 (2017) 47–53
51
Table 2 Statistical significant differences in preparedness for practice between intensivists with background in anaesthesiology or internal medicine (P b .05) Anesthesiology
Internal medicine
PValue
Transfer. admission and discharge Admitting a patient to the ICU
4.66
4.40
0.03
Communication with patients and their family Speaking on the phone with a patient”s family members Supporting a patient”s family
4.11 4.14
4.47 4.47
0.02 0.04
IC-specific tasks Insertion of a central venous line Initial care of the ICU patient Postoperative care of the ICU patient Leading a resuscitation procedure (BLS/ALS) Leading a team in the initial care of the unstable ICU patients or resuscitation Setting the indication and the use of sedation/pain management protocols Setting the mechanical ventilation in a complicated patient (e.g. ARDS) Performing airway management Ultrasound guided insertion of central venous lines Prevention and treatment of infections in the ICU patient Providing ICU support during procedures in a non-ICU environment (eg. PCI or ERCP) Setting the indications and adjustment of technical equipment in organ dysfunction Setting the ventilation in a patient with severe COPD Interpreting imaging Preoperative optimization of IC patients Supervising a MICU transport Performing echocardiography Performing abdominal ultrasound
4.78 4.76 4.66 4.59 4.53 4.57 4.47 4.76 4.70 4.00 4.43 3.89 4.32 3.87 4.37 4.15 2.92 2.28
4.40 4.30 4.40 4.17 4.03 3.97 4.03 3.57 3.80 4.33 3.59 4.30 3.83 4.27 3.42 3.35 2.03 1.61
b0.01 b0.01 0.03 b0.01 b0.01 b0.01 0.02 b0.01 b0.01 0.04 b0.01 0.02 0.05 0.01 b0.01 0.03 b0.01 b0.01
Professional Development Providing feedback to colleague medical specialists
3.58
3.13
0.04
burnout [15,18]. As the leadership role for physicians is becoming increasingly important, initiatives are being developed internationally to improve residents' training in management and leadership skills. [19] The results of our study call for a similar increased attention to and training in management and leadership skills in ICM training programmes. Respondents in our study reported low preparedness scores for research related tasks. Although roughly a quarter of our respondents worked in academic centers in which research is an integral part of medical practice, the majority of respondents worked in general hospitals. Because performing research generally forms a smaller part of daily practice in general hospitals than in academic centers, poor preparedness for research related tasks may not be a major concern in all cases. However, because of the increasingly rapidly expanding body of knowledge in medical science in general and in ICM in particular, the demands for applying evidence based medical practice have risen substantially and will probably continue to do so. The ability to perform research may positively influence the ability to judge the quality and implication of research, and to apply possible findings in daily practice. From this perspective low preparedness scores on research related tasks are undesirable, and systematic training and practice of evidence based medicine skills in ICM training is strongly recommended. Within the professional development cluster, three tasks received insufficiently preparedness scores, and four tasks were graded as marginally sufficiently prepared. The relatively low scores of providing feedback to colleagues, dealing with feedback from colleagues and discussing critical incidents, medical errors or complaints during a conference in a structured fashion are of interest. Open communication within health care teams are of key importance in team-based patient safety behavior and hence related to patient-outcomes [20]. In addition, allowing and encouraging team members to express thoughts and to provide feedback are important aspects of a safe and inspiring learning climate [21]. Hence, we feel that it is mandatory to prepare trainees optimally for such tasks. As the learning climate is deeply embedded in the working climate of a specific department or ICU [22], it is unlikely that the acquisition and sufficient training of these professional
development skills can be obtained through specific courses. Because the working climate is dependent on the values and beliefs of all team members, particularly those of the most influential team members [23,24], improvements in the graduate training of ICM trainees require an integral approach in which faculty, trainees and nursing personnel work together to achieve a safe and inspiring working climate. Examples of such integral interprofessional approaches moreover have proven to be beneficial for patient-outcomes [25]. Respondents of our study felt well prepared for the majority of the tasks within the Intensive Care specific task cluster. The low scores on performing ultrasound examinations can be explained by the fact that these techniques were not incorporated in the curriculum during the period that our respondents were trained. Because these techniques have now generally been added to the training programme, and are practiced regularly by ICM residents in their daily work, we expect graduate intensivists to be better prepared for ultrasound examination tasks in the coming years. Similarly, respondents felt insufficiently prepared for dealing with patients in intensive care follow-up clinics. The awareness of post intensive care syndrome [26], and the need for follow up on patients with physical and mental health problems after ICU treatment has only recently led to the development of post-ICU clinics in some hospitals in the Netherlands [27]. The comparisons between respondents with a background specialty in anaesthesiology and internal medicine showed several differences, particularly within the Intensive Care specific tasks cluster. The increased exposure to ICM specific skills such as airway management and vital function instability during base specialty training of anaesthesists as compared to internal medicine specialists likely accounts for most of these differences. It is of concern, however, that these differences are still significant at completion of ICM training. Therefore, to attain more uniform outcomes, individual training programmes need to be customized to each trainee's specific needs more extensively. This requires accurate assessment of knowledge, skills and behavior prior to admission and individualized learning trajectories, adequate supervision and assessment during training. Defining ICM specific skills as entrustable professional activities (EPA), [28] may help to ensure adequate training, supervision and assessment of
52
I.S. Dijkstra et al. / Journal of Critical Care 42 (2017) 47–53
competence to reduce these significant differences between ICM specialists from different backgrounds [11]. Although there is debate whether competency-based education provides better training than traditional time-based models, [29,30] ICM training may be specifically suitable for adopting EPA's due to its technical and well defined characteristics.
6. Take home message
4.1. Limitations
Funding
In absence of suitable objective data, we relied on self-report measures to measure the alignment between ICM training en independent practice. Therefore the results of our study do not necessarily reflect competence or performance. Feedback from former trainees is however valuable as they can reflect on the content of their training in relation to their experiences in independent practice. Our study hence provides valuable information about how well ICM training programmes prepare trainees for practice. The sample size was relatively small and nonresponse may therefore have affected the results of our study. Respondents were equally distributed across the different training programmes in the Netherlands and our sample seems representative for the population in terms of biographic parameters. Moreover the results of our study were in line with comparable studies [10]. Therefore we believe that non response has not distorted the results of our study meaningfully. During the administration of our questionnaire and onwards, there has been a continuing debate about the distribution and allocation of ICU's in the Netherlands and obligations for smaller ICU's to discuss or transfer certain patients to larger ICU's on a regular basis. An ICU is of key importance for hospitals and this debate may therefore have created feelings of insecurity about future employment of respondents in smaller hospitals particularly. Therefore this debate may have influenced the response rate.
This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.
5. Conclusions In line with findings in others specialties [10], graduate intensivists generally feel well prepared for independent ICM practice, but they feel insufficiently prepared for tasks related to leadership and management, science, and professional development. These results call for a reevaluation and update of the current (Dutch) CoBaTrICE framework. Although the medical expert role should evidently remain the core of ICM training, ICM training frameworks should expand its focus to the generic roles of independent practice more explicitly. It is however questionable whether a two-year training programme provides enough time and opportunities to train fellow intensivists better for these generic roles without compromising the development of ICM specific medical expertise. Additionally, our finding that trainees from different background specialties attain different preparedness rates for ICM specific tasks especially, may imply that the current two-year training programme is too short to prepare trainees optimally for the full scope of independent practice. Alternatively it could be argued that training programmes do not have to prepare trainees for every task to the same extend, provided that they receive sufficient support to develop expertise when confronted with these tasks in independent practice through continuing medical education or on the job support from senior co-workers. Although in the last decade much has been changed in undergraduate and postgraduate medical education to pay more attention to the generic roles of medical practice, new consultants apparently still report deficiencies in their training. As such our findings may guide discussions about the alignment and integration of the different phases of the medical education continuum, the required length of ICM training, or the desirability for ICM to become an independent specialty with corresponding training length.
Although graduated intensivist feel well prepared for the majority of the tasks of independent practice, they report deficiencies regarding the preparation for tasks related to leadership, management, science and professional development. Therefore ICM training programmes should address these generic roles of independent practice more explicitly.
Conflicts of interest None. Acknowledgements We would like to acknowledge all programme leaders of ICM training programmes in the Netherlands for their participation to our study. References [1] Frank J. The CanMEDS 2005 physician competency framework. Better standards. Better physicians. Better care; 2005. [2] Frank JR, Danoff D. The CanMEDS initiative: implementing an outcomes-based framework of physician competencies. Med Teach 2007;29:642–7. [3] Bion JF, Barrett H. Development of core competencies for an international training programme in intensive care medicine. Intensive Care Med 2006;32:1371–83. [4] Bion JF, Libreau G, Barrett H. CoBaTrICE competency based training in intensive care medicine in Europe: creating the European intensivist. Care Crit Ill 2004;20:57–8 [2 pp.]. [5] International standards for programmes of training in intensive care medicine in EuropeIntensive Care Med 2011;37:385–93. [6] The educational environment for training in intensive care medicine: structures, processes, outcomes and challenges in the European regionIntensive Care Med 2009;35:1575–83. [7] The views of patients and relatives of what makes a good intensivist: a European surveyIntensive Care Med 2007;33:1913–20. [8] Bion J, Rothen HU. Competency-based training in intensive care medicine, 12; 2012. [9] Dijkstra IS, Pols J, Remmelts P, Bakker B, Mooij JJ, Borleffs JC, et al. What are we preparing them for? Development of an inventory of tasks for medical, surgical and supportive specialties. Med Teach 2013;35:e1068–77. [10] Dijkstra IS, Pols J, Remmelts P, Brand PL. Preparedness for practice: a systematic cross-specialty evaluation of the alignment between postgraduate medical education and independent practice. Med Teach 2015;37:153–61. [11] Tulleken JE. Intensive care medicine curricula in Europe. Intensive Care Med 2015; 41:2241. [12] Bland JM, Altman DG. Cronbach's alpha. BMJ 1997;314:572. [13] Ten Cate O. Why the ethics of medical education research differs from that of medical research. Med Educ 2009;43:608–10. [14] Morrow G, Illing J, Redfern N, Burford B, Kergon C, Briel R. Are specialist registrars fully prepared for the role of consultant? Clin Teach 2009;6:87–90. [15] Westerman M, Teunissen PW, Fokkema JPI, van der Vleuten CP, Scherpbier AJ, Siegert CE, et al. The transition to hospital consultant and the influence of preparedness, social support, and perception: a structural equation modelling approach. Med Teach 2013;35:320–7. [16] Berkenbosch L, Brouns JW, Heyligers I, Busari JO. How Dutch medical residents perceive their competency as manager in the revised postgraduate medical curriculum. Postgrad Med J 2011;87:680–7. [17] Howieson B, Thiagarajah T. What is clinical leadership? A journal-based metareview. Int J Clin Leadersh 2011;17:7–18. [18] Westerman M, Teunissen PW, van der Vleuten CP, Scherpbier AJ, Siegert CE, van der Lee N, et al. Understanding the transition from resident to attending physician: a transdisciplinary, qualitative study. Acad Med 2010;85:1914–9. [19] Blumenthal DM, Bernard K, Bohnen J, Bohmer R. Addressing the leadership gap in medicine: Residents' need for systematic leadership development training. Acad Med 2012;87:513–22. [20] Brock D, Abu-Rish E, Chiu CR, Hammer D, Wilson S, Vorvick L, et al. Interprofessional education in team communication: working together to improve patient safety. BMJ Qual Saf 2013;22:414–23. [21] Roff S, McAleer S, Skinner A. Development and validation of an instrument to measure the postgraduate clinical learning and teaching educational environment for hospital-based junior doctors in the UK. Med Teach 2005;27:326–31. [22] Genn JM. AMEE medical education guide no. 23 (part 1): curriculum, environment, climate, quality and change in medical education—a unifying perspective. Med Teach 2001;23:337–44.
I.S. Dijkstra et al. / Journal of Critical Care 42 (2017) 47–53 [23] Paice E, Heard S, Moss F. How important are role models in making good doctors? BMJ 2002;325:707–10. [24] Passi V, Johnson S, Peile E, Wright S, Hafferty F, Johnson N. Doctor role modelling in medical education: BEME guide no. 27. Med Teach 2013;35:e1422–36. [25] Zwarenstein M, Goldman J, Reeves S. Interprofessional collaboration: effects of practice-based interventions on professional practice and healthcare outcomes. Cochrane Database Syst Rev 2009:CD000072. [26] Griffiths J, Fortune G, Barber V, Young JD. The prevalence of post-traumatic stress disorder in survivors of ICU treatment: a systematic review. Intensive Care Med 2007;33:1506–18.
53
[27] Van Der Schaaf M, Bakhshi-Raiez F, Van DS, Dongelmans DA, De Keizer NF. Recommendations for intensive care follow-up clinics; report from a survey and conference of Dutch intensive cares. Minerva Anestesiol 2015;81:135–44. [28] Ten Cate O, Chen HC, Hoff RG, Peters H, Bok H, van der Schaaf M. Curriculum development for the workplace using entrustable professional activities (EPAs): AMEE guide no. 99. Med Teach 2015;37:983–1002. [29] Hodges BD. A tea-steeping or i-doc model for medical education? Acad Med 2010; 85:S34–44. [30] Touchie C, ten Cate O. The promise, perils, problems and progress of competencybased medical education. Med Educ 2016;50:93–100.