- Email: [email protected]
Simulation Testing of Pediatric Rapid Response Teams: Can Simulation Be Used to Determine the Best Team Structure?
THE JOURNAL OF PEDIATRICS • www.jpeds.com
EDITORIALS
Simulation Testing of Pediatric Rapid Response Teams: Can Simulation Be Used to Determine the Best Team Structure?
M
variability in training. Common RRT activation criteria are variedical simulation and rapid response teams (RRTs) able but typically include objective measures (ie, changes in are 2 subjects whose application and effectiveness vital signs) as well as subjective concerns from staff and family. have been criticized. However, both are increasThe use of a RRT allows prompt patient evaluingly being integrated and expanded into the See related article, p ••• ation by more skilled providers, which may healthcare environment, including instituresult in reassurance or escalation of care. The tions that care for children. Added to this prewidespread acceptance of RRTs has been based on previous dicament is the constant effort to restructure responsibilities evidence that these teams improve clinical outcomes, lower costs, among healthcare providers including within medical reand the Institute for Healthcare Improvement recommendasponse teams. Furthermore, there are no studies using simution to implement rapid response systems as a key part of lation scenarios to compare the performance of RRTs led by the 100 000 Lives Campaign.4 However, conflicting reports intensivists-in-training and nurse practitioners (NPs). In this 1 make RRT adoption challenging such as 1 study5 reporting a volume of The Journal, Fehr et al studies the use of simulation to assess the performance of RRTs. significant reduction in pediatric intensive care unit (PICU) Medical simulation can be used for many functions includmortality rate after readmission from a medical or surgical unit, ing personnel training and assessment as well as system which is in sharp contrast to another report of an observaevaluations.2 Medical simulation is typically used for formative tional study6 in a children’s hospital that did not employ a RRT purposes, which emphasize future improvement of the indibut still identified a significant reduction in mortality over the vidual, course, and/or system. However, medical simulation can equivalent time period in which other pediatric centers dealso be used for high-stake, summative testing. In general, limicreased mortality in association with RRT implementation. In tations of simulation include developing realistic and relevant addition, the most recent American Heart Association’s Pescenarios, learner and leadership buy-in of the process and rediatric Advanced Life Support guidelines7 report inconclusive evidence for the use of RRTs and recommend that pediatric sources, and ultimately creating a culture of safety for the learnRRTs “may be considered in facilities where children with ers. A big debate within medical simulation is the challenge of high-risk illnesses are cared for on general in-patient units.” extrapolating data obtained in the simulated environment Therefore, it would be valuable to determine which RRT coninto the clinical setting. In addition, tools used in medical simufigurations are the most effective, which is the basis for the lation to evaluate the effectiveness of training typically incorstudy by Fehr et al.1 porate 1, if not several, of the 4 levels in the Kirkpatrick 3 Fehr et al1 evaluated 2 distinct RRT structures to deterModel. The most common Kirkpatrick levels used in medical simulation include level 1 (reaction) and level 2 (learning). mine if the performance of the group differed among teams Achievement of level 1 could be obtained by surveying the learner led by intensivists-in-training vs NPs. It should be noted that group regarding their reactions to training whereas level 2 could there is limited evidence regarding the use of NPs and phybe obtained by assessing the knowledge of each learner by the sician assistants, particularly for children, within the acute and administration of a post-test. The ideal strategy in the medical critical care settings.8 This is relevant to all institutions because various staffing modeling are being used to provide approsimulation community would be to generate evidence at priate personnel to care for our patients effectively despite staffKirkpatrick level 4 (results). Kirkpatrick level 4 would include ing and financial challenges. demonstrating targeted results (ie, patient outcomes) as a direct The authors developed the simulation scenarios from review result of medical simulation training. Establishing these types of critical events within their institution that triggered actual of direct outcomes becomes very challenging and complex to RRT calls. Ten simulation scenarios were subsequently develsubstantiate. Because of the previously stated concerns, it is oped that met several curriculum objectives including a wide equally very interesting and difficult to study the issue of using age range, diverse medical conditions among cardiac, respimedical simulation to determine the best team structure. ratory, and neurologic categories, and a significant degree of There has been continued interest in the use of RRTs within medical complexity. The age range for the simulation patients the healthcare system. This is despite the evidence being unclear was from 24 days to 10 years. The conditions covered medical about their effectiveness. The lack of RRT effectiveness can be and surgical disorders including asthma exacerbation, supradue to several factors including inconsistent activation criteria, ventricular tachycardia, cardiomyopathy, opioid overdose, various medical conditions, varying team compositions, and
NPs PICU RRTs
Nurse practitioners Pediatric intensive care unit Rapid response teams
The author declares no conflicts of interest. 0022-3476/$ - see front matter. © 2017 Elsevier Inc. All rights reserved. http://dx.doi.org10.1016/j.jpeds.2017.04.029
1 EDI 5.4.0 DTD ■ YMPD9138_proof ■ May 5, 2017
THE JOURNAL OF PEDIATRICS • www.jpeds.com respiratory failure, seizure disorder, airway foreign body, septic shock, and increased intracranial pressure. The scenarios were designed to necessitate the use of established algorithmic treatment plans such as Pediatric Advanced Life Support as well as using nonstandardized therapies because of atypical clinical presentations and having incorrect initial diagnostic information provided to the learners. After development by the authors, the scenarios were reviewed and pilot tested by members of their institution’s RRT. Scoring rubrics were developed to assess the global performance of the RRT during each scenario. This rubric was specific to each scenario and resulted in an overall global score from 1 (unsatisfactory) to 9 (superior). Subject volunteers were obtained from the authors’ institution, which had the following backgrounds: PICU nurses, respiratory therapists, PICU intensivists-in-training, and NPs. These teams compromise the typical RRT at the authors’ institution. Each team consisted of 3 members: a PICU nurse, respiratory therapist, and a PICU intensivist-in-training or an NP. All subjects in this study participated in an introductory simulation scenario on asthma, which was not included in the evaluation. The remaining 9 scenarios were divided into sets of 3, which included 1 scenario per set with incomplete information that required additional investigation by the RRT to identify the correct medical condition. These 3 challenging scenarios included undiagnosed or incorrectly diagnosed congestive heart failure, airway foreign body, and coarctation of the aorta. From a curriculum development perspective, I especially value the deliberate use of incorrect information in the scenario to mimic realistic situations encountered within the clinical environment. Six teams were led by an NP, and 11 were led by a PICU intensivist-in-training. The PICU intensivist-in-training was a fifth postgraduate year, 9 of the 11 times; this would typically correspond to a second year PICU fellow. The NPs were currently performing in an NP role for an average of 4.8 years and previously in a nursing role for an average of 15 years. All simulation scenarios occurred within a simulation center, and the sessions were video recorded. Each team completed 2 randomly assigned sets of scenarios (a total of 6 scenarios) plus the introductory session on asthma. These 17 teams were evaluated for a total of 102 study scenarios. Debriefing occurred only after completing the introductory session on asthma and the 6 scenarios. Key action items and a global performance score was assigned by 1 of the authors. The global score for each RRT was based on efficiency as well as adherence of the medical care to each specific scenario checklist. Analysis of the RRT performance used the overall global scores. Both types of RRTs promptly and effectively managed most of the simulated scenarios. The authors report the largest differences among the 2 teams particularly during the increased intracranial pressure ([mean ± SD for intensivist-intraining {7.43 ± 0.79}] vs [mean ± SD for NP {5.00 ± 1.73}]) and coarctation of the aorta ([intensivist-in-training {6.57 ± 1.40}] vs [NP {4.33 ± 1.53}]) scenarios. However, the overall global performance for all 9 scenarios among both groups was similar (intensivist-in-training [7.44 ± 1.25]) vs (NP [6.75 ± 1.46]). The global scores for the 9 scenarios ranged from the lowest for coarctation of the aorta (5.9 ± 1.70) to the highest
Volume ■■ for supraventricular tachycardia (8.30 ± 0.80). The reported overall reliability of the assessment (r = 0.57) implies that no significant judgements should be made regarding either RRT. The overall global scores made it difficult to determine the specific factor(s), which resulted in the variation among the groups. The authors conclude several limitations of this study while also emphasizing the many benefits from the use of medical simulation for RRT training.9 The relatively low number of learners, the limited amount of simulation scenarios, and the diverse clinical as well as simulation experiences of the team members limits the interpretation of the results. However, significant benefits from this study include another training opportunity for exposure to critical medical conditions using an active learning format as well as enhancing the culture for the use of medical simulation to promote teamwork. In my experience, medical simulation can be used as an additional tool to effectively introduce and start the discussions on how to best train our providers to enhance their future performance. In addition, many of the systematic approaches used to manage these specific clinical dilemmas within the simulated venue can be frequently generalized for use in other situations within the actual clinical environment. Future studies related to this topic could be developed to evaluate other medical conditions to perhaps address additional conditions commonly seen (ie, nonpatients with medical conditions within a pediatric setting such as the parent with a new-onset seizure). Studies could also be developed to assess additional pediatric RRT structures (such as physician assistant vs NP or PICU intensivist-in-training versus pediatric hospitalist). If the main question is if medical simulation can be used as an effective evaluation tool in general, the answer appears to be yes. If the question is modified to ask if medical simulation can be used to determine the best team structure for pediatric RRTs, the answer is less clear. I would like to commend Fehr et al1 in expanding the application of medical simulation from the traditional, formative focused training modality for use in the formal evaluation of team framework. I think the overall take home message is that medical simulation and RRTs are both here to stay and only expanding in the opportunities available to enhance our educational activities, shape our institutions, and promote better care to our patients. ■ David A. Young, MD, MEd, MBA, FAAP, CHSE Department of Pediatric Anesthesiology Texas Children’s Hospital/Baylor College of Medicine Houston, Texas Reprint requests: David A. Young, MD, MEd, MBA, FAAP, CHSE, Department of Pediatric Anesthesiology, Texas Children’s Hospital/Baylor College of Medicine, 6621 Fannin Street; Suite A-3300, Houston, TX 77030. E-mail: [email protected]
References 1. Fehr JJ, McBride ME, Boulet JR, Murray DJ. The simulation-based assessment of pediatric rapid response teams. J Pediatr 2017; doi:10.1016/ j.jpeds.2017.03.030.
2 EDI 5.4.0 DTD ■ YMPD9138_proof ■ May 5, 2017
■■ 2017
EDITORIALS
2. Eppich W, Howard V, Vozenilek J, Curran I. Simulation-based team training in healthcare. Simul Healthc 2011;6(Suppl):S14-9. 3. Cox T, Seymour N, Stefanidis D. Moving the needle: simulation’s impact on patient outcomes. Surg Clin North Am 2015;95:827-38. 4. Berwick DM, Calkins DR, McCannon CJ, Hackbarth AD. The 100,000 lives campaign: setting a goal and a deadline for improving health care quality. JAMA 2006;295:324-7. 5. Kotsakis A, Lobos AT, Parshuram C, Gilleland J, Gaiteiro R, MohseniBod H, et al. Implementation of a multicenter rapid response system in pediatric academic hospitals is effective. Pediatrics 2011;128:72. 6. Joffe AR, Anton NR, Burkholder SC. Reduction in hospital mortality over time in a hospital without a pediatric medical emergency team: limitations
of before-and-after study designs. Arch Pediatr Adolesc Med 2011;165: 419. 7. de Caen AR, Berg MD, Chameides L, Gooden CK, Hickey RW, Scott HF, et al. Part 12: pediatric advanced life support: 2015 American Heart Association guidelines update for cardiopulmonary resuscitation and emergency cardiovascular care. Circulation 2015;132(18 Suppl 2):S52642. 8. Kleinpell RM, Ely EW, Grabenkort R. Nurse practitioners and physician assistants in the intensive care unit: an evidence-based review. Crit Care Med 2008;36:2888-97. 9. Paige JT, Garbee DD, Brown KM. Using simulation in interprofessional education. Surg Clin North Am 2015;95:751-66.
3 EDI 5.4.0 DTD ■ YMPD9138_proof ■ May 5, 2017
EDITORIALS
Simulation Testing of Pediatric Rapid Response Teams: Can Simulation Be Used to Determine the Best Team Structure?
M
variability in training. Common RRT activation criteria are variedical simulation and rapid response teams (RRTs) able but typically include objective measures (ie, changes in are 2 subjects whose application and effectiveness vital signs) as well as subjective concerns from staff and family. have been criticized. However, both are increasThe use of a RRT allows prompt patient evaluingly being integrated and expanded into the See related article, p ••• ation by more skilled providers, which may healthcare environment, including instituresult in reassurance or escalation of care. The tions that care for children. Added to this prewidespread acceptance of RRTs has been based on previous dicament is the constant effort to restructure responsibilities evidence that these teams improve clinical outcomes, lower costs, among healthcare providers including within medical reand the Institute for Healthcare Improvement recommendasponse teams. Furthermore, there are no studies using simution to implement rapid response systems as a key part of lation scenarios to compare the performance of RRTs led by the 100 000 Lives Campaign.4 However, conflicting reports intensivists-in-training and nurse practitioners (NPs). In this 1 make RRT adoption challenging such as 1 study5 reporting a volume of The Journal, Fehr et al studies the use of simulation to assess the performance of RRTs. significant reduction in pediatric intensive care unit (PICU) Medical simulation can be used for many functions includmortality rate after readmission from a medical or surgical unit, ing personnel training and assessment as well as system which is in sharp contrast to another report of an observaevaluations.2 Medical simulation is typically used for formative tional study6 in a children’s hospital that did not employ a RRT purposes, which emphasize future improvement of the indibut still identified a significant reduction in mortality over the vidual, course, and/or system. However, medical simulation can equivalent time period in which other pediatric centers dealso be used for high-stake, summative testing. In general, limicreased mortality in association with RRT implementation. In tations of simulation include developing realistic and relevant addition, the most recent American Heart Association’s Pescenarios, learner and leadership buy-in of the process and rediatric Advanced Life Support guidelines7 report inconclusive evidence for the use of RRTs and recommend that pediatric sources, and ultimately creating a culture of safety for the learnRRTs “may be considered in facilities where children with ers. A big debate within medical simulation is the challenge of high-risk illnesses are cared for on general in-patient units.” extrapolating data obtained in the simulated environment Therefore, it would be valuable to determine which RRT coninto the clinical setting. In addition, tools used in medical simufigurations are the most effective, which is the basis for the lation to evaluate the effectiveness of training typically incorstudy by Fehr et al.1 porate 1, if not several, of the 4 levels in the Kirkpatrick 3 Fehr et al1 evaluated 2 distinct RRT structures to deterModel. The most common Kirkpatrick levels used in medical simulation include level 1 (reaction) and level 2 (learning). mine if the performance of the group differed among teams Achievement of level 1 could be obtained by surveying the learner led by intensivists-in-training vs NPs. It should be noted that group regarding their reactions to training whereas level 2 could there is limited evidence regarding the use of NPs and phybe obtained by assessing the knowledge of each learner by the sician assistants, particularly for children, within the acute and administration of a post-test. The ideal strategy in the medical critical care settings.8 This is relevant to all institutions because various staffing modeling are being used to provide approsimulation community would be to generate evidence at priate personnel to care for our patients effectively despite staffKirkpatrick level 4 (results). Kirkpatrick level 4 would include ing and financial challenges. demonstrating targeted results (ie, patient outcomes) as a direct The authors developed the simulation scenarios from review result of medical simulation training. Establishing these types of critical events within their institution that triggered actual of direct outcomes becomes very challenging and complex to RRT calls. Ten simulation scenarios were subsequently develsubstantiate. Because of the previously stated concerns, it is oped that met several curriculum objectives including a wide equally very interesting and difficult to study the issue of using age range, diverse medical conditions among cardiac, respimedical simulation to determine the best team structure. ratory, and neurologic categories, and a significant degree of There has been continued interest in the use of RRTs within medical complexity. The age range for the simulation patients the healthcare system. This is despite the evidence being unclear was from 24 days to 10 years. The conditions covered medical about their effectiveness. The lack of RRT effectiveness can be and surgical disorders including asthma exacerbation, supradue to several factors including inconsistent activation criteria, ventricular tachycardia, cardiomyopathy, opioid overdose, various medical conditions, varying team compositions, and
NPs PICU RRTs
Nurse practitioners Pediatric intensive care unit Rapid response teams
The author declares no conflicts of interest. 0022-3476/$ - see front matter. © 2017 Elsevier Inc. All rights reserved. http://dx.doi.org10.1016/j.jpeds.2017.04.029
1 EDI 5.4.0 DTD ■ YMPD9138_proof ■ May 5, 2017
THE JOURNAL OF PEDIATRICS • www.jpeds.com respiratory failure, seizure disorder, airway foreign body, septic shock, and increased intracranial pressure. The scenarios were designed to necessitate the use of established algorithmic treatment plans such as Pediatric Advanced Life Support as well as using nonstandardized therapies because of atypical clinical presentations and having incorrect initial diagnostic information provided to the learners. After development by the authors, the scenarios were reviewed and pilot tested by members of their institution’s RRT. Scoring rubrics were developed to assess the global performance of the RRT during each scenario. This rubric was specific to each scenario and resulted in an overall global score from 1 (unsatisfactory) to 9 (superior). Subject volunteers were obtained from the authors’ institution, which had the following backgrounds: PICU nurses, respiratory therapists, PICU intensivists-in-training, and NPs. These teams compromise the typical RRT at the authors’ institution. Each team consisted of 3 members: a PICU nurse, respiratory therapist, and a PICU intensivist-in-training or an NP. All subjects in this study participated in an introductory simulation scenario on asthma, which was not included in the evaluation. The remaining 9 scenarios were divided into sets of 3, which included 1 scenario per set with incomplete information that required additional investigation by the RRT to identify the correct medical condition. These 3 challenging scenarios included undiagnosed or incorrectly diagnosed congestive heart failure, airway foreign body, and coarctation of the aorta. From a curriculum development perspective, I especially value the deliberate use of incorrect information in the scenario to mimic realistic situations encountered within the clinical environment. Six teams were led by an NP, and 11 were led by a PICU intensivist-in-training. The PICU intensivist-in-training was a fifth postgraduate year, 9 of the 11 times; this would typically correspond to a second year PICU fellow. The NPs were currently performing in an NP role for an average of 4.8 years and previously in a nursing role for an average of 15 years. All simulation scenarios occurred within a simulation center, and the sessions were video recorded. Each team completed 2 randomly assigned sets of scenarios (a total of 6 scenarios) plus the introductory session on asthma. These 17 teams were evaluated for a total of 102 study scenarios. Debriefing occurred only after completing the introductory session on asthma and the 6 scenarios. Key action items and a global performance score was assigned by 1 of the authors. The global score for each RRT was based on efficiency as well as adherence of the medical care to each specific scenario checklist. Analysis of the RRT performance used the overall global scores. Both types of RRTs promptly and effectively managed most of the simulated scenarios. The authors report the largest differences among the 2 teams particularly during the increased intracranial pressure ([mean ± SD for intensivist-intraining {7.43 ± 0.79}] vs [mean ± SD for NP {5.00 ± 1.73}]) and coarctation of the aorta ([intensivist-in-training {6.57 ± 1.40}] vs [NP {4.33 ± 1.53}]) scenarios. However, the overall global performance for all 9 scenarios among both groups was similar (intensivist-in-training [7.44 ± 1.25]) vs (NP [6.75 ± 1.46]). The global scores for the 9 scenarios ranged from the lowest for coarctation of the aorta (5.9 ± 1.70) to the highest
Volume ■■ for supraventricular tachycardia (8.30 ± 0.80). The reported overall reliability of the assessment (r = 0.57) implies that no significant judgements should be made regarding either RRT. The overall global scores made it difficult to determine the specific factor(s), which resulted in the variation among the groups. The authors conclude several limitations of this study while also emphasizing the many benefits from the use of medical simulation for RRT training.9 The relatively low number of learners, the limited amount of simulation scenarios, and the diverse clinical as well as simulation experiences of the team members limits the interpretation of the results. However, significant benefits from this study include another training opportunity for exposure to critical medical conditions using an active learning format as well as enhancing the culture for the use of medical simulation to promote teamwork. In my experience, medical simulation can be used as an additional tool to effectively introduce and start the discussions on how to best train our providers to enhance their future performance. In addition, many of the systematic approaches used to manage these specific clinical dilemmas within the simulated venue can be frequently generalized for use in other situations within the actual clinical environment. Future studies related to this topic could be developed to evaluate other medical conditions to perhaps address additional conditions commonly seen (ie, nonpatients with medical conditions within a pediatric setting such as the parent with a new-onset seizure). Studies could also be developed to assess additional pediatric RRT structures (such as physician assistant vs NP or PICU intensivist-in-training versus pediatric hospitalist). If the main question is if medical simulation can be used as an effective evaluation tool in general, the answer appears to be yes. If the question is modified to ask if medical simulation can be used to determine the best team structure for pediatric RRTs, the answer is less clear. I would like to commend Fehr et al1 in expanding the application of medical simulation from the traditional, formative focused training modality for use in the formal evaluation of team framework. I think the overall take home message is that medical simulation and RRTs are both here to stay and only expanding in the opportunities available to enhance our educational activities, shape our institutions, and promote better care to our patients. ■ David A. Young, MD, MEd, MBA, FAAP, CHSE Department of Pediatric Anesthesiology Texas Children’s Hospital/Baylor College of Medicine Houston, Texas Reprint requests: David A. Young, MD, MEd, MBA, FAAP, CHSE, Department of Pediatric Anesthesiology, Texas Children’s Hospital/Baylor College of Medicine, 6621 Fannin Street; Suite A-3300, Houston, TX 77030. E-mail: [email protected]
References 1. Fehr JJ, McBride ME, Boulet JR, Murray DJ. The simulation-based assessment of pediatric rapid response teams. J Pediatr 2017; doi:10.1016/ j.jpeds.2017.03.030.
2 EDI 5.4.0 DTD ■ YMPD9138_proof ■ May 5, 2017
■■ 2017
EDITORIALS
2. Eppich W, Howard V, Vozenilek J, Curran I. Simulation-based team training in healthcare. Simul Healthc 2011;6(Suppl):S14-9. 3. Cox T, Seymour N, Stefanidis D. Moving the needle: simulation’s impact on patient outcomes. Surg Clin North Am 2015;95:827-38. 4. Berwick DM, Calkins DR, McCannon CJ, Hackbarth AD. The 100,000 lives campaign: setting a goal and a deadline for improving health care quality. JAMA 2006;295:324-7. 5. Kotsakis A, Lobos AT, Parshuram C, Gilleland J, Gaiteiro R, MohseniBod H, et al. Implementation of a multicenter rapid response system in pediatric academic hospitals is effective. Pediatrics 2011;128:72. 6. Joffe AR, Anton NR, Burkholder SC. Reduction in hospital mortality over time in a hospital without a pediatric medical emergency team: limitations
of before-and-after study designs. Arch Pediatr Adolesc Med 2011;165: 419. 7. de Caen AR, Berg MD, Chameides L, Gooden CK, Hickey RW, Scott HF, et al. Part 12: pediatric advanced life support: 2015 American Heart Association guidelines update for cardiopulmonary resuscitation and emergency cardiovascular care. Circulation 2015;132(18 Suppl 2):S52642. 8. Kleinpell RM, Ely EW, Grabenkort R. Nurse practitioners and physician assistants in the intensive care unit: an evidence-based review. Crit Care Med 2008;36:2888-97. 9. Paige JT, Garbee DD, Brown KM. Using simulation in interprofessional education. Surg Clin North Am 2015;95:751-66.
3 EDI 5.4.0 DTD ■ YMPD9138_proof ■ May 5, 2017