- Email: [email protected]
Identifying Adverse Events in Pediatric Surgery: Comparing Morbidity and Mortality Conference with the NSQIP-Pediatric System
Accepted Manuscript Identifying Adverse Events in Pediatric Surgery: Comparing Morbidity and Mortality Conference to the NSQIP–Pediatric Barrett P. Cromeens, DO, PhD, Richard E. Lisciandro, BSN, Richard J. Brilli, MD, Johanna R. Askegard-Giesmann, MD, Brian D. Kenney, MD, FACS, Gail E. Besner, MD, FACS PII:
S1072-7515(17)30192-8
DOI:
10.1016/j.jamcollsurg.2017.02.008
Reference:
ACS 8681
To appear in:
Journal of the American College of Surgeons
Received Date: 19 December 2016 Revised Date:
5 February 2017
Accepted Date: 6 February 2017
Please cite this article as: Cromeens BP, Lisciandro RE, Brilli RJ, Askegard-Giesmann JR, Kenney BD, Besner GE, Identifying Adverse Events in Pediatric Surgery: Comparing Morbidity and Mortality Conference to the NSQIP–Pediatric, Journal of the American College of Surgeons (2017), doi: 10.1016/ j.jamcollsurg.2017.02.008. This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
ACCEPTED MANUSCRIPT
Identifying Adverse Events in Pediatric Surgery: Comparing Morbidity and Mortality Conference to the NSQIP–Pediatric
RI PT
Barrett P Cromeensa, DO, PhD, Richard E Lisciandrob, BSN, Richard J Brillib, MD, Johanna R Askegard-Giesmanna, MD; Brian D Kenneya, MD, FACS, Gail E Besnera, MD, FACS
Department of Pediatric Surgery; bDepartment of Quality Improvement Services, Nationwide
SC
a
M AN U
Children’s Hospital, Columbus, OH
Disclosure Information: Nothing to disclose.
Presented at the American College of Surgeons NSQIP 12th Annual Conference, San Diego CA,
TE D
July 2016.
AC C
EP
Correspondence address: Gail E. Besner, MD, FACS Department of Pediatric Surgery FB6135 Nationwide Children’s Hospital 700 Children’s Drive Columbus, Ohio, 43205, USA E-mail: [email protected] Phone: (614) 722-3930 FAX: (614) 722-3903
Brief Title: Identifying Adverse Events in Pediatric Surgery
ACCEPTED MANUSCRIPT 2
Abstract: Background
Recent improvements to M&M conference have focused on the case
review system. However, case selection occurs by physician reporting which is limited by
RI PT
selection bias. We compared the effectiveness of our M&M conference to the NSQIP-P system for identifying adverse events. Study Design
Complications from January 2010-September 2015 were compared
SC
between M&M and NSQIP-P. Only M&M patients meeting NSQIP-P criteria were compared to patients from the NSQIP-P system; exclusions were studied separately. Complication rates in
M AN U
M&M Conference before and after a 2012 format change designed to increase case reporting were also compared. Results
Detection of mortality in M&M Conference and NSQIP-P was not different.
Morbidity events identified by NSQIP-P were significantly higher than M&M conference over
TE D
the entire study period (194 vs. 100 occurrences/1000 cases) (p<0.0001). Morbidity occurrences in M&M Conference increased with the 2012 improvements, however, they still remained less than that identified by NSQIP-P (226 vs. 141 occurrences/1000 cases) (p<0.0001). Of 863
EP
patients presented in M&M Conference, 210 were excluded from direct comparison because they did not meet NSQIP-P criteria. These included 62 deaths and 287 occurrences of morbidity.
AC C
Their analysis in M&M Conference resulted in 32 action initiatives directed at system failures. Conclusions NSQIP-P identified more complications than M&M. M&M Conference improvements increased reported cases, but they still remained lower than NSQIP-P. However, M&M Conference identified events resulting in systems changes which would not have been identified by NSQIP-P. While NSQIP-P captures occurrences to compare large patient cohorts,
2
ACCEPTED MANUSCRIPT 3
M&M analyzes singular failures and initiates direct interventions. Integration of these systems
Keywords
RI PT
may optimize their usefulness in quality improvement.
Morbidity; Mortality; M&M Conference; NSQIP; Pediatric Surgery; Adverse
Abbreviations
SC
Events
ACS, American College of Surgeons; CPT, Current Procedural
M AN U
Terminology; ECMO, Extra-Corporeal Membrane Oxygenation; M&M, Morbidity & Mortality; NSQIP-P, National Surgical Quality Improvement Program – Pediatric; SCR, Surgical Clinical
AC C
EP
TE D
Reviewer
3
ACCEPTED MANUSCRIPT 4
Introduction Morbidity and Mortality (M&M) conference has been the traditional mechanism by which surgeons have analyzed complications in order to better understand potential causes of
RI PT
individual or system failures, and to implement changes that prevent repeated occurrences. With recent emphasis on improvement science in medicine, and because surgical M&M conference has been in place at many institutions for decades, there have been recent focused efforts to
SC
optimize the conference as a more useful tool to drive changes in clinical practice and hospital systems.(1-13) These efforts range from better defining the objectives of the conference to more
M AN U
structured initiatives including the use of root cause analysis, plan-do-check-act (PDCA) cycles, cause and effect diagrams, physician report cards, and taxonomies of failure analysis. Although these changes have resulted in important improvements to the conference, they primarily focus on case review and failure classification systems. Few efforts have been reported that ensure
TE D
adequate case selection despite mounting evidence that potential biases in case selection for M&M conferences result in many missed complications.(14-21) The pediatric surgery M&M Conference at our institution has undergone several changes
EP
intended to focus the conference as a tool to identify individual and system failures which can be used to initiate interventions to improve outcomes. In 2012, conference frequency was increased
AC C
from bimonthly to weekly and the presentation format was altered to include detailed and focused reviews. As a result, relatively simple complications were presented quickly, while more complex cases received detailed review. The intent was to increase the number of cases presented within a limited conference time. In 2013, a detailed case analysis system was introduced that facilitated the identification of individual and system failures so that corrective actions could be initiated and tracked to completion.(6) However, at our institution, little was
4
ACCEPTED MANUSCRIPT 5
known about whether the M&M Conference effectively identified all complications that could benefit from case review. Cases in which a complication has occurred in our institution are selected for presentation
RI PT
at M&M conference by pediatric surgery fellows. This contrasts with the process used by the American College of Surgeons (ACS) National Surgical Quality Improvement Program –
Pediatric (NSQIP-P). NSQIP-P is a validated national database where standardized definitions
SC
and specific inclusion and exclusion criteria are used to prospectively collect outcomes data by trained reviewers.(22-24) This database is then leveraged to track surgical complications as
M AN U
interventions are implemented to decrease the incidence of the complications. While M&M conference is also intended to assist in identifying the causes of complications so that interventions can be implemented to prevent future complications, to do so effectively requires reliable identification of complications. The lack of strictly defined criteria for complications in
TE D
M&M Conference such as that utilized by NSQIP-P calls into question the ability of M&M Conference to capture all complications, and thus its effectiveness as a tool for quality improvement. The purpose of this study was to identify the incidence and type of morbidity and
EP
mortality events occurring on our pediatric surgery service as identified by the pediatric surgery
AC C
M&M Conference compared to those identified by the nationally validated NSQIP-P system.
Methods
Data Collection
Data were collected from patients on the general pediatric surgery service
from January 1, 2010 to September 30, 2015. For M&M conference, complications were compiled by the pediatric surgery fellows using the following general guidelines: all deaths; unplanned readmissions; unplanned reoperation within 30 days of initial operation; any adverse
5
ACCEPTED MANUSCRIPT 6
event, error, or delay in diagnosis resulting in unexpected morbidity; unplanned invasive intervention; need for escalation of care; or length of stay (LOS) outside of what would be reasonably expected for a given disease process. Compared to the very strict criteria utilized by
RI PT
NSQIP-P, some components of the general M&M guidelines are subjective in nature (e.g.
prolonged LOS). The general presentation format of the pediatric surgery M&M conference at our institution has been described previously.(6) In brief, detailed M&M case discussion
SC
summaries and action item implementation plans are carefully recorded. In 2012, there was a change to the conference format intended to increase the number of cases discussed. Prior to
M AN U
June 2012, M&M Conference was held on a bimonthly basis. After June 2012, the conference frequency was increased to weekly. In addition, “Focused Reviews” of more straight forward complications were added to the case review process (e.g. superficial wound infection), providing increased time for “Detailed Reviews” of more complicated cases.
TE D
Data collection methods for NSQIP-P have been previously reported.(22-24) In brief, the first 35 procedures performed on patients <18 years of age and falling within eligible Current Procedural Terminology (CPT) codes are sampled on an eight-day cycle by trained Surgical
EP
Clinical Reviewers (SCR). From these cases, 147 variables are abstracted from the medical record including preoperative risk factors, intraoperative variables, and 30-day postoperative
AC C
outcome variables. Not all of the variables recorded by NSQIP (e.g. pre-operative lab values and co-morbidities) indicate the occurrence of an adverse event. Because the current study was designed to compare morbidity and mortality between M&M Conference and NSQIP-P, only NSQIP-P variables corresponding to the occurrence of an adverse event were used for comparison purposes. These include all 30-day postoperative outcome variables, intra-op cardiac arrests requiring CPR, intra-op deaths, intra-op unplanned extubations, and intra-op “other”.
6
ACCEPTED MANUSCRIPT 7
Although NSQIP-P tracks cases from multiple surgical services, only cases abstracted within NSQIP-P that were performed by the pediatric surgery service were included.
Because M&M Conference
RI PT
Comparison of M&M Conference and NSQIP-P occurrences
identifies complications from the entire population of patients on the pediatric general surgery service and NSQIP-P includes only a sample of patients meeting specified criteria, these two
SC
populations of patients are not directly comparable. Abbreviated criteria for NSQIP-P data
extraction are provided in Table 1. It is important to note that outcome variables in NSQIP-P are
M AN U
only tracked up to 30 days. To allow for direct M&M Conference and NSQIP-P comparison, the same inclusion and exclusion criteria utilized for NSQIP-P sampling were applied to cases presented at M&M Conference during the study period, and only those M&M cases meeting NSQIP-P criteria were included for comparative analyses. To understand complications that
TE D
were not being captured by NSQIP-P but reported through the M&M Conference process, all M&M cases excluded from direct comparative analysis because they did not meet NSQIP-P
EP
inclusion criteria were analyzed separately.
Determination of Morbidity and Mortality Rates
NSQIP-P utilizes a sampling
AC C
algorithm to collect data on a fraction of operative patients that intends to be reflective of the institutional experience. In contrast, M&M conference attempts to comprehensively capture all complications occurring within the institution. Given these differences, complication rates were analyzed rather than absolute numbers in order to compare the two systems. Morbidity and mortality occurrence rates were tabulated for M&M Conference and for NSQIP-P as follows:
7
ACCEPTED MANUSCRIPT 8
M&M Conference Occurrence Rates
The same outcome variable definitions tracked by
NSQIP-P were applied to the M&M analysis to determine the rate of M&M Conference occurrences. The total number of morbidity occurrences was obtained by summing all the
RI PT
defined NSQIP-P intra-op and post-op morbidity variables. To further break down morbidity into categories, the defined NSQIP-P intra-op and post-op morbidity variables were further grouped into types of complications (Table 2). The total number of mortality occurrences was obtained by
SC
summing all the defined NSQIP-P intra-op and post-op mortality variables. Morbidity and
mortality rates were calculated as the total number of occurrences identified divided by the total
M AN U
number of pediatric surgery cases meeting NSQIP-P criteria during the study period. The Department of Pediatric Surgery performed 23,902 operations during the study period. Of these, 9,603 met NSQIP-P study criteria and served as the denominator for calculating occurrence rates for M&M Conference.
Occurrence rates were calculated as the total number of
TE D
NSQIP-P Occurrence Rates
occurrences identified divided by the total number of cases extracted for the pediatric surgery service during the study period. NSQIP-P abstracted 2,973 cases during the study period, which
EP
served as the denominator for calculating occurrence rates for NSQIP-P. Statistical Analyses
Morbidity and mortality rates were compared between the two systems
AC C
using univariate chi-square analysis with p<0.05 considered statistically significant. All rates were reported as the number of occurrences per 1000 cases.
Analysis of 2012 M&M Conference Format Change
Because the M&M
Conference format was changed in 2012 to increase the number of complications reported, complication rates reported before 2012 (pre-2012 M&M cohort) and after 2012 (post-2012
8
ACCEPTED MANUSCRIPT 9
M&M cohort) were compared. In addition to comparing M&M Conference and NSQIP-P occurrences over the entire study period, post-2012 cohorts for both systems were also
RI PT
compared.
Results
Comparison of M&M Conference and NSQIP-P Occurrences From January 1, 2010
SC
through September 30, 2015 the department of pediatric surgery held 151 M&M Conferences during which 863 patients were discussed. When the NSQIP-P selection criteria were applied,
M AN U
210 patients presented at M&M Conference were excluded. The most common reason for exclusion was for undergoing a procedure not tracked by NSQIP-P (Figure 1). One exclusion criterion for NSQIP-P is >18 years of age. While there were patients presented in M&M Conference who were >18 years of age, they all met at least one additional NSQIP-P exclusion
TE D
criterion and therefore are represented in the multifactorial category of Figure 1. Morbidity and mortality rates were calculated and compared between the two systems (Figure 2). Mortality rates identified by M&M Conference and NSQIP-P were not different (8 vs.
EP
10 deaths/1000 cases) (p=0.28). Morbidity events identified by NSQIP-P were significantly higher than M&M Conference identified events over the entire study period (194
AC C
occurrences/1000 cases vs. 100 occurrences/1000 cases) (p<0.0001) (Figure 2A). Morbidity and mortality rates identified solely by M&M Conference were compared before and after the 2012 conference format change (Figure 2B). There was no difference in mortality identified between the pre- and post-2012 M&M cohorts. However, morbidities identified in the post-2012 M&M cohort were significantly greater than the pre-2012 cohort (141 vs. 46 occurrences/1000 cases) (p<0.0001). Morbidity and mortality rates identified by the post-2012 M&M cohort were
9
ACCEPTED MANUSCRIPT 10
compared to the post-2012 NSQIP-P cohort (Figure 2C). There was no difference in mortality identified between M&M Conference and NSQIP-P in the post-2012 time period. However, morbidities identified by post-2012 NSQIP-P still remained significantly higher than post-2012
RI PT
M&M (226 vs. 141 occurrences/1000 cases) (p<0.0001).
Morbidity rates by category between post-2012 M&M and post-2012 NSQIP-P cohorts were compared (Figure 3). There were no differences between M&M Conference and NSQIP-P
SC
regarding wound, neurologic, renal complications, or with reoperations. NSQIP-P identified significantly higher rates than M&M conference regarding respiratory, cardiovascular, and other
M AN U
infectious complications, as well as readmissions. M&M Conference did identify significantly higher “other” complication rates compared to NSQIP-P. This “other” category includes a wide range of adverse events that do not fall into a well-defined NSQIP-P variable, and are assigned at
TE D
the discretion of the SCR as a marker of poor outcome.
Analysis of M&M Conference Cases Excluded by NSQIP-P
When analyzing the 210
M&M conference cases excluded by NSQIP-P, there were 62 deaths and 287 incidents of
EP
morbidity identified (Table 3). Of the 62 deaths, 46 patients were non-operative cases while 16 patients were operative cases, the majority of which were excluded because they were operative
AC C
trauma cases (NSQIP-P does not track trauma-related occurrences). Of the 210 patients excluded, 118 were excluded because they underwent a procedure not tracked by NSQIP-P. Over half of these patients (n=70) fell within one of four types of procedures: circumcision (n=10), soft tissue abscess drainage (n=16), central line placement (n=31), and extra-corporeal membrane oxygenation (ECMO) cannulation (n=13). From these 70 cases, 54 unplanned reoperations were performed.
10
ACCEPTED MANUSCRIPT 11
Ninety-one M&M conference patients excluded by NSQIP-P criteria underwent review by our comprehensive M&M case analysis system which was initiated in 2013 and designed to identify individual and system failures so that corrective actions could be initiated and
RI PT
completed.(6) Analysis of these 91 patients resulted in the initiation of 66 action items. The majority of these action items were education initiatives (n=34), however the remainder were directed at implementing systems changes (n=32). These systems changes included creating
SC
automatic triggers for consultation of the pediatric surgery service, replacing outdated and insufficient equipment, modifying order sets to ensure appropriate content, updating and
M AN U
automating burn resuscitation protocols, and working to expedite patient transfers from referring institutions. Below is an example of a case excluded from NSQIP-P where failures were identified during M&M Conference resulting in corrective action: Sample M&M Case
A 17 month old female presented with a foreign body ingestion. A
TE D
foreign body series, which included a chest x-ray and abdominal x-ray, did not identify a foreign body. The mother provided a plastic disc identical to the ingested object which was confirmed to be radiopaque. Although no foreign body was identified radiologically, the patient was observed
EP
overnight. Upon continuing to refuse oral intake, she underwent endoscopy, with the foreign body identified in the pyriform sinus. Discussion in M&M Conference identified a system error
AC C
involving lack of appropriate radiologic protocol. It was decided that a lateral view of the neck and soft tissue be added to the foreign body series.
Discussion
Originally known as the Anesthesia Study Commission, early M&M conferences were some of the first quality improvement initiatives before quality improvement science in medicine
11
ACCEPTED MANUSCRIPT 12
became an established field. Originally composed of surgeons, anesthesiologists, and internal medicine doctors, these conferences were purposed for robust discussion of unexpected and poor outcomes so that their etiology could be determined and thus avoided in the future.(25, 26) These
RI PT
conferences were quick to catch on in many surgical departments across the country and became the traditional mechanism for case review. As emphasis on quality improvement science in
medicine grew, the conferences were an attractive tool for gathering data and implementing
SC
quality initiatives. However, there has been growing evidence that data capture by M&M
Conference is not reflective of the entire institutional experience, and has led some to conclude
M AN U
that M&M Conference is an inadequate system to drive change.(14-21)
The pediatric surgery M&M Conference at our institution is a robust multidisciplinary conference that has been in place for decades. In recent years, format changes were implemented that increased the efficiency of case presentation and case analysis, assigned and tracked
TE D
corrective actions, and documented action item follow-up.(6) In the first year after implementation of these conference improvements, 42 action items directed at correcting system failures were identified. Before this system was implemented, interventions to correct errors
EP
were rarely initiated and more importantly, they were not documented or tracked to completion. Reports from other institutions of M&M conferences under-reporting mortality by as much as
AC C
50% and morbidity by as much as 75%, indicated significant unidentified potential for improvement of patient care.(15, 16, 19) Based on these reports, we sought to determine the extent to which our M&M conference was under-reporting complications. Our institution has participated in NSQIP-P since 2010. Since NSQIP-P is a validated
data collection system that tracks a robust set of complications, we chose to use this system as the standard to which we compared our M&M Conference. Similar comparisons have been made
12
ACCEPTED MANUSCRIPT 13
between M&M conferences and the adult NSQIP system in other hospitals.(15, 16) However, comparable studies in pediatric hospitals are lacking. In the current study, there was no difference in mortality rates identified by M&M Conference or NSQIP-P. This is in contrast to
RI PT
studies in the adult setting, where M&M Conference under-reports mortality by as much as 50% compared to NSQIP.(16, 19) At our institution, the pediatric surgery service is managed by two clinical fellows who are responsible for M&M case selection. Because they are aware of all
SC
patients on the service, consistency is likely maintained with some control for selection bias, which may contribute to the more robust mortality reporting seen in our M&M Conference.
M AN U
During the total study period there were significantly fewer morbidities identified by M&M Conference than by NSQIP-P, and this held true even after the 2012 changes designed to increase the number of cases presented. Although our M&M Conference was equivalent to NSQIP-P at identifying mortality, it significantly under-reported morbidity, similar to other
TE D
reports. Possible explanations for under-reporting of complications at M&M Conferences likely include insufficient and biased sampling. Most M&M conferences rely on physician selfreporting. As the physician directly taking care of patients is the most likely to know whether or
EP
not a complication occurred, they are the logical person to choose complications for presentation. However, time pressures, fears of embarrassment in front of one’s peers, punishment, and legal
AC C
implications have been suggested to decrease the likelihood of open self-reporting.(27-30) Another possible contribution is the lack of clearly defined criteria for what should and should not be considered a complication. When the decision as to whether an untoward outcome should be presented at conference is left in the hands of each surgeon, there is inherent opportunity for disagreement between individuals that will likely result in inconsistent reporting. Additionally, there is bias by each physician to report that which they find most pertinent for discussion. When
13
ACCEPTED MANUSCRIPT 14
looking at the categories of complications identified by M&M Conference, it is not surprising that categories most directly linked to the operation and thus most likely of interest to the reporting surgeon were not under-reported compared to NSQIP-P. These categories include
RI PT
postoperative mortality, unplanned reoperations and wound complications. Contrast these
limitations to other data capturing systems such as NSQIP-P. In this system, trained reviewers systematically review charts and collect data based on defined criteria in an attempt to control for
SC
the above stated biases. This approach consistently captures data that reflect the institutional experience.(22, 23, 31, 32) This consistent and defined approach results in superior capture of
M AN U
complications compared to M&M Conference. It is this under-reporting of M&M Conference compared to NSQIP that lead Miller, et. al.(15) to conclude that NSQIP may be the better option for quality improvement endeavors in their institution. However, while our study confirms reporting limitations of M&M Conference, other benefits of M&M Conference are apparent
TE D
upon further analysis.
The NSQIP-P system was designed to capture the majority of clinically relevant complications that occur in the field of pediatric surgery. Because there are exclusion criteria in
EP
NSQIP-P, it is inevitable that complications will occur in excluded patients and will therefore not be captured. However, the extent of the complications not captured by NSQIP-P, and the value
AC C
which they may hold for quality improvement, is difficult to quantify. By analyzing the patients excluded from M&M Conference for not meeting NSQIP-P criteria, the extent and value of the complications missed by NSQIP-P becomes apparent. Applying the NSQIP-P exclusion criteria to M&M Conference resulted in the exclusion of 210 patients from comparative analysis. By excluding these patients, NSQIP-P did not capture the 62 deaths or 287 incidents of morbidity that occurred in these 210 patients. M&M Conference has the added benefit of real time direct
14
ACCEPTED MANUSCRIPT 15
discussion with the involved parties regarding the cause of a complication. At our institution, this discussion is facilitated by a comprehensive failure analysis including mechanisms for implementing action initiatives and tracking them to completion. Of the 210 M&M Conference
RI PT
patients in our study that were excluded from comparative analysis due to not meeting NSQIP-P inclusion criteria, 91 underwent comprehensive failure analysis. This resulted in 66 action
initiatives, 32 of which were directed at correcting systems failures. These were cases with
SC
substantial system failures that necessitated corrective action. Although NSQIP-P captures more complications than M&M Conference, it is only able to capture those complications falling
M AN U
within the definitions of the system. M&M Conference on the other hand identifies numerous complications otherwise missed by NSQIP-P that hold substantial value for quality improvement, as evidenced by the many actions initiated to correct identified failures. Based on the above findings, it is clear that M&M conference and NSQIP-P are not
TE D
equivalent systems. However, neither are comprehensive quality improvement tools and each provide specific strengths and weaknesses. Being able to harness and integrate the benefits of each system would be ideal for optimizing improvement endeavors. Because NSQIP-P has
EP
proven more rigorous at identifying complications, we are going to start forwarding complications identified by NSQIP-P to M&M conference. Of course NSQIP-P is only a
AC C
sampling, thus there will be complications missed by both systems. However, any complication identified by NSQIP-P that may not have already been identified by M&M conference will benefit from the comprehensive failure analysis provided by the M&M process. We are initially trialing this effort with surgical site infections but will hopefully expand this further to other adverse events. One drawback to using the NSQIP-P system in this manner is that often the complications are not identified until 90 days have passed, making M&M discussion remote
15
ACCEPTED MANUSCRIPT 16
from when the surgery occurred, potentially hindering the failure analysis process secondary to recall limitations by the involved parties. Unfortunately, we have not found a means to overcome this time gap. Although this limitation remains, delayed analysis is superior to no analysis.
RI PT
Additionally, ongoing efforts to define and standardize the M&M selection process are being made to improve capture.
The utilization of these quality improvement tools is becoming even more important. The
SC
American College of Surgeons in conjunction with the Task Force for Children’s Surgical Care have developed the Children’s Surgery Verification (CSV) Quality Improvement Program to
M AN U
define optimal resources and standardize surgical care for children. Part of this verification process requires that the institution have a Performance Improvement and Patient Safety (PIPS) program that encompasses all surgical disciplines. As our institution is navigating the verification process, our M&M conference has served as an important platform on which to meet
TE D
the PIPS requirements. Currently our M&M platform is being used as a model for all surgical (and medical) divisions at our institution. This will allow us to develop a case registry from which cases will be selected for presentation at our quarterly PIPS meeting, with emphasis on
AC C
Conclusions
EP
cases in which systems errors have been identified and corrected.
M&M Conference is equivalent to NSQIP-P at identifying mortality on the pediatric
surgery service at our institution. However, M&M Conference significantly under-reports morbidity compared to the NSQIP-P system. Focused efforts to improve M&M Conference significantly increased the morbidities identified, however, this was still significantly less than those identified by the NSQIP-P system. Although M&M Conference should ideally capture all
16
ACCEPTED MANUSCRIPT 17
the complications on the service, in reality, M&M Conference is hindered by inadequate sampling and sampling biases. However, to dismiss M&M Conference because it under-reports complications would be short sighted. The data presented here demonstrate that although M&M
RI PT
Conference under-reports complications compared to NSQIP-P, each of these two systems has its own strengths and weaknesses. NSQIP-P is a robust data collection system that samples data that reflect institutional experience, making it an ideal system for comparing variables between
SC
large cohorts of patients to identify and track areas for improvement. However, data collection is constrained to the parameters that define the system, therefore missing some complications and
M AN U
opportunities for important systems changes. Although M&M Conference does not capture complications to the extent of NSQIP-P, it is not constrained by the same definitions as the NSIQP-P system, and thus captures outcomes otherwise missed by NSQIP-P. In addition, M&M Conference performs real-time analysis of complications so that failures are identified and
TE D
corrective actions are implemented. Use of both systems in order to harness their strengths is
AC C
EP
likely to prove useful in order to optimize their roles in improvement of patient care.
17
ACCEPTED MANUSCRIPT 18
References 1.
Anderson CI, Nelson CS, Graham CF, et al. Disorganized care: the findings of an
iterative, in-depth analysis of surgical morbidity and mortality. J Surg Res 2012;177:43-48. Antonacci AC, Lam S, Lavarias V, et al. A report card system using error profile analysis
RI PT
2.
and concurrent morbidity and mortality review: surgical outcome analysis, part II. J Surg Res 2009;153:95-104.
Antonacci AC, Lam S, Lavarias V, et al. A morbidity and mortality conference-based
SC
3.
classification system for adverse events: surgical outcome analysis: part I. J Surg Res
4.
M AN U
2008;147:172-177.
Batthish M, Tse SM, Feldman BM, et al. Trying to improve care: the Morbidity and
Mortality Conference in a division of rheumatology. J Rheumatol 2014;41:2452-2458. 5.
Cifra CL, Bembea MM, Fackler JC, Miller MR. Transforming the morbidity and
TE D
mortality conference to promote safety and quality in a PICU. Pediatr Crit Care Med 2016;17:58-66. 6.
Cromeens B, Brilli R, Kurtovic K, et al. Implementation of a pediatric surgical quality
7.
EP
improvement (QI)-driven M&M conference. J Pediatr Surg 2016;51:137-142. Deis JN, Smith KM, Warren MD, et al. Transforming the morbidity and mortality
8.
AC C
conference into an instrument for systemwide improvement culture and redesign. 2008 Aug. Gonzalo JD, Bump GM, Huang GC, Herzig SJ. Implementation and evaluation of a
multidisciplinary systems-focused internal medicine morbidity and mortality conference. J Grad Med Educ 2014;6:139-146.
18
ACCEPTED MANUSCRIPT 19
9.
Johna S, Tang T, Saidy M. Patient safety in surgical residency: root cause analysis and
the surgical morbidity and mortality conference--case series from clinical practice. Perm J 2012;16:67-69. McDonnell C, Laxer RM, Roy WL. Redesigning a morbidity and mortality program in a
RI PT
10.
university-affiliated pediatric anesthesia department. Jt Comm J Qual Patient Saf 2010;36:117125.
Rabizadeh S, Gower WA, Payton K, et al. Restructuring the morbidity and mortality
SC
11.
(Phila) 2012;51:1079-1086. 12.
M AN U
conference in a department of pediatrics to serve as a vehicle for system changes. Clin Pediatr
Szostek JH, Wieland ML, Loertscher LL, et al. A systems approach to morbidity and
mortality conference. Am J Med 2010;123:663-668. 13.
Vogel P, Vassilev G, Kruse B, Cankaya Y. Morbidity and mortality conference as part of
2011;396:1009-1015. 14.
TE D
PDCA cycle to decrease anastomotic failure in colorectal surgery. Langenbecks Arch Surg
McVeigh TP, Waters PS, Murphy R, et al. Increasing reporting of adverse events to
2013;216:50-56.
Miller DC, Filson CP, Wallner LP, et al. Comparing performance of morbidity and
AC C
15.
EP
improve the educational value of the morbidity and mortality conference. J Am Coll Surg
mortality conference and National Surgical Quality Improvement Program for detection of complications after urologic surgery. Urology 2006;68:931-937. 16.
Hutter MM, Rowell KS, Devaney LA, et al. Identification of surgical complications and
deaths: an assessment of the traditional surgical morbidity and mortality conference compared
19
ACCEPTED MANUSCRIPT 20
with the American College of Surgeons-National Surgical Quality Improvement Program. J Am Coll Surg 2006;203:618-624. 17.
Falcone JL, Lee KK, Billiar TR, Hamad GG. Practice-based learning and improvement: a
RI PT
two-year experience with the reporting of morbidity and mortality cases by general surgery residents. J Surg Educ 2012;69:385-392. 18.
Thompson JS, Prior MA. Quality assurance and morbidity and mortality conference. J
19.
SC
Surg Res 1992;52:97-100.
Bilimoria KY, Kmiecik TE, DaRosa DA, et al. Development of an online morbidity,
M AN U
mortality, and near-miss reporting system to identify patterns of adverse events in surgical patients. Arch Surg 2009;144:305-311; discussion 311. 20.
Feldman L, Barkun J, Barkun A, et al. Measuring postoperative complications in general
surgery patients using an outcomes-based strategy: comparison with complications presented at
21.
TE D
morbidity and mortality rounds. Surgery 1997;122:711-719; discussion 719-720. Wanzel KR, Jamieson CG, Bohnen JM. Complications on a general surgery service:
incidence and reporting. Can J Surg 2000;43:113-117. Raval MV, Dillon PW, Bruny JL, et al. Pediatric American College of Surgeons National
EP
22.
Surgical Quality Improvement Program: feasibility of a novel, prospective assessment of surgical
23.
AC C
outcomes. J Pediatr Surg 2011;46:115-121. Raval MV, Dillon PW, Bruny JL, et al. American College of Surgeons National Surgical
Quality Improvement Program Pediatric: a phase 1 report. J Am Coll Surg 2011;212:1-11. 24.
Dillon P, Hammermeister K, Morrato E, et al. Developing a NSQIP module to measure
outcomes in children's surgical care: opportunity and challenge. Semin Pediatr Surg 2008;17:131-140.
20
ACCEPTED MANUSCRIPT 21
25.
Ruth HS, Haugen FP, Grove DD. Anesthesia Study Commission; findings of 11 years'
activity. J Am Med Assoc 1947;135:881-884. 26.
Deshpande JK, Throop PG, Slayton JM. Standardization of case reviews (morbidity and
27.
RI PT
mortality rounds) promotes patient safety. Pediatr Clin North Am 2012;59:1307-1315.
Cullen DJ, Bates DW, Small SD, et al. The incident reporting system does not detect
adverse drug events: a problem for quality improvement. Jt Comm J Qual Improv 1995;21:541-
28.
SC
548.
Mariner WK. Medical error reporting: professional tensions between confidentiality &
M AN U
liability. Issue Brief (Mass Health Policy Forum) 2001 Nov 6(13):1-35. 29.
Leape LL. Reporting of adverse events. N Engl J Med 2002;347:1633-1638.
30.
Weissman JS, Annas CL, Epstein AM, et al. Error reporting and disclosure systems:
views from hospital leaders. JAMA 2005;293:1359-1366.
Sharp NE, Knott EM, Iqbal CW, et al. Accuracy of American College of Surgeons
TE D
31.
National Surgical Quality Improvement Program Pediatric for laparoscopic appendectomy at a single institution. J Surg Res 2013;184:318-321.
Duggan EM, Gates DW, Slayton JM, Blakely ML. Is NSQIP Pediatric review
EP
32.
representative of total institutional experience for children undergoing appendectomy? J Pediatr
AC C
Surg 2014;49:1292-1294.
21
ACCEPTED MANUSCRIPT 22
Table 1. Abbreviated NSQIP-Pediatric Inclusion/Exclusion Criteria
AC C
EP
TE D
M AN U
SC
RI PT
Patient must be < 18 years of age Patient must have undergone an operation Operation must fall within the listing of tracked CPT codes Operation cannot be associated with excluded ICD-9 codes (800-933; 938-959.9; 995.50995.59) Patient must have an American Society of Anesthesiologists score < 6
22
ACCEPTED MANUSCRIPT 23
Table 2. NSQIP-Pediatric Outcomes Variables Grouped by Morbidity Category
SC
Neurologic
M AN U
Respiratory
Cardiovasular
TE D
Renal/urinary
AC C
EP
Other infectious
Other
Outcomes variables Postoperative superficial Incisional SSI; postoperative deep incisional SSI; postoperative organ/space SSI; postoperative deep wound disruption; postoperative superficial wound disruption Postoperative coma > 24 h; postoperative CVA/stroke/intracranial hemorrhage; postoperative seizure; postoperative nerve injury; postoperative intraventricular hemorrhage grade 1-4; postoperative intraventricular hemorrhage grade not specified Intraoperative unplanned extubation; postoperative pneumonia; postoperative unplanned intubation; postoperative pulmonary embolism Intraoperative cardiac arrest requiring CPR; postoperative cardiac arrest requiring CPR; intraoperative/postoperative transfusion within 72 h; postoperative venous thrombosis requiring therapy Postoperative progressive renal insufficiency; postoperative acute renal failure; postoperative urinary tract infection Postoperative systemic sepsis; postoperative septic shock; postoperative central-line associated blood stream infection Intraoperative other occurrence; postoperative other occurrence; postoperative graft/prosthesis/flap failure
RI PT
Morbity category Wound
Readmissions Reoperations SSI, surgical site infection
23
ACCEPTED MANUSCRIPT 24
Table 3. Morbidity Identified by Morbidity and Mortality Conference But Excluded by NSQIP-Pediatric
RI PT
Occurrences/1,000 Cases 2.10 0.28 0.14 0.84 0.14 0.14 6.71 2.46 7.27 20.10
SC
Occurrences 30 4 2 12 2 2 96 35 104 287
AC C
EP
TE D
M AN U
Morbidities Wound Neurologic Respiratory Cardiovascular Renal/urinary Other infectious Other Readmissions Reoperations Total
24
ACCEPTED MANUSCRIPT 25
Figure Legends Figure 1. Distribution of morbidity and mortality cases excluded from comparative analysis based on NSQIP-Pediatric (NSQIP-P) exclusion criteria. Each category shown is a reason for
RI PT
exclusion from NSQIP-P selection that corresponds to the NSQIP-P criteria shown in Table 1. The most common reason for exclusion of patients presented at morbidity and mortality
conferences (M&M) was for undergoing a procedure not tracked by NSQIP-P. This was
SC
followed by not having undergone an operation, meeting multiple exclusion criteria, undergoing a procedure linked with an excluded ICD-9 code, or having a complication >30 days after their
M AN U
operation. Not represented are patients >18 years of age and patients with an American Society of Anesthesiologists (ASA) score of 6. Although there were patients >18 years old excluded, they also met other exclusion criteria, putting them in the “multifactorial” category. There were
TE D
no patients operated on with an ASA score of 6.
Figure 2. Morbidity and mortality comparisons between morbidity and mortality (M&M) conference and NSQIP-Pediatric (NSQIP-P) by time period. (A) M&M vs NSQIP-P
EP
complication rates for the entire study period. There was no difference between the 2 systems at identifying mortality, however, NSQIP-P identified significantly more occurrences of morbidity
AC C
over the entire study period. (B) M&M complication rates before and after the 2012 format change. Improvements to the M&M conference in 2012, designed to increase the number of cases presented, significantly increased the occurrences of morbidity captured. (C) M&M vs NSQIP-P complication rates after the 2012 conference improvements. There was no difference between the 2 systems at identifying mortality. NSQIP-P still identified significantly more
25
ACCEPTED MANUSCRIPT 26
occurrences of morbidity during the post-2012 time period. The numbers above the bars represent the total number of occurrences identified /1,000 cases. *p≤0.05.
RI PT
Figure 3. Morbidity and mortality comparisons between morbidity and mortality (M&M)
conference, gray bar, and NSQIP-Pediatric (NSQIP-P), black bar, after the 2012 format change by complication type. Significant differences between systems were identified in the following
AC C
EP
TE D
M AN U
SC
categories: respiratory, cardiovascular, other infectious, other, and readmissions. *p≤0.05.
26
ACCEPTED MANUSCRIPT 27
Precis While the morbidity and mortality conference under-reported events, NSQIP-Pediatric excluded
AC C
EP
TE D
M AN U
SC
strengths that should be harnessed for quality improvement.
RI PT
events identified by conference that resulted in systems changes. Each system has its own
27
AC C
EP
TE D
M AN U
SC
RI PT
ACCEPTED MANUSCRIPT
AC C
EP
TE D
M AN U
SC
RI PT
ACCEPTED MANUSCRIPT
AC C
EP
TE D
M AN U
SC
RI PT
ACCEPTED MANUSCRIPT
S1072-7515(17)30192-8
DOI:
10.1016/j.jamcollsurg.2017.02.008
Reference:
ACS 8681
To appear in:
Journal of the American College of Surgeons
Received Date: 19 December 2016 Revised Date:
5 February 2017
Accepted Date: 6 February 2017
Please cite this article as: Cromeens BP, Lisciandro RE, Brilli RJ, Askegard-Giesmann JR, Kenney BD, Besner GE, Identifying Adverse Events in Pediatric Surgery: Comparing Morbidity and Mortality Conference to the NSQIP–Pediatric, Journal of the American College of Surgeons (2017), doi: 10.1016/ j.jamcollsurg.2017.02.008. This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
ACCEPTED MANUSCRIPT
Identifying Adverse Events in Pediatric Surgery: Comparing Morbidity and Mortality Conference to the NSQIP–Pediatric
RI PT
Barrett P Cromeensa, DO, PhD, Richard E Lisciandrob, BSN, Richard J Brillib, MD, Johanna R Askegard-Giesmanna, MD; Brian D Kenneya, MD, FACS, Gail E Besnera, MD, FACS
Department of Pediatric Surgery; bDepartment of Quality Improvement Services, Nationwide
SC
a
M AN U
Children’s Hospital, Columbus, OH
Disclosure Information: Nothing to disclose.
Presented at the American College of Surgeons NSQIP 12th Annual Conference, San Diego CA,
TE D
July 2016.
AC C
EP
Correspondence address: Gail E. Besner, MD, FACS Department of Pediatric Surgery FB6135 Nationwide Children’s Hospital 700 Children’s Drive Columbus, Ohio, 43205, USA E-mail: [email protected] Phone: (614) 722-3930 FAX: (614) 722-3903
Brief Title: Identifying Adverse Events in Pediatric Surgery
ACCEPTED MANUSCRIPT 2
Abstract: Background
Recent improvements to M&M conference have focused on the case
review system. However, case selection occurs by physician reporting which is limited by
RI PT
selection bias. We compared the effectiveness of our M&M conference to the NSQIP-P system for identifying adverse events. Study Design
Complications from January 2010-September 2015 were compared
SC
between M&M and NSQIP-P. Only M&M patients meeting NSQIP-P criteria were compared to patients from the NSQIP-P system; exclusions were studied separately. Complication rates in
M AN U
M&M Conference before and after a 2012 format change designed to increase case reporting were also compared. Results
Detection of mortality in M&M Conference and NSQIP-P was not different.
Morbidity events identified by NSQIP-P were significantly higher than M&M conference over
TE D
the entire study period (194 vs. 100 occurrences/1000 cases) (p<0.0001). Morbidity occurrences in M&M Conference increased with the 2012 improvements, however, they still remained less than that identified by NSQIP-P (226 vs. 141 occurrences/1000 cases) (p<0.0001). Of 863
EP
patients presented in M&M Conference, 210 were excluded from direct comparison because they did not meet NSQIP-P criteria. These included 62 deaths and 287 occurrences of morbidity.
AC C
Their analysis in M&M Conference resulted in 32 action initiatives directed at system failures. Conclusions NSQIP-P identified more complications than M&M. M&M Conference improvements increased reported cases, but they still remained lower than NSQIP-P. However, M&M Conference identified events resulting in systems changes which would not have been identified by NSQIP-P. While NSQIP-P captures occurrences to compare large patient cohorts,
2
ACCEPTED MANUSCRIPT 3
M&M analyzes singular failures and initiates direct interventions. Integration of these systems
Keywords
RI PT
may optimize their usefulness in quality improvement.
Morbidity; Mortality; M&M Conference; NSQIP; Pediatric Surgery; Adverse
Abbreviations
SC
Events
ACS, American College of Surgeons; CPT, Current Procedural
M AN U
Terminology; ECMO, Extra-Corporeal Membrane Oxygenation; M&M, Morbidity & Mortality; NSQIP-P, National Surgical Quality Improvement Program – Pediatric; SCR, Surgical Clinical
AC C
EP
TE D
Reviewer
3
ACCEPTED MANUSCRIPT 4
Introduction Morbidity and Mortality (M&M) conference has been the traditional mechanism by which surgeons have analyzed complications in order to better understand potential causes of
RI PT
individual or system failures, and to implement changes that prevent repeated occurrences. With recent emphasis on improvement science in medicine, and because surgical M&M conference has been in place at many institutions for decades, there have been recent focused efforts to
SC
optimize the conference as a more useful tool to drive changes in clinical practice and hospital systems.(1-13) These efforts range from better defining the objectives of the conference to more
M AN U
structured initiatives including the use of root cause analysis, plan-do-check-act (PDCA) cycles, cause and effect diagrams, physician report cards, and taxonomies of failure analysis. Although these changes have resulted in important improvements to the conference, they primarily focus on case review and failure classification systems. Few efforts have been reported that ensure
TE D
adequate case selection despite mounting evidence that potential biases in case selection for M&M conferences result in many missed complications.(14-21) The pediatric surgery M&M Conference at our institution has undergone several changes
EP
intended to focus the conference as a tool to identify individual and system failures which can be used to initiate interventions to improve outcomes. In 2012, conference frequency was increased
AC C
from bimonthly to weekly and the presentation format was altered to include detailed and focused reviews. As a result, relatively simple complications were presented quickly, while more complex cases received detailed review. The intent was to increase the number of cases presented within a limited conference time. In 2013, a detailed case analysis system was introduced that facilitated the identification of individual and system failures so that corrective actions could be initiated and tracked to completion.(6) However, at our institution, little was
4
ACCEPTED MANUSCRIPT 5
known about whether the M&M Conference effectively identified all complications that could benefit from case review. Cases in which a complication has occurred in our institution are selected for presentation
RI PT
at M&M conference by pediatric surgery fellows. This contrasts with the process used by the American College of Surgeons (ACS) National Surgical Quality Improvement Program –
Pediatric (NSQIP-P). NSQIP-P is a validated national database where standardized definitions
SC
and specific inclusion and exclusion criteria are used to prospectively collect outcomes data by trained reviewers.(22-24) This database is then leveraged to track surgical complications as
M AN U
interventions are implemented to decrease the incidence of the complications. While M&M conference is also intended to assist in identifying the causes of complications so that interventions can be implemented to prevent future complications, to do so effectively requires reliable identification of complications. The lack of strictly defined criteria for complications in
TE D
M&M Conference such as that utilized by NSQIP-P calls into question the ability of M&M Conference to capture all complications, and thus its effectiveness as a tool for quality improvement. The purpose of this study was to identify the incidence and type of morbidity and
EP
mortality events occurring on our pediatric surgery service as identified by the pediatric surgery
AC C
M&M Conference compared to those identified by the nationally validated NSQIP-P system.
Methods
Data Collection
Data were collected from patients on the general pediatric surgery service
from January 1, 2010 to September 30, 2015. For M&M conference, complications were compiled by the pediatric surgery fellows using the following general guidelines: all deaths; unplanned readmissions; unplanned reoperation within 30 days of initial operation; any adverse
5
ACCEPTED MANUSCRIPT 6
event, error, or delay in diagnosis resulting in unexpected morbidity; unplanned invasive intervention; need for escalation of care; or length of stay (LOS) outside of what would be reasonably expected for a given disease process. Compared to the very strict criteria utilized by
RI PT
NSQIP-P, some components of the general M&M guidelines are subjective in nature (e.g.
prolonged LOS). The general presentation format of the pediatric surgery M&M conference at our institution has been described previously.(6) In brief, detailed M&M case discussion
SC
summaries and action item implementation plans are carefully recorded. In 2012, there was a change to the conference format intended to increase the number of cases discussed. Prior to
M AN U
June 2012, M&M Conference was held on a bimonthly basis. After June 2012, the conference frequency was increased to weekly. In addition, “Focused Reviews” of more straight forward complications were added to the case review process (e.g. superficial wound infection), providing increased time for “Detailed Reviews” of more complicated cases.
TE D
Data collection methods for NSQIP-P have been previously reported.(22-24) In brief, the first 35 procedures performed on patients <18 years of age and falling within eligible Current Procedural Terminology (CPT) codes are sampled on an eight-day cycle by trained Surgical
EP
Clinical Reviewers (SCR). From these cases, 147 variables are abstracted from the medical record including preoperative risk factors, intraoperative variables, and 30-day postoperative
AC C
outcome variables. Not all of the variables recorded by NSQIP (e.g. pre-operative lab values and co-morbidities) indicate the occurrence of an adverse event. Because the current study was designed to compare morbidity and mortality between M&M Conference and NSQIP-P, only NSQIP-P variables corresponding to the occurrence of an adverse event were used for comparison purposes. These include all 30-day postoperative outcome variables, intra-op cardiac arrests requiring CPR, intra-op deaths, intra-op unplanned extubations, and intra-op “other”.
6
ACCEPTED MANUSCRIPT 7
Although NSQIP-P tracks cases from multiple surgical services, only cases abstracted within NSQIP-P that were performed by the pediatric surgery service were included.
Because M&M Conference
RI PT
Comparison of M&M Conference and NSQIP-P occurrences
identifies complications from the entire population of patients on the pediatric general surgery service and NSQIP-P includes only a sample of patients meeting specified criteria, these two
SC
populations of patients are not directly comparable. Abbreviated criteria for NSQIP-P data
extraction are provided in Table 1. It is important to note that outcome variables in NSQIP-P are
M AN U
only tracked up to 30 days. To allow for direct M&M Conference and NSQIP-P comparison, the same inclusion and exclusion criteria utilized for NSQIP-P sampling were applied to cases presented at M&M Conference during the study period, and only those M&M cases meeting NSQIP-P criteria were included for comparative analyses. To understand complications that
TE D
were not being captured by NSQIP-P but reported through the M&M Conference process, all M&M cases excluded from direct comparative analysis because they did not meet NSQIP-P
EP
inclusion criteria were analyzed separately.
Determination of Morbidity and Mortality Rates
NSQIP-P utilizes a sampling
AC C
algorithm to collect data on a fraction of operative patients that intends to be reflective of the institutional experience. In contrast, M&M conference attempts to comprehensively capture all complications occurring within the institution. Given these differences, complication rates were analyzed rather than absolute numbers in order to compare the two systems. Morbidity and mortality occurrence rates were tabulated for M&M Conference and for NSQIP-P as follows:
7
ACCEPTED MANUSCRIPT 8
M&M Conference Occurrence Rates
The same outcome variable definitions tracked by
NSQIP-P were applied to the M&M analysis to determine the rate of M&M Conference occurrences. The total number of morbidity occurrences was obtained by summing all the
RI PT
defined NSQIP-P intra-op and post-op morbidity variables. To further break down morbidity into categories, the defined NSQIP-P intra-op and post-op morbidity variables were further grouped into types of complications (Table 2). The total number of mortality occurrences was obtained by
SC
summing all the defined NSQIP-P intra-op and post-op mortality variables. Morbidity and
mortality rates were calculated as the total number of occurrences identified divided by the total
M AN U
number of pediatric surgery cases meeting NSQIP-P criteria during the study period. The Department of Pediatric Surgery performed 23,902 operations during the study period. Of these, 9,603 met NSQIP-P study criteria and served as the denominator for calculating occurrence rates for M&M Conference.
Occurrence rates were calculated as the total number of
TE D
NSQIP-P Occurrence Rates
occurrences identified divided by the total number of cases extracted for the pediatric surgery service during the study period. NSQIP-P abstracted 2,973 cases during the study period, which
EP
served as the denominator for calculating occurrence rates for NSQIP-P. Statistical Analyses
Morbidity and mortality rates were compared between the two systems
AC C
using univariate chi-square analysis with p<0.05 considered statistically significant. All rates were reported as the number of occurrences per 1000 cases.
Analysis of 2012 M&M Conference Format Change
Because the M&M
Conference format was changed in 2012 to increase the number of complications reported, complication rates reported before 2012 (pre-2012 M&M cohort) and after 2012 (post-2012
8
ACCEPTED MANUSCRIPT 9
M&M cohort) were compared. In addition to comparing M&M Conference and NSQIP-P occurrences over the entire study period, post-2012 cohorts for both systems were also
RI PT
compared.
Results
Comparison of M&M Conference and NSQIP-P Occurrences From January 1, 2010
SC
through September 30, 2015 the department of pediatric surgery held 151 M&M Conferences during which 863 patients were discussed. When the NSQIP-P selection criteria were applied,
M AN U
210 patients presented at M&M Conference were excluded. The most common reason for exclusion was for undergoing a procedure not tracked by NSQIP-P (Figure 1). One exclusion criterion for NSQIP-P is >18 years of age. While there were patients presented in M&M Conference who were >18 years of age, they all met at least one additional NSQIP-P exclusion
TE D
criterion and therefore are represented in the multifactorial category of Figure 1. Morbidity and mortality rates were calculated and compared between the two systems (Figure 2). Mortality rates identified by M&M Conference and NSQIP-P were not different (8 vs.
EP
10 deaths/1000 cases) (p=0.28). Morbidity events identified by NSQIP-P were significantly higher than M&M Conference identified events over the entire study period (194
AC C
occurrences/1000 cases vs. 100 occurrences/1000 cases) (p<0.0001) (Figure 2A). Morbidity and mortality rates identified solely by M&M Conference were compared before and after the 2012 conference format change (Figure 2B). There was no difference in mortality identified between the pre- and post-2012 M&M cohorts. However, morbidities identified in the post-2012 M&M cohort were significantly greater than the pre-2012 cohort (141 vs. 46 occurrences/1000 cases) (p<0.0001). Morbidity and mortality rates identified by the post-2012 M&M cohort were
9
ACCEPTED MANUSCRIPT 10
compared to the post-2012 NSQIP-P cohort (Figure 2C). There was no difference in mortality identified between M&M Conference and NSQIP-P in the post-2012 time period. However, morbidities identified by post-2012 NSQIP-P still remained significantly higher than post-2012
RI PT
M&M (226 vs. 141 occurrences/1000 cases) (p<0.0001).
Morbidity rates by category between post-2012 M&M and post-2012 NSQIP-P cohorts were compared (Figure 3). There were no differences between M&M Conference and NSQIP-P
SC
regarding wound, neurologic, renal complications, or with reoperations. NSQIP-P identified significantly higher rates than M&M conference regarding respiratory, cardiovascular, and other
M AN U
infectious complications, as well as readmissions. M&M Conference did identify significantly higher “other” complication rates compared to NSQIP-P. This “other” category includes a wide range of adverse events that do not fall into a well-defined NSQIP-P variable, and are assigned at
TE D
the discretion of the SCR as a marker of poor outcome.
Analysis of M&M Conference Cases Excluded by NSQIP-P
When analyzing the 210
M&M conference cases excluded by NSQIP-P, there were 62 deaths and 287 incidents of
EP
morbidity identified (Table 3). Of the 62 deaths, 46 patients were non-operative cases while 16 patients were operative cases, the majority of which were excluded because they were operative
AC C
trauma cases (NSQIP-P does not track trauma-related occurrences). Of the 210 patients excluded, 118 were excluded because they underwent a procedure not tracked by NSQIP-P. Over half of these patients (n=70) fell within one of four types of procedures: circumcision (n=10), soft tissue abscess drainage (n=16), central line placement (n=31), and extra-corporeal membrane oxygenation (ECMO) cannulation (n=13). From these 70 cases, 54 unplanned reoperations were performed.
10
ACCEPTED MANUSCRIPT 11
Ninety-one M&M conference patients excluded by NSQIP-P criteria underwent review by our comprehensive M&M case analysis system which was initiated in 2013 and designed to identify individual and system failures so that corrective actions could be initiated and
RI PT
completed.(6) Analysis of these 91 patients resulted in the initiation of 66 action items. The majority of these action items were education initiatives (n=34), however the remainder were directed at implementing systems changes (n=32). These systems changes included creating
SC
automatic triggers for consultation of the pediatric surgery service, replacing outdated and insufficient equipment, modifying order sets to ensure appropriate content, updating and
M AN U
automating burn resuscitation protocols, and working to expedite patient transfers from referring institutions. Below is an example of a case excluded from NSQIP-P where failures were identified during M&M Conference resulting in corrective action: Sample M&M Case
A 17 month old female presented with a foreign body ingestion. A
TE D
foreign body series, which included a chest x-ray and abdominal x-ray, did not identify a foreign body. The mother provided a plastic disc identical to the ingested object which was confirmed to be radiopaque. Although no foreign body was identified radiologically, the patient was observed
EP
overnight. Upon continuing to refuse oral intake, she underwent endoscopy, with the foreign body identified in the pyriform sinus. Discussion in M&M Conference identified a system error
AC C
involving lack of appropriate radiologic protocol. It was decided that a lateral view of the neck and soft tissue be added to the foreign body series.
Discussion
Originally known as the Anesthesia Study Commission, early M&M conferences were some of the first quality improvement initiatives before quality improvement science in medicine
11
ACCEPTED MANUSCRIPT 12
became an established field. Originally composed of surgeons, anesthesiologists, and internal medicine doctors, these conferences were purposed for robust discussion of unexpected and poor outcomes so that their etiology could be determined and thus avoided in the future.(25, 26) These
RI PT
conferences were quick to catch on in many surgical departments across the country and became the traditional mechanism for case review. As emphasis on quality improvement science in
medicine grew, the conferences were an attractive tool for gathering data and implementing
SC
quality initiatives. However, there has been growing evidence that data capture by M&M
Conference is not reflective of the entire institutional experience, and has led some to conclude
M AN U
that M&M Conference is an inadequate system to drive change.(14-21)
The pediatric surgery M&M Conference at our institution is a robust multidisciplinary conference that has been in place for decades. In recent years, format changes were implemented that increased the efficiency of case presentation and case analysis, assigned and tracked
TE D
corrective actions, and documented action item follow-up.(6) In the first year after implementation of these conference improvements, 42 action items directed at correcting system failures were identified. Before this system was implemented, interventions to correct errors
EP
were rarely initiated and more importantly, they were not documented or tracked to completion. Reports from other institutions of M&M conferences under-reporting mortality by as much as
AC C
50% and morbidity by as much as 75%, indicated significant unidentified potential for improvement of patient care.(15, 16, 19) Based on these reports, we sought to determine the extent to which our M&M conference was under-reporting complications. Our institution has participated in NSQIP-P since 2010. Since NSQIP-P is a validated
data collection system that tracks a robust set of complications, we chose to use this system as the standard to which we compared our M&M Conference. Similar comparisons have been made
12
ACCEPTED MANUSCRIPT 13
between M&M conferences and the adult NSQIP system in other hospitals.(15, 16) However, comparable studies in pediatric hospitals are lacking. In the current study, there was no difference in mortality rates identified by M&M Conference or NSQIP-P. This is in contrast to
RI PT
studies in the adult setting, where M&M Conference under-reports mortality by as much as 50% compared to NSQIP.(16, 19) At our institution, the pediatric surgery service is managed by two clinical fellows who are responsible for M&M case selection. Because they are aware of all
SC
patients on the service, consistency is likely maintained with some control for selection bias, which may contribute to the more robust mortality reporting seen in our M&M Conference.
M AN U
During the total study period there were significantly fewer morbidities identified by M&M Conference than by NSQIP-P, and this held true even after the 2012 changes designed to increase the number of cases presented. Although our M&M Conference was equivalent to NSQIP-P at identifying mortality, it significantly under-reported morbidity, similar to other
TE D
reports. Possible explanations for under-reporting of complications at M&M Conferences likely include insufficient and biased sampling. Most M&M conferences rely on physician selfreporting. As the physician directly taking care of patients is the most likely to know whether or
EP
not a complication occurred, they are the logical person to choose complications for presentation. However, time pressures, fears of embarrassment in front of one’s peers, punishment, and legal
AC C
implications have been suggested to decrease the likelihood of open self-reporting.(27-30) Another possible contribution is the lack of clearly defined criteria for what should and should not be considered a complication. When the decision as to whether an untoward outcome should be presented at conference is left in the hands of each surgeon, there is inherent opportunity for disagreement between individuals that will likely result in inconsistent reporting. Additionally, there is bias by each physician to report that which they find most pertinent for discussion. When
13
ACCEPTED MANUSCRIPT 14
looking at the categories of complications identified by M&M Conference, it is not surprising that categories most directly linked to the operation and thus most likely of interest to the reporting surgeon were not under-reported compared to NSQIP-P. These categories include
RI PT
postoperative mortality, unplanned reoperations and wound complications. Contrast these
limitations to other data capturing systems such as NSQIP-P. In this system, trained reviewers systematically review charts and collect data based on defined criteria in an attempt to control for
SC
the above stated biases. This approach consistently captures data that reflect the institutional experience.(22, 23, 31, 32) This consistent and defined approach results in superior capture of
M AN U
complications compared to M&M Conference. It is this under-reporting of M&M Conference compared to NSQIP that lead Miller, et. al.(15) to conclude that NSQIP may be the better option for quality improvement endeavors in their institution. However, while our study confirms reporting limitations of M&M Conference, other benefits of M&M Conference are apparent
TE D
upon further analysis.
The NSQIP-P system was designed to capture the majority of clinically relevant complications that occur in the field of pediatric surgery. Because there are exclusion criteria in
EP
NSQIP-P, it is inevitable that complications will occur in excluded patients and will therefore not be captured. However, the extent of the complications not captured by NSQIP-P, and the value
AC C
which they may hold for quality improvement, is difficult to quantify. By analyzing the patients excluded from M&M Conference for not meeting NSQIP-P criteria, the extent and value of the complications missed by NSQIP-P becomes apparent. Applying the NSQIP-P exclusion criteria to M&M Conference resulted in the exclusion of 210 patients from comparative analysis. By excluding these patients, NSQIP-P did not capture the 62 deaths or 287 incidents of morbidity that occurred in these 210 patients. M&M Conference has the added benefit of real time direct
14
ACCEPTED MANUSCRIPT 15
discussion with the involved parties regarding the cause of a complication. At our institution, this discussion is facilitated by a comprehensive failure analysis including mechanisms for implementing action initiatives and tracking them to completion. Of the 210 M&M Conference
RI PT
patients in our study that were excluded from comparative analysis due to not meeting NSQIP-P inclusion criteria, 91 underwent comprehensive failure analysis. This resulted in 66 action
initiatives, 32 of which were directed at correcting systems failures. These were cases with
SC
substantial system failures that necessitated corrective action. Although NSQIP-P captures more complications than M&M Conference, it is only able to capture those complications falling
M AN U
within the definitions of the system. M&M Conference on the other hand identifies numerous complications otherwise missed by NSQIP-P that hold substantial value for quality improvement, as evidenced by the many actions initiated to correct identified failures. Based on the above findings, it is clear that M&M conference and NSQIP-P are not
TE D
equivalent systems. However, neither are comprehensive quality improvement tools and each provide specific strengths and weaknesses. Being able to harness and integrate the benefits of each system would be ideal for optimizing improvement endeavors. Because NSQIP-P has
EP
proven more rigorous at identifying complications, we are going to start forwarding complications identified by NSQIP-P to M&M conference. Of course NSQIP-P is only a
AC C
sampling, thus there will be complications missed by both systems. However, any complication identified by NSQIP-P that may not have already been identified by M&M conference will benefit from the comprehensive failure analysis provided by the M&M process. We are initially trialing this effort with surgical site infections but will hopefully expand this further to other adverse events. One drawback to using the NSQIP-P system in this manner is that often the complications are not identified until 90 days have passed, making M&M discussion remote
15
ACCEPTED MANUSCRIPT 16
from when the surgery occurred, potentially hindering the failure analysis process secondary to recall limitations by the involved parties. Unfortunately, we have not found a means to overcome this time gap. Although this limitation remains, delayed analysis is superior to no analysis.
RI PT
Additionally, ongoing efforts to define and standardize the M&M selection process are being made to improve capture.
The utilization of these quality improvement tools is becoming even more important. The
SC
American College of Surgeons in conjunction with the Task Force for Children’s Surgical Care have developed the Children’s Surgery Verification (CSV) Quality Improvement Program to
M AN U
define optimal resources and standardize surgical care for children. Part of this verification process requires that the institution have a Performance Improvement and Patient Safety (PIPS) program that encompasses all surgical disciplines. As our institution is navigating the verification process, our M&M conference has served as an important platform on which to meet
TE D
the PIPS requirements. Currently our M&M platform is being used as a model for all surgical (and medical) divisions at our institution. This will allow us to develop a case registry from which cases will be selected for presentation at our quarterly PIPS meeting, with emphasis on
AC C
Conclusions
EP
cases in which systems errors have been identified and corrected.
M&M Conference is equivalent to NSQIP-P at identifying mortality on the pediatric
surgery service at our institution. However, M&M Conference significantly under-reports morbidity compared to the NSQIP-P system. Focused efforts to improve M&M Conference significantly increased the morbidities identified, however, this was still significantly less than those identified by the NSQIP-P system. Although M&M Conference should ideally capture all
16
ACCEPTED MANUSCRIPT 17
the complications on the service, in reality, M&M Conference is hindered by inadequate sampling and sampling biases. However, to dismiss M&M Conference because it under-reports complications would be short sighted. The data presented here demonstrate that although M&M
RI PT
Conference under-reports complications compared to NSQIP-P, each of these two systems has its own strengths and weaknesses. NSQIP-P is a robust data collection system that samples data that reflect institutional experience, making it an ideal system for comparing variables between
SC
large cohorts of patients to identify and track areas for improvement. However, data collection is constrained to the parameters that define the system, therefore missing some complications and
M AN U
opportunities for important systems changes. Although M&M Conference does not capture complications to the extent of NSQIP-P, it is not constrained by the same definitions as the NSIQP-P system, and thus captures outcomes otherwise missed by NSQIP-P. In addition, M&M Conference performs real-time analysis of complications so that failures are identified and
TE D
corrective actions are implemented. Use of both systems in order to harness their strengths is
AC C
EP
likely to prove useful in order to optimize their roles in improvement of patient care.
17
ACCEPTED MANUSCRIPT 18
References 1.
Anderson CI, Nelson CS, Graham CF, et al. Disorganized care: the findings of an
iterative, in-depth analysis of surgical morbidity and mortality. J Surg Res 2012;177:43-48. Antonacci AC, Lam S, Lavarias V, et al. A report card system using error profile analysis
RI PT
2.
and concurrent morbidity and mortality review: surgical outcome analysis, part II. J Surg Res 2009;153:95-104.
Antonacci AC, Lam S, Lavarias V, et al. A morbidity and mortality conference-based
SC
3.
classification system for adverse events: surgical outcome analysis: part I. J Surg Res
4.
M AN U
2008;147:172-177.
Batthish M, Tse SM, Feldman BM, et al. Trying to improve care: the Morbidity and
Mortality Conference in a division of rheumatology. J Rheumatol 2014;41:2452-2458. 5.
Cifra CL, Bembea MM, Fackler JC, Miller MR. Transforming the morbidity and
TE D
mortality conference to promote safety and quality in a PICU. Pediatr Crit Care Med 2016;17:58-66. 6.
Cromeens B, Brilli R, Kurtovic K, et al. Implementation of a pediatric surgical quality
7.
EP
improvement (QI)-driven M&M conference. J Pediatr Surg 2016;51:137-142. Deis JN, Smith KM, Warren MD, et al. Transforming the morbidity and mortality
8.
AC C
conference into an instrument for systemwide improvement culture and redesign. 2008 Aug. Gonzalo JD, Bump GM, Huang GC, Herzig SJ. Implementation and evaluation of a
multidisciplinary systems-focused internal medicine morbidity and mortality conference. J Grad Med Educ 2014;6:139-146.
18
ACCEPTED MANUSCRIPT 19
9.
Johna S, Tang T, Saidy M. Patient safety in surgical residency: root cause analysis and
the surgical morbidity and mortality conference--case series from clinical practice. Perm J 2012;16:67-69. McDonnell C, Laxer RM, Roy WL. Redesigning a morbidity and mortality program in a
RI PT
10.
university-affiliated pediatric anesthesia department. Jt Comm J Qual Patient Saf 2010;36:117125.
Rabizadeh S, Gower WA, Payton K, et al. Restructuring the morbidity and mortality
SC
11.
(Phila) 2012;51:1079-1086. 12.
M AN U
conference in a department of pediatrics to serve as a vehicle for system changes. Clin Pediatr
Szostek JH, Wieland ML, Loertscher LL, et al. A systems approach to morbidity and
mortality conference. Am J Med 2010;123:663-668. 13.
Vogel P, Vassilev G, Kruse B, Cankaya Y. Morbidity and mortality conference as part of
2011;396:1009-1015. 14.
TE D
PDCA cycle to decrease anastomotic failure in colorectal surgery. Langenbecks Arch Surg
McVeigh TP, Waters PS, Murphy R, et al. Increasing reporting of adverse events to
2013;216:50-56.
Miller DC, Filson CP, Wallner LP, et al. Comparing performance of morbidity and
AC C
15.
EP
improve the educational value of the morbidity and mortality conference. J Am Coll Surg
mortality conference and National Surgical Quality Improvement Program for detection of complications after urologic surgery. Urology 2006;68:931-937. 16.
Hutter MM, Rowell KS, Devaney LA, et al. Identification of surgical complications and
deaths: an assessment of the traditional surgical morbidity and mortality conference compared
19
ACCEPTED MANUSCRIPT 20
with the American College of Surgeons-National Surgical Quality Improvement Program. J Am Coll Surg 2006;203:618-624. 17.
Falcone JL, Lee KK, Billiar TR, Hamad GG. Practice-based learning and improvement: a
RI PT
two-year experience with the reporting of morbidity and mortality cases by general surgery residents. J Surg Educ 2012;69:385-392. 18.
Thompson JS, Prior MA. Quality assurance and morbidity and mortality conference. J
19.
SC
Surg Res 1992;52:97-100.
Bilimoria KY, Kmiecik TE, DaRosa DA, et al. Development of an online morbidity,
M AN U
mortality, and near-miss reporting system to identify patterns of adverse events in surgical patients. Arch Surg 2009;144:305-311; discussion 311. 20.
Feldman L, Barkun J, Barkun A, et al. Measuring postoperative complications in general
surgery patients using an outcomes-based strategy: comparison with complications presented at
21.
TE D
morbidity and mortality rounds. Surgery 1997;122:711-719; discussion 719-720. Wanzel KR, Jamieson CG, Bohnen JM. Complications on a general surgery service:
incidence and reporting. Can J Surg 2000;43:113-117. Raval MV, Dillon PW, Bruny JL, et al. Pediatric American College of Surgeons National
EP
22.
Surgical Quality Improvement Program: feasibility of a novel, prospective assessment of surgical
23.
AC C
outcomes. J Pediatr Surg 2011;46:115-121. Raval MV, Dillon PW, Bruny JL, et al. American College of Surgeons National Surgical
Quality Improvement Program Pediatric: a phase 1 report. J Am Coll Surg 2011;212:1-11. 24.
Dillon P, Hammermeister K, Morrato E, et al. Developing a NSQIP module to measure
outcomes in children's surgical care: opportunity and challenge. Semin Pediatr Surg 2008;17:131-140.
20
ACCEPTED MANUSCRIPT 21
25.
Ruth HS, Haugen FP, Grove DD. Anesthesia Study Commission; findings of 11 years'
activity. J Am Med Assoc 1947;135:881-884. 26.
Deshpande JK, Throop PG, Slayton JM. Standardization of case reviews (morbidity and
27.
RI PT
mortality rounds) promotes patient safety. Pediatr Clin North Am 2012;59:1307-1315.
Cullen DJ, Bates DW, Small SD, et al. The incident reporting system does not detect
adverse drug events: a problem for quality improvement. Jt Comm J Qual Improv 1995;21:541-
28.
SC
548.
Mariner WK. Medical error reporting: professional tensions between confidentiality &
M AN U
liability. Issue Brief (Mass Health Policy Forum) 2001 Nov 6(13):1-35. 29.
Leape LL. Reporting of adverse events. N Engl J Med 2002;347:1633-1638.
30.
Weissman JS, Annas CL, Epstein AM, et al. Error reporting and disclosure systems:
views from hospital leaders. JAMA 2005;293:1359-1366.
Sharp NE, Knott EM, Iqbal CW, et al. Accuracy of American College of Surgeons
TE D
31.
National Surgical Quality Improvement Program Pediatric for laparoscopic appendectomy at a single institution. J Surg Res 2013;184:318-321.
Duggan EM, Gates DW, Slayton JM, Blakely ML. Is NSQIP Pediatric review
EP
32.
representative of total institutional experience for children undergoing appendectomy? J Pediatr
AC C
Surg 2014;49:1292-1294.
21
ACCEPTED MANUSCRIPT 22
Table 1. Abbreviated NSQIP-Pediatric Inclusion/Exclusion Criteria
AC C
EP
TE D
M AN U
SC
RI PT
Patient must be < 18 years of age Patient must have undergone an operation Operation must fall within the listing of tracked CPT codes Operation cannot be associated with excluded ICD-9 codes (800-933; 938-959.9; 995.50995.59) Patient must have an American Society of Anesthesiologists score < 6
22
ACCEPTED MANUSCRIPT 23
Table 2. NSQIP-Pediatric Outcomes Variables Grouped by Morbidity Category
SC
Neurologic
M AN U
Respiratory
Cardiovasular
TE D
Renal/urinary
AC C
EP
Other infectious
Other
Outcomes variables Postoperative superficial Incisional SSI; postoperative deep incisional SSI; postoperative organ/space SSI; postoperative deep wound disruption; postoperative superficial wound disruption Postoperative coma > 24 h; postoperative CVA/stroke/intracranial hemorrhage; postoperative seizure; postoperative nerve injury; postoperative intraventricular hemorrhage grade 1-4; postoperative intraventricular hemorrhage grade not specified Intraoperative unplanned extubation; postoperative pneumonia; postoperative unplanned intubation; postoperative pulmonary embolism Intraoperative cardiac arrest requiring CPR; postoperative cardiac arrest requiring CPR; intraoperative/postoperative transfusion within 72 h; postoperative venous thrombosis requiring therapy Postoperative progressive renal insufficiency; postoperative acute renal failure; postoperative urinary tract infection Postoperative systemic sepsis; postoperative septic shock; postoperative central-line associated blood stream infection Intraoperative other occurrence; postoperative other occurrence; postoperative graft/prosthesis/flap failure
RI PT
Morbity category Wound
Readmissions Reoperations SSI, surgical site infection
23
ACCEPTED MANUSCRIPT 24
Table 3. Morbidity Identified by Morbidity and Mortality Conference But Excluded by NSQIP-Pediatric
RI PT
Occurrences/1,000 Cases 2.10 0.28 0.14 0.84 0.14 0.14 6.71 2.46 7.27 20.10
SC
Occurrences 30 4 2 12 2 2 96 35 104 287
AC C
EP
TE D
M AN U
Morbidities Wound Neurologic Respiratory Cardiovascular Renal/urinary Other infectious Other Readmissions Reoperations Total
24
ACCEPTED MANUSCRIPT 25
Figure Legends Figure 1. Distribution of morbidity and mortality cases excluded from comparative analysis based on NSQIP-Pediatric (NSQIP-P) exclusion criteria. Each category shown is a reason for
RI PT
exclusion from NSQIP-P selection that corresponds to the NSQIP-P criteria shown in Table 1. The most common reason for exclusion of patients presented at morbidity and mortality
conferences (M&M) was for undergoing a procedure not tracked by NSQIP-P. This was
SC
followed by not having undergone an operation, meeting multiple exclusion criteria, undergoing a procedure linked with an excluded ICD-9 code, or having a complication >30 days after their
M AN U
operation. Not represented are patients >18 years of age and patients with an American Society of Anesthesiologists (ASA) score of 6. Although there were patients >18 years old excluded, they also met other exclusion criteria, putting them in the “multifactorial” category. There were
TE D
no patients operated on with an ASA score of 6.
Figure 2. Morbidity and mortality comparisons between morbidity and mortality (M&M) conference and NSQIP-Pediatric (NSQIP-P) by time period. (A) M&M vs NSQIP-P
EP
complication rates for the entire study period. There was no difference between the 2 systems at identifying mortality, however, NSQIP-P identified significantly more occurrences of morbidity
AC C
over the entire study period. (B) M&M complication rates before and after the 2012 format change. Improvements to the M&M conference in 2012, designed to increase the number of cases presented, significantly increased the occurrences of morbidity captured. (C) M&M vs NSQIP-P complication rates after the 2012 conference improvements. There was no difference between the 2 systems at identifying mortality. NSQIP-P still identified significantly more
25
ACCEPTED MANUSCRIPT 26
occurrences of morbidity during the post-2012 time period. The numbers above the bars represent the total number of occurrences identified /1,000 cases. *p≤0.05.
RI PT
Figure 3. Morbidity and mortality comparisons between morbidity and mortality (M&M)
conference, gray bar, and NSQIP-Pediatric (NSQIP-P), black bar, after the 2012 format change by complication type. Significant differences between systems were identified in the following
AC C
EP
TE D
M AN U
SC
categories: respiratory, cardiovascular, other infectious, other, and readmissions. *p≤0.05.
26
ACCEPTED MANUSCRIPT 27
Precis While the morbidity and mortality conference under-reported events, NSQIP-Pediatric excluded
AC C
EP
TE D
M AN U
SC
strengths that should be harnessed for quality improvement.
RI PT
events identified by conference that resulted in systems changes. Each system has its own
27
AC C
EP
TE D
M AN U
SC
RI PT
ACCEPTED MANUSCRIPT
AC C
EP
TE D
M AN U
SC
RI PT
ACCEPTED MANUSCRIPT
AC C
EP
TE D
M AN U
SC
RI PT
ACCEPTED MANUSCRIPT