- Email: [email protected]
Examining The Myth Of The “July Phenomenon” in Surgical Patients: An Analysis Utilizing The National Surgical Quality Improvement Program Database
196
ASSOCIATION FOR ACADEMIC SURGERY AND SOCIETY OF UNIVERSITY SURGEONS—ABSTRACTS
16.7. Evaluating A New Technique For Improving The Reliability Of Surgical Outcomes Measurement. A. A. Ghaferi,1 C. Y. Ko,2 J. B. Dimick1; 1University of Michigan, Ann Arbor, MI; 2American College of Surgeons, Chicago, IL Introduction: Reliability adjustment, a method for better accounting for statistical ‘‘noise’’, is becoming more widely used outside surgery. We sought to determine the impact of this technique in surgery. Methods: We used the American College of Surgeons - National Surgical Quality Improvement Program (ACS-NSQIP) to identify all patients undergoing colon resection in 2007 (n¼179 hospitals, n¼16,262 patients). We first used standard methods to generate risk-adjusted mortality for each hospital. We then used empirical Bayes techniques to adjust for reliability. Hospitals were grouped into five equal-sized groups (quintiles). To evaluate the impact of reliability adjustment, we estimated the extent to which hospital-level variation was reduced and compared quintile rankings with and without reliability adjustment. Results: Adjustment for reliability greatly diminished the apparent variation in risk-adjusted mortality across hospitals. Before adjustment for reliability, risk-adjusted mortality varied nearly 5-fold (1.8% to 8.4%) from the best (top 20%) to worst (bottom 20%) quintiles. After adjusting for reliability, the mortality rates varied less than 2-fold (3.3% to 5.7%) from the best to worst quintiles. Reliability adjustment resulted in large changes in hospital quintile rankings. Of the hospitals in the top 20% using standard methods, only 45% (20/44) remain in the top 20% after reliability adjustment. Likewise, of the hospitals in the bottom 20% using standard methods, only 55% (25/45) remained in the bottom 20% after reliability adjustment. Conclusions: Reliability adjustment results in large changes in hospital performance for colon resection. This technique provides more accurate estimates of hospital performance and should be considered in the place of standard methods.
16.8. Examining The Myth Of The ‘‘July Phenomenon’’ in Surgical Patients: An Analysis Utilizing The National Surgical Quality Improvement Program Database. B. A. Ehlert, J. T. Nelson, C. E. Goettler, F. M. Parker, W. M. Bogey, C. S. Powell, M. C. Stoner; East Carolina University, Greenville, NC Introduction: Conventional wisdom states the start of the academic year in July is a precarious time to be hospitalized due to the introduction of new residents, described as the ‘‘July Phenomenon’’ in the literature. There is a paucity of literature regarding this observation in surgical practice. The objective of this study is to evaluate morbidity and mortality rates in surgical patients at the beginning of the academic year. Methods: The National Surgical Quality Improvement Program (NSQIP) database was utilized to gather data on the ten most common inpatient surgical procedures from 2005-2007. Patients were then divided into populations based on quarter of admission and their comorbid profile was assessed with a modified Charlson Comorbidity Index (CCI). Study end points included mortality, serious adverse events (SAE) and all morbidities. Univariate and multivariate analysis of mortality and morbidity rates was conducted examining the total population, and then subsequently stratified by operation to assess for significant differences in end points (P<0.05). Results: A total of 89,473 patients were identified. 26,287 patients underwent surgery during the first academic quarter, defined as July-September, and 63,186 were operated on throughout the remainder of the year. Demographic and pre-operative CCI were not significantly different among the patient populations. During the first academic quarter, the mortality rate was 2.02% and throughout the remainder of the year was 2.05% (P¼0.793). Overall SAE and morbidity rates were non-significant among the two populations (11.48% vs. 11.39%, P¼0.697 and 18.34% vs. 17.84%, P¼ 0.076, respectively). When stratified by operation, the only significant findings associated with first quarter operation were an increased mortality rate for ‘‘artery bypass grafts’’ (3.74% vs. 2.86%, P¼0.039) and ‘‘repair bowel opening’’ (1.13% vs. 0.56, P¼.033). Multivariate analysis performed on the ‘‘artery by-
pass graft’’ subset of patients confirmed the deleterious effect of first quarter admission (OR¼1.35, CI 1.023–1.774); however, first quarter admission was not a correlate of mortality in the ‘‘repair bowel opening’’ patient subset (OR¼1.880 CI 0.903–3.912). Conclusions: By in large, these data refute the ‘‘July Phenomenon’’. Although the ‘‘repair bowel opening’’ subset showed a significant increase in mortality during the first academic quarter, multivariate analysis revealed patient disease to have a larger impact than timing of operation. The ‘‘artery bypass graft’’ population was affected by timing of operation and this may reflect new surgical residents being unfamiliar with the management of patients with complex cardiovascular disease. Alternatively, this finding may be a reflection of disease process or patient factors not described in the NSQIP dataset. Further research is warranted to validate these findings in the vascular subset of patients.
16.9. Ranking Hospitals On Surgical Morbidity: Does The Type Of Complication Matter?. J. R. East,1 G. Rangrass,1 C. Y. Ko,2 J. B. Dimick1; 1University of Michigan, Ann Arbor, MI; 2American College of Surgeons, Chicago, IL Introduction: The American College of Surgeons National Surgical Quality Improvement Program (ACS NQSIP) reports rates of riskadjusted morbidity for each participating hospital. However, their approach does not take into account the type of complication that occurred. We sought to determine whether alternative approaches that consider the severity and/or number of complications would impact hospital rankings. Methods: We used data from the ACS NSQIP of all patients undergoing colectomy in 2007 (181 hospitals, 18,455 patients). We divided complications into minor and severe based on consensus of a panel of clinicians. We first evaluated the extent to which minor and severe complications increased a patient’s risk of mortality and prolonged length of stay. We then assessed whether using an alternative approach for estimating hospital morbidity (based on the severity and number of complications) would impact hospital rankings. We assessed agreement among these alternative approaches based on the proportion of patients that moved out of the top 20% and bottom 20% of hospitals. Results: Patients with severe complications have a higher mortality (15.7% vs. 6.9%) and prolonged length of stay (61% vs. 43%) compared to patients with minor complications. Patients with multiple complications (2 or more) had a higher mortality (19% vs. 6.1%) and prolonged length of stay (68% vs. 33%) than patients with only one complication. Compared to the standard approach for assessing morbidity, ranking hospitals on severe complications resulted in 13 (35%) hospitals moving out of the top 20% and 17 (47%) hospitals moving out of the bottom 20%. A similar degree of reclassification was found when comparing the standard morbidity rankings to an alternative approach that considered the number of different complications. Conclusions: Although the number and severity of complications impact mortality and prolonged length of stay, existing approaches to morbidity assessment do not take these factors into account. Quality measurement platforms should consider weighting complications according to severity and number.
16.10. How Widely Has ACS-NSQIP Disseminated? A Market Share Analysis In The National Medicare Population. N. H. Osborne,1,2 G. Rangrass,1,2 C. Y. Ko,3,4 J. B. Dimick1,2; 1 Center for Healthcare Outcomes & Policy (CHOP), Ann Arbor, MI; 2University of Michigan Medical School, Department of Surgery, Ann Arbor, MI; 3Center of Surgical Outcomes and Quality, Los Angeles, CA; 4American College of Surgeons National Surgical Quality Improvement Program, Chicago, IL Introduction: Although the American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP) is the most recognized outcomes registry in surgery, it is unclear how widely the program has disseminated. We sought to determine the
ASSOCIATION FOR ACADEMIC SURGERY AND SOCIETY OF UNIVERSITY SURGEONS—ABSTRACTS
16.7. Evaluating A New Technique For Improving The Reliability Of Surgical Outcomes Measurement. A. A. Ghaferi,1 C. Y. Ko,2 J. B. Dimick1; 1University of Michigan, Ann Arbor, MI; 2American College of Surgeons, Chicago, IL Introduction: Reliability adjustment, a method for better accounting for statistical ‘‘noise’’, is becoming more widely used outside surgery. We sought to determine the impact of this technique in surgery. Methods: We used the American College of Surgeons - National Surgical Quality Improvement Program (ACS-NSQIP) to identify all patients undergoing colon resection in 2007 (n¼179 hospitals, n¼16,262 patients). We first used standard methods to generate risk-adjusted mortality for each hospital. We then used empirical Bayes techniques to adjust for reliability. Hospitals were grouped into five equal-sized groups (quintiles). To evaluate the impact of reliability adjustment, we estimated the extent to which hospital-level variation was reduced and compared quintile rankings with and without reliability adjustment. Results: Adjustment for reliability greatly diminished the apparent variation in risk-adjusted mortality across hospitals. Before adjustment for reliability, risk-adjusted mortality varied nearly 5-fold (1.8% to 8.4%) from the best (top 20%) to worst (bottom 20%) quintiles. After adjusting for reliability, the mortality rates varied less than 2-fold (3.3% to 5.7%) from the best to worst quintiles. Reliability adjustment resulted in large changes in hospital quintile rankings. Of the hospitals in the top 20% using standard methods, only 45% (20/44) remain in the top 20% after reliability adjustment. Likewise, of the hospitals in the bottom 20% using standard methods, only 55% (25/45) remained in the bottom 20% after reliability adjustment. Conclusions: Reliability adjustment results in large changes in hospital performance for colon resection. This technique provides more accurate estimates of hospital performance and should be considered in the place of standard methods.
16.8. Examining The Myth Of The ‘‘July Phenomenon’’ in Surgical Patients: An Analysis Utilizing The National Surgical Quality Improvement Program Database. B. A. Ehlert, J. T. Nelson, C. E. Goettler, F. M. Parker, W. M. Bogey, C. S. Powell, M. C. Stoner; East Carolina University, Greenville, NC Introduction: Conventional wisdom states the start of the academic year in July is a precarious time to be hospitalized due to the introduction of new residents, described as the ‘‘July Phenomenon’’ in the literature. There is a paucity of literature regarding this observation in surgical practice. The objective of this study is to evaluate morbidity and mortality rates in surgical patients at the beginning of the academic year. Methods: The National Surgical Quality Improvement Program (NSQIP) database was utilized to gather data on the ten most common inpatient surgical procedures from 2005-2007. Patients were then divided into populations based on quarter of admission and their comorbid profile was assessed with a modified Charlson Comorbidity Index (CCI). Study end points included mortality, serious adverse events (SAE) and all morbidities. Univariate and multivariate analysis of mortality and morbidity rates was conducted examining the total population, and then subsequently stratified by operation to assess for significant differences in end points (P<0.05). Results: A total of 89,473 patients were identified. 26,287 patients underwent surgery during the first academic quarter, defined as July-September, and 63,186 were operated on throughout the remainder of the year. Demographic and pre-operative CCI were not significantly different among the patient populations. During the first academic quarter, the mortality rate was 2.02% and throughout the remainder of the year was 2.05% (P¼0.793). Overall SAE and morbidity rates were non-significant among the two populations (11.48% vs. 11.39%, P¼0.697 and 18.34% vs. 17.84%, P¼ 0.076, respectively). When stratified by operation, the only significant findings associated with first quarter operation were an increased mortality rate for ‘‘artery bypass grafts’’ (3.74% vs. 2.86%, P¼0.039) and ‘‘repair bowel opening’’ (1.13% vs. 0.56, P¼.033). Multivariate analysis performed on the ‘‘artery by-
pass graft’’ subset of patients confirmed the deleterious effect of first quarter admission (OR¼1.35, CI 1.023–1.774); however, first quarter admission was not a correlate of mortality in the ‘‘repair bowel opening’’ patient subset (OR¼1.880 CI 0.903–3.912). Conclusions: By in large, these data refute the ‘‘July Phenomenon’’. Although the ‘‘repair bowel opening’’ subset showed a significant increase in mortality during the first academic quarter, multivariate analysis revealed patient disease to have a larger impact than timing of operation. The ‘‘artery bypass graft’’ population was affected by timing of operation and this may reflect new surgical residents being unfamiliar with the management of patients with complex cardiovascular disease. Alternatively, this finding may be a reflection of disease process or patient factors not described in the NSQIP dataset. Further research is warranted to validate these findings in the vascular subset of patients.
16.9. Ranking Hospitals On Surgical Morbidity: Does The Type Of Complication Matter?. J. R. East,1 G. Rangrass,1 C. Y. Ko,2 J. B. Dimick1; 1University of Michigan, Ann Arbor, MI; 2American College of Surgeons, Chicago, IL Introduction: The American College of Surgeons National Surgical Quality Improvement Program (ACS NQSIP) reports rates of riskadjusted morbidity for each participating hospital. However, their approach does not take into account the type of complication that occurred. We sought to determine whether alternative approaches that consider the severity and/or number of complications would impact hospital rankings. Methods: We used data from the ACS NSQIP of all patients undergoing colectomy in 2007 (181 hospitals, 18,455 patients). We divided complications into minor and severe based on consensus of a panel of clinicians. We first evaluated the extent to which minor and severe complications increased a patient’s risk of mortality and prolonged length of stay. We then assessed whether using an alternative approach for estimating hospital morbidity (based on the severity and number of complications) would impact hospital rankings. We assessed agreement among these alternative approaches based on the proportion of patients that moved out of the top 20% and bottom 20% of hospitals. Results: Patients with severe complications have a higher mortality (15.7% vs. 6.9%) and prolonged length of stay (61% vs. 43%) compared to patients with minor complications. Patients with multiple complications (2 or more) had a higher mortality (19% vs. 6.1%) and prolonged length of stay (68% vs. 33%) than patients with only one complication. Compared to the standard approach for assessing morbidity, ranking hospitals on severe complications resulted in 13 (35%) hospitals moving out of the top 20% and 17 (47%) hospitals moving out of the bottom 20%. A similar degree of reclassification was found when comparing the standard morbidity rankings to an alternative approach that considered the number of different complications. Conclusions: Although the number and severity of complications impact mortality and prolonged length of stay, existing approaches to morbidity assessment do not take these factors into account. Quality measurement platforms should consider weighting complications according to severity and number.
16.10. How Widely Has ACS-NSQIP Disseminated? A Market Share Analysis In The National Medicare Population. N. H. Osborne,1,2 G. Rangrass,1,2 C. Y. Ko,3,4 J. B. Dimick1,2; 1 Center for Healthcare Outcomes & Policy (CHOP), Ann Arbor, MI; 2University of Michigan Medical School, Department of Surgery, Ann Arbor, MI; 3Center of Surgical Outcomes and Quality, Los Angeles, CA; 4American College of Surgeons National Surgical Quality Improvement Program, Chicago, IL Introduction: Although the American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP) is the most recognized outcomes registry in surgery, it is unclear how widely the program has disseminated. We sought to determine the