Utilization of the NSQIP-Pediatric Database in Development and Validation of a New Predictive Model of Pediatric Postoperative Wound Complications

SOUTHERN SURGICAL ASSOCIATION ARTICLE

Utilization of the NSQIP-Pediatric Database in Development and Validation of a New Predictive Model of Pediatric Postoperative Wound Complications Ilan I Maizlin, MD, David T Redden, Robert T Russell, MD, MPH, FACS

PhD,

Elizabeth A Beierle,

MD, FACS,

Mike K Chen,

MD, FACS,

Surgical wound classification, introduced in 1964, stratifies the risk of surgical site infection (SSI) based on a clinical estimate of the inoculum of bacteria encountered during the procedure. Recent literature has questioned the accuracy of predicting SSI risk based on wound classification. We hypothesized that a more specific model founded on specific patient and perioperative factors would more accurately predict the risk of SSI. STUDY DESIGN: Using all observations from the 2012 to 2014 pediatric National Surgical Quality Improvement Program-Pediatric (NSQIP-P) Participant Use File, patients were randomized into model creation and model validation datasets. Potential perioperative predictive factors were assessed with univariate analysis for each of 4 outcomes: wound dehiscence, superficial wound infection, deep wound infection, and organ space infection. A multiple logistic regression model with a step-wise backwards elimination was performed. A receiver operating characteristic curve with c-statistic was generated to assess the model discrimination for each outcome. RESULTS: A total of 183,233 patients were included. All perioperative NSQIP factors were evaluated for clinical pertinence. Of the original 43 perioperative predictive factors selected, 6 to 9 predictors for each outcome were significantly associated with postoperative SSI. The predictive accuracy level of our model compared favorably with the traditional wound classification in each outcome of interest. CONCLUSIONS: The proposed model from NSQIP-P demonstrated a significantly improved predictive ability for postoperative SSIs than the current wound classification system. This model will allow providers to more effectively counsel families and patients of these risks, and more accurately reflect true risks for individual surgical patients to hospitals and payers. (J Am Coll Surg 2017; -:1e13.
2017 by the American College of Surgeons. Published by Elsevier Inc. All rights reserved.)

BACKGROUND:

hospital costs associated with an SSI can be up to $25,000 per event.1-3 Significant time and effort, often by large collaborative groups, have been focused on bundles of care surrounding perioperative care to reduce the incidence of SSIs. Currently, the most widely used Association of periOperative Registered Nurses (AORN) wound classification system stratifies surgical wounds into 4 classes: Class I, clean; Class II, clean-contaminated; Class III, contaminated; and Class IV, dirty. This classification system is based on the presumed bacterial load of the surgical wound.4,5 In 1985, the CDC guidelines provided updated, estimated postoperative rates of SSIs to 1% to 5% for clean, 3% to 11% for clean-contaminated, 10% to 17%

Surgical site infection (SSI) has become an important metric as an indicator of quality of care. It can result in significant morbidity and incur much higher costs. The Disclosure Information: Nothing to disclose. Presented at the Southern Surgical Association 128th Annual Meeting, Palm Beach, FL, December 2016. Received December 19, 2016; Accepted December 19, 2016. From the Division of Pediatric Surgery, Children’s Hospital of Alabama (Maizlin, Beierle, Chen, Russell) and the Department of Biostatistics, School of Public Health (Redden), University of Alabama at Birmingham, Birmingham, AL. Correspondence address: Robert T Russell, MD, MPH, FACS, Division of Pediatric Surgery, Children’s Hospital of Alabama, University of Alabama at Birmingham, 1600 7th Ave South, Lowder Building, Suite 300, Birmingham, AL 35233. email: [email protected]

ª 2017 by the American College of Surgeons. Published by Elsevier Inc. All rights reserved.

1

http://dx.doi.org/10.1016/j.jamcollsurg.2016.12.022 ISSN 1072-7515/16

2

Maizlin et al

Predictive Model of Pediatric Wound Infection

Abbreviations and Acronyms

AUROC ¼ area under the receiver operating characteristic NSQIP-P ¼ National Surgical Quality Improvement Program-Pediatric SSI ¼ surgical site infection

for contaminated, and more than 27% for dirty.3,6 The objectives of this wound classification schema are to allow for better prediction of wounds that have an increased risk for infection, and increase awareness of this possibility of infection for those providing postoperative care. However, a recent collaborative study of pediatric institutions demonstrated inconsistency and inaccuracy in how wounds were being classified.7 Furthermore, Gonzalez and colleagues8 and Oyetunji and associates9 recently demonstrated with National Surgical Quality Improvement Program-Pediatric (NSQIP-P) data that current wound classification systems do not reflect the true risk of SSIs and are inadequate measures for benchmarking surgical care in children. As the focus in health care shifts to improving the quality of care and value-based care, appropriate risk prediction models that properly benchmark quality will be essential for both providers and payers. The aim of this study was to evaluate the ability of current wound classification systems to appropriately predict postoperative SSIs in children. Furthermore, this study developed multivariate models, based on NSQIP-P data, incorporating important perioperative variables to better predict SSI occurrence postoperatively in pediatric surgical patients.

METHODS Data source and patients A retrospective analysis was performed using NSQIP-P data from the 2012 to 2014 Participant Use File. The NSQIP-P collects patient-level clinical data including demographics, comorbidities, laboratory values, and outcomes, and identifies cases by CPT codes. These data are reliable because they are rigorously defined, and collected and recorded by trained surgical clinical reviewers, who undergo training and examination in variable definitions. In addition, the program performs random audits to check for data validity and definition compliance. Cases are systematically sampled across all specialties at each participating hospital, following an 8-day cycle.10 All cases are followed for 30 days using the medical record or patient outreach to verify the presence or absence of adverse events at the 30-day postoperative time point.

J Am Coll Surg

Predictors and outcomes Factors considered for calculating patient-specific risk of surgical outcomes were demographic, preoperative, and perioperative clinical variables. The final variables were chosen based on previous literature,6,10-12 predictive value, and clinical face validity. In addition to these patientspecific risk factors, a procedure-specific risk variable was included based on subgrouping of CPT codes. Consequently, out of a total of 218 preoperative and intraoperative variables, 43 variables were selected (Table 1). Four outcomes were then individually modeled: superficial, deep, organ space SSI, and surgical wound dehiscence. The exact NSQIP-P definitions of each outcome of interest are provided in Table 2, as based on the American College of Surgeons NSQIP User Guide for the 2014 Participant Use File. Development and validation cohorts All records were randomized into development (50% of records) and validation (50% of records) dataset groups. The cohorts were compared for the variables in question, to determine that they contained no statistical difference in predictor or outcome variables. The predictive (logistic model) equations were estimated from the development dataset and applied to the validation dataset. Statistical analysis Each outcome model was evaluated for applicability of all clinical variables listed in Table 1. Univariate analysis of all predictive variables was performed to compare risk factors among patients who did or did not experience each specific outcome. Variables found to be significantly associated with the outcome on univariate analysis subsequently underwent a multivariate logistic regression analysis with a stepwise backward elimination. Any multivariate model bears concerns of collinearity. In our model, we evaluated adjusted R2 values of weighed information matrixes of the predictive variables. Specifically, we performed weighted regression of X’WX information matrixes, in which X indicates raw predictors and W is a diagonal matrix of weights that is determined by the fitting algorithm of each iteration. Consequently, the R2 value demonstrated degree of collinearity (with 0 equaling no collinearity and 1 equaling complete collinearity). This value was also used to determine a degree of tolerance (1
R2). Statistical Analysis System (SAS version 9.4) was used to perform the analyses. Chi-square test was used to analyze categorical variables. A Kruskal-Wallis test was used to evaluate continuous variables. A value of p < 0.05 was considered statistically significant.

Vol.

-,

No.

-, -

2017

Maizlin et al

Table 1. All Preoperative and Perioperative NSQIP Variables Deemed to Have Potential Clinical Predictive Value for the Outcomes of Interest Variable name

Principal operative procedure (CPT code subgroup) Diabetes mellitus Premature birth Ventilator dependence Current pneumonia Oxygen support Tracheostomy Esophageal/gastric/intestinal disease Biliary/liver/pancreatic disease Cardiac risk factors Acute renal failure Currently on dialysis CVA/stroke or traumatic/acquired brain injury with resulting neurologic deficit Developmental delay/impaired cognitive status Cerebral palsy Immune disease/immunosuppressant use Steroid use (within 30 d) Bone marrow transplant Solid organ transplant Open wound Weight loss or failure to thrive Nutritional support Bleeding disorders Hematologic disorder Chemotherapy for malignancy (within 30 d) Radiotherapy for malignancy (within 90 d) SIRS/sepsis/septic shock within 48 h before surgery Inotropic support at time of surgery Previous CPR (within 7 d) Previous operation (within 30 d) Congenital malformation Blood transfusions (within 48 h) Childhood malignancy Case status (elective/emergent) ASA classification Preoperative serum albumin Preoperative WBC Duration from anesthesia start to surgery start Duration from surgery to anesthesia stop Duration patient is in operating room Duration of anesthesia Total operation time

Variable type

Categorical Categorical Categorical Categorical Categorical Categorical Categorical Categorical Categorical Categorical Categorical Categorical Categorical Categorical Categorical Categorical Categorical Categorical Categorical Categorical Categorical Categorical Categorical Categorical Categorical Categorical Categorical Categorical Categorical Categorical Categorical Categorical Categorical Categorical Categorical Continuous Continuous Continuous Continuous Continuous Continuous Continuous

ASA, American Society of Anesthesiologists; CVA, cerebrovascular accident; SIRS, systemic inflammatory response syndrome.

Predictive Model of Pediatric Wound Infection

3

Model performance and comparison to traditional wound classification The models’ discriminative ability was measured using the receiver operating characteristic curve with c-statistic, also referred to as the area under the receiver operating characteristic (AUROC) curve (the plot of sensitivity vs 1-specificity). The c-statistic indicates how well the model does in predicting membership in 1 of 2 groups (the binary response). It takes values from 0.5 to 1.0, with 0.5 indicating the model is no better than chance and 1.0 indicating perfect prediction. In general, models with a c-statistic greater than 0.7 are considered predictive, and models are considered strongly predictive when the c-statistic exceeds 0.8.13 Each model’s discriminative ability (ie c-statistic) was then compared with the discriminative ability of the traditional wound classification scheme, based on the classification variable in the NSQIP-P Participant Use File.

RESULTS During the 3 years investigated, there were 183,233 patients entered in the NSQIP-P database. All patients were under the age of 18 years of age. The rates of occurrence of all predictive variables in the overall dataset are summarized in Table 3. The entire dataset was equally randomized into a development dataset (91,616 patients) used to develop the predictive model, and a validation dataset (91,617 patients) used to validate the predictive power of the model. The 2 subsets were further compared in all 43 predictor variables to assure appropriate randomization in all variables. The results of the randomization are also reported in Table 3, demonstrating no significance difference in occurrence of any of the predictors between the subsets and in relation to the overall patient population. Table 4 displays the distribution of the outcomes of interest (superficial SSI, deep SSI, organ space SSI, and wound dehiscence) in the overall dataset and in each of the subsets. Once again, the randomization process provided equal distribution of the outcomes between the development and validation subsets. Overall wound complications occurred in 2.4% of NSQIP-P patients, with superficial SSIs (1.0%) and organ-space SSIs (0.8%) the most common complications. The rates of outcomes of interest were then determined based on the traditional wound classifications (Table 5). Similar to results shown in previous publications,14 the rate of overall wound complications increased with increase in wound class. The rates noted in each wound class in our patient cohort are lower than those in the

4

Maizlin et al

Table 2.

NSQIP Definitions of the Outcomes of Interest

Outcome

Superficial SSI

Deep SSI

Organ space SSI

Wound dehiscence

Predictive Model of Pediatric Wound Infection

Definition

Infection that involves only skin or subcutaneous tissue at the site of incision, with either: superficial purulent drainage, positive culture from fluid or superficial tissue, or localized symptoms of infection (pain, swelling, erythema, or heat). Infection that involves deep soft tissues (eg fascial or muscle layers) at the site of incision, with either: purulent drainage from deep incision, an abscess, or evidence of infection at deep tissue layer but not at the organ space component of the surgical site. Infection that involves any organ, cavity, or anatomic potential space that was opened or manipulated during an operation, with either: purulent drainage from a surgical drain, positive culture from organ space fluid or organ tissue, or an abscess in the organ space. Spontaneous separation of the layers of a surgical wound, which may occur at superficial or deep layers.

Definitions from American College of Surgeons NSQIP User Guide for the 2014 Participant Use File. SSI, surgical site infection.

original wound classification studies.4,6 Nonetheless, the overall trend is confirmed and maintained in all specific outcomes variables, providing external validation to the dataset as an appropriate distribution of wound site complication outcomes, in concordance with previously established outcome patterns. The 1 exception is wound dehiscence, which appeared to be lower in the contaminated group. After a multivariate analysis of the development dataset for the 43 potential predictive variables with stepwise regressive elimination, each outcome of interest was deemed to have a unique model of 6 to 9 predictive variables (Table 6). In the evaluation of collinearity, none of the consequent R2 variables were significantly higher than 0, so no variables demonstrated any significant collinearity. Similarly, tolerance for all variables was noted to be above 0.75. Performing AUROC, to determine each model’s discriminatory ability, we established c-statistics for each outcome variable, ranging from 0.701 for superficial SSI to 0.839 for organ space SSI. Similar AUROC is performed for traditional wound classification, demonstrating significantly lower c-statistics of the traditional wound classification model for each outcome of interest (Fig. 1A to D).

J Am Coll Surg

Lastly, the percent b-coefficient estimate of each variable in each model was used to establish a generalized linear model predictability equation ðY½% ¼ b0 þ b1 X 1 þ b2 X 2 . þ bn Xn Þ. A list of all predictive coefficients of each variable in each model is summarized in Table 7. The higher the coefficient, the more likely is the event of interest to occur. The coefficients for the same variable differ between the different outcome models, because the same variable may contribute differently to the probability of each outcome occurrence. The intercept coefficient (b0) is the likelihood of the outcome (Y) occurrence without any of the factors being present (ie all X ¼ 0). Therefore, the lower or more negative the intercept, the less likely is the overall likelihood of the outcome. Using these equations in a clinical example, a 10-yearold child with ulcerative colitis, on steroids and parenteral nutrition, undergoing emergent 3-hour total abdominal colectomy for sepsis due to colon perforation, would have a 6.47% likelihood of a superficial infection (y ¼
0.451 þ 2.108 þ 1.299 þ 1.632 þ 1.483 þ 0.402), a 10.27% likelihood of a deep tissue infection (y ¼
0.545 þ 2.120 þ 3.671 þ 1.342 þ 3.090 þ 0.588), a 14.17% likelihood of an intra-abdominal infection (y ¼
0.841 þ 2.909 þ 1.020 þ 2.580 þ 1.813 þ 2.533 þ 1.330 þ 2.327 þ 0.495), and a 2.53% likelihood of wound dehiscence (y ¼
0.589 þ 1.541
1.400 þ 1.263 þ 1.171 þ 0.546). With the current wound classification schema, providers could only estimate his risk of any wound infection to be greater than 27%. However, the same 10-yearold child with familial adenomatous polyposis undergoing 3-hour elective total abdominal colectomy, without those risks factors, would have a 4.17% likelihood of a superficial infection, a 4.33% likelihood of a deep tissue, a 5.72% likelihood of an intra-abdominal infection, and a 0.94% likelihood of wound dehiscence. Using current wound classification estimates, the estimated risk would be between 3% and 11%. The consequent predictive models, based on the coefficients summarized in Table 7, were confirmed in the validation dataset by demonstration of high rates of predicted versus actual occurrence rates for each outcome of interest.

DISCUSSION The traditional surgical wound classification, based on extent of wound contamination, was first introduced by Stone and colleagues14 in 1976. Since then, several epidemiologic studies in adults have demonstrated its direct correlation with the rate of postoperative surgical infections.3 As a result, it became the standard of care for estimating the likelihood of postoperative infections in the adult

Vol.

-,

No.

Table 3.

-, -

2017

Maizlin et al

Predictive Model of Pediatric Wound Infection

5

Distribution of Predictor Variables in Total Dataset and Each Data Subset

Variable

Total, n (%) Principal operative procedure Breast Cardiothoracic Esophageal Hepatobiliary Spleen Stomach Small bowel and colon Rectal and anal Lymphatic Skin and soft tissue Oral cavity Pharynx and thyroid Musculoskeletal Other Age of patient, y, mean (SD) Diabetes mellitus, n (%) No Noninsulin agents Insulin Premature birth, n (%) Ventilator dependence, n (%) Current pneumonia, n (%) Oxygen support, n (%) Tracheostomy, n (%) Esophageal/gastric/intestinal disease, n (%) Biliary/liver/pancreatic disease, n (%) Cardiac risk factors, n (%) None Minor Major Severe Acute renal failure, n (%) Currently on dialysis, n (%) CVA/stroke or traumatic/acquired brain injury with resulting neurological deficit, n (%) Developmental delay/impaired cognitive status, n (%) Cerebral palsy, n (%) Immune disease/immunosuppressant use, n (%) Steroid use, within 30 d, n (%) Bone marrow transplant, n (%) Solid organ transplant, n (%) Open wound, n (%) Weight loss or failure to thrive, n (%) Nutritional support, n (%) Bleeding disorders, n (%)

Total

183,233 (100.0)

Development dataset

Validation dataset

p Value

91,616 (50.0)

91,617 (50.0)

(0.3) (1.7) (1.8) (17.5) (0.2) (6.3) (45.3) (1.0) (0.2) (8.2) (4.8) (5.2) (6.6) (0.8) (5.61)

255 1,632 1,659 15,984 216 5,862 41,462 941 221 7,469 4,432 4,780 5,988 715 6.67

(0.3) (1.8) (1.9) (17.4) (0.2) (6.4) (45.3) (1.0) (0.2) (8.2) (4.8) (5.2) (6.5) (0.8) (5.61)

227 1,518 1,695 16,066 222 5,759 41,496 909 233 7,578 4,449 4,729 6,028 708 6.64

(0.2) (1.7) (1.9) (17.5) (0.2) (6.3) (45.3) (1.0) (0.3) (8.3) (4.9) (5.2) (6.6) (0.8) (5.62)

182,510 202 521 43,258 5,180 683 6,207 1,985 33,119 4,011

(99.6) (0.1) (0.3) (23.6) (2.8) (0.4) (3.4) (1.1) (18.1) (2.2)

91,249 98 269 11,872 2,605 364 3,106 988 16,586 1,984

(99.6) (0.1) (0.3) (23.9) (2.8) (0.4) (3.4) (1.1) (18.1) (2.2)

91,261 104 252 21,386 2,575 319 3,101 997 16,533 2,027

(99.6) (0.1) (0.3) (23.3) (2.8) (0.3) (3.4) (1.1) (18.0%) (2.2%)

166,471 9,518 5,918 1,326 347 582

(90.9) (5.2) (3.2) (0.7) (0.2) (0.3)

83,284 4,722 2,917 693 167 299

(90.9) (5.2) (3.2) (0.8) (0.2) (0.3)

83,187 4,796 3,001 633 180 283

(90.8) (5.2) (3.3) (0.7) (0.2) (0.3)

4,191 23,969 6,684 1,911 4,673 348 449 3,013 5,856 14,910 1,201

(2.3) (13.1) (3.6) (1.0) (2.6) (0.2%) (0.2%) (1.6) (3.2) (8.1) (0.7)

2,092 12,095 3,348 988 2,342 183 236 1,547 2,941 7,461 649

(2.3) (13.2) (3.7) (1.1) (2.6) (0.2%) (0.3%) (1.7) (3.2) (8.1) (0.7)

2,099 11,874 3,336 923 2,331 165 213 1,466 2,915 7,449 552

(2.3) (13.0) (3.6) (1.0) (2.5) (0.2) (0.2) (1.6) (3.2) (8.1) (0.6)

0.433 482 3,150 3,354 32,050 432 11,621 82,958 1,850 454 15,047 8,881 9,509 12,016 1,423 6.65

0.801 0.788

0.662 0.845 0.215 0.724 0.494 0.127 0.236 0.118

0.486 0.239 0.685 0.125 0.364 0.228 0.337 0.252 0.197 0.257 0.638 0.457 0.457 (Continued)

6

Maizlin et al

Table 3.

Continued

J Am Coll Surg

Predictive Model of Pediatric Wound Infection

Variable

Development dataset

Total

Hematologic disorder, n (%) Chemotherapy for malignancy within 30 d, n (%) Radiotherapy for malignancy in last 90 d, n (%) SIRS/sepsis/septic shock, n (%) None SIRS Sepsis Septic shock Inotropic support at time of surgery, n (%) Previous CPR within 7 d before surgery, n (%) Previous operation within 30 d, n (%) Congenital malformation, n (%) Blood transfusions within 48 h before surgery, n (%) Childhood malignancy, n (%) No current or past Current Past Preoperative serum albumin, mean (SD) Preoperative WBC, mean (SD) Case status, n (%) Elective Urgent Emergent ASA classification, n (%) I II III IV V Duration from anesthesia start to surgery start, min, mean (SD) Duration from surgery to anesthesia stop, min, mean (SD) Duration patient is in operating room, h, mean (SD) Duration of anesthesia, h, mean (SD) Total operation time, h, mean (SD)

5,745 (3.1) 1,019 (0.6) 155 (0.1)

2,854 (3.1) 521 (0.6) 75 (0.1)

Validation dataset

2,891 (3.2) 498 (0.5) 80 (0.1)

171,105 5,911 5,891 326 1,386 265 70,666 53,217 1,956

(93.4) (3.2) (3.2) (0.2) (0.8) (0.1) (38.6) (29.0) (1.1)

85,545 2,981 2,925 165 676 119 35,367 26,689 978

(93.4) (3.3) (3.2) (0.2) (0.7) (0.1) (38.6) (29.1) (1.1)

85,560 2,930 2,966 161 710 146 35,299 26,528 978

(93.4) (3.2) (3.2) (0.2) (0.8) (0.2) (38.5) (29.0) (1.1)

177,969 4,198 1,066 3.44 10.90

(97.1) (2.3) (0.6) (0.81) (5.60)

88,932 2,123 561 3.44 10.90

(97.1) (2.3) (0.6) (0.81) (5.64)

89,037 2,075 505 3.44 10.90

(97.2) (2.3) (0.6) (0.82) (5.65)

134,082 (73.2) 18,371 (10.0) 30,780 (16.8)

67,144 (73.3) 9,185 (10.0) 15,287 (16.7)

66,938 (73.1) 9,186 (10.0) 15,493 (16.9)

30,971 38,299 19,388 2,560 398 35.7 18.6 2.4 2.5 1.6

31,276 38,036 19,434 2,497 374 35.6 18.6 2.4 2.5 1.6

p Value

0.229 0.412 0.529 0.236

0.341 0.109 0.231 0.355 0.786 0.415

0.854 0.782 0.214

0.426 62,247 76,335 38,822 5,057 772 36.0 19.2 2.4 2.5 1.6

(34.0) (41.7) (21.2) (2.8) (0.4) (32.8) (16.8) (1.9) (1.9) (1.5)

(33.8) (41.8) (21.2) (2.8) (0.4) (34.6) (16.3) (1.9) (1.9) (1.5)

(34.1) (41.5) (21.2) (2.7) (0.4) (32.6) (16.5) (1.9) (1.9) (1.5)

0.126 0.452 0.897 0.749 0.965

ASA, American Society of Anesthesiologists; CVA, cerebrovascular accident; SIRS, systemic inflammatory response syndrome.

Table 4.

Distribution of Outcomes Variables in Total Dataset and Each Data Subset Development dataset, n (%)

Total, n (%) Variable

Superficial SSI Deep SSI Organ-space SSI Wound dehiscence SSI, surgical site infection.

Validation dataset, n (%)

n

%

n

%

n

%

p Value

1,886 407 1,384 770

1.0 0.2 0.8 0.4

945 200 699 377

1.0 0.2 0.8 0.4

941 207 685 393

1.0 0.2 0.7 0.4

0.763 0.454 0.128 0.289

Vol.

-,

No.

Table 5.

-, -

Maizlin et al

2017

Predictive Model of Pediatric Wound Infection

7

Comparison of Outcome Variables Based on Traditional Wound Classification Clean (n ¼ 90,330) n %

Outcomes

Superficial SSI Deep SSI Organ-space SSI Wound dehiscence Total

747 205 319 348 1,619

0.8 0.2 0.4 0.4 1.8

Clean-contaminated (n ¼ 68,361) n %

801 117 360 331 1,609

1.2 0.2 0.5 0.4 2.4

Contaminated (n ¼ 14,439) n %

Dirty/infected (n ¼ 10,103) n %

Total (N ¼ 183,233) n %

183 34 144 44 405

15 51 561 47 814

1,886 407 1,384 770 4,447

1.3 0.2 1.0 0.3 2.8

1.5 0.5 5.6 0.5 8.1

1.0 0.2 0.8 0.4 2.4

SSI, surgical site infection.

Table 6.

Comparison of Surgical Site Infection Predictability of the New Model to Traditional Wound Classification

Type of wound complication

Wound dehiscence

Superficial

Deep

Organ space

Predictive factors

Operative CPT code Increasing age at time of surgery Preoperative presence of open wound Hematologic disorder SIRS/sepsis/septic shock within 48 h of operation ASA class Duration of procedure Operative CPT code Premature birth Esophageal/gastric/intestinal disease Preoperative presence of open wound Nutritional support Childhood malignancy ASA class Duration of procedure Operative CPT code Developmental delay Immune disease/immunosuppressant use SIRS/sepsis/septic shock within 48 h of operation ASA class Duration of procedure Operative CPT code Increasing age at time of surgery Currently on dialysis Steroid use (within 30 d) Nutritional support SIRS/sepsis/septic shock within 48 h of operation Case status ASA class Duration of procedure

New model ROC c-statistic

Traditional wound classification ROC c-statistic

0.754

0.549

0.701

0.570

0.755

0.569

0.839

0.727

ASA, American Society of Anesthesiologists; ROC, receiver operating characteristic; SIRS, systemic inflammatory response syndrome.

8

Maizlin et al

Predictive Model of Pediatric Wound Infection

J Am Coll Surg

Figure 1. Receiver operating characteristic (ROC) plots with discriminatory measures for each outcome of interest, based on traditional wound classification vs the new predictive model. (A) Superficial infection, area under the curve (AUC): wound class ¼ 0.5701; new model ¼ 0.7012. (B) Deep tissue infection, AUC: wound class ¼ 0.5687; new model ¼ 0.7546. (C) Organ space infection, AUC: wound class ¼ 0.7273; new model ¼ 0.8385. (D) Wound dehiscence, AUC: wound class ¼ 0.5493; new model ¼ 0.7540. Deep blue solid line indicates the new predictive model ROC curve; light dashed line indicates the traditional wound classification ROC curve.

population.15 However, the recent emphasis on quality of care stressed the importance of perioperative risk estimation16 and challenged this classification in both adult1 and pediatric populations.9 Moreover, the current system provides only predictive rates for overall postoperative complications, without differentiating them into type or severity. A mild superficial infection requiring a short course of oral antibiotics, for example, is an equivalent outcome measure in the current system to an extensive multifocal intra-abdominal abscess requiring surgical exploration. Consequently, a new predictive model is

required for estimation of postoperative infections, with higher predictive power and the ability to differentiate various types of wound infections and complications. It is important to emphasize the necessity of developing a specialized pediatric predictive model. The original wound classification was developed in the adult population3,4,14 and then extrapolated to the pediatric population, with no pediatric-specific validation.9 As a result, traditional wound classification has been deemed inaccurate, specifically in the pediatric population,8,9 and especially for procedures in neonates and young infants.17

Vol.

-,

No.

Table 7.

-, -

2017

Predictive Model of Pediatric Wound Infection

9

Predictive Regression Models of Each Outcome, with Intercepts and Variable-Specific Coefficients

Type of wound complication

Wound dehiscence

Superficial

Maizlin et al

Predictive factors

Predictive % coefficient

Intercept Operative CPT code Skin and soft tissue Breast Cardiothoracic Esophageal Hepatobiliary Spleen Stomach Small bowel and colon Rectal and anal Lymphatic Oral cavity Pharynx and thyroid Musculoskeletal Other Increasing age at time of surgery (in y) Preoperative presence of open wound Hematologic disorder SIRS/sepsis/septic shock within 48 h of operation None SIRS Sepsis Septic shock ASA class I II III IV V Duration of procedure (in h) Intercept Operative CPT code Skin and soft tissue Breast Cardiothoracic Esophageal Hepatobiliary Spleen Stomach Small bowel and colon Rectal and anal Lymphatic Oral cavity Pharynx and thyroid Musculoskeletal Other


0.589 Reference* 0.171 0.085 0.672 0.459 0.010 0.314 1.541 3.154 0.181 2.344 0.383 0.945 1.090
0.140 (per y) 3.360 1.743 Reference 0.693 1.263 3.189 Reference 0.381 0.844 1.171 1.841 0.182 (per h)
0.451 Reference 0.909 0.768 0.982 1.439 0.619 1.671 2.108 3.730 0.523 0.229 0.748 1.569 0.895 (Continued)

Maizlin et al

10

Table 7.

Predictive Model of Pediatric Wound Infection

J Am Coll Surg

Continued

Type of wound complication

Deep

Organ space

Predictive factors

Premature birth Esophageal/gastric/intestinal disease Preoperative presence of open wound Nutritional support Childhood malignancy No current or past Current Past ASA class I II III IV V Duration of procedure (in h) Intercept Operative CPT code Skin and soft tissue Breast Cardiothoracic Esophageal Hepatobiliary Spleen Stomach Small bowel and colon Rectal and anal Lymphatic Oral cavity Pharynx and thyroid Musculoskeletal Other Developmental delay Immune disease/immunosuppressant use SIRS/sepsis/septic shock within 48 h of operation None SIRS Sepsis Septic shock ASA class I II III IV V Duration of procedure (in h) Intercept Operative CPT code Skin and soft tissue

Predictive % coefficient

1.198 1.299 1.891 1.632 Reference 1.536 0.538 Reference 0.455 0.816 1.483
0.534 0.134 (per h)
0.545 Reference 0.301 0.504 0.700 1.127 0.101 1.365 2.120 4.960 0.190 0.924 1.960 0.928 1.229 1.713 3.671 Reference 0.294 1.342 3.610 Reference 1.443 2.169 3.090
0.656 0.196 (per h)
0.841 Reference (Continued)

Vol.

-,

No.

Table 7.

-, -

2017

Maizlin et al

Predictive Model of Pediatric Wound Infection

11

Continued

Type of wound complication

Predictive factors

Predictive % coefficient

Breast Cardiothoracic Esophageal Hepatobiliary Spleen Stomach Small bowel and colon Rectal and anal Lymphatic Oral cavity Pharynx and thyroid Musculoskeletal Other Increasing age at time of surgery (in y) Currently on dialysis Steroid use (within 30 d) Nutritional support SIRS/sepsis/septic shock within 48 h of operation None SIRS Sepsis Septic shock Case status Elective Urgent Emergent ASA class I II III IV V Duration of procedure (in h)

0.010 0.298 1.088 1.546 0.913 1.147 2.909 4.087 0.491 0.655 0.445 0.635 1.817 0.102 (per y) 2.178 2.580 1.813 Reference 1.036 2.533 8.768 Reference 0.540 1.330 Reference 1.413 2.136 2.327 1.648 0.165 (per h)

*Reference ¼ 0. ASA, American Society of Anesthesiologists; SIRS, systemic inflammatory response syndrome.

For example, children are less likely than adults to have acquired cardiovascular pathologies, but they may have congenital pathologies that could influence their surgical outcomes. These issues prompted us to design and validate our model exclusively in a pediatric cohort, with a pediatric-specific predictive algorithm. Before developing our model, we examined the predictability of the current wound classification system by applying it to our patient cohort. The resulting total rates of wound complications were significantly lower than the previously mentioned predicted rates. Our findings are equivalent to those rates published by Ortega and colleagues1 with NSQIP-P data. In establishing external

validation, those lower rates were also consistent with the large single-institution review by Gonzalez and associates,8 although they did report higher rates of organ space infections in their perforated appendicitis group (15%). Although improvement in surgical procedures, such as increased use of laparoscopy and improvements in sterile techniques over the past several decades, may have contributed to the lower adult SSI rates,18 the lower infection rates seen may be due to the specialization of the NSQIP patient cohort. The NSQIP dataset does not include trauma-related cases and therefore contains slightly lower rates of emergent procedures, potentially skewing the data toward individuals medically optimized

12

Maizlin et al

Predictive Model of Pediatric Wound Infection

for their procedure and possibly at a lower risk of postoperative complications.19 Nonetheless, several evaluations of the NSQIP dataset have deemed it an appropriate representation of the natural surgical scope of practice.1,9 It is also possible that wound class misclassification may play a role in the SSI rate discrepancy. A recent multicenter review found only 50% concordance between actual and documented classifications.20 Gonzalez and colleagues8 similarly demonstrated rates of wound classification discordance to vary between 2% and 62%. Therefore, the current wound class system appears to be not only inaccurate, but also unreliable and of poor reproducibility. As previously mentioned, the NSQIP dataset may present a higher concordance rate because it is set with an internal validation system, by which all submissions are reviewed and adjusted by a surgical clinical reviewer.10 The predictive model presented provides several benefits over the traditional wound classification. First, it stratifies the outcomes based on extent and nature of the wound complication. As such, it provides and prepares the clinician not only for the probability of a complication, but also for its type and therefore, the potential resources and type of intervention required. The model also incorporates the traditional component of procedure type (CPT bins variable) in an effort to account for a variety of preoperative and intraoperative variables that might affect surgical outcome. The resulting model is straightforward to use in the clinical setting because it requires only a short list of easily obtainable predictors. The new predictive model’s risk discrimination compares favorably with wound classification in every outcome of interest. Although the traditional wound classification’s c-statistic bordered on 0.5 (the statistical equivalent of flipping a coin), our model’s c-statistic ranged between 0.7 and 0.9, considered to provide reasonable to strong level of discrimination. The model is especially useful in discrimination of deep and organspace SSIs. Considering that these outcomes result in the most significant use of resources and quality-of-life detriments, we believe the current model’s predictive power to be especially important. Three variablesdCPT code, American Society of Anesthesiologists class, and duration of surgerydare universally present across all models. It is understandable that CPT code and surgery duration would have an effect on SSI because they act as a surrogate for complexity of both the pathology and the consequent operative intervention. The American Society of Anesthesiologists class has been suggested, in several earlier SSI models, to be at least as good as the traditional wound classification because it stands as a measure of intrinsic host susceptibility.1,6

J Am Coll Surg

Several limitations of this study are those inherent to both its retrospective nature and the limitations of the database used. The database collects only data for 30day outcomes, which will omit those complications occurring beyond 1 month after surgery. Similarly, NSQIP-P does not record specific medications, so differences in preoperative antibiotic administration could affect the data. In addition, Pediatric NSQIP participants are mostly tertiary centers, making generalization of these findings more difficult. Finally, the surgical decision-making process is complex, and we cannot capture all of the measured and unmeasured variables affecting that process. However, considering the extensive range of potential predictive variables and the volume of the available cases and patients, we believe that NSQIP-P is the best available dataset to use to both design and verify our model. Furthermore, considering the yearly iterations of the dataset, we will be presented with the opportunity to further verify our model against those upcoming cohorts.

CONCLUSIONS The proposed model derived from the NSQIP-P dataset demonstrated a significantly improved predictive ability for postoperative SSIs than the current wound classification system. This model tailors the risks for each specific patient and allows calculation of specific risks for each type of specified SSI within the current NSQIP-P definitions. This model will allow providers to more effectively counsel families and patients and more accurately reflect true risks for individual surgical patients to hospitals and payers. Author Contributions Study conception and design: Maizlin, Redden, Beierle, Chen, Russell Acquisition of data: Maizlin, Redden, Russell Analysis and interpretation of data: Maizlin, Redden, Beierle, Chen, Russell Drafting of manuscript: Maizlin, Redden, Beierle, Chen, Russell Critical revision: Maizlin, Redden, Beierle, Chen, Russell REFERENCES 1. Ortega G, Rhee DS, Papandria DJ, et al. An evaluation of surgical site infections by wound classification system using the ACS-NSQIP. J Surg Res 2012;174:33e38. 2. Levy SM, Holzmann-Pazgal G, Lally KP, et al. Quality check of a quality measure: surgical wound classification discrepancies impact risk-stratified surgical site infection rates in pediatric appendicitis. J Am Coll Surg 2013;217:969e973. 3. Cruse PJ, Foord R. The epidemiology of wound infection. A 10-year prospective study of 62,939 wounds. Surg Clin North Am 1980;60:27e40.

Vol.

-,

No.

-, -

2017

Maizlin et al

4. Hart D, Postlethwait RW, Brown IW Jr, et al. Postoperative wound infections: a further report on ultraviolet irradiation with comments on the recent (1964) National Research Council cooperative study report. Ann Surg 1968;167:728e743. 5. Berard F, Gandon J. Postoperative wound infections: the influence of ultraviolet irradiation of the operating room and of various other factors. Ann Surg 1964;160:1e192. 6. Culver DH, Horan TC, Gaynes RP, et al. Surgical wound infection rates by wound class, operative procedure, and patient risk index. National Nosocomial Infections Surveillance System. Am J Med 1991;91:152Se157S. 7. Putnam LR, Levy SM, Blakely ML, et al. A multicenter, pediatric quality improvement initiative improves surgical wound class assignment, but is it enough? J Pediatr Surg 2016;51: 639e644. 8. Gonzalez KW, Dalton BG, Kurtz B, et al. Operative wound classification: an inaccurate measure of pediatric surgical morbidity. J Pediatr Surg 2016;51:1900e1903. 9. Oyetunji TA, Gonzalez DO, Gonzalez KW, et al. Wound classification in pediatric surgical procedures: Measured and found wanting. J Pediatr Surg 2016;51:1014e1016. 10. Saito JM, Chen LE, Hall BL, et al. Risk-adjusted hospital outcomes for children’s surgery. Pediatrics 2013;132:e677ee688. 11. Kraemer K, Cohen ME, Liu Y, et al. Development and evaluation of the American College of Surgeons NSQIP Pediatric Surgical Risk Calculator. J Am Coll Surg 2016;223:685e693.

Predictive Model of Pediatric Wound Infection

13

12. Dillon P, Hammermeister K, Morrato E, et al. Developing a NSQIP module to measure outcomes in children’s surgical care: opportunity and challenge. Semin Pediatr Surg 2008; 17:131e140. 13. Zweig MH, Campbell G. Receiver-operating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine. Clin Chem 1993;39:561e577. 14. Stone AM, Tucci VJ, Isenberg HD, et al. Wound infection: a prospective study of 7519 operations. Am Surg 1976;42: 849e852. 15. Devaney L, Rowell KS. Improving surgical wound classificationewhy it matters. AORN J 2004;80:208e209. 212e223. 16. Mitchell TO, Holihan JL, Askenasy EP, et al. Do risk calculators accurately predict surgical site occurrences? J Surg Res 2016;203:56e63. 17. Vu LT, Nobuhara KK, Lee H, et al. Conflicts in wound classification of neonatal operations. J Pediatr Surg 2009;44: 1206e1211. 18. Barnes S. What’s new in SSI prevention? AORN J 2015;101: P10eP12. 19. Blair LJ, Huntington CR, Cox TC, et al. Risk factors for postoperative sepsis in laparoscopic gastric bypass. Surg Endosc 2016;30:1287e1293. 20. Levy SM, Lally KP, Blakely ML, et al. Surgical wound misclassification: a multicenter evaluation. J Am Coll Surg 2015;220: 323e329.