Use of out-of-hospital variables to predict severity of injury in pediatric patients involved in motor vehicle crashes

PEDIATRICS/INJURY PREVENTION/ORIGINAL CONTRIBUTION

Use of Out-of-Hospital Variables to Predict Severity of Injury in Pediatric Patients Involved in Motor Vehicle Crashes

From the Department of Emergency Medicine, Harbor–UCLA Medical Center, Torrance, CA*; UCLA School of Medicine, University of California– Los Angeles, Los Angeles, CA‡; and the Department of Emergency Medicine, George Washington University Medical Center, Washington, DC.§

Craig D. Newgard, MD, MPH* Roger J. Lewis, MD, PhD*‡ B. Tilman Jolly, MD§

Author contributions are provided at the end of this article. Received for publication January 5, 2001. Revisions received November 7, 2001, and January 14, 2002. Accepted for publication January 27, 2002. Presented at the Society for Academic Emergency Medicine annual meeting, Boston, MA, May 1999, and at the Ambulatory Pediatric Association national conference, San Francisco, CA, May 1999. Supported by grant number F32 HS00148 from the Agency for Healthcare Research and Quality, and by the Research Training Grant from the Society for Academic Emergency Medicine. Address for reprints: Craig D. Newgard, MD, MPH, Department of Emergency Medicine, Harbor–UCLA Medical Center, Box 21, 1000 West Carson Street, Torrance, CA 90509; 310-222-3666, fax 310-782-1763; E-mail [email protected]. Copyright © 2002 by the American College of Emergency Physicians. 0196-0644/2002/$35.00 + 0 47/1/123549 doi:10.1067/mem.2002.123549

MAY 2002

39:5

ANNALS OF EMERGENCY MEDICINE

Study objective: We sought to create a clinical decision rule, on the basis of variables available to out-of-hospital personnel, that could be used to accurately predict severe injury in pediatric patients involved in motor vehicle crashes as occupants. Methods: We analyzed the National Automotive Sampling System database, a national probability sample, using pediatric patients up to 15 years old (occupants only) involved in motor vehicle crashes from January 1993 to December 1999. The National Automotive Sampling System database includes patients from regions throughout the country, weighted to represent a nationwide sample. Twelve out-of-hospital variables were used in classification and regression tree analysis to create a decision rule separating children with severe injuries (Injury Severity Score [ISS] ≥16) from those with minor injuries (ISS<16). Misclassification costs and complexity parameters were selected to yield a decision tree with appropriate sensitivity and specificity for the identification of severely injured patients, while also being simple and practical for outof-hospital use. Probability weights were used throughout the analysis to account for the sampling design and sampling weights. Results: Using a sample size of 8,392 children, we constructed a decision rule using 3 out-of-hospital variables (Glasgow Coma Scale score, passenger space intrusion ≥6 in [≥15 cm], and restraint use) to predict those patients with an ISS of 16 or more. Internal cross-validation was used to determine the sensitivity and specificity, yielding values of 92% and 73%, respectively, for the prediction of patients with an ISS of 16 or more. Conclusion: Out-of-hospital variables available to field personnel could be used to effectively triage pediatric motor vehicle crash patients using the decision rule developed here. Prospective trials would be needed to test this decision rule in actual use.

4 8 1

PREDICTING SEVERITY OF INJURY IN PEDIATRIC PATIENTS Newgard, Lewis & Jolly

[Newgard CD, Lewis RJ, Jolly BT. Use of out-of-hospital variables to predict severity of injury in pediatric patients involved in motor vehicle crashes. Ann Emerg Med. May 2002;39:481-491.] INTRODUCTION

Unintentional injury is the leading cause of death in children older than 1 year.1 Although motor vehicle occupant injuries constitute 12% of all pediatric trauma requiring hospitalization in the United States,2 this mechanism of injury represents the leading cause of death in children between 6 and 14 years old2,3 and is responsible for 30% of all childhood injury deaths.2 Compared with other forms of trauma in children, motor vehicle occupant trauma is responsible for more severe injuries, the highest absolute mortality, the greatest costs, and the largest number of productive life years lost in the pediatric population.2 In 1999, 872 children were injured and 7 children were killed every day as an occupant in a motor vehicle crash (MVC) in the United States.3 Several studies have demonstrated the benefits of treating severely injured pediatric trauma patients at designated pediatric trauma centers as opposed to adult trauma centers or nontrauma centers.4-7 The beneficial effect of pediatric trauma center care on mortality in seriously injured children appears to be most profound in cases of blunt trauma7 and head injury.4,6 The magnitude of this effect differs by type and mechanism (ie, blunt versus penetrating) of injury, with some studies demonstrating a 60% reduction in mortality in head-injured patients,6 and other studies showing a survival benefit 10 times greater for children with head or internal injuries.4 Accurate identification of severely injured pediatric patients, with selective transport to pediatric trauma centers, would improve those patients’ outcomes while minimizing trauma center resources spent on patients who do not require trauma center care. Preventable morbidity and mortality occurs with undertriage (severely injured patients sent to nonpediatric trauma centers or nontrauma centers), whereas increased costs and inappropriate use of emergency medical services (EMS) and trauma center resources occur with overtriage (children with minor injuries sent to pediatric trauma centers). The costs of overtriage are especially high in rural settings, where excessive transport times may remove out-of-hospital providers for significant periods of time, placing the rest of the community at risk. The goal of an effective triage tool is to minimize both undertriage and overtriage.

4 8 2

Although there is a relatively large body of literature involving triage rules for trauma triage in adults, there is currently no validated out-of-hospital triage tool that produces acceptable rates of both undertriage and overtriage for pediatric trauma patients. Numerous studies have examined various pediatric trauma triage rules, yielding sensitivities and specificities for identifying children with an ISS of 16 or more between 27% to 100% and 25% to 99%, respectively (Table 1).8-16 Many EMS systems simply extrapolate existing adult triage criteria to children. The use of adult criteria is problematic, because significant differences exist between children and adults in physiology, anatomy, and in the biomechanics of traumatic injury. When adult triage criteria are applied to children, an unacceptably high rate of undertriage occurs.9-12,17 All published pediatric-specific triage rules to date entail significant problems with undertriage, overtriage, or both, 8-16 the analyses of which have been hindered by various methodologic flaws. Existing trauma triage studies that include children have been limited by retrospective analyses, 12-14,18 small sample sizes, 13,16,19,20 missing data,12,13,18,21-23 biased trauma center populations,9-11,13-15,18,24 and different outcome measures.8,14-16 As a result, the reported accuracy of a given rule may be misleading. Finally, most pediatric trauma triage studies have applied triage criteria to all trauma patients, rather than separating patients by mechanism of injury.8-16 Although this design allows broader application of the rule, inclusion of a wide variety of mechanisms of injury results in a triage tool that is less accurate for individual mechanisms of injury. Our objective in this study was to create a clinical decision rule, on the basis of variables reliably available to out-of-hospital personnel, which could be used to accurately predict severe injury in pediatric patients involved as occupants in MVCs and to provide an unbiased estimate of the rule’s sensitivity and specificity in a representative population. M AT E R I A L S A N D M E T H O D S

The analysis was conducted using prospectively collected data from the National Automotive Sampling System Crashworthiness Data System (NASS CDS) database, a comprehensive national probability sample of patients involved in MVCs. We selected pediatric patients up to 15 years old involved in MVCs (occupants only) during a 7year period, from January 1993 through December 1999.

ANNALS OF EMERGENCY MEDICINE

39:5 MAY 2002

PREDICTING SEVERITY OF INJURY IN PEDIATRIC PATIENTS Newgard, Lewis & Jolly

This age group was selected to represent the typical age range of children transported to emergency departments and trauma centers approved for pediatric patients, and because the pattern of covariates and missing data in older adolescent occupants prevented their inclusion in the analysis. The NASS CDS database represents a 3-stage sampling of MVC patients from specific regions throughout the country, weighted to represent a nationwide sample of patients (Appendix).25 The sampling system used to create the NASS CDS database was designed to ensure validity of the data without requiring investigation of each MVC in the country. Each case recorded in the NASS CDS database is weighted to represent the nationwide incidence of similar cases. The sampling system for the NASS CDS database is designed to oversample more severely injured persons, allowing adequate representation of their associated characteristics and covariate patterns.25 These patients are then underweighted to reflect an accurate estimate of the national incidence of similar crashes and injuries.25 The patients contained in the severely injured outcome group in this analysis are actually composed of many more sampled patients who were assigned fractional (ie, less than one) weights in the dataset. The NASS CDS database includes patients who were evaluated at a medi-

cal facility and whose vehicle was towed because of damage (and was thus available for crash investigation). Our hospital’s Institution Review Board approved the study. Out-of-hospital variables that were both available in the NASS CDS database and easily obtained by on-scene EMS personnel were selected for analysis as potential predictors of injury. The 12 variables analyzed were: age, sex, weight, Glasgow Coma Scale (GCS) score, primary point of vehicular impact, rollover, magnitude of passenger space intrusion (PSI), intrusion location, restraint use, seat location (front-middle, front-right, rear-left, rear-middle, rearright, rear-other), entrapment, and air bag deployment.26 Age, weight, GCS score, PSI, and rollover (number of quarter turns) were coded as continuous variables, whereas sex, primary point of vehicular impact, intrusion location, restraint use, seat location, entrapment, and air bag deployment were coded as categorical variables. GCS score was recorded from the initial evaluation at a medical facility. In cases of multiple impacts or multiple areas of intrusion, the primary impact site (defined as the site of the most impact force, the most severe damage, or with the highest calculated change in velocity [∆v]) and the primary intrusion site (defined as the area of greatest intrusion within the passenger compartment) were used for the analysis,

Table 1.

Studies of pediatric trauma triage criteria.*

Principal Author

No. of Patients Studied

Triage Criteria Studied

Tepas et al8 Eichelberger et al9†

615 1,334

Chan et al10† Kaufmann et al11†

1,116 376

Phillips et al12 Qazi et al13† Aprahamian et al14† Engum et al15†

1,505 143 144 1,285

PTS <8 TS ≤14 RTS ≤11 PTS ≤8 TS ≤12 PTS ≤8 RTS ≤11 “Trauma Scorecard” Mechanism of injury criteria alone PTS ≤8 Simplified version of ACS criteria

Qazi et al16†

192

Paramedic judgment

Outcome Studied ISS >10 ISS ≥16 ISS ≥16 ISS ≥16 ISS ≥16 ISS ≥16 ISS ≥16 ISS ≥16 ISS ≥16 AIS ≥3 Death, ICU, major nonorthopedic operation ICU or major nonorthopedic operation

Sensitivity, %

Specificity, %

Undertriage (1–Sensitivity), %

Overtriage (1–Specificity), %

56 72 73 78 27 85 76 78 44 52 100

96 75 74 75 99 57 81 84 25 98 29

44 28 27 22 73 15 24 22 56 48 0

4 25 26 25 1 43 19 16 75 2 71

50

88

50

12

Adapted with permission from Phillips S, Rond PC, Kelly SM, et al. The need for pediatric-specific triage criteria: results from the Florida trauma triage study. Pediatr Emerg Care. 1996;12:394-398. Lippincott, Williams & Wilkins, 1996. PTS, Pediatric Trauma Score; ISS, Injury Severity Score; TS, Trauma Score; RTS, Revised Trauma Score; AIS, Abbreviated Injury Scale; ACS, American College of Surgeons. *Studies assessing death as the sole outcome variable and studies that failed to separate adult and pediatric patients have been excluded. †Denotes studies assessing trauma center populations only.

MAY 2002

39:5

ANNALS OF EMERGENCY MEDICINE

4 8 3

PREDICTING SEVERITY OF INJURY IN PEDIATRIC PATIENTS Newgard, Lewis & Jolly

respectively. Restrained occupants were considered occupants secured in a childseat or by lap-shoulder belt use. Blood pressure, pulse rate, respiratory rate, and physiologic scores (Trauma Score, Revised Trauma Score, Pediatric Trauma Score) were unavailable in the database. An Injury Severity Score (ISS) of 16 or more was used to define severe injury and the need for trauma center care because of its availability in the database, validation of the ISS as a measure of injury and mortality in children, and to allow for comparison with previously published pediatric triage rules.27-29 Other indicators of injury severity (eg, Trauma Score, Revised Trauma Score, Pediatric Trauma Score) and resource use (eg, ICU care, nonorthopedic operation, neurosurgic care) were unavailable in the database. Proc Univariate (version 8.1, SAS Institute, Inc., Cary, NC) was used for assessment of continuous variables, which allowed integration of probability weights. Categorical variables were assessed using Proc FREQ with probability weights. Probability weights were calculated by multiplying the weight factor for each observation by a fixed proportion (actual sample size÷expanded sample size) to ensure validity in the analysis and appropriate calculation of variances. Probability weights were used in all analyses to account for both sampling design and sampling weights, while allowing a fixed sample size.30,31 Classification and regression tree (CART) analysis, (CART version 4.0, Salford Systems, San Diego, CA) was used to create a decision tree to separate children with severe injury (ISS ≥16) from those with minor injury (ISS <16). CART analysis is a nonparametric method of statistical analysis used to classify observations on the basis of a large number of possible predictive variables.32 We used a version of CART that allows the direct analysis of a weighted database to avoid a bias that can be introduced by replicating observations according to their weights before analysis.33-35 CART analysis begins by selecting the single best predictor for separating patients into severely and nonseverely injured groups. For each of these subgroups, the best predictors for further subdividing the groups are selected until further “splits” would not improve accuracy. Each variable is considered at each decision point, regardless of whether it had been used earlier. After a full decision tree is created, it is examined to determine whether it can be simplified or “pruned” without significant loss of predictive power. Misclassification costs and tree complexity parameters were empirically selected to generate a practical (ie, not overly complex) decision tree with adequate sensitivity and specificity. The decision tree presented represents a balance between predictive power and simplicity.

4 8 4

The sensitivity and specificity of the decision tree were calculated by cross-validation.36-38 Cross-validation uses a randomly selected subset of the study sample termed a “learning sample” (approximately 90% of the original sample) to create a tree in a manner identical to that used to create the original model. This tree is used to predict the outcomes in the excluded, independent sample of patients (10% of the original sample), that is, the validation set. This same procedure is repeated until all observations have been excluded once from the model creation process. Confidence intervals have been omitted because the current version of CART does not provide the information necessary to calculate valid confidence intervals and because alternative means of calculating these intervals suggested an inaccurate measure of precision. When a missing variable is encountered at a branch point in the decision tree, CART analysis can use a different variable most closely resembling the missing variable in its ability to make a similar decision split in the data (ie, a “surrogate” variable). Because of this, missing values do not require withdrawal of the patient and missing values did not detract from the integrity of the decision tree. Potential effect of the triage rule was assessed by comparison with 2 published triage rules,12,15 reflecting trauma center criteria and accuracy used in various EMS systems, using the NASS CDS database national estimates. R E S U LT S

During the 7-year period, 8,394 pediatric patients (1 month to 15 years old) were entered into the NASS CDS database. Two (0.02%) of these patients were missing outcome data, allowing 8,392 children to be included in this analysis. Forty-seven (0.6%) children were severely injured, defined as an ISS of 16 or more. There were 15 fatalities (0.2% of the total sample; 32% of severely injured children) included in the outcome group. The 47 severely injured patients in the probability-adjusted dataset were composed of 370 severely injured children (including 138 fatalities) sampled in the original database. The discrepancy between the number of severely injured patients sampled and those represented in the probability-adjusted dataset reflects the fractional probability weights assigned to these observations to account for the sampling design. Fatalities at the scene of an MVC are not included in the NASS CDS database and are thus omitted from this analysis. Covariate patterns between the 2 groups (ISS <16 versus ISS ≥16) and among the entire sample are listed in

ANNALS OF EMERGENCY MEDICINE

39:5 MAY 2002

PREDICTING SEVERITY OF INJURY IN PEDIATRIC PATIENTS Newgard, Lewis & Jolly

Table 2. The median ISS for the groups (ISS <16 and ISS ≥16) were 0 (interquartile range [IQR] 0 to 1) and 26 (IQR 19 to 43), respectively. All 12 variables were considered in the CART analysis used to generate the final decision tree. Three variables, GCS score, PSI, and restraint use, were retained in the final tree used to predict those patients with an ISS of 16 or more (Figure).

Table 2.

Covariate patterns among patients.* Characteristic Probability-weighted sample Male, % Median age, y Median weight, kg Median GCS score GCS score <15, % Restrained, % Air bag deployment, % Seat location, % Front-middle Front-right Rear-left Rear-middle Rear-right Rear-other Impact type, % Front Rear Left lateral Right lateral Top Undercarriage Rollover, % Magnitude of intrusion, % 1–3 in (3–8 cm) 3–6 in (8–15 cm) 6–12 in (15–30 cm) 12–18 in (30–46 cm) 18–24 in (46–61 cm) >24 in (>61 cm) Catastrophic Intrusion location, % Front seat-left Front seat-middle Front seat-right Rear seat-left Rear seat-middle Rear seat-right Rear seat-other Catastrophic Entrapment, % Median ISS (IQR) Deaths (%)

Entire Population

ISS <16

ISS ≥16

N=8,392 50 8 30 15 2 72 3

N=8,345 50 8 30 15 0.8 72 3

N=47 51 10 34 13 57 27 4

1.8 37 23 12 22 5

1.8 37 23 12 22 5

1.8 45 18 9 21 5

50 12 13 13 10 1.3 10

50 12 13 13 10 1.3 10

38 8 11 25 19 0.0 24

33 27 26 10 3 1.1 0.04

34 27 26 9 3 0.9 0.01

2 12 28 31 11 14 1.5

32 4 37 12 1.2 12 1.2 0.04 2 0 15 (0.2)

32 4 37 12 1.1 12 1.2 0.01 2 0 (0–1) 0 (0)

22 2 37 12 4 20 2 1.5 13 26 (17–43) 15 (32)

All numbers reflect those in the probability-adjusted dataset. *Percentages refer to column percentages; all percentages are rounded-off, unless <2%.

MAY 2002

39:5

ANNALS OF EMERGENCY MEDICINE

Cross-validation analysis of the decision rule yielded an estimated sensitivity and specificity of 92% and 73%, respectively, for identifying patients with an ISS of 16 or more. The overall positive predictive value (PPV) and negative predictive value (NPV) of the tree are 1.9% and 99.9%, whereas the positive and negative likelihood ratios are 3.4 and 0.1, respectively. Performance measures of the clinical decision rule are listed in Table 3. The variables included in the final tree were selected in the CART analysis for their predictive power and ability to create a tree with reasonable simplicity. Variation of assumed misclassification costs resulted in the creation of different trees that used additional mechanistic variables and decision nodes, yielding different values for sensitivity and specificity in the cross-validation analysis. We favored a tree with higher sensitivity, thus minimizing the number of seriously injured children who are undertriaged. Although anatomic injury was a strong predictor of injury severity, this variable was not considered in the analysis because accurate information on anatomic injury is not reliably available to out-of-hospital personnel. A GCS score of less than 15 is the initial decision point, and portends the greatest predictive value for identifying injured children. For those children with a GCS score of less than 15 recorded in the database, 62% (15 of 24 children) were seriously injured. Similar to previous studies regarding physiologic variables, the GCS score was highly specific for injured children.8-11,14 A PSI of 6 in (15 cm) or more is the second-level variable, serving to identify an additional 23 injured children, although with less predictive power. The last decision point in the decision tree entails restraint use. In patients with a normal GCS score and minimal or no PSI (ie, <6 in [<15 cm]), lack of appropriate restraint use suggested a higher probability of severe injury. Conversely, those children with a normal GCS score, minimal or no PSI, and appropriate restraint use had a very low probability of severe injury. This last group, identified by the 3 variables, represents a low-risk group that could safely be evaluated at a nontrauma center. The percentage of observations with a given variable available in the dataset varied widely. Fourteen percent of observations had a GCS score recorded; 27% had PSI of any degree recorded (passengers with no PSI or missing values for PSI were both recorded as missing in the dataset); and 76% had restraint use recorded. Eighty-one percent of observations had at least 1 of the 3 defining variables available in the dataset (81% of uninjured children; 99% of injured children). Surrogate markers were partially useful in circumventing the problem of missing

4 8 5

PREDICTING SEVERITY OF INJURY IN PEDIATRIC PATIENTS Newgard, Lewis & Jolly

data; however, they were used in classifying patients primarily at the final decision node (restraint use). The surrogate markers used in the analysis to make similar splits in the data include: rollover (number of quarter turns) as a surrogate for PSI of 6 in (15 cm) or more in 28 observations (3% of those classified at the second decision node), and rollover and seat location as surrogates for restraint use in 1,943 observations (26% of those classified at the third decision node). No surrogates were used for a GCS score of less than 15; patients with missing GCS values were treated as if their GCS score was 15. We assessed the potential effect of the rule by comparing it with 2 published pediatric triage rules currently in use.12,15 These 2 rules were selected to represent different EMS systems with different sets of triage criteria. The comparisons require the assumption that the published triage rules would have similar accuracy (ie, sensitivity, specificity) when applied to a national sample of pediatric MVC occupants. When comparing our clinical decision rule with that of Phillips et al12 (sensitivity 78%; specificity 84%), there would be 497 children per year potentially undertriaged (ie, seriously injured children mistriaged to a nontrauma center) by the Florida criteria who would have been appropriately triaged under the current

Figure.

rule. Assuming a 32% mortality rate in the seriously injured group (evident in the NASS CDS database estimates) and a 50% mortality reduction when this group is cared for at a pediatric trauma center,4,6 there would be 80 potentially preventable fatalities per year in children up to 15 years (7% of MVC occupant fatalities in this group) by using the new triage rule. The lower specificity of our decision rule compared with that of Phillips et al would result in the overtriage of 69,227 children annually (11% of those uninjured). The triage criteria published by Engum et al15 uses a different outcome definition, but is representative of many urban EMS systems with extensive trauma center criteria that likely capture most seriously injured patients, at the expense of significant overtriage (ie, low rule specificity). The criteria presented by Engum et al, with 100% sensitivity (no seriously injured patients mistriaged), would capture 284 children annually (8% of all injured children) who our decision rule would undertriage. This high sensitivity comes at the expense of specificity (29%), which would extend unnecessary trauma center care and resources to 276,906 children with minor injuries per year (44% of those uninjured) when compared with our decision rule.

ISS <16: 8,345 (99.4%) ISS ≥16: 47 (0.6%) N=8,392

Clinical decision rule for identifying severely injured (ISS ≥16) pediatric passengers involved in MVCs. Trauma center denotes the need for pediatric trauma center care.

Child occupant up to 15 y in MVC

Yes

Trauma center ISS <16: 9 (38%) ISS ≥16: 15 (62%) N=24

No

GCS score <15?

Yes

PSI ≥6 in (≥15 cm)?

Yes

Trauma center ISS <16: 897 (97.5%) ISS ≥16: 23 (2.5%) N=920

4 8 6

No

Unrestrained?

No

Trauma center

Nontrauma center

ISS <16: 1,418 (99.6%) ISS ≥16: 6 (0.4%) N=1,423

ISS <16: 6,021 (99.9%) ISS ≥16: 3 (0.1%) N=6,024

ANNALS OF EMERGENCY MEDICINE

39:5 MAY 2002

PREDICTING SEVERITY OF INJURY IN PEDIATRIC PATIENTS Newgard, Lewis & Jolly

DISCUSSION

This study is unique in the field of pediatric trauma triage for a number of reasons. First, we are aware of no pediatric trauma triage studies that use CART analysis as a method of creating a clinical decision rule. CART analysis has several advantages over the more commonly used multivariate logistic regression models for the creation of clinical decision rules. CART analysis is specifically tailored to the decisionmaking process, yielding a decision tree that is simple to incorporate into clinical decision making (ie, trauma or nontrauma center) and allowing for the concise presentation of a decision rule.39 The results of logistic regression models, which produce a probability of an outcome, are often difficult for clinicians to incorporate into the clinical decisionmaking process.40 CART analysis does not require that predictive variables be selected in advance, is nonparametric, and unlike logistic regression, allows for the reliable selection of a small number of best predictive variables out of a large group of potential candidates. CART analysis is also highly suited to sorting out complex interactions between predictive variables and precludes the need to arbitrarily establish preanalysis cutoff points for continuous variables. The decision tree allows specific, easily followed combinations of variables to increase predictive power. Many previous triage studies have illustrated the benefit of combining variables as opposed to using isolated predictors.18,20,23,24,41-44 One important distinction between CART and other programs that use the CART procedure is the ability to internally cross-validate the final decision tree. The interpretation of many triage studies is complicated by the lack of an independent dataset for validation of the presented rule. The acquisition of an independent dataset for validation is time and effort consuming. The method of cross-validation used within CART allows for an independ-

Table 3.

Performance measures of the clinical decision rule. Cross-Validation Results

Value

Sensitivity Specificity PPV NPV Positive likelihood ratio Negative likelihood ratio

92% 73% 1.9% 99.9% 3.4 0.1

MAY 2002

39:5

ANNALS OF EMERGENCY MEDICINE

ent validation of the triage rule, providing an unbiased estimate of its true ability to differentiate the 2 populations without requiring a separate dataset and precluding the need for other methods of internal validation (eg, split-sample methods) that decrease sample size. Furthermore, this method of cross-validation decreases the risks of overfitting the dataset, of excluding important predictors, and of inflating the accuracy of the decision rule. This is the largest pediatric MVC trauma triage study to date. In addition, the probabilistic design of the NASS CDS database allows the data to be representative of the entire US pediatric population involved in MVCs, improving the generalizability of the results. Furthermore, because the database is produced from crash team investigators, it is very comprehensive and more accurate than data collected from police reports, ambulance reports, or ED records.45 Finally, the field of pediatric trauma triage is largely dominated by studies focusing on physiologic triage tools, which have consistently produced inadequate sensitivities in identifying injured children when used alone.8-11,14 There are very few studies looking at childhood trauma that combine physiologic and mechanistic variables to produce triage tools. We attempted to explore the potentially helpful combination of variables readily available to out-of-hospital personnel for triaging children. Although many combinations of the 12 variables surfaced, the 3 variables included in the final decision tree provide the best balance of tree accuracy and simplicity. Although other pediatric trauma triage studies have included children affected by all types of trauma, there is no other rule that performs as well, on the basis of so few variables for victims of MVCs. This finding is partially explained by restricting the analysis to victims of one particular mechanism of trauma (occupants of MVCs). Nonetheless, the decision tree shown here encompasses most components of an effective clinical decision rule, including development in an area where there is an obvious need, clinical sensibility, derivation with rigorous methodologic standards, sensibility, accuracy, ease of use, validation, and a clear course of action.40,46 The predictive power of the decision tree results from the ability to combine mechanism of injury with a physiologic variable. The GCS score of a child proved to be an important component of the clinical assessment. Multiple studies have suggested the GCS score to be one of the most specific indicators of serious injury in children, even when compared with other physiologic variables.12,15,47,48 Our analysis supports the conclusion

4 8 7

PREDICTING SEVERITY OF INJURY IN PEDIATRIC PATIENTS Newgard, Lewis & Jolly

that any child involved in an MVC with an abnormal GCS score mandates pediatric trauma center evaluation. Although PSI was a poor predictor of severe injury in a prior trauma triage study,41 this variable was consistently included as an important decision point in multiple decision trees. The predictive power of PSI likely represents its ability to serve as a surrogate marker of ∆v, and thus impact severity. Prior studies have demonstrated a strong correlation between ∆v and increased morbidity and mortality,49 as well as between ∆v and injury severity (ISS ≥16), noting that “vehicle crush” approximates ∆v and is easier to measure.50 Jones and Champion50 demonstrated that an external vehicle crush of 15 in (38 cm) to 28 in (71 cm) (depending on impact site) correlated with an ISS of 16 or greater in more than 90% of patients, but the extent of PSI affected by vehicle crush was not noted. It is possible that these measurements would correlate to our criterion of PSI of 6 in (15 cm) or more. It is unrealistic to assume that there is a set of triage criteria or a triage rule that will triage patients with 100% sensitivity and 100% specificity. Generally, an EMS system will favor either sensitivity or specificity, at the expense of the other, on the basis of the characteristics of their population and surrounding community and available resources. There is no single triage rule that will work well in all settings. An urban trauma system with shorter transport times and more resources may favor a rule with high sensitivity, while accepting the financial burden of caring for many patients with only minor injuries at trauma centers. A rural trauma system, on the other hand, may opt for a less sensitive rule that is more specific for seriously injured children. Such an EMS system may have longer transport times, no readily available trauma center, and minimal resources, all of which generate a need for trauma center criteria with higher specificity and PPV. The consequence of these decisions is exemplified by the comparison between the decision rule presented here and 2 different published rules encompassing different performance parameters. One limitation of the study is the use of a national complex sampling database. Although this sample accurately reflects the national incidence of MVCs, its associated covariate pattern may not represent that of a population served by a given EMS system. Further, the selection of patients for whom a vehicle was towed may constitute a selection bias toward more significant crashes in the database. Also, the NASS CDS database does not include physiologic variables other than GCS score (eg, respiratory rate, systolic blood pressure) used in calculating the Trauma Score, Revised Trauma Score, or Pediatric Trauma

4 8 8

Score, all of which have been shown to aid in the identification of injured children. Although some studies have suggested that the GCS score is the most predictive physiologic variable,12,15,47,48 the addition of respiratory rate and blood pressure may increase the accuracy of the tool. Furthermore, there could be a discrepancy between the GCS score recorded in the out-of-hospital setting and the initial GCS score recorded at a medical receiving facility (as recorded in the NASS CDS database). Another limitation of the NASS CDS database is the amount of missing data for pertinent variables. Although there were large percentages of missing data for the variables included in the first 2 decision nodes (GCS score and PSI), surrogate markers were used in only 3% of children classified by these 2 nodes. Consequently, the missing values for GCS score and PSI had little effect on the accuracy of the final tree. Furthermore, 81% of seriously injured children were effectively classified by these 2 variables, despite the number of missing values. The last decision node (restraint use) was left to classify the majority of observations, but only a fraction of injured children. The discrepancy between the percentage missing all 3 variables in noninjured (19%) and injured (1%) children likely reflects a nonrandom pattern of the missing values, making it strictly impossible to assess the accuracy of the tree had these variables been present (ie, without the use of surrogate markers). However, we believe that less missing values would likely improve the performance of the decision rule, especially with respect to identifying additional injured children in the first 2 decision nodes, on the basis of the predictive power of the GCS score and PSI. The importance of these 2 variables in assessing the likelihood of serious injury in children involved in MVCs may reinforce the importance of their complete collection in future out-of-hospital studies, which may help resolve the true effect of missing values in this dataset. Some of the variables included in the American College of Surgeon’s trauma triage criteria (eg, anatomic injuries, crash speed, comorbidities) were excluded from the analysis because of the difficulty in obtaining accurate and timely on-scene information regarding these variables. Other potential aids such as online medical control51,52 and paramedic judgment of injury severity,53,54 suggested by some as additional methods of further improving triage in the adult population, were not assessed. It should be noted that paramedic judgment in the setting of pediatric trauma has not been shown to be a sensitive triage criterion.16 The use of the ISS has been suggested by some to be a poor predictor of the need for hospital resources.55 An

ANNALS OF EMERGENCY MEDICINE

39:5 MAY 2002

PREDICTING SEVERITY OF INJURY IN PEDIATRIC PATIENTS Newgard, Lewis & Jolly

outcome marker of ISS of 16 or more was used in this study because of its validation in pediatric trauma populations as a measure of injury severity and mortality,27-29,56 because the variable allowed comparison with previously published triage rules, and because it was available in the database. However, some authors suggest that a constellation of additional outcome variables, including the need for ICU care, major nonorthopedic operative interventions (ie, craniotomy, laparotomy, thoracotomy, spinal stabilization), and the interval from ED presentation to operation, may predict more accurately the need for specialized trauma center resources.41,47,55 The clinical decision rule presented here is not intended to be used on all trauma patients, but rather on children involved as passengers in MVCs. When a triage rule focuses on a single mechanistic class of trauma (ie, MVCs), the accuracy of the tool will likely be increased because of the more homogeneous population to which it is being applied. The fact that MVCs involving occupants are one of the most frequent mechanisms of pediatric trauma and are responsible for the greatest morbidity and mortality in childhood unintentional injury justifies specific emphasis on this subgroup of trauma patients in developing a clinical decision rule.2,3 Although pedestrian injuries in children as a result of motor vehicles also represent an important form of preventable injury, the total number of fatalities, costs, and years of productive life lost are less than one third of that for motor vehicle occupant injury.2 Regarding the percentage of severely injured patients, the incidence in the NASS CDS database (0.6%) is lower than that seen in previous studies.8-11 The low incidence of severely injured children and the moderate specificity of the rule contribute to the low PPV and high NPV of the rule. A higher incidence of seriously injured children in a population served by a given EMS system would increase the PPV. Increased rule specificity (as may occur with the inclusion of other physiologic variables) would also increase the PPV. The performance measures in this rule may not suit the needs of all EMS systems, as each system has its own unique priorities, goals, and available resources. Lastly, the internal cross-validation analysis may not accurately represent the rule’s function in actual practice. The ability of a clinical decision rule to perform well can only truly be assessed through prospective evaluation in a patient population different from that in which it was originally developed and validated.40,46 As with any rule, factors encountered in clinical practice may substantially lessen the accuracy of the tool, especially in the out-ofhospital setting. Paramedic compliance, accurate report-

MAY 2002

39:5

ANNALS OF EMERGENCY MEDICINE

ing of on-scene variables, precise clinical assessment of the patient (including GCS score), critically ill patients, and time constraints in the field, among other things, may detract from the usefulness and accuracy of the rule. Also, the complexity of using “surrogate markers” to overcome the problem of missing variables (as accomplished through the CART program) may prohibit their use by field personnel, making thorough on-scene data collection imperative for application of the rule. These issues will only be fully addressed by prospective validation of the decision rule in out-of-hospital trials. Despite these limitations, the clinical decision rule presented here is a product of the largest pediatric trauma triage study to date and is the only pediatric triage rule developed and internally validated in a population-based database. If the accuracy and ease of use of this rule are recognized in prospective validation efforts, these results may serve to significantly improve the current practice of pediatric trauma triage, potentially reducing morbidity and mortality of children involved as occupants in MVCs. Author contributions: CDN and RJL conceived the study and designed the analysis. BTJ assisted with data access and critical steps in initiating the project. CDN performed the database management, data coding, and statistical analysis. RJL provided statistical advice and guidance for the analysis. CDN drafted the manuscript and all authors contributed substantially to its revision. CDN and RJL take responsibility for the paper as a whole.

REFERENCES 1. Federal Interagency Forum on Child and Family Statistics. America’s Children: Key National Indicators of Well-Being, 2001. Washington, DC: Centers for Disease Control and Prevention, National Center for Health Statistics, National Vital Statistics System; 1998. 2. Guyer B, Ellers B. Childhood injuries in the United States–mortality, morbidity, and cost. Am J Dis Child. 1990;144:649-652. 3. National Highway Traffic Safety Administration. Traffic Safety Facts 1999–Children. Washington, DC: US Department of Transportation; 1999. Publication DOT HS 809 087. 4. Cooper A, Barlow B, DiScala C, et al. Efficacy of pediatric trauma care: results of a population-based study. J Pediatr Surg. 1993;28:299-303. 5. Hulka F, Mullins RJ, Mann NC, et al. Influence of a statewide trauma system on pediatric hospitalization and outcome. J Trauma. 1997;42:514-519. 6. Johnson DL, Krishnamurthy S. Send severely head-injured children to a pediatric trauma center. Pediatr Neurosurg. 1996;25:309-314. 7. Hall JR, Reyes HM, Meller JL, et al. The outcome for children with blunt trauma is best at a pediatric trauma center. J Pediatr Surg. 1996;31:72-77. 8. Tepas JJ, Ramenofsky ML, Mollitt DL, et al. The Pediatric Trauma Score as a predictor of injury severity: an objective assessment. J Trauma. 1988;28:425-429. 9. Eichelberger MR, Gotschall CS, Sacco WJ, et al. A comparison of the Trauma Score, the Revised Trauma Score, and the Pediatric Trauma Score. Ann Emerg Med. 1989;18:1053-1058. 10. Chan BSH, Walker PJ, Cass DT. Urban trauma: an analysis of 1,116 pediatric cases. J Trauma. 1989;29:1540-1547. 11. Kaufmann CR, Maier RV, Rivara FP, et al. Evaluation of the Pediatric Trauma Score. JAMA. 1990;263:69-72. 12. Phillips S, Rond PC, Kelly SM, et al. The need for pediatric-specific triage criteria: results from the Florida trauma triage study. Pediatr Emerg Care. 1996;12:394-398.

4 8 9

PREDICTING SEVERITY OF INJURY IN PEDIATRIC PATIENTS Newgard, Lewis & Jolly

13. Qazi K, Wright MS, Kippes C. Stable pediatric blunt trauma patients: is trauma team activation always necessary? J Trauma. 1998;45:562-564.

38. Geisser S. The predictive sample reuse method with applications. J Am Statist Assoc. 1975;70:320-328.

14. Aprahamian C, Cattey RP, Walker AP, et al. Pediatric Trauma Score. Arch Surg. 1990;125:1128-1131.

39. McConnochie KM, Roghmann KJ, Pasternack J. Developing prediction rules and evaluating observation patterns using categorical clinical markers: 2 complimentary procedures. Med Decision Making 1993;13:30-42.

15. Engum SA, Mitchell MK, Scherer LR, et al. Prehospital triage in the injured pediatric patient. J Pediatr Surg. 2000;35:82-87. 16. Qazi K, Kempf JA, Christopher NC, et al. Paramedic judgment of the need for trauma team activation for pediatric patients. Acad Emerg Med. 1998;5:1002-1007.

40. Stiell IG, Wells GA. Methodologic standards for the development of clinical decision rules in emergency medicine. Ann Emerg Med. 1999;33:437-447. 41. Henry MC, Hollander JE, Alicandro JM, et al. Incremental benefit of individual American College of Surgeons’ trauma triage criteria. Acad Emerg Med. 1996;3:992-1000.

17. Nayduch DA, Moylan J, Rutledge R, et al. Comparison of the ability of Adult and Pediatric Trauma Scores to predict pediatric outcome following major trauma. J Trauma. 1991;31:452458.

42. Bond RJ, Kortbeek JB, Preshaw RM. Field trauma triage: combining mechanism of injury with the out-of-hospital index for an improved trauma triage tool. J Trauma. 1997;43:283-287.

18. Cottington EM, Young JC, Shufflebarger CM, et al. The utility of physiologic status, injury site, and injury mechanism in identifying patients with major trauma. J Trauma. 1988;28:305-311.

43. West JG, Murdock MA, Baldwin LC, et al. A method for evaluating field triage criteria. J Trauma. 1986;26:655-659.

19. Cooper ME, Yarbrough DR, Zone-Smith L, et al. Application of field triage guidelines by outof-hospital personnel: is mechanism of injury a valid guideline for patient triage? Am Surg. 1995;61:363-371.

44. Phillips JA, Buchman TG. Optimizing out-of-hospital triage criteria for trauma team alerts. J Trauma. 1993;34:127-132.

20. Knopp R, Yanagi A, Kallsen G, et al. Mechanism of injury and anatomic injury as criteria for out-of-hospital trauma triage. Ann Emerg Med. 1988;17:895-902. 21. Esposito TJ, Offner PJ, Jurkovich GJ, et al. Do out-of-hospital trauma center triage criteria identify major trauma victims? Arch Surg. 1995;130:171-176. 22. Lowe DK, Oh GR, Neely KW, et al. Evaluation of injury mechanism as a criterion in trauma triage. Am J Surg. 1986;152:6-10. 23. Kane G, Engelhardt R, Celentano J, et al. Empirical development and evaluation of out-ofhospital trauma triage instruments. J Trauma. 1985;25:482-489. 24. Long WB, Bachulis BL, Hynes GD. Accuracy and relationship of mechanisms of injury, Trauma Score, and injury severity score in identifying major trauma. Am J Surg. 1986;151:581584. 25. National Accident Sampling System (NASS) Crashworthiness Data System. Analytic User’s Manual, 1997 File. Washington, DC: US Department of Transportation; National Highway Traffic Safety Administration; National Center for Statistics and Analysis; 1997. 26. National Accident Sampling System (NASS) Crashworthiness Data System. NASS CDS Variable-Attribute Structure Manual, 1988-1996. Washington, DC: US Department of Transportation, National Highway Traffic Safety Administration; National Center for Statistics and Analysis, Crash Investigation Division; 1998. 27. Champion HR, Copes WS, Sacco WJ, et al. The major trauma outcome study: establishing national norms for trauma care. J Trauma. 1990;30:1356-1365. 28. Furnival RA, Schunk JE. ABCs of scoring systems for pediatric trauma. Pediatr Emerg Care. 1999;15:215-223. 29. Eichelberger MR, Mangubat EA, et al. Outcome analysis of blunt injury in children. J Trauma. 1988;28:1109-1117. 30. Korn EL, Graubard BI. Epidemiologic studies utilizing surveys: accounting for the sampling design. Am J Public Health. 1991;81:1166-1173.

45. Grant RJ, Gregor MA, Beck P, et al. A comparison of data sources for motor vehicle crash characteristic accuracy. Acad Emerg Med. 2000;7:892-897. 46. Laupacis A, Sekar N, Stiell IG. Clinical prediction rules–a review and suggested modifications of methodological standards. JAMA. 1997;277:488-494. 47. Henry MC, Alicandro JM, Hollander JE, et al. Evaluation of American College of Surgeons trauma triage criteria in a suburban and rural setting. Am J Emerg Med. 1996;14:124-129. 48. Moront ML, Gotschall CS, Eichelberger MR. Helicopter transport of injured children: system effectiveness and triage criteria. J Pediatr Surg. 1996;31:1183-1188. 49. Dischinger PC, Siegel JH, Ho SM, et al. Effect of change in velocity on the development of medical complications in patients with multisystem trauma sustained in vehicular crashes. Accid Anal Prev. 1998;30:831-837. 50. Jones IS, Champion HR. Trauma triage: vehicle damage as an estimate of injury severity. J Trauma. 1989;29:646-653. 51. Norcross ED, Ford DW, Cooper ME, et al. Application of American College of Surgeons’ field triage guidelines by out-of-hospital personnel. J Am Coll Surg. 1995;181:539-544. 52. Gerndt SJ, Conley JL, Lowell MJ, et al. Out-of-hospital classification combined with an inhospital trauma radio response reduces cost and duration of evaluation of the injured patient. Surgery. 1995;118:789-796. 53. Simmons E, Hedges JR, Irwin L, et al. Paramedic injury severity perception can aid trauma triage. Ann Emerg Med. 1995;26:461-468. 54. Fries GR, McCalla G, Levitt MA, et al. A prospective comparison of paramedic judgement and the trauma triage rule in the out-of-hospital setting. Ann Emerg Med. 1994;24:885-889. 55. Baxt WG, Upenieks V. The lack of full correlation between the injury severity score and the resource needs of injured patients. Ann Emerg Med. 1990;19:1396-1400. 56. Baker SP, O’Neil B, Haddon W Jr, et al. The injury severity score: a method for describing patients with multiple injuries and evaluating emergency care. J Trauma. 1974;14:187-196.

31. Lemeshow S, Letenneur L, Dartigues J, et al. Illustration of analysis taking into account complex survey considerations: the association between wine consumption and dementia in the PAQUID study. Am J Epidemiol. 1998;148:298-306. 32. Breiman L, Freidman JH, Olshen RA, et al. Classification and Regression Trees. New York, NY: Chapman & Hall; 1984. 33. Lewis RJ, Newgard CD. An error in research: admission, anxiety, and action. Acad Emerg Med. 2000;7:1177-1179. 34. Newgard CD, Lewis RJ. The National Accident Sampling System (NASS) database: an example of the pitfalls of analyzing weighted databases [abstract-errata]. Acad Emerg Med. 2000;7:1179. 35. Newgard CD, Lewis RJ. Use of prehospital variables to predict significant spinal injury in children involved in motor vehicle collisions [abstract]. Acad Emerg Med. 2000;7:480-481. 36. Efron B. The jackknife, the bootstrap and other resampling plans. In: The Regional Conference Series in Applied Mathematics. Philadelphia, PA: Society for Industrial and Applied Mathematics; 1982. 37. Stone M. Cross-validatory choice and the assessment of statistical predictions. J Roy Statist Soc Ser B. 1974;36:111-147.

4 9 0

ANNALS OF EMERGENCY MEDICINE

39:5 MAY 2002

PREDICTING SEVERITY OF INJURY IN PEDIATRIC PATIENTS Newgard, Lewis & Jolly

APPENDIX. National Automotive Sampling System Crashworthiness Data System database. The NASS CDS database is operated and maintained by the National Center for Statistics and Analysis, which operates under the National Highway Traffic Safety Administration.26 To qualify for entry into the NASS CDS database, the MVC must have a police report, be reported to the state, involve a “harmful event” (defined as property damage, personal injury, or both) and occur as a result of an “accident” defined as at least one harmful event produced by an unstabilized situation. Crashes resulting from a diseased condition (ie, cerebral hemorrhage, myocardial infarction, seizure), deliberate intent, or an ongoing natural cataclysm (ie, earthquake, cyclone, flood, hurricane) are excluded. The crash must involve a motor vehicle in transport, must have occurred on a public trafficway, and must involve at least one towed vehicle. One of 24 teams of crash researchers throughout the country is tasked with investigation and data collection for these crashes. At each sampling site the research team investigates a sample of the respective police reported crashes. The investigation consists of a detailed review of police accident reports, hospital records, out-of-hospital care records, photographs of the vehicles, and the vehicles themselves. Because the NASS CDS database only represents a sampling of MVCs, the data are then statistically weighted to represent the nationwide incidence of crashes and resulting injuries.26 The sampling process is accomplished in stages. The first stage consists of dividing the country into equal regions (primary sample units [PSU]) on the basis of the geographic region and degree of urbanization. Twenty-four PSUs are selected by a proportional probability to represent all the PSUs. The 24 investigational research teams correspond to the 24 PSUs. Because it is not practical to investigate every crash in a given PSU, there is a second stage of sampling. In this stage, the police agencies within a PSU are categorized by the number and type of police accident reports they produce, then agencies are randomly selected by a factor that allows for appropriate representation of the number and severity of crashes in that PSU. The final sampling stage is made within these selected police agencies. All police-reported crashes are classified and stratified by 5 criteria in an attempt to establish a sample that is representative of all types of crashes and injuries. These 5 criteria are: type of vehicle (eg, passenger cars, light trucks, vans); tow status of vehicle (must have been towed to allow data collection from the vehicle); most severe police-reported injury (fatalities at the scene are not included in the NASS CDS database); disposition of the injured (whether or not the patient is taken to a treatment facility); and model year of the vehicle (“late model year vehicles” produced from 1993 to 1999 or “nonlate model year vehicles” produced during 1992 or before). Each category is assigned a weight factor to allow representation of all types and severities of crashes in the United States. A random selection is made from within each category to produce the cases included in the NASS CDS database. If the passenger is taken to a treatment facility, there is a “second phase of sampling” and stratification to ensure an adequate representation of all types and severity of injuries. Patients who were treated and released (ie, no overnight hospital stay) are included in the NASS CDS database; however, patients not evaluated at a medical facility after the crash are not represented in the database.

MAY 2002

39:5

ANNALS OF EMERGENCY MEDICINE

4 9 1