- Email: [email protected]
Prognosticating Clinical Prediction Scores Without Clinical Gestalt for Patients With Chest Pain in the Emergency Department
The Journal of Emergency Medicine, Vol. -, No. -, pp. 1–10, 2017 Ó 2017 Elsevier Inc. All rights reserved. 0736-4679/$ - see front matter
https://doi.org/10.1016/j.jemermed.2017.10.006
Original Contributions
PROGNOSTICATING CLINICAL PREDICTION SCORES WITHOUT CLINICAL GESTALT FOR PATIENTS WITH CHEST PAIN IN THE EMERGENCY DEPARTMENT Chin Pang Wong, MBCHB, MCEM,* Chun Tat Lui, FHKCEM, FHKAM,* Jonathan Gabriel Sung, MBCHB, MCRP,† Ho Lam, FHKCP, FHKAM,† Hin Tat Fung, FHKCEM, FHKAM,* and Ping Wa Yam, FHKCP, FHKAM† *Accident and Emergency Department, Tuen Mun Hospital, Hong Kong and †Division of Cardiology, Department of Medicine and Geriatrics, Tuen Mun Hospital, Hong Kong Reprint Address: Chun Tat Lui, FHKCEM, FHKAM, Accident and Emergency Department, Tuen Mun Hospital, Hong Kong
, Abstract—Background: Assessment of patients with chest pain is a regular challenge in the emergency department (ED). Recent guidelines recommended quantitative assessment of ischemic risk by means of risk scores. Objective: Our aim was to assess the performance of Thrombosis in Myocardial Infarction (TIMI); Global Registry of Acute Coronary Events (GRACE); history, electrocardiogram, age, risk factors, and troponin (HEART) scores; and the North America Chest Pain Rule (NACPR) without components of clinical gestalt in predicting 30-day major adverse cardiac events (MACE). Methods: We performed a prospective cohort study in adult patients who attended the ED with undifferentiated chest pain. Clinical prediction rules were applied and calculated. The clinical prediction rules were modified from the original ones, excluding components requiring judgment by clinical gestalt. The primary outcome was MACE. Performance of the tests were evaluated by receive operating characteristic curves and the area under curves (AUC). Results: There were 1081 patients included in the study. Thirty-day MACE occurred in 164 (15.2%) patients. The AUC of the GRACE score was 0.756, which was inferior to the TIMI score (AUC 0.809) and the HEART score (AUC 0.845). A TIMI score $ 1 had a sensitivity of 97% and a specificity of 45.7%. A GRACE score $ 50 had a sensitivity of 99.4% and a specificity of 7.5%. A HEART score $ 1 had a sensitivity of 98.8% and
a specificity of 11.7%. The NACPR had a sensitivity of 93.3% and a specificity of 51.5%. Conclusions: Without clinical gestalt, the modified HEART score had the best discriminative capacity in predicting 30-day MACE. Ó 2017 Elsevier Inc. All rights reserved. , Keywords—acute coronary syndrome; chest pain; decision support techniques; myocardial infarction
INTRODUCTION Patients with undifferentiated chest pain account for a significant proportion of attendance and burden of the emergency department (ED) (1). Prompt and accurate diagnosis of acute coronary syndrome (ACS) is crucial for patients’ immediate management to achieve better outcome. However, within the heavy patient load, genuine cardiac events contribute to only a minor proportion (2,3). It is well known that normal initial electrocardiogram (ECG) and biomarker do not exclude ACS. As a result, serial blood tests and investigations are required. This leads to prolonged length of stay and ED overcrowding, which are of tremendous concern (4). In the era of high-sensitive troponin, the falsepositive rate further aggravates the burden to the health care system. Therefore, an objective, reproducible tool for clinical risk stratification is useful to tackle this
This research project has been presented on October 28–29, 2016 in the Scientific Symposium on Emergency Medicine of Hong Kong.
RECEIVED: 20 June 2017; FINAL SUBMISSION RECEIVED: 6 September 2017; ACCEPTED: 7 October 2017 1
2
C. P. Wong et al.
diagnostic challenge. As stated in their recent guidelines, both the American Heart Association and the European Society of Cardiology recommended the use of riskstratification models to guide management in patients with chest pain (5,6). It ensures rapid accurate diagnosis of ACS and appropriate discharge of low-risk patients. These risk scores can also be used to assess prognosis in ACS patients. However, most of the existing scores contained elements of clinical gestalt with subjective input from attending physicians, which may affect the consistency and reproducibility of the scores. In this study, we compared the diagnostic accuracies of four commonly used scores, with removal of the components of clinical gestalt. The assessed scores are the Thrombosis in Myocardial Infarction (TIMI) score; the Global Registry of Acute Coronary Events (GRACE) score; the history, electrocardiogram, age, risk factors, and troponin (HEART) score; and the North America Chest Pain rule (NACPR) (7–10). Initially, the TIMI and the GRACE scores were developed for post-ACS prognostication, these scores were then validated for the prediction of major adverse cardiac events (MACE) for patients with undifferentiated chest pain (11). Our aim was to identify the best clinical prediction score without components of clinical gestalt for early and safe discharge of low-risk patients. MATERIALS AND METHODS
risk factors were recorded. The presenting vital signs were recorded for score calculations. A standard 12-lead ECG was performed for each included patient. We adopted troponin as the cardiac marker in calculation of scores. The troponin level was measured after initial assessment by the attending clinician. We used the Abbott ARCHITECT STATÒ high-sensitive troponin I assay in our study. A level > 99th percentile was considered positive. Sex-specific cutoffs were adopted according to manufacturer’s recommendation. The upper reference limits were 34.2 ng/L for male and 15.6 ng/L for female. The coefficient of variation of the troponin assay was < 4% and the lower limit of detection was 10 ng/L. Outcome variables were traced by reviewing all patients’ written and electronic hospital records, laboratory test results, intervention reports, and death registries. Clinical Prediction Scores The components of the evaluated clinical prediction models were shown in Table 1. The initial definitions of these scores included subjective components that required the attending clinician’s subjective judgment. In our study, we made modifications to these scores so that these subjective components were omitted. In this manner, we were able to explore the accuracy risk scores based only on objective parameters without clinical gestalt.
Study Design and Setting
TIMI Score
We performed a prospective cohort study in the ED of a tertiary referral hospital with daily attendance of > 600 patients. The study period was from February 2016 to June 2016. Patients aged > 18 years who complained of chest pain in the triage were included. Twelve-lead ECGs were obtained. Patients were excluded if STelevation myocardial infarction (STEMI) was diagnosed, or there were clearly established alternative diagnoses on presentation not related to cardiac ischemia (e.g., aortic dissection, pulmonary embolism, pneumothorax, and herpes zoster). Ethics approval was obtained from the local Institutional Review Board (CREC/16093). Verbal consent was obtained from the participants of the study.
The TIMI score for unstable angina/non-STEMI was introduced by Antman et al. in 2000 (7). It consists of seven risk factors weighed one mark each. Total marks of one or less indicate lower risk at 14 days in terms of all-cause mortality, acute myocardial infarction (AMI), and severe recurrent ischemia prompting urgent revascularization. In our study, we had modified ‘‘the use of aspirin in the past 7 days’’ into ‘‘the use of antiplatelets in the past 7 days.’’ Patients on aspirin, clopidogrel, Ticagrelor, and other antiplatelets were defined as positive exposures to antiplatelets. We omitted the item ‘‘severe angina $2 episodes in 24 hours,’’ which involved the clinician’s subjective judgment. It was also inapplicable for cross-sectional risk stratification of chest pain patients in the ED in a single time point.
Data Collection Data were collected prospectively in form of standardized data collection sheets filled by the attending clinician. Consecutive samples were recruited. Prior training sessions had been arranged for the clinicians in the study center for familiarization of the data collection form. Patients’ demographic data, relevant medical history, drug history, smoking status, and other cardiovascular
GRACE Score This score was developed by the Global Registry of Acute Coronary Events. It is an international registry designed to track in-hospital and long-term outcomes of patients presenting with ACS (12). It comprises 8 components with different weighing. Web-based calculator is readily
Prognosticating Scores for Chest Pain
3
Table 1. Components and Definitions of Clinical Prediction Scores for Chest Pain Risk Stratification Prediction Rules TIMI score*
GRACE score
HEART score†
North America Chest Pain Rulek
Items
Score
Age $ 65 years Three or more risk factors for CAD Family history of CAD Hypertension Hyperlipidemia Diabetes mellitus Tobacco use ST deviation $ 0.5 mm Elevated cardiac marker level Known CAD (stenosis $ 50%) Aspirin or other antiplatelet use in the past 7 days Age Heart rate Systolic blood pressure Creatinine Congestive heart failure: Killip class I/II/III/IV Cardiac arrest at presentation ST-segment deviation Elevated cardiac enzymes/ markers ECG Significant ST depression‡ Nonspecific repolarization disturbance/LBBB/LVH Normal Age $ 65 years 45–65 years # 45 years Risk factors $ 3 risk factors§ or history of atherosclerotic disease 1 or 2 risk factors No risk factors known Troponin $ 3 normal limit 13 normal limit Less than or equal normal limit New ischemia on initial ECG{ History of coronary artery disease Initial cardiac troponin is positive and Age # 40 years or Age 4150 years and repeat troponin at least 3 h from symptom onset is negative.
1 1
available for its complicated score calculation. The probability of death or death/AMI occurring up to given time points after admission will be shown after corresponding parameters are entered (13). The Killip class was determined during the initial patient assessment by the attending clinician. HEART Score
1 1 1 1 0–100 0–46 0–58 1–28 0–59 39 28 14 2 1 0 2 1 0 2 1 0 2 1 0
CAD = coronary artery disease; ECG = electrocardiogram; GRACE = Global Registry of Acute Coronary Events; LBBB = left bundle branch block; LVH = left ventricular hypertrophy; TIMI = Thrombosis in Myocardial Infarction. * Excluded the component ‘‘Severe angina $ 2 episodes in 24 hours.’’ † Excluded the component ‘‘History of highly, moderately and slightly suspicious.’’ ‡ ST segment depression $ 0.05 mV in two or more contiguous leads, in the appropriate clinical context or ST depression $ 0.1 mV. § Risk factors: hypercholesterolemia, hypertension, diabetes mellitus, cigarette smoking, positive family history, and obesity. k Excluded the component of ‘‘Pain is typical for acute coronary syndrome.’’ Repeat troponin is changed from original 6 h to 3 h with adoption of high-sensitive troponin assay in the current study. { ST segment deviation $ 1 mm or T-wave inversion $ 0.2 mm in at least two contiguous leads.
The HEART score was developed by Six et al. in 2008 (9). It was intended for aiding the diagnosis of ACS in the ED. It comprises five predictors with the same weighing. Each predictor is given 0–2 points, depending on the factor severity. It uses MACE as the clinical end point. In our study, we omitted the component ‘‘highly, moderately and slightly suspicious history,’’ which required subjective clinician judgment. NACPR This rule was also developed for risk stratification of patients with undifferentiated chest pain in the ED. It was published in 2015 by Hess et al. (10). They aimed to codify the existing common practice to identify low-risk patients for safe discharge. It includes five clinical criteria. A patient with chest pain can be discharged safely if all of these criteria are negative. We omitted the criterion ‘‘Pain is typical for ACS’’ in our study because it was a subjective component. In addition, the time interval of serial troponin levels was changed from 6 h to 3 h in the current study to adopt the recommendation from the manufacturer of high-sensitive troponin assays. Outcomes The primary outcome was MACE at 30 days. MACE were defined as AMI, percutaneous coronary intervention, coronary bypass graft, and death within 30 days of presentation to the ED. The diagnosis of AMI for the index attendance of the patient was based on the third universal definition, assessed independently by the attending clinician and an independent blinded cardiologist (14). In case of disagreement, third adjudication from an independent cardiologist would be sought. In Hong Kong, emergency medical services would deliver patients to 1 of 18 EDs under the Hospital Authority. All cardiac events or attendance of EDs were traced from the corporate electronic patient record system from all 18 acute hospitals in Hong Kong. Deaths were confirmed with the Death Registry under the Department of Health of Hong Kong. The outcome assessment was performed after patient discharge and reconfirmed 6 months after completion of case recruitment through the electronic patient record system.
4
C. P. Wong et al.
Statistical Analysis MedCalc for Windows, version 9.6.4.0 (MedCalc Software, Ostend, Belgium) and SPSS (IBM SPSS Statistics for Windows, Version 22.0, IBM Corp, Armonk, NY) were employed for analysis. The statistical analysis was performed by one of the investigators, independent of the outcome assessors and attending physicians. Age, as a continuous variable, was expressed in mean and standard deviation with normal distribution, and compared by independent sample t-test. Categorical variables were shown in proportions and percentages. Their comparisons were made by c2 test or Fisher exact test, where appropriate. A p value < 0.05 was taken to imply statistical significance. The receiver operating characteristic (ROC) curve for the prediction of the outcome was computed for each score. The area under curve (AUC) was evaluated for diagnostic accuracies. Pairwise comparisons of AUC were conducted for different scores using nonparametric approach (15). The sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), positive likelihood ratio (PLR), and negative likelihood ratio (NLR) for the primary outcome at different risk score cutoff values were calculated. Power analysis was performed with PASS 2011 software (NCSS, Kaysville, UT). RESULTS
Figure 1. Enrollment and outcome of the study cohort. AMI = acute myocardial infarction; MACE = major adverse cardiac events; NSTEMI = non–ST-elevation myocardial infarction; STEMI = ST-elevation myocardial infarction.
had a statistically significant higher rates of 30-day MACE.
Patient Enrollment and Outcome The flow of study subjects was shown in Figure 1. A total of 1081 patients were included in the study, of which 52.3% were male. Thirty-day MACE occurred in 164 patients (15.2%). Non-STEMI and unstable angina were diagnosed in 104 and 54 patients, respectively, for the index attendance and admission. There was one case reattended within 30 days for ventricular fibrillation and mortality. There were five cases of non-fatal AMI within 30 days requiring vascular interventions. Clinical Risk Factors, ECG Features, and Biomarker Level Patients’ baseline characteristics and risk factors were analyzed for the primary outcome (30-day MACE). Advanced age, ever-smoker, hypertension, diabetes mellitus, hyperlipidemia, known coronary artery disease, and renal impairment were found statistically significant as risk factors of 30-day MACE (Table 2). Patients who were on antiplatelets, with ST depression in ECG, with new onset of T-wave changes in ECG, with the first high-sensitive troponin I > 99th percentile and with the repeated high-sensitive troponin I > 99th percentile
Proportion of 30-Day MACE in Different Ranges of Risk Scores Figure 2 shows the bar chart representing the correlation between different scores and 30-day MACE. All risk scores had a positive correlation with the primary outcome, that is, the higher the score, the greater the proportion of 30-day MACE. Comparison of TIMI, GRACE, HEART, and NACPR Figure 3 shows ROC curves of the 3 predictive scores. The AUC of HEART score was 0.845 (95% confidence interval [CI] 0.812–0.878); TIMI score was 0.809 (95% CI 0.777–0.841); and GRACE score was 0.756 (95% CI 0.717–0.795). Pairwise comparison with nonparametric method demonstrated statistically significant difference in AUCs (TIMI > GRACE, AUC difference 0.053 [95% CI 0.013–0.094], p = 0.01; HEART > TIMI, AUC difference 0.036 [95% CI 0.004–0.068], p = 0.027; HEART > GRACE, AUC difference 0.089 [95% CI 0.054–0.125]; p < 0.001). Table 3 shows the sensitivity, specificity, PPV, NPV, PLR, and the NLR. As we aim to identify low-risk
Prognosticating Scores for Chest Pain
5
Table 2. Comparison of Demographic, Clinical, Electrocardiogram Features, and Biomarkers in Patients With and Without 30-Day Major Adverse Cardiac Events 30-Day MACE Parameters Demographics Age, mean 6 SD Age $ 65 years, n (%) Sex, male, n (%) Risk factors and comorbidities, n (%) Family history of early-onset CAD Current smoker Ever smoker Hypertension Diabetes mellitus Lipid Known CAD Renal impairment On pacemaker Current medications, n (%) On antiplatelets On warfarin On novel oral anticoagulants Duration of chest pain, h, median (IQR) Presenting vital signs, mean 6 SD Systolic blood pressure, mm Hg Diastolic blood pressure, mm Hg Heart rate, beats/min Killip heart failure class I II III ECG features, n (%) ST elevation $ 0.5 mm ST depression $ 0.5 mm New-onset T-wave changes Troponin level, n (%) First troponin > 99th percentile Repeated troponin > 99th percentile Creatinine, mmol/L, median (IQR)
All (n = 1081)
Yes (n = 164)
No (n = 917)
p Value
48 6 27 443 (41) 565 (52.3)
59 6 23 89 (54.3) 97 (59.1)
46 6 27 354 (38.6) 468 (51)
< 0.001 < 0.001 0.055
4 (0.4) 147 (13.6) 298 (27.6) 515 (47.6) 213 (19.7) 412 (38.1) 230 (21.3) 151 (14) 16 (1.5)
1 (0.6) 31 (18.9) 66 (40.2) 105 (64) 57 (34.8) 107 (65.2) 72 (43.9) 53 (32.3) 4 (2.4)
4 (0.4) 116 (12.6) 232 (25.3) 410 (44.7) 156 (17) 305 (33.3) 158 (17.2) 98 (10.7) 12 (1.3)
1.000 0.031 < 0.001 < 0.001 < 0.001 < 0.001 < 0.001 < 0.001 0.285
346 (32) 18 (1.7) 10 (0.9) 7.5 (1.8–62.9)
89 (54.3) 2 (1.2) 1 (0.6) 7.2 (1.8–60.9)
257 (28) 16 (1.7) 9 (1) 11.1 (1.9–65.4)
< 0.001 1.000 1.000 0.194
147 6 27 81 6 16 82 6 16
149 6 28 81 6 17 82 6 16
139 6 22 82 6 15 85 6 16
< 0.001 0.834 0.324 < 0.001
1024 (94.7) 28 (2.6) 29 (2.7)
142 (86.6) 9 (5.5) 13 (7.9)
882 (96.2) 19 (2.1) 16 (1.7)
10 (0.9) 36 (3.3) 61 (5.6)
4 (2.4) 23 (14) 32 (19.5)
6 (0.7) 13 (1.4) 29 (3.2)
0.051 < 0.001 < 0.001
80 (8.7) 91 (9.9) 69 (62–79)
< 0.001 < 0.001 < 0.001
199 (18.4) 229 (21.2) 76 (66–94)
119 (72.6) 138 (84.1) 80 (68–100)
CAD = coronary artery disease; EGC = electrocardiogram; IQR = interquartile range; MACE = major adverse cardiac event; SD = standard deviation.
patients for safe discharge, we are looking for the cutoff points at the best sensitivity of various risk scores. A TIMI score $ had a sensitivity of 97% (95% CI 92.7– 98.9%) and a specificity of 45.7% (95% CI 42.4–49%). A GRACE score $50 had a sensitivity of 99.4% (95% CI 96.1–100%) and a specificity of 7.5% (95% CI 5.9– 9.5%). A HEART score $1 had a sensitivity of 98.8% (95% CI 95.2–99.8%) and a specificity of 11.7% (95% CI 9.7–14%). The NACPR had a sensitivity of 93.3% (95% CI 66–99.7%) and specificity of 51.5% (95% CI 44.9–58%). Power Analysis A sample of 164 from the positive group and 917 from the negative group achieved 99% and 67% power to detect the differences between a diagnostic test with an AUROC of 0.845 and another diagnostic test with an AUC of 0.756
and 0.809 using a two-sided z-test at a significance level of 0.05, for continuous scores. DISCUSSION Recent recommended models for risk stratifications of chest pain patients involves adoption of a validated risk assessment score (5,6). Existing scores, such as HEART score, integrated components of clinical gestalt with attending subjective input from physicians (9). There is doubt in the reproducibility and consistency of the scores with evaluation from different physicians, which contributes to variations in standardization of the risk stratification of chest pain. Clinical risk scores should be applicable only for the group with undifferentiated chest pain. For patients in obvious clinical unstable angina or other diseases causing chest pain, such as pneumothorax, clinical decision scores should not be applied, but patient
6
C. P. Wong et al.
Figure 2. Proportion of 30-day major adverse cardiac events (MACE) in different ranges of each clinical prediction rule. GRACE = Global Registry of Acute Coronary Events; HEART = history, electrocardiogram, age, risk factors, and troponin; TIMI = Thrombosis in Myocardial Infarction.
should be treated directly. For the diagnosis of ‘‘undifferentiated chest pain,’’ which implies the attending physician could not identify the apparent cause, further input of clinical gestalt in the clinical risk scores would commit logical fallacy. Instead, clinical scores with objective components should be sought for standardized stratification of the ischemic risk. We have made modifications to the existing risk scores in our study. All components that required patients’ subjective recalling or the attending clinician’s judgment were omitted. This prospective study illustrated the accuracy of clinical prediction scores without subjective clinical gestalt components in risk stratification of chest pain patients in ED. Scores without clinical gestalt would be expected to have better reproducibility and reliability with lower inter-observer bias. All risk scores remained in good diagnostic accuracies as reflected by the AUC (0.756–0.845). Their sensitivities remained high (93.3–99.4%), which ensured the safe discharge of low-risk patients. This might potentially change the current practice as the risk stratification might depend only on discreet information. Time might be saved at the initial patient assessment, as the clinician might not need to clarify much on the severity and the nature of pain. These reporting of pain intensity from patients were often inaccurate and difficult to quantify.
The evaluation of these parts of risk scores might also subject to variation depending on the clinician’s experience. Therefore, our finding suggested that a risk score based solely on objective parameters without clinical gestalt is practical to identify low-risk patients for early discharge from the ED. In this study, 164 of 1081 patients (15.2%) had reached the primary outcome. This rate was in keeping with prior studies of similar nature (13–17%) (16,17). Generally speaking, the TIMI, GRACE, and HEART scores all performed well in predicting 30-day MACE with respect to the AUROC curve. This demonstration of safety of these scores was also shown in previous studies (18–20). We found that the HEART score had the best discriminative capacity (AUC 0.845), followed by the TIMI score (AUC 0.809). The GRACE score had a relatively inferior performance (AUC 0.756). This concurred with previous studies (21). Our postulation is that for the GRACE score, it does not involve medical risk factors (e.g., hypertension and diabetes mellitus) in the calculations. Unlike the other two risk scores, the GRACE score is calculated based on the clinical findings in a single time point only. In fact, cardiovascular risk factors do predict the likelihood of MACE/ACS well.
Prognosticating Scores for Chest Pain
7
Figure 3. Receiver operating characteristic curve of predictive scores predicting 30-day major adverse cardiac events. GRACE = Global Registry of Acute Coronary Events; HEART = history, electrocardiogram, age, risk factors, and troponin; TIMI = Thrombosis in Myocardial Infarction.
In this study, the HEART score had the best performance in predicting 30-day MACE. For a HEART score $ 1, its sensitivity, NPV, and NLR were 98.8%, 98.2%, and 0.1, respectively. For a HEART score $ 2, its sensitivity, NPV, and NLR were 98.2%, 99%, and 0.1, respectively. This was in keeping with various validation studies using 30-day MACE as the primary outcome. In these preceding literature, the sensitivity and NPV reached 99% and 98–98.2%, respectively (16,21,22). The HEART score involves graded calculation of each component from 0–2 points. This slightly complicated calculation might contribute to its superiority to TIMI. It is not unexpected that HEART outperformed TIMI and GRACE, as HEART was originally derived to predict MACE, while TIMI and GRACE were
originally derived for prognostication of patients with unstable angina and non-STEMI. For the TIMI score, its diagnostic accuracy came next to the HEART score. A TIMI score $ 1 showed a sensitivity, NPV, and NLR of 97%, 98.8%, and 0.1, respectively. These results concurred with prior similar studies (21,23). A TIMI score $ 2 gave a better specificity (68.8%), but compromised the sensitivity significantly (76.8%). Therefore, only the cutoff score of 1 was useful in identifying low-risk patients for discharge. The GRACE score showed inferiority in term of diagnostic accuracy in this study. Its sensitivity, NPV and NLR at cutoff of $ 50 were 99.4%, 98.6%, and 0.1, respectively. It gave a lower sensitivity (92.1%) at the $ 75 cutoff. Although shown inferior in terms of AUC, it did perform well with high sensitivity and NPV. The NACPR had sensitivity inferior to other risk scores in this study. The sensitivity, NPV, and NLR were 93.3%, 99.2%, and 0.1, respectively. Regarding the original study, for the cutoff at age # 40 years, the sensitivity and NPV were both up to 100% (10). This discrepancy might possibly occur because of omitting one of the five items ‘‘Pain is typical for ACS.’’ The best of clinical scoring systems for risk stratification would theoretically had the highest sensitivities and NPV in order not to miss patients with ACS. Cutoff of the prediction models had been set to achieve the lowest false-negative rate. However, in the reality situation, one may need to consider from the view of costeffectiveness and resources utilization, particularly in the less-wealthy cities. For example, with the HEART score, HEART score $ 1 achieved 98.8% sensitivity with low specificity of 11.7%. If we sacrificed sensitivity to 98.2% by adopting cutoff of 2, specificity increased dramatically to 32.8%, with much lower rate of falsepositive prediction. Rapid risk stratification of chest pain patients in ED is always a challenge. Recent attempts to further enhance the accuracies of the predictive models include dual
Table 3. Diagnostic Characteristics of Clinical Prediction Scores Predicting 30-Day Major Adverse Cardiac Events at Various Cutoffs Variable
Sensitivity, % (95% CI)
Specificity, % (95% CI)
PPV, % (95% CI)
NPV, % (95% CI)
PLR (95% CI)
NLR (95% CI)
TIMI $ 1 TIMI $ 2 GRACE $ 50 GRACE $ 75 GRACE $ 100 HEART $ 1 HEART $ 2 NACPR
97 (92.7–98.9) 76.8 (69.5–82.9) 99.4 (96.1–100) 92.1 (86.5–95.5) 76.2 (68.8–82.4) 98.8 (95.2–99.8) 98.2 (94.3–99.5) 93.3 (66–99.7)
45.7 (42.4–49) 68.8 (65.7–71.8) 7.5 (5.9–9.5) 32.5 (29.5–35.7) 61.9 (58.7–65.1) 11.7 (9.7–14) 32.8 (29.8–36) 51.5 (44.9–58)
24.2 (21–27.7) 30.6 (26.2–35.3) 16.1 (13.9–18.6) 19.6 (16.9–22.6) 26.4 (22.5–30.6) 16.7 (14.4–19.2) 20.7 (18–23.8) 10.9 (6.3–18)
98.8 (97.1–99.6) 94.3 (92.2–95.9) 98.6 (91.2–99.9) 95.8 (92.8–97.7) 93.6 (91.2–95.3) 98.2 (92.9–99.7) 99 (96.9–99.7) 99.2 (94.8–100)
1.8 (1.7–1.9) 2.5 (2.2–2.8) 1.1 (1.1–1.1) 1.4 (1.3–1.5) 2 (1.8–2.3) 1.1 (1.1–1.2) 1.5 (1.4–1.5) 1.9 (1.6–2.3)
0.1 (0–0.2) 0.3 (0.3–0.4) 0.1 (0–0.6) 0.2 (0.1–0.4) 0.4 (0.3–0.5) 0.1 (0–0.4) 0.1 (0–0.2) 0.1 (0–0.9)
CAD = coronary artery disease; CI = confidence interval; GRACE = Global Registry of Acute Coronary Events; HEART = history, electrocardiogram, age, risk factors, and troponin; NACPR = North America Chest Pain Rule; NLR = negative likelihood ratio; NPV = negative predictive value; PLR = positive likelihood ratio; PPV = positive predictive value; TIMI = Thrombosis in Myocardial Infarction.
8
C. P. Wong et al.
cardiac marker strategies, combination of two risk scores, and accelerated diagnostic protocols (22,24,25). Accelerated diagnostic protocol integrating HEART score with objective components and newer generation of cardiac markers may be the future direction to explore for a more accurate predictive model, thus enhancing the efficiency of chest pain stratification and possibly reducing hospital admissions.
Chun Tat managed the statistics method and analyzed the data. Wong Chin Pang drafted the manuscript and all authors contributed substantially to its revision. Wong Chin Pang takes responsibility for the paper as a whole. All authors had substantial contribution in authorship of the paper in terms of idea generation, study design and logistics, data collection and monitoring, analysis, and manuscript drafting.
REFERENCES
Limitations Firstly, there were no universal clinical follow-ups for included patients for the assessment of the outcome (30-day MACE). All cardiac events or attendance of EDs were traced from the corporate electronic patient record system from all acute hospitals in Hong Kong. However, patients may still have events not traceable in the system when they attended private non-acute hospitals. Secondly, we aimed to evaluate the accuracy of risk scores without subjective component and clinical gestalt. In the GRACE score, the item Killip class involved clinician’s judgment by physical examination on whether the patient was in heart failure. It was subjected to interassessor variation. Objective assessments of heart failure, such as echocardiogram and natriuretic peptide level, were not adopted. Same problem existed in the HEART score. The attending clinician had to judge whether the patient was obese or not in order to give a score on the ‘‘risk factor’’ item. As there was no definition of obesity given in the HEART score, it was also a subjective judgment made by the attending clinician. Thirdly, highsensitive troponin I was adopted in this study. It might not be universally applicable to other centers using other types of biomarker assays. Lastly, it was a study that recruited patients in only one regional hospital. The CI of the current study remained variable, particularly with unbalanced number of outcome events. Further multicenter research with larger sample size is needed for validation and demonstration of universal applicability. CONCLUSIONS The modified HEART score without components of clinical gestalt had the best discriminative capacity in predicting 30-day MACE. Acknowledgments—Author contributions: Lui Chun Tat, Lam Ho, Wong Chin Pang and Jonathan Gabriel Sung conceived the study and designed the research project. Lui Chun Tat formulated the research question and supervised the conduct of the research project. Wong Chin Pang and Jonathan Gabriel Sung performed the data collection. Wong Chin Pang, Jonathan Gabriel Sung, Lui Chun Tat, and Lam Ho undertook recruitment of patients and managed the data, including quality control. Lui
1. Goodacre S, Cross E, Arnold J, et al. The health care burden of acute chest pain. Heart 2005;91:229–30. 2. Launbjerg J, Fruergaard P, Hesse B, et al. Long-term risk of death, cardiac events and recurrent chest pain in patients with acute chest pain of different origin. Cardiology 1996;87:60–6. 3. Hollander JE. Risk stratification of emergency department patients with chest pain: the need for standardized reporting guidelines. Ann Emerg Med 2004;43:68–70. 4. Bernstein SL, Aronsky D, Duseja R, et al. The effect of emergency department crowding on clinically oriented outcomes. Acad Emerg Med 2009;16:1–10. 5. Amsterdam EA, Wenger NK, Brindis RG, et al. 2014 AHA/ ACC guideline for the management of patients with non-STelevation acute coronary syndromes: executive summary: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. Circulation 2014;130:2354–94. 6. Roffi M, Patrono C, Collet JP, et al. Management of acute coronary syndromes in patients presenting without persistent ST-segment elevation of the European Society of Cardiology. Eur Heart J 2016;37:267–315. 7. Antman EM, Cohen M, Bernink PJ, et al. The TIMI risk score for unstable angina/non-ST elevation MI: a method for prognostication and therapeutic decision making. JAMA 2000;284:835–42. 8. Granger CB, Goldberg RJ, Dabbous O, et al. Predictors of hospital mortality in the global registry of acute coronary events. Arch Intern Med 2003;163:2345–53. 9. Six AJ, Backus BE, Kelder JC. Chest pain in the emergency room: value of the HEART score. Neth Heart J 2008;16:191–6. 10. Hess EP, Brison RJ, Perry JJ, et al. Development of a clinical prediction rule for 30-day cardiac events in emergency department patients with chest pain and possible acute coronary syndrome. Ann Emerg Med 2012;59:115–25. 11. Ramsay G, Podogrodzka M, McClure C, et al. Risk prediction in patients presenting with suspected cardiac pain: the GRACE and TIMI risk scores versus clinical evaluation. QJM 2007;100:11–8. 12. Granger CB. Strategies of patient care in acute coronary syndromes: rationale for the Global Registry of Acute Coronary Events (GRACE) registry. Am J Cardiol 2000;86:4M–9. 13. GRACE 2.0 Calculator. Available at: http://www.gracescore.org/ WebSite/WebVersion.aspx. Accessed September 20, 2016. 14. Thygesen K, Alpert JS, Jaffe AS, et al. Third universal definition of myocardial infarction. Circulation 2012;126:2020–35. 15. DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 1988;44:837–45. 16. Backus BE, Six AJ, Kelder JC, et al. A prospective validation of the HEART score for chest pain patients at the emergency department. Int J Cardiol 2013;168:2153–8. 17. Six AJ, Cullen L, Backus BE, et al. The HEART score for the assessment of patients with chest pain in the emergency department: a multinational validation study. Crit Pathw Cardiol 2013; 12:121–6. 18. Pollack CV Jr, Sites FD, Shofer FS, et al. Application of the TIMI risk score for unstable angina and non-ST elevation acute coronary syndrome to an unselected emergency department chest pain population. Acad Emerg Med 2006;13:13–8.
Prognosticating Scores for Chest Pain 19. Mahler SA, Riley RF, Hiestand BC, et al. The HEART Pathway randomized trial: identifying emergency department patients with acute chest pain for early discharge. Circ Cardiovasc Qual Outcomes 2015;8:195–203. 20. de Arau´jo Gonc¸alves P, Ferreira J, Aguiar C, et al. TIMI, PURSUIT, and GRACE risk scores: sustained prognostic value and interaction with revascularization in NSTE-ACS. Eur Heart J 2005;26:865–72. 21. Sakamoto JT, Liu N, Koh ZX, et al. Comparing HEART, TIMI, and GRACE scores for prediction of 30-day major adverse cardiac events in high acuity chest pain patients in the emergency department. Int J Cardiol 2016;221:759–64.
9 22. Marcoon S, Chang AM, Lee B, et al. HEART score to further risk stratify patients with low TIMI scores. Crit Pathw Cardiol 2013;12:1–5. 23. Sun BC, Laurie A, Fu R, et al. Comparison of the HEART and TIMI risk scores for suspected acute coronary syndrome in the emergency department. Crit Pathw Cardiol 2016;15:1–5. 24. Lui CT, Lam H, Cheung KH, et al. Role of copeptin in dual-cardiac marker strategy for patients with chest pain presented to ED. Am J Emerg Med 2015;33:1732–6. 25. Than M, Cullen L, Reid CM, et al. A 2-h diagnostic protocol to assess patients with chest pain symptoms in the Asia-Pacific region (ASPECT): a prospective observational validation study. Lancet 2011;377:1077–84.
10
C. P. Wong et al.
ARTICLE SUMMARY 1. Why is this topic important? Risk stratification of patients attending emergency department with chest pain is a common challenge and prevalent clinical situation. A highly reproducible and reliable predictive score would be critical in rapid risk stratification in the emergency department. 2. What does this study attempt to show? This prospective study demonstrated the accuracy of clinical prediction scores without subjective clinical gestalt components in risk stratification of chest pain patients in emergency department. 3. What are the key findings? With components of clinical gestalt, HEART (history, electrocardiogram, age, risk factors, and troponin) score had the best accuracy in predicting major adverse cardiac events compared to TIMI (Thrombosis in Myocardial Infarction) and GRACE (Global Registry of Acute Coronary Events) in risk stratification of chest pain patients in emergency department. 4. How is patient care impacted? Predictive model without clinical gestalt would be expected to have better reproducibility and reliability with lower inter-observer bias. Adoption of the predictive scores would potentially impact on better care for patients attending emergency departments for chest pain.
https://doi.org/10.1016/j.jemermed.2017.10.006
Original Contributions
PROGNOSTICATING CLINICAL PREDICTION SCORES WITHOUT CLINICAL GESTALT FOR PATIENTS WITH CHEST PAIN IN THE EMERGENCY DEPARTMENT Chin Pang Wong, MBCHB, MCEM,* Chun Tat Lui, FHKCEM, FHKAM,* Jonathan Gabriel Sung, MBCHB, MCRP,† Ho Lam, FHKCP, FHKAM,† Hin Tat Fung, FHKCEM, FHKAM,* and Ping Wa Yam, FHKCP, FHKAM† *Accident and Emergency Department, Tuen Mun Hospital, Hong Kong and †Division of Cardiology, Department of Medicine and Geriatrics, Tuen Mun Hospital, Hong Kong Reprint Address: Chun Tat Lui, FHKCEM, FHKAM, Accident and Emergency Department, Tuen Mun Hospital, Hong Kong
, Abstract—Background: Assessment of patients with chest pain is a regular challenge in the emergency department (ED). Recent guidelines recommended quantitative assessment of ischemic risk by means of risk scores. Objective: Our aim was to assess the performance of Thrombosis in Myocardial Infarction (TIMI); Global Registry of Acute Coronary Events (GRACE); history, electrocardiogram, age, risk factors, and troponin (HEART) scores; and the North America Chest Pain Rule (NACPR) without components of clinical gestalt in predicting 30-day major adverse cardiac events (MACE). Methods: We performed a prospective cohort study in adult patients who attended the ED with undifferentiated chest pain. Clinical prediction rules were applied and calculated. The clinical prediction rules were modified from the original ones, excluding components requiring judgment by clinical gestalt. The primary outcome was MACE. Performance of the tests were evaluated by receive operating characteristic curves and the area under curves (AUC). Results: There were 1081 patients included in the study. Thirty-day MACE occurred in 164 (15.2%) patients. The AUC of the GRACE score was 0.756, which was inferior to the TIMI score (AUC 0.809) and the HEART score (AUC 0.845). A TIMI score $ 1 had a sensitivity of 97% and a specificity of 45.7%. A GRACE score $ 50 had a sensitivity of 99.4% and a specificity of 7.5%. A HEART score $ 1 had a sensitivity of 98.8% and
a specificity of 11.7%. The NACPR had a sensitivity of 93.3% and a specificity of 51.5%. Conclusions: Without clinical gestalt, the modified HEART score had the best discriminative capacity in predicting 30-day MACE. Ó 2017 Elsevier Inc. All rights reserved. , Keywords—acute coronary syndrome; chest pain; decision support techniques; myocardial infarction
INTRODUCTION Patients with undifferentiated chest pain account for a significant proportion of attendance and burden of the emergency department (ED) (1). Prompt and accurate diagnosis of acute coronary syndrome (ACS) is crucial for patients’ immediate management to achieve better outcome. However, within the heavy patient load, genuine cardiac events contribute to only a minor proportion (2,3). It is well known that normal initial electrocardiogram (ECG) and biomarker do not exclude ACS. As a result, serial blood tests and investigations are required. This leads to prolonged length of stay and ED overcrowding, which are of tremendous concern (4). In the era of high-sensitive troponin, the falsepositive rate further aggravates the burden to the health care system. Therefore, an objective, reproducible tool for clinical risk stratification is useful to tackle this
This research project has been presented on October 28–29, 2016 in the Scientific Symposium on Emergency Medicine of Hong Kong.
RECEIVED: 20 June 2017; FINAL SUBMISSION RECEIVED: 6 September 2017; ACCEPTED: 7 October 2017 1
2
C. P. Wong et al.
diagnostic challenge. As stated in their recent guidelines, both the American Heart Association and the European Society of Cardiology recommended the use of riskstratification models to guide management in patients with chest pain (5,6). It ensures rapid accurate diagnosis of ACS and appropriate discharge of low-risk patients. These risk scores can also be used to assess prognosis in ACS patients. However, most of the existing scores contained elements of clinical gestalt with subjective input from attending physicians, which may affect the consistency and reproducibility of the scores. In this study, we compared the diagnostic accuracies of four commonly used scores, with removal of the components of clinical gestalt. The assessed scores are the Thrombosis in Myocardial Infarction (TIMI) score; the Global Registry of Acute Coronary Events (GRACE) score; the history, electrocardiogram, age, risk factors, and troponin (HEART) score; and the North America Chest Pain rule (NACPR) (7–10). Initially, the TIMI and the GRACE scores were developed for post-ACS prognostication, these scores were then validated for the prediction of major adverse cardiac events (MACE) for patients with undifferentiated chest pain (11). Our aim was to identify the best clinical prediction score without components of clinical gestalt for early and safe discharge of low-risk patients. MATERIALS AND METHODS
risk factors were recorded. The presenting vital signs were recorded for score calculations. A standard 12-lead ECG was performed for each included patient. We adopted troponin as the cardiac marker in calculation of scores. The troponin level was measured after initial assessment by the attending clinician. We used the Abbott ARCHITECT STATÒ high-sensitive troponin I assay in our study. A level > 99th percentile was considered positive. Sex-specific cutoffs were adopted according to manufacturer’s recommendation. The upper reference limits were 34.2 ng/L for male and 15.6 ng/L for female. The coefficient of variation of the troponin assay was < 4% and the lower limit of detection was 10 ng/L. Outcome variables were traced by reviewing all patients’ written and electronic hospital records, laboratory test results, intervention reports, and death registries. Clinical Prediction Scores The components of the evaluated clinical prediction models were shown in Table 1. The initial definitions of these scores included subjective components that required the attending clinician’s subjective judgment. In our study, we made modifications to these scores so that these subjective components were omitted. In this manner, we were able to explore the accuracy risk scores based only on objective parameters without clinical gestalt.
Study Design and Setting
TIMI Score
We performed a prospective cohort study in the ED of a tertiary referral hospital with daily attendance of > 600 patients. The study period was from February 2016 to June 2016. Patients aged > 18 years who complained of chest pain in the triage were included. Twelve-lead ECGs were obtained. Patients were excluded if STelevation myocardial infarction (STEMI) was diagnosed, or there were clearly established alternative diagnoses on presentation not related to cardiac ischemia (e.g., aortic dissection, pulmonary embolism, pneumothorax, and herpes zoster). Ethics approval was obtained from the local Institutional Review Board (CREC/16093). Verbal consent was obtained from the participants of the study.
The TIMI score for unstable angina/non-STEMI was introduced by Antman et al. in 2000 (7). It consists of seven risk factors weighed one mark each. Total marks of one or less indicate lower risk at 14 days in terms of all-cause mortality, acute myocardial infarction (AMI), and severe recurrent ischemia prompting urgent revascularization. In our study, we had modified ‘‘the use of aspirin in the past 7 days’’ into ‘‘the use of antiplatelets in the past 7 days.’’ Patients on aspirin, clopidogrel, Ticagrelor, and other antiplatelets were defined as positive exposures to antiplatelets. We omitted the item ‘‘severe angina $2 episodes in 24 hours,’’ which involved the clinician’s subjective judgment. It was also inapplicable for cross-sectional risk stratification of chest pain patients in the ED in a single time point.
Data Collection Data were collected prospectively in form of standardized data collection sheets filled by the attending clinician. Consecutive samples were recruited. Prior training sessions had been arranged for the clinicians in the study center for familiarization of the data collection form. Patients’ demographic data, relevant medical history, drug history, smoking status, and other cardiovascular
GRACE Score This score was developed by the Global Registry of Acute Coronary Events. It is an international registry designed to track in-hospital and long-term outcomes of patients presenting with ACS (12). It comprises 8 components with different weighing. Web-based calculator is readily
Prognosticating Scores for Chest Pain
3
Table 1. Components and Definitions of Clinical Prediction Scores for Chest Pain Risk Stratification Prediction Rules TIMI score*
GRACE score
HEART score†
North America Chest Pain Rulek
Items
Score
Age $ 65 years Three or more risk factors for CAD Family history of CAD Hypertension Hyperlipidemia Diabetes mellitus Tobacco use ST deviation $ 0.5 mm Elevated cardiac marker level Known CAD (stenosis $ 50%) Aspirin or other antiplatelet use in the past 7 days Age Heart rate Systolic blood pressure Creatinine Congestive heart failure: Killip class I/II/III/IV Cardiac arrest at presentation ST-segment deviation Elevated cardiac enzymes/ markers ECG Significant ST depression‡ Nonspecific repolarization disturbance/LBBB/LVH Normal Age $ 65 years 45–65 years # 45 years Risk factors $ 3 risk factors§ or history of atherosclerotic disease 1 or 2 risk factors No risk factors known Troponin $ 3 normal limit 13 normal limit Less than or equal normal limit New ischemia on initial ECG{ History of coronary artery disease Initial cardiac troponin is positive and Age # 40 years or Age 4150 years and repeat troponin at least 3 h from symptom onset is negative.
1 1
available for its complicated score calculation. The probability of death or death/AMI occurring up to given time points after admission will be shown after corresponding parameters are entered (13). The Killip class was determined during the initial patient assessment by the attending clinician. HEART Score
1 1 1 1 0–100 0–46 0–58 1–28 0–59 39 28 14 2 1 0 2 1 0 2 1 0 2 1 0
CAD = coronary artery disease; ECG = electrocardiogram; GRACE = Global Registry of Acute Coronary Events; LBBB = left bundle branch block; LVH = left ventricular hypertrophy; TIMI = Thrombosis in Myocardial Infarction. * Excluded the component ‘‘Severe angina $ 2 episodes in 24 hours.’’ † Excluded the component ‘‘History of highly, moderately and slightly suspicious.’’ ‡ ST segment depression $ 0.05 mV in two or more contiguous leads, in the appropriate clinical context or ST depression $ 0.1 mV. § Risk factors: hypercholesterolemia, hypertension, diabetes mellitus, cigarette smoking, positive family history, and obesity. k Excluded the component of ‘‘Pain is typical for acute coronary syndrome.’’ Repeat troponin is changed from original 6 h to 3 h with adoption of high-sensitive troponin assay in the current study. { ST segment deviation $ 1 mm or T-wave inversion $ 0.2 mm in at least two contiguous leads.
The HEART score was developed by Six et al. in 2008 (9). It was intended for aiding the diagnosis of ACS in the ED. It comprises five predictors with the same weighing. Each predictor is given 0–2 points, depending on the factor severity. It uses MACE as the clinical end point. In our study, we omitted the component ‘‘highly, moderately and slightly suspicious history,’’ which required subjective clinician judgment. NACPR This rule was also developed for risk stratification of patients with undifferentiated chest pain in the ED. It was published in 2015 by Hess et al. (10). They aimed to codify the existing common practice to identify low-risk patients for safe discharge. It includes five clinical criteria. A patient with chest pain can be discharged safely if all of these criteria are negative. We omitted the criterion ‘‘Pain is typical for ACS’’ in our study because it was a subjective component. In addition, the time interval of serial troponin levels was changed from 6 h to 3 h in the current study to adopt the recommendation from the manufacturer of high-sensitive troponin assays. Outcomes The primary outcome was MACE at 30 days. MACE were defined as AMI, percutaneous coronary intervention, coronary bypass graft, and death within 30 days of presentation to the ED. The diagnosis of AMI for the index attendance of the patient was based on the third universal definition, assessed independently by the attending clinician and an independent blinded cardiologist (14). In case of disagreement, third adjudication from an independent cardiologist would be sought. In Hong Kong, emergency medical services would deliver patients to 1 of 18 EDs under the Hospital Authority. All cardiac events or attendance of EDs were traced from the corporate electronic patient record system from all 18 acute hospitals in Hong Kong. Deaths were confirmed with the Death Registry under the Department of Health of Hong Kong. The outcome assessment was performed after patient discharge and reconfirmed 6 months after completion of case recruitment through the electronic patient record system.
4
C. P. Wong et al.
Statistical Analysis MedCalc for Windows, version 9.6.4.0 (MedCalc Software, Ostend, Belgium) and SPSS (IBM SPSS Statistics for Windows, Version 22.0, IBM Corp, Armonk, NY) were employed for analysis. The statistical analysis was performed by one of the investigators, independent of the outcome assessors and attending physicians. Age, as a continuous variable, was expressed in mean and standard deviation with normal distribution, and compared by independent sample t-test. Categorical variables were shown in proportions and percentages. Their comparisons were made by c2 test or Fisher exact test, where appropriate. A p value < 0.05 was taken to imply statistical significance. The receiver operating characteristic (ROC) curve for the prediction of the outcome was computed for each score. The area under curve (AUC) was evaluated for diagnostic accuracies. Pairwise comparisons of AUC were conducted for different scores using nonparametric approach (15). The sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), positive likelihood ratio (PLR), and negative likelihood ratio (NLR) for the primary outcome at different risk score cutoff values were calculated. Power analysis was performed with PASS 2011 software (NCSS, Kaysville, UT). RESULTS
Figure 1. Enrollment and outcome of the study cohort. AMI = acute myocardial infarction; MACE = major adverse cardiac events; NSTEMI = non–ST-elevation myocardial infarction; STEMI = ST-elevation myocardial infarction.
had a statistically significant higher rates of 30-day MACE.
Patient Enrollment and Outcome The flow of study subjects was shown in Figure 1. A total of 1081 patients were included in the study, of which 52.3% were male. Thirty-day MACE occurred in 164 patients (15.2%). Non-STEMI and unstable angina were diagnosed in 104 and 54 patients, respectively, for the index attendance and admission. There was one case reattended within 30 days for ventricular fibrillation and mortality. There were five cases of non-fatal AMI within 30 days requiring vascular interventions. Clinical Risk Factors, ECG Features, and Biomarker Level Patients’ baseline characteristics and risk factors were analyzed for the primary outcome (30-day MACE). Advanced age, ever-smoker, hypertension, diabetes mellitus, hyperlipidemia, known coronary artery disease, and renal impairment were found statistically significant as risk factors of 30-day MACE (Table 2). Patients who were on antiplatelets, with ST depression in ECG, with new onset of T-wave changes in ECG, with the first high-sensitive troponin I > 99th percentile and with the repeated high-sensitive troponin I > 99th percentile
Proportion of 30-Day MACE in Different Ranges of Risk Scores Figure 2 shows the bar chart representing the correlation between different scores and 30-day MACE. All risk scores had a positive correlation with the primary outcome, that is, the higher the score, the greater the proportion of 30-day MACE. Comparison of TIMI, GRACE, HEART, and NACPR Figure 3 shows ROC curves of the 3 predictive scores. The AUC of HEART score was 0.845 (95% confidence interval [CI] 0.812–0.878); TIMI score was 0.809 (95% CI 0.777–0.841); and GRACE score was 0.756 (95% CI 0.717–0.795). Pairwise comparison with nonparametric method demonstrated statistically significant difference in AUCs (TIMI > GRACE, AUC difference 0.053 [95% CI 0.013–0.094], p = 0.01; HEART > TIMI, AUC difference 0.036 [95% CI 0.004–0.068], p = 0.027; HEART > GRACE, AUC difference 0.089 [95% CI 0.054–0.125]; p < 0.001). Table 3 shows the sensitivity, specificity, PPV, NPV, PLR, and the NLR. As we aim to identify low-risk
Prognosticating Scores for Chest Pain
5
Table 2. Comparison of Demographic, Clinical, Electrocardiogram Features, and Biomarkers in Patients With and Without 30-Day Major Adverse Cardiac Events 30-Day MACE Parameters Demographics Age, mean 6 SD Age $ 65 years, n (%) Sex, male, n (%) Risk factors and comorbidities, n (%) Family history of early-onset CAD Current smoker Ever smoker Hypertension Diabetes mellitus Lipid Known CAD Renal impairment On pacemaker Current medications, n (%) On antiplatelets On warfarin On novel oral anticoagulants Duration of chest pain, h, median (IQR) Presenting vital signs, mean 6 SD Systolic blood pressure, mm Hg Diastolic blood pressure, mm Hg Heart rate, beats/min Killip heart failure class I II III ECG features, n (%) ST elevation $ 0.5 mm ST depression $ 0.5 mm New-onset T-wave changes Troponin level, n (%) First troponin > 99th percentile Repeated troponin > 99th percentile Creatinine, mmol/L, median (IQR)
All (n = 1081)
Yes (n = 164)
No (n = 917)
p Value
48 6 27 443 (41) 565 (52.3)
59 6 23 89 (54.3) 97 (59.1)
46 6 27 354 (38.6) 468 (51)
< 0.001 < 0.001 0.055
4 (0.4) 147 (13.6) 298 (27.6) 515 (47.6) 213 (19.7) 412 (38.1) 230 (21.3) 151 (14) 16 (1.5)
1 (0.6) 31 (18.9) 66 (40.2) 105 (64) 57 (34.8) 107 (65.2) 72 (43.9) 53 (32.3) 4 (2.4)
4 (0.4) 116 (12.6) 232 (25.3) 410 (44.7) 156 (17) 305 (33.3) 158 (17.2) 98 (10.7) 12 (1.3)
1.000 0.031 < 0.001 < 0.001 < 0.001 < 0.001 < 0.001 < 0.001 0.285
346 (32) 18 (1.7) 10 (0.9) 7.5 (1.8–62.9)
89 (54.3) 2 (1.2) 1 (0.6) 7.2 (1.8–60.9)
257 (28) 16 (1.7) 9 (1) 11.1 (1.9–65.4)
< 0.001 1.000 1.000 0.194
147 6 27 81 6 16 82 6 16
149 6 28 81 6 17 82 6 16
139 6 22 82 6 15 85 6 16
< 0.001 0.834 0.324 < 0.001
1024 (94.7) 28 (2.6) 29 (2.7)
142 (86.6) 9 (5.5) 13 (7.9)
882 (96.2) 19 (2.1) 16 (1.7)
10 (0.9) 36 (3.3) 61 (5.6)
4 (2.4) 23 (14) 32 (19.5)
6 (0.7) 13 (1.4) 29 (3.2)
0.051 < 0.001 < 0.001
80 (8.7) 91 (9.9) 69 (62–79)
< 0.001 < 0.001 < 0.001
199 (18.4) 229 (21.2) 76 (66–94)
119 (72.6) 138 (84.1) 80 (68–100)
CAD = coronary artery disease; EGC = electrocardiogram; IQR = interquartile range; MACE = major adverse cardiac event; SD = standard deviation.
patients for safe discharge, we are looking for the cutoff points at the best sensitivity of various risk scores. A TIMI score $ had a sensitivity of 97% (95% CI 92.7– 98.9%) and a specificity of 45.7% (95% CI 42.4–49%). A GRACE score $50 had a sensitivity of 99.4% (95% CI 96.1–100%) and a specificity of 7.5% (95% CI 5.9– 9.5%). A HEART score $1 had a sensitivity of 98.8% (95% CI 95.2–99.8%) and a specificity of 11.7% (95% CI 9.7–14%). The NACPR had a sensitivity of 93.3% (95% CI 66–99.7%) and specificity of 51.5% (95% CI 44.9–58%). Power Analysis A sample of 164 from the positive group and 917 from the negative group achieved 99% and 67% power to detect the differences between a diagnostic test with an AUROC of 0.845 and another diagnostic test with an AUC of 0.756
and 0.809 using a two-sided z-test at a significance level of 0.05, for continuous scores. DISCUSSION Recent recommended models for risk stratifications of chest pain patients involves adoption of a validated risk assessment score (5,6). Existing scores, such as HEART score, integrated components of clinical gestalt with attending subjective input from physicians (9). There is doubt in the reproducibility and consistency of the scores with evaluation from different physicians, which contributes to variations in standardization of the risk stratification of chest pain. Clinical risk scores should be applicable only for the group with undifferentiated chest pain. For patients in obvious clinical unstable angina or other diseases causing chest pain, such as pneumothorax, clinical decision scores should not be applied, but patient
6
C. P. Wong et al.
Figure 2. Proportion of 30-day major adverse cardiac events (MACE) in different ranges of each clinical prediction rule. GRACE = Global Registry of Acute Coronary Events; HEART = history, electrocardiogram, age, risk factors, and troponin; TIMI = Thrombosis in Myocardial Infarction.
should be treated directly. For the diagnosis of ‘‘undifferentiated chest pain,’’ which implies the attending physician could not identify the apparent cause, further input of clinical gestalt in the clinical risk scores would commit logical fallacy. Instead, clinical scores with objective components should be sought for standardized stratification of the ischemic risk. We have made modifications to the existing risk scores in our study. All components that required patients’ subjective recalling or the attending clinician’s judgment were omitted. This prospective study illustrated the accuracy of clinical prediction scores without subjective clinical gestalt components in risk stratification of chest pain patients in ED. Scores without clinical gestalt would be expected to have better reproducibility and reliability with lower inter-observer bias. All risk scores remained in good diagnostic accuracies as reflected by the AUC (0.756–0.845). Their sensitivities remained high (93.3–99.4%), which ensured the safe discharge of low-risk patients. This might potentially change the current practice as the risk stratification might depend only on discreet information. Time might be saved at the initial patient assessment, as the clinician might not need to clarify much on the severity and the nature of pain. These reporting of pain intensity from patients were often inaccurate and difficult to quantify.
The evaluation of these parts of risk scores might also subject to variation depending on the clinician’s experience. Therefore, our finding suggested that a risk score based solely on objective parameters without clinical gestalt is practical to identify low-risk patients for early discharge from the ED. In this study, 164 of 1081 patients (15.2%) had reached the primary outcome. This rate was in keeping with prior studies of similar nature (13–17%) (16,17). Generally speaking, the TIMI, GRACE, and HEART scores all performed well in predicting 30-day MACE with respect to the AUROC curve. This demonstration of safety of these scores was also shown in previous studies (18–20). We found that the HEART score had the best discriminative capacity (AUC 0.845), followed by the TIMI score (AUC 0.809). The GRACE score had a relatively inferior performance (AUC 0.756). This concurred with previous studies (21). Our postulation is that for the GRACE score, it does not involve medical risk factors (e.g., hypertension and diabetes mellitus) in the calculations. Unlike the other two risk scores, the GRACE score is calculated based on the clinical findings in a single time point only. In fact, cardiovascular risk factors do predict the likelihood of MACE/ACS well.
Prognosticating Scores for Chest Pain
7
Figure 3. Receiver operating characteristic curve of predictive scores predicting 30-day major adverse cardiac events. GRACE = Global Registry of Acute Coronary Events; HEART = history, electrocardiogram, age, risk factors, and troponin; TIMI = Thrombosis in Myocardial Infarction.
In this study, the HEART score had the best performance in predicting 30-day MACE. For a HEART score $ 1, its sensitivity, NPV, and NLR were 98.8%, 98.2%, and 0.1, respectively. For a HEART score $ 2, its sensitivity, NPV, and NLR were 98.2%, 99%, and 0.1, respectively. This was in keeping with various validation studies using 30-day MACE as the primary outcome. In these preceding literature, the sensitivity and NPV reached 99% and 98–98.2%, respectively (16,21,22). The HEART score involves graded calculation of each component from 0–2 points. This slightly complicated calculation might contribute to its superiority to TIMI. It is not unexpected that HEART outperformed TIMI and GRACE, as HEART was originally derived to predict MACE, while TIMI and GRACE were
originally derived for prognostication of patients with unstable angina and non-STEMI. For the TIMI score, its diagnostic accuracy came next to the HEART score. A TIMI score $ 1 showed a sensitivity, NPV, and NLR of 97%, 98.8%, and 0.1, respectively. These results concurred with prior similar studies (21,23). A TIMI score $ 2 gave a better specificity (68.8%), but compromised the sensitivity significantly (76.8%). Therefore, only the cutoff score of 1 was useful in identifying low-risk patients for discharge. The GRACE score showed inferiority in term of diagnostic accuracy in this study. Its sensitivity, NPV and NLR at cutoff of $ 50 were 99.4%, 98.6%, and 0.1, respectively. It gave a lower sensitivity (92.1%) at the $ 75 cutoff. Although shown inferior in terms of AUC, it did perform well with high sensitivity and NPV. The NACPR had sensitivity inferior to other risk scores in this study. The sensitivity, NPV, and NLR were 93.3%, 99.2%, and 0.1, respectively. Regarding the original study, for the cutoff at age # 40 years, the sensitivity and NPV were both up to 100% (10). This discrepancy might possibly occur because of omitting one of the five items ‘‘Pain is typical for ACS.’’ The best of clinical scoring systems for risk stratification would theoretically had the highest sensitivities and NPV in order not to miss patients with ACS. Cutoff of the prediction models had been set to achieve the lowest false-negative rate. However, in the reality situation, one may need to consider from the view of costeffectiveness and resources utilization, particularly in the less-wealthy cities. For example, with the HEART score, HEART score $ 1 achieved 98.8% sensitivity with low specificity of 11.7%. If we sacrificed sensitivity to 98.2% by adopting cutoff of 2, specificity increased dramatically to 32.8%, with much lower rate of falsepositive prediction. Rapid risk stratification of chest pain patients in ED is always a challenge. Recent attempts to further enhance the accuracies of the predictive models include dual
Table 3. Diagnostic Characteristics of Clinical Prediction Scores Predicting 30-Day Major Adverse Cardiac Events at Various Cutoffs Variable
Sensitivity, % (95% CI)
Specificity, % (95% CI)
PPV, % (95% CI)
NPV, % (95% CI)
PLR (95% CI)
NLR (95% CI)
TIMI $ 1 TIMI $ 2 GRACE $ 50 GRACE $ 75 GRACE $ 100 HEART $ 1 HEART $ 2 NACPR
97 (92.7–98.9) 76.8 (69.5–82.9) 99.4 (96.1–100) 92.1 (86.5–95.5) 76.2 (68.8–82.4) 98.8 (95.2–99.8) 98.2 (94.3–99.5) 93.3 (66–99.7)
45.7 (42.4–49) 68.8 (65.7–71.8) 7.5 (5.9–9.5) 32.5 (29.5–35.7) 61.9 (58.7–65.1) 11.7 (9.7–14) 32.8 (29.8–36) 51.5 (44.9–58)
24.2 (21–27.7) 30.6 (26.2–35.3) 16.1 (13.9–18.6) 19.6 (16.9–22.6) 26.4 (22.5–30.6) 16.7 (14.4–19.2) 20.7 (18–23.8) 10.9 (6.3–18)
98.8 (97.1–99.6) 94.3 (92.2–95.9) 98.6 (91.2–99.9) 95.8 (92.8–97.7) 93.6 (91.2–95.3) 98.2 (92.9–99.7) 99 (96.9–99.7) 99.2 (94.8–100)
1.8 (1.7–1.9) 2.5 (2.2–2.8) 1.1 (1.1–1.1) 1.4 (1.3–1.5) 2 (1.8–2.3) 1.1 (1.1–1.2) 1.5 (1.4–1.5) 1.9 (1.6–2.3)
0.1 (0–0.2) 0.3 (0.3–0.4) 0.1 (0–0.6) 0.2 (0.1–0.4) 0.4 (0.3–0.5) 0.1 (0–0.4) 0.1 (0–0.2) 0.1 (0–0.9)
CAD = coronary artery disease; CI = confidence interval; GRACE = Global Registry of Acute Coronary Events; HEART = history, electrocardiogram, age, risk factors, and troponin; NACPR = North America Chest Pain Rule; NLR = negative likelihood ratio; NPV = negative predictive value; PLR = positive likelihood ratio; PPV = positive predictive value; TIMI = Thrombosis in Myocardial Infarction.
8
C. P. Wong et al.
cardiac marker strategies, combination of two risk scores, and accelerated diagnostic protocols (22,24,25). Accelerated diagnostic protocol integrating HEART score with objective components and newer generation of cardiac markers may be the future direction to explore for a more accurate predictive model, thus enhancing the efficiency of chest pain stratification and possibly reducing hospital admissions.
Chun Tat managed the statistics method and analyzed the data. Wong Chin Pang drafted the manuscript and all authors contributed substantially to its revision. Wong Chin Pang takes responsibility for the paper as a whole. All authors had substantial contribution in authorship of the paper in terms of idea generation, study design and logistics, data collection and monitoring, analysis, and manuscript drafting.
REFERENCES
Limitations Firstly, there were no universal clinical follow-ups for included patients for the assessment of the outcome (30-day MACE). All cardiac events or attendance of EDs were traced from the corporate electronic patient record system from all acute hospitals in Hong Kong. However, patients may still have events not traceable in the system when they attended private non-acute hospitals. Secondly, we aimed to evaluate the accuracy of risk scores without subjective component and clinical gestalt. In the GRACE score, the item Killip class involved clinician’s judgment by physical examination on whether the patient was in heart failure. It was subjected to interassessor variation. Objective assessments of heart failure, such as echocardiogram and natriuretic peptide level, were not adopted. Same problem existed in the HEART score. The attending clinician had to judge whether the patient was obese or not in order to give a score on the ‘‘risk factor’’ item. As there was no definition of obesity given in the HEART score, it was also a subjective judgment made by the attending clinician. Thirdly, highsensitive troponin I was adopted in this study. It might not be universally applicable to other centers using other types of biomarker assays. Lastly, it was a study that recruited patients in only one regional hospital. The CI of the current study remained variable, particularly with unbalanced number of outcome events. Further multicenter research with larger sample size is needed for validation and demonstration of universal applicability. CONCLUSIONS The modified HEART score without components of clinical gestalt had the best discriminative capacity in predicting 30-day MACE. Acknowledgments—Author contributions: Lui Chun Tat, Lam Ho, Wong Chin Pang and Jonathan Gabriel Sung conceived the study and designed the research project. Lui Chun Tat formulated the research question and supervised the conduct of the research project. Wong Chin Pang and Jonathan Gabriel Sung performed the data collection. Wong Chin Pang, Jonathan Gabriel Sung, Lui Chun Tat, and Lam Ho undertook recruitment of patients and managed the data, including quality control. Lui
1. Goodacre S, Cross E, Arnold J, et al. The health care burden of acute chest pain. Heart 2005;91:229–30. 2. Launbjerg J, Fruergaard P, Hesse B, et al. Long-term risk of death, cardiac events and recurrent chest pain in patients with acute chest pain of different origin. Cardiology 1996;87:60–6. 3. Hollander JE. Risk stratification of emergency department patients with chest pain: the need for standardized reporting guidelines. Ann Emerg Med 2004;43:68–70. 4. Bernstein SL, Aronsky D, Duseja R, et al. The effect of emergency department crowding on clinically oriented outcomes. Acad Emerg Med 2009;16:1–10. 5. Amsterdam EA, Wenger NK, Brindis RG, et al. 2014 AHA/ ACC guideline for the management of patients with non-STelevation acute coronary syndromes: executive summary: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. Circulation 2014;130:2354–94. 6. Roffi M, Patrono C, Collet JP, et al. Management of acute coronary syndromes in patients presenting without persistent ST-segment elevation of the European Society of Cardiology. Eur Heart J 2016;37:267–315. 7. Antman EM, Cohen M, Bernink PJ, et al. The TIMI risk score for unstable angina/non-ST elevation MI: a method for prognostication and therapeutic decision making. JAMA 2000;284:835–42. 8. Granger CB, Goldberg RJ, Dabbous O, et al. Predictors of hospital mortality in the global registry of acute coronary events. Arch Intern Med 2003;163:2345–53. 9. Six AJ, Backus BE, Kelder JC. Chest pain in the emergency room: value of the HEART score. Neth Heart J 2008;16:191–6. 10. Hess EP, Brison RJ, Perry JJ, et al. Development of a clinical prediction rule for 30-day cardiac events in emergency department patients with chest pain and possible acute coronary syndrome. Ann Emerg Med 2012;59:115–25. 11. Ramsay G, Podogrodzka M, McClure C, et al. Risk prediction in patients presenting with suspected cardiac pain: the GRACE and TIMI risk scores versus clinical evaluation. QJM 2007;100:11–8. 12. Granger CB. Strategies of patient care in acute coronary syndromes: rationale for the Global Registry of Acute Coronary Events (GRACE) registry. Am J Cardiol 2000;86:4M–9. 13. GRACE 2.0 Calculator. Available at: http://www.gracescore.org/ WebSite/WebVersion.aspx. Accessed September 20, 2016. 14. Thygesen K, Alpert JS, Jaffe AS, et al. Third universal definition of myocardial infarction. Circulation 2012;126:2020–35. 15. DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 1988;44:837–45. 16. Backus BE, Six AJ, Kelder JC, et al. A prospective validation of the HEART score for chest pain patients at the emergency department. Int J Cardiol 2013;168:2153–8. 17. Six AJ, Cullen L, Backus BE, et al. The HEART score for the assessment of patients with chest pain in the emergency department: a multinational validation study. Crit Pathw Cardiol 2013; 12:121–6. 18. Pollack CV Jr, Sites FD, Shofer FS, et al. Application of the TIMI risk score for unstable angina and non-ST elevation acute coronary syndrome to an unselected emergency department chest pain population. Acad Emerg Med 2006;13:13–8.
Prognosticating Scores for Chest Pain 19. Mahler SA, Riley RF, Hiestand BC, et al. The HEART Pathway randomized trial: identifying emergency department patients with acute chest pain for early discharge. Circ Cardiovasc Qual Outcomes 2015;8:195–203. 20. de Arau´jo Gonc¸alves P, Ferreira J, Aguiar C, et al. TIMI, PURSUIT, and GRACE risk scores: sustained prognostic value and interaction with revascularization in NSTE-ACS. Eur Heart J 2005;26:865–72. 21. Sakamoto JT, Liu N, Koh ZX, et al. Comparing HEART, TIMI, and GRACE scores for prediction of 30-day major adverse cardiac events in high acuity chest pain patients in the emergency department. Int J Cardiol 2016;221:759–64.
9 22. Marcoon S, Chang AM, Lee B, et al. HEART score to further risk stratify patients with low TIMI scores. Crit Pathw Cardiol 2013;12:1–5. 23. Sun BC, Laurie A, Fu R, et al. Comparison of the HEART and TIMI risk scores for suspected acute coronary syndrome in the emergency department. Crit Pathw Cardiol 2016;15:1–5. 24. Lui CT, Lam H, Cheung KH, et al. Role of copeptin in dual-cardiac marker strategy for patients with chest pain presented to ED. Am J Emerg Med 2015;33:1732–6. 25. Than M, Cullen L, Reid CM, et al. A 2-h diagnostic protocol to assess patients with chest pain symptoms in the Asia-Pacific region (ASPECT): a prospective observational validation study. Lancet 2011;377:1077–84.
10
C. P. Wong et al.
ARTICLE SUMMARY 1. Why is this topic important? Risk stratification of patients attending emergency department with chest pain is a common challenge and prevalent clinical situation. A highly reproducible and reliable predictive score would be critical in rapid risk stratification in the emergency department. 2. What does this study attempt to show? This prospective study demonstrated the accuracy of clinical prediction scores without subjective clinical gestalt components in risk stratification of chest pain patients in emergency department. 3. What are the key findings? With components of clinical gestalt, HEART (history, electrocardiogram, age, risk factors, and troponin) score had the best accuracy in predicting major adverse cardiac events compared to TIMI (Thrombosis in Myocardial Infarction) and GRACE (Global Registry of Acute Coronary Events) in risk stratification of chest pain patients in emergency department. 4. How is patient care impacted? Predictive model without clinical gestalt would be expected to have better reproducibility and reliability with lower inter-observer bias. Adoption of the predictive scores would potentially impact on better care for patients attending emergency departments for chest pain.