Comparison of Stroke Prediction Accuracy of ABCD2 and ABCD3-I in Patients with Transient Ischemic Attack: A Meta-Analysis

ARTICLE IN PRESS

Comparison of Stroke Prediction Accuracy of ABCD2 and ABCD3-I in Patients with Transient Ischemic Attack: A Meta-Analysis Meng Zhao, MD,*,†,‡ Shuo Wang, MD,*,†,‡ Dong Zhang, MD,*,†,‡ Yan Zhang, MD,*,†,‡ Xiaofeng Deng, MD,*,†,‡ and Jizong Zhao, MD*,†,‡

Background: A direct comparison of the stroke prediction utility of the ABCD2 and ABCD3-I scores has not been performed. Thus, we conducted a diagnostic meta-analysis and applied the results to a hypothetical cohort of 1000 patients with transient ischemic attack (TIA) to assess the power of stroke prediction by ABCD2 and ABCD3-I scores. Methods: Medline, PubMed, Embase, and manuscript references were searched to identify studies that directly compared the stroke predictive powers of ABCD2 and ABCD3-I scores. We conducted a diagnostic metaanalysis using bivariate random effects models, and the predictive powers of ABCD2 and ABCD3-I scores were assessed by their summary sensitivity and specificity. Then, we applied the results to a hypothetical cohort of 1000 patients with TIA to calculate the effect per 1000 patients triaged for stroke prevention in a virtual setting. Results: Of the 35 identified studies on ABCD2 and ABCD3-I, 6 studies (7364 participants) directly compared the diagnostic accuracies of ABCD2 and ABCD3-I scores for occurrence of a future stroke. The pooled sensitivities of ABCD2 versus ABCD3-I were 79.9% (62.2%-90.6%) versus 96.1% (90.2%-98.5%) at 7 days (P = .022), and 76.6% (63.8%-85.8%) versus 94.6% (88.9%-97.5%) at 90 days (P = .001). The pooled specificities of ABCD2 versus ABCD3-I were 29.2% (18.2%-43.3%) versus 17.7% (8.5%-33.3%) at 7 days (P = .214), and 40.3% (25.0%-57.7%) versus 20.2% (12.6%-30.6%) at 90 days (P = .032). Conclusions: ABCD3-I scores had a better sensitivity but poorer specificity than ABCD2 scores. However, in community-based referring settings, it is more suitable to use ABCD2 at initial triage and deciding on urgency of specialist assessment. The prognostic utility of each of the components of the scores should be carefully considered rather than dichotomized scores during clinical triage. Key Words: Transient ischemic attack (TIA)—ischemic stroke—cerebrovascular disease—stroke—ABCD. © 2017 National Stroke Association. Published by Elsevier Inc. All rights reserved.

From the *Department of Neurosurgery, Beijing Tiantan Hospital, Capital Medical University, Beijing, China; †China National Clinical Research Center for Neurological Diseases, Beijing, China; and ‡Center of Stroke, Beijing Institute for Brain Disorders, Beijing, China. Received April 22, 2017; revision received May 10, 2017; accepted May 21, 2017. Grant support: This study was funded by the “13th Five-Year Plan” on National Science and Technology supporting plan (2015BAI12B04), the National Natural Science Foundation of China (81371292), and the Beijing Municipal Administration of Hospitals’ Mission Plan (SML2015050). Disclosure: The authors report no conflict of interest concerning the materials or methods used in this study or the findings specified in this paper. Address correspondence to Jizong Zhao MD, Department of Neurosurgery, Beijing Tiantan Hospital, Capital Medical University, No.6 Tiantanxili, Dongcheng District, Beijing 100050, China. E-mail:[email protected]. 1052-3057/$ - see front matter © 2017 National Stroke Association. Published by Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.jstrokecerebrovasdis.2017.05.030

Journal of Stroke and Cerebrovascular Diseases, Vol. ■■, No. ■■ (■■), 2017: pp ■■–■■

1

ARTICLE IN PRESS M. ZHAO ET AL.

2

Introduction Transient ischemic attacks (TIAs) are common, affecting at least 240,000 people annually in the United States.1 Although the symptoms of TIA can resolve within 24 hours, the risk of a subsequent stroke is high following TIA, varying from 1.7% to 20.6% at 90 days.2-6 Accurate prediction of stroke risk after TIA is essential for clinical decision-making.7 Thus, risk scores were developed to help physicians stratify stroke risk and to guide the triage of suitable interventions. The ABCD2 score5 is one of the most validated triage tools for predicting stroke risk after TIA. However, the predictive power of ABCD2 has recently been questioned.8-11 The ABCD3-I score has been introduced to include dual TIA, vascular, and brain imaging findings in addition to the ABCD2, to improve risk stratification.12,13 However, the ABCD3-I score also has limitations. Additional prognostic utility of the imaging markers used in the ABCD3-I score is generally not available to community-based clinicians making referrals in pre-hospital settings. Moreover, ABCD3-I might classify many patients as high risk for future stroke, resulting in unnecessary hospitalization and exposure to potential side effects from antiplatelet therapy. A direct comparison of the stroke prediction utility and clinical guidance ability of the ABCD2 and ABCD3-I scores has not been performed. Whether ABCD3-I is a better scoring system that could replace ABCD2 in clinical applications to stratify stroke risk among patients with TIA is unknown. Therefore, we conducted a systematic review and metaanalysis to compare the stroke prediction accuracy of ABCD2 scores and ABCD3-I scores for stratifying stroke risks of patients with TIA.

Methods Search Strategy The results are reported according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. 14 Potential studies were identified from MEDLINE (Ovid), Web of Science, and Embase using the search terms “TIA,” “ABCD2,” and “ABCD3*” (detailed search methods are given in Appendix S1); reference lists of relevant original articles and reviews were crosschecked. The following study inclusion criteria were required: (1) assessed patients with TIA by both ABCD2 and ABCD3-I; (2) included stroke events as outcomes; (3) reported stroke risks at 7 or 90 days; and (4) conducted a direct comparison of ABCD2 and ABCD3-I in the same population. We excluded papers that (1) were not written in English, and (2) did not report results from an independent cohort.

setting, number of patients, patient characteristics, interventions after admission, ABCD2 or ABCD3-I scores, and the number of future strokes at 7 or 90 days. Both ABCD2 and ABCD3-I scores were stratified into low-risk (0-3), medium-risk (4-5 for ABCD2, 4-7 for ABCD3-I), and highrisk (6-7 for ABCD2, 8-13 for ABCD3-I) stroke categories when reported in these studies. The ABCD2 was dichotomized at 4 points or higher and less than 4 points, which was the cut point of low and medium risk for meta-analysis, respectively, as used in the majority of studies and recommended in the UK guidelines.8,15,16 The ABCD3-I was also dichotomized at the cut point of low and medium risk (also 4 points) for analysis.12 As ABCD3-I had more possible points than ABCD2, an alternative cut point of 8 points or higher and less than 8 points for the ABCD3-I score was also analyzed. We used QUADAS-217 to assess the methodological quality of the studies. Two authors assessed the methodological quality, and any disagreement was resolved by consensus.

Statistical Analysis The 2 scoring systems were compared by assessing summary points (sensitivities and specificities) using bivariate models18 and the areas under the hierarchical summary receiver operating characteristic (HSROC) curves using hierarchical models.19 Moreover, the positive likelihood ratio (PLR), negative likelihood ratio (NLR), and diagnostic odds ratio (DOR) were also calculated to evaluate the diagnostic accuracy of the 2 scores. Formulae and definitions of predictive values are provided in supplementary Table S3. PLR ≥ 10, NLR ≤ .1, and DOR > 1 indicate good predictive value. To improve the clinical relevance of this meta-analysis, we applied the summary sensitivities, specificities, and pooled prevalence of stroke to a hypothetical cohort of 1000 patients with TIA. Cochran Q statistic of DOR after visual inspection of forest plots was used to assess the heterogeneity.20,21 For Cochran Q, P < .10 was considered statistically significant for heterogeneity. Furthermore, we conducted a leave-one-out sensitivity analysis and a subgroup analysis to detect heterogeneity. The study sample sizes, study settings, and study regions were determined to explore heterogeneity before subgroup analyses and meta-regression. Data were analyzed using RevMan (Nordic Cochrane Center, version 5.0, Copenhagen, Denmark) and R (R Core Team, version 3.3, Vienna, Austria) statistical programs.

Results Study Identification and Selection

Data Extraction and Quality Assessment Two authors independently extracted the following information from included studies: study methods, study

The literature search was performed in November 2016. The initial search yielded a total of 869 publications, of which we identified 204 publications as duplicates. Another

ARTICLE IN PRESS STROKE PREDICTION ACCURACY OF ABCD2 AND ABCD3-I

609 studies were excluded because of lack of relevance to this study after title and abstract screening. The full texts of 56 papers were obtained for formal assessment, of which another 50 papers were excluded (details in flow diagram in supplementary Figure S1). We included a total of 6 studies (7364 patients) that conducted direct comparisons of ABCD2 and ABCD3-I.

Study Characteristics and Methodological Quality All 6 studies were prospective in design. The summary characteristics of the included studies are summarized in Table 1. Three studies reported a total of 6165 participants with 188 strokes at 7 days,22-24 and 5 studies22,23,25-27 reported a total of 4349 participants with 285 stroke outcomes at 90 days (Table 2). In 5 studies, stroke specialists or neurologists used the time-based definition of TIA to make the TIA diagnoses. One study24 did not report the definition of TIA or the person who made the TIA diagnosis in its cohort. The QUADAS-2 scores for each domain are shown in supplementary Figure S2. The included studies met most of the criteria of QUADAS-2; thus, they were labeled as high-quality studies. All studies directly compared ABCD2 and ABCD3-I in the same cohort and reported outcomes using the same cutoff values.

Diagnostic Accuracy Four studies22,25-27 reported stroke outcome at 90 days. One study23 reported outcomes at 90 ± 14 days and was included as reporting 90-day outcomes for analysis. One study22 reported outcomes precisely at 7 days. Two studies23,24 reported “early” stroke outcomes (during hospitalization), and we also included these outcomes as 7-day outcomes in the meta-analysis. No significant heterogeneity in diagnostic accuracy was detected based on the 7-day or 90-day stroke outcomes in the ABCD2 or the ABCD3-I studies (all P > .10). The leave-one-out sensitivity analysis demonstrated that the estimates for the summary sensitivity and specificity were stable (supplementary Table S1). When we left out an Austrian study,23 the summary specificity was slightly elevated, although the difference was not significant. In the subgroup analysis (supplementary Table S2), the sample size and study region had no significant effects on the summary sensitivities and specificities. The results from the stroke unit study23 demonstrated a lower specificity compared with the other studies conducted in hospital or in emergency settings. We conducted the meta-analysis and summarized the extracted sensitivity and specificity based on the 7-day and 90-day outcomes (Table 3). The pooled sensitivities of ABCD2 versus ABCD3-I were 79.9% (62.2%90.6%) versus 96.1% (90.2%-98.5%) at 7 days (P = .022), and 76.6% (63.8%-85.8%) versus 94.6% (88.9%-97.5%) at 90 days (P = .001). The pooled specificities of ABCD2 versus ABCD3-I were 29.2% (18.2%-43.3%) versus 17.7% (8.5%-

3

33.3%) at 7 days (P = .214), and 40.3% (25.0%-57.7%) versus 20.2% (12.6%-30.6%) at 90 days (P = .032). The other parameters of diagnostic accuracy are shown in supplementary Table S3. The pooled PLR of ABCD2 versus ABCD3-I were 1.498 (1.257-1.785) versus 1.163 (1.0581.279) at 7 days, and 1.340 (1.114-1.613) versus 1.200 (1.0511.370) at 90 days. The HSROC curves for ABCD2 and ABCD3-I are shown in Figure 1. The areas under HSROC curves for ABCD2 versus ABCD3-I were .578 (.3190.821) versus .520 (.124-0.940) at 7 days, and .597 (.4240.730) versus .572 (.258-0.960) at 90 days. We also considered the scenario in which the cutoff point of ABCD3-I for low risk and high risk was set to 8 points. At 7 days, the pooled proportion of patients with recurrent stroke was 10% (95% confidence interval [CI], 4%-22%) for patients with ABCD3-I of 8 or higher and 4% (95%CI: 2%-10%) for patients with ABCD3-I lower than 8; at 90 days, it these values were 17% (95%CI: 9%-30%) and 5% (95%CI: 3%-9%), respectively. The pooled sensitivity and specificity were, respectively, .244 (95% CI: .083-0.533) and .892 (95% CI: .718-0.964) at 7 days, and .454 (95% CI: .371-0.540) and .822 (95% CI: .750-0.877) at 90 days. We applied the summary sensitivities and specificities of ABCD2 and ABCD3-I to a hypothetical cohort of 1000 patients with TIA (Table 4). Based on the pooled proportion of future stroke risk at day 7 (5%), the ABCD3-I score identified 8 more patients who would have a stroke event than the ABCD2 score. However, 109 more patients would be falsely identified as high risk by the ABCD3-I score. Based on the pooled proportion of future stroke risk at day 90 (7%), the ABCD3-I score identified 12 more patients who would have stroke events than the ABCD2 score. However, 187 more patients would be falsely identified as high risk by the ABCD3-I owing to its unsatisfactory specificity.

Discussion The comparison revealed that the ABCD3-I score had slightly better sensitivity but poorer specificity for predicting occurrence of a future stroke than ABCD2. However, both scores had relatively unsatisfactory predictive accuracy. Although the risk scores were predominantly intended for use during the initial triage and determining the urgency of specialist assessment, physicians should not entirely rely on risk scores for clinical decisionmaking. Moreover, the prognostic utility of each of the components of the scores should be carefully considered rather than dichotomized scores during clinical triage. The studies included in this meta-analysis comprised a wide patient population in terms of study countries, treatment care, settings, use of anticoagulants, and followup time points. No significant heterogeneity, however, was detected across studies. The following key features could partially explain the study homogeneity: all studies used the time-based definition for TIA; diagnoses of TIA and

4

Table 1. The summary characteristics of the included studies Admission from onset

Evaluating clinician

Outcome ascertainment

Chatzikonstantinou et al

Prospective

Time based

Not reported

Not reported

In hospital

Not reported

Dai et al

Prospective

Time based

Within 3 d

Neurologist

Clinical review or telephone

De Marchis et al

Prospective

Time based

Within 24 h

Stroke physicians

Telephone

At discharge: mono antiplatelets (47.9%); dual antiplatelets (40.4%); anticoagulants (4.1%). Not reported

Kiyohara et al

Prospective

Time based

Within 7 d

Neurologist

Telephone

Knoflach et al

Prospective

Time based

Within 6 h

Neurologist

In person with the treating physician or by telephone

Song et al

Prospective

Time based

Within 7 d

Stroke specialist

Face to face

Drugs

At discharge: antiplatelets (30.3%) or anticoagulants (23.7%). At discharge: aspirin or clopidogrel (74.1%); a combination of both (5.2%); a combination of aspirin and dipyridamole (4.5%); oral anticoagulation (15.4%). Lipidlowering drugs were prescribed to 70.9% and antihypertensive drugs

Lipid-lowering drug or antihypertension medication

DWI timing Performed either directly on admission or within 24 h from admission Within 3 d

Carotid stenosis criteria

Stroke definition

ECST criteria for carotid stenosis

Focal neurological deficit persisting for >24 h

≥50% luminal narrowing of the ipsilateral ICA

Focal neurological deficit persisting for >24 h

On admission

Not clear

On admission

≥50% luminal narrowing of the ipsilateral ICA ECST ≥ 70% (NASCET ≥ 50%)

Focal neurological deficit persisting for >24 h Focal neurological deficit persisting for >24 h

Not reported

Within 7 d

NASCET ≥ 50%

Early stroke was defined as a new-onset ischemic stroke manifesting after the index TIA or minor stroke during the stroke unit stay or as a progressive stroke emerging from initial minor stroke (worsening of more than 1 point on the NIHSS between admission and discharge from the stroke unit). Focal neurological deficit persisting for >24 h

Abbreviations: ECST, European Carotid Surgery Trial; ICA, internal carotid artery; NASCET, North American Symptomatic Carotid Endarterectomy Trial; NIHSS, National Institutes of Health Stroke Scale; TIA, transient ischemic attack.

ARTICLE IN PRESS

TIA definition

M. ZHAO ET AL.

Study design

Study

ARTICLE IN PRESS STROKE PREDICTION ACCURACY OF ABCD2 AND ABCD3-I

Table 2.

5

Number of stokes and pooled proportion with ABCD2 or ABCD3-I score of 4 points or higher or less than 4 points at 7 days and 90 days 7d

ABCD2

ABCD3-I

All ≥4 <4 All ≥4 <4

90 d

n (%)

Strokes

Pooled proportion % (95% CI)

n (%)

Strokes

Pooled proportion % (95% CI)

6165 4842 (78.5) 1323 (21.5) 6154 5430 (88.2) 724 (11.8)

188 159 29 187 181 6

4.9 (2.1-11.0) 5.5 (2.3-12.2) 3.3 (1.1-9.3) 4.9 (2.1-10.9) 5.8 (2.3-13.6) 1.0 (.4-2.5)

4349 3071 (70.6) 1278 (29.39) 4259 3632 (85.27) 627 (14.72)

285 226 59 281 271 10

7.5 (4.6-12.0) 9.9 (5.3-17.9) 4.5 (2.8-7.3) 7.3 (4.4-12.1) 8.7 (4.8-15.3) 2.1 (1.0-4.8)

score evaluations were performed by specialists or neurologists, except for 1 study that did not report diagnoses in detail; and all studies were conducted in a prospective cohort. Furthermore, we found no statistical evidence that sensitivity or specificity differed according to the results of the leave-one-out sensitivity analysis. In the subgroup analysis, the results of ABCD2 and ABCD3-I scores from a stroke unit study showed a lower specificity. The author suggested that the risk factors that were once considered to be associated with future stroke tend to be less effective in stroke unit settings, as patients can receive more professional care. As only 1 study23 was conducted in a stroke unit, further studies are needed before firm conclusions can be drawn. The optimum scoring system for predicting stroke is one that maximizes both sensitivity and specificity. In this

study, better sensitivity means the scoring system misses fewer future stroke patients, and better specificity means a patient with ABCD2 or ABCD3-I score higher than or equal to the cut point (4 or 8) is more likely to experience a future stroke. Several studies have suggested that ABCD3-I scores are superior to ABCD2 scores in terms of the predictive accuracy of stroke.22,25 Studies have drawn this conclusion mainly by identifying a significant association of the added terms in the ABCD3-I scoring system and future stroke. However, which specific term elevates the predictive power remains controversial. A recent pooled analysis of individual patient data suggested the positive predictive values of all 3 additional items—diffusion-weighted imaging (DWI; OR 3.8; 2.17.0; P < .001), dual TIA (OR: 3.3; 95%CI: 1.8-5.8; P < .001), and ipsilateral carotid stenosis (OR: 4.7; 95%CI: 2.6-8.6;

Table 3. Stroke prediction accuracy of ABCD2 and ABCD3-I for individual study Study ABCD2 (7 d) Kiyohara et al Chatzikonstantinou et al Knoflach et al ABCD3-I (7 d) Kiyohara et al Chatzikonstantinou et al Knoflach et al ABCD2 (90 d) Kiyohara et al Song et al De Marchis et al Dai et al Knoflach et al ABCD3-I (90 d) Kiyohara et al Song et al De Marchis et al Dai et al Knoflach et al

Year

Setting

Patients

TP

FN

FP

TN

Sens %

Spec %

2013 2013 2016

Hospital Stroke center or unit Stroke center or unit

693 235 5237

37 12 110

11 5 13

467 124 4092

178 94 1022

77.1 70.6 89.4

27.6 43.1 20.0

2013 2013 2016

Hospital Stroke center or unit Stroke center or unit

682 235 5237

45 16 120

2 1 3

544 145 4560

91 73 554

95.7 94.1 97.6

14.3 33.5 10.8

2013 2013 2014 2015 2016

Hospital Hospital Emergency department Hospital Stroke center or unit

693 239 302 658 2457

58 23 8 44 93

13 6 3 26 11

446 86 167 238 1908

176 124 124 350 445

81.7 79.3 72.7 62.9 89.4

28.3 59.0 42.6 59.5 18.9

2013 2013 2014 2015 2016

Hospital Hospital Emergency department Hospital Stroke center or unit

682 239 223 658 2457

67 29 5 68 102

5 0 1 2 2

522 161 157 394 2127

88 49 60 194 226

92.5 98.3 78.6 96.5 97.6

14.5 23.5 27.8 33.0 9.6

Abbreviations: FN, false negative; FP, false positive; Sens, sensitivity; Spec, specificity.; TN, true negative; TP, true positive.

ARTICLE IN PRESS 6

M. ZHAO ET AL.

Figure 1. Summary points and hierarchical summary receiver operating characteristic (HSROC) with 95% confidence contour illustrating the study estimate of the sensibility and specificity for ABCD2 and ABCD3-I. (A) ABCD2 at 7 days; (B) ABCD3-I at 7 days; (C) ABCD2 at 90 days; (D) ABCD3-I at 90 days.

P < .001)—could contribute to the improved stroke risk prediction of ABCD3-I compared with ABCD2 and ABCD2-I after TIA at 7 days. Song et al,26 Dai et al,25 and Knoflach et al23 suggested that acute DWI hyperintensity after TIA is significantly associated with stroke. However, Wardlaw et al16 reported that two thirds of patients with TIA do not have an acute ischemic lesion on DWI and suggested that DWI is not cost-effective for secondary stroke prevention. Similarly, dual TIA is not always reliable when used as a stand-alone parameter. Kiyohara et al and Song et al reported that dual TIA was a significant risk factor for stroke at 90 days. However, in Knoflach et al’s and Dai et al’s cohorts, the results were negative. Severe carotid stenosis has long been recognized as a risk factor for stroke events after TIA.28-30 Diener and Frank31 suggested that the ABCD2 score should be improved, as the score might miss patients with symptomatic carotid stenosis who need urgent therapy, such as surgery or stenting. One in 5 patients with an ABCD2 score below 4 has carotid stenosis of more than 50%.8 It is conceivable that adding carotid artery imaging to the

ABCD2 scoring system could elevate the score’s predictive power.22,25,32,33 In Asian countries, however, intracranial artery stenosis is more prevalent than carotid stenosis.34,35 A Japanese study developed a new score, ABCD-I(d,c/ i), by adding intracranial arterial stenosis to the ABCD3-I score. This novel scoring system was validated in Japanese patients and found to be significantly associated with stroke, even in the long term for up to 3 years.22 In our study, both scoring systems demonstrated good sensitivities but poor specificities. The area under the HSROC curves for both scores were not satisfactory. These results, together with the PLRs and NLRs, suggested that neither the ABCD2 nor the ABCD3-I scores achieved good predictive power of future stroke. For ABCD3-I, the specificity was worse (P = .03), which barely reached 20% based on 7-day and 90-day outcomes. Moreover, ABCD3-I has no utility in the pre-hospital setting, as DWI is generally not available to community-based clinicians who make referrals. We believe it is better to use ABCD2 at initial triage and deciding on urgency of specialist assessment in community-based referring settings. The benefit of the

ARTICLE IN PRESS 54 (45,60) 16 (10, 25) 555 (393, 697) 375 (232, 537) 66 (62, 68) 4 (2, 8) 742 (645, 813) 188 (117, 285) 12 more 187 more

ABCD2 ABCD3-I

38 (32, 43) 12 (7, 18) 567 (402, 712) 383 (238,548) 47 (44, 49) 3 (1, 6) 758 (659, 830) 192 (120, 291) 9 more 191 more 56 (44, 63) 14 (7, 26) 658 (527, 761) 272 (169, 403) 67 (63, 69) 3 (1, 7) 765 (620, 851) 165 (79, 310) 9 more 107 more

ABCD2 ABCD3-I ABCD2 ABCD3-I

Prevalence 5% Prevalence 7%

*True positive, received appropriate treatment. †False negative, did not receive required treatment. ‡False positive, received unnecessary treatment. §True negative, did not receive unnecessary treatment. ‖Pooled proportion.

40 (31, 45) 10 (5, 19) 673 (539, 777) 277 (173, 411) 48 (45, 49) 2 (1, 5) 782 (634, 869) 168 (81, 316) 8 more 109 more TP* FN† FP‡ TN§ Mean difference in TP Mean difference in FP

ABCD2 ABCD3-I

Prevalence 5%‖

7d

Table 4. Effect of ABCD2 and ABCD3-I score in a hypothetical cohort of 1000 patients with TIA

90 d

Prevalence 7%‖

STROKE PREDICTION ACCURACY OF ABCD2 AND ABCD3-I

7

better sensitivity of ABCD3-I is offset because more patients are unnecessarily identified as high risk. The ABCD3-I comprises more points than ABCD2, indicating that it would identify more patients as high risk of stroke for the same cut point as ABCD2. However, identifying more patients as high risk did not necessarily lead to lower sensitivity, which depends on the predictive utility of the additional points of the ABCD3-I system. To address this issue, we further considered the scenario in which the cut point for high risk was set to 8 points. Although the specificity increased significantly, the poor sensitivity was unacceptable for a score aimed to triage stroke risk, failing to identify more than half of future stroke patients. For a score intended for initial triage and deciding on urgency of specialist assessment, we could not dichotomize the ABCD3-I at 8 points because of low sensitivity. We then evaluated the ABCD2 and ABCD3-I scores in a hypothetical cohort of 1000 patients with TIA to better understand their clinical applications. The satisfying sensitivities of the ABCD2 and ABCD3-I scores enabled them to identify most patients with future strokes. However, the unsatisfactory specificities of both scores led to far too many false positives, which suggested possible iatrogenic harm, such as unnecessary hospitalization, healthcare-associated infections, and side effects from aggressive antiplatelet therapy, for more than half of patients with TIA. Although risk scores have limited sensitivity and specificity, the use of a triage tool may still help improve outcomes without risking iatrogenic harm and raising healthcare costs. In contrast to National Institute for Health and Care Excellence guidelines, however, the American Heart Association/American Stroke Association guidelines stated that no clinical risk tool has sufficient predictive power to be recommended. No randomized trial has evaluated the benefit of hospitalization or the utility of the ABCD2 or ABCD3-I scores for assisting with triage decisions. To date, there is only 1 randomized controlled trial (RCT) assessing a TIA triage tool: the TIA/Stroke Electronic Support Tool. The use of the TIA/Stroke Electronic Support Tool resulted in better patient outcomes and lower treatment cost.36 In future studies, it is essential to consider RCTs that occur in a setting where the ABCD2 and ABCD3-I scores are actually used to make clinical decisions and include a placebo group. Our study included some limitations. First, the number of studies included in our meta-analysis was limited. However, these 6 studies were well-designed, prospective, high-quality studies that directly compared the 2 target scores. No significant heterogeneity was detected across the studies. Second, most TIA diagnoses were made by specialists in the included studies. The results of the included studies are heavily confounded by the fact that essentially all included patients received hyperacute TIA management in a tertiary hospital or a stroke center regardless of their scores. The scores and initial triage were

ARTICLE IN PRESS M. ZHAO ET AL.

8

more often performed by nonspecialists in outpatient settings in common practice.37,38 The results might be difficult to replicate in clinical applications. In conclusion, ABCD3-I scores had a better sensitivity but poorer specificity than ABCD2 scores when the common cut points were used. RCTs assessing the stroke prediction scores and actual patient outcomes are needed.

Appendix: Supplementary Material Supplementary data to this article can be found online at doi:10.1016/j.jstrokecerebrovasdis.2017.05.030.

15.

16.

17.

18.

References 1. Johnston SC, Fayad PB, Gorelick PB, et al. Prevalence and knowledge of transient ischemic attack among US adults. Neurology 2003;60:1429-1434. 2. Giles MF, Rothwell PM. Risk of stroke early after transient ischaemic attack: a systematic review and meta-analysis. Lancet Neurol 2007;6:1063-1072. 3. Correia M, Silva MR, Magalhães R, et al. Transient ischemic attacks in rural and urban Northern Portugal. Stroke 2006;37:50-55. 4. Cucchiara BL, Messe SR, Taylor RA, et al. Is the ABCD score useful for risk stratification of patients with acute transient ischemic attack? Stroke 2006;37:1710-1714. 5. Johnston SC, Rothwell PM, Nguyen-Huynh MN, et al. Validation and refinement of scores to predict very early stroke risk after transient ischaemic attack. Lancet 2007;369:283-292. 6. Kernan WN. Stroke after transient ischaemic attack: dealing in futures. Lancet 2007;369:251-252. 7. Kernan WN, Ovbiagele B, Black HR, et al. Guidelines for the prevention of stroke in patients with stroke and transient ischemic attack: a guideline for healthcare professionals from the American Heart Association/ American Stroke Association. Stroke 2014;45:2160-2236. 8. Wardlaw JM, Brazzelli M, Chappell FM, et al. ABCD2 score and secondary stroke prevention: meta-analysis and effect per 1000 patients triaged. Neurology 2015;85:373-380. 9. Amarenco P, Labreuche J, Lavallée PC. Patients with transient ischemic attack with ABCD2 <4 can have similar 90-day stroke risk as patients with transient ischemic attack with ABCD2 ≥4. Stroke 2012;43:863-865. 10. Sanders LM, Srikanth VK, Blacker DJ, et al. Performance of the ABCD2 score for stroke risk post TIA: meta-analysis and probability modeling. Neurology 2012;79:971-980. 11. Perry JJ, Sharma M, Sivilotti MLA, et al. Prospective validation of the ABCD2 score for patients in the emergency department with transient ischemic attack. Can Med Assoc J 2011;183:1137-1145. 12. Merwick Ã, Albers GW, Amarenco P, et al. Addition of brain and carotid imaging to the ABCD2 score to identify patients at early risk of stroke after transient ischaemic attack: a multicentre observational study. Lancet Neurol 2010;9:1060-1069. 13. Kelly PJ, Albers GW, Chatzikonstantinou A, et al. Validation and comparison of imaging-based scores for prediction of early stroke risk after transient ischaemic attack: a pooled analysis of individual-patient data from cohort studies. Lancet Neurol 2016;15:1238-1247. 14. Moher D, Liberati A, Tetzlaff J, et al. Preferred reporting items for systematic reviews and meta-analyses: the

19.

20.

21.

22.

23.

24.

25.

26.

27.

28.

29.

30. 31.

32.

33.

PRISMA statement. Ann Intern Med 2009;151:264-269. W64. Thorneycroft D. National clinical guideline for diagnosis and initial management of acute stroke and transient ischaemic attack (TIA). 1-123, 2008. Wardlaw J, Brazzelli M, Miranda H, et al. An assessment of the cost-effectiveness of magnetic resonance, including diffusion-weighted imaging, in patients with transient ischaemic attack and minor stroke: a systematic review, meta-analysis and economic evaluation. Health Technol Assess 2014;18:1-368, v-vi. Whiting PF, Rutjes AWS, Westwood ME, et al. QUADAS2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med 2011;155:529-536. Reitsma JB, Glas AS, Rutjes AWS, et al. Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews. J Clin Epidemiol 2005;58:982-990. Rutter CM, Gatsonis CA. A hierarchical regression approach to meta-analysis of diagnostic test accuracy evaluations. Stat Med 2001;20:2865-2884. Takwoingi Y, Riley RD, Deeks JJ. Meta-analysis of diagnostic accuracy studies in mental health. Evid Based Ment Health 2015;18:103-109. Kim KW, Lee J, Choi SH, et al. Systematic review and meta-analysis of studies evaluating diagnostic test accuracy: a practical review for clinical researchers—part I. Gen Guid Tips 2015;16:1175-1187. Kiyohara T, Kamouchi M, Kumai Y, et al. ABCD3 and ABCD3-I scores are superior to ABCD2 score in the prediction of short- and long-term risks of stroke after transient ischemic attack. Stroke 2014;45:418-425. Knoflach M, Lang W, Seyfang L, et al. Predictive value of ABCD2 and ABCD3-I scores in TIA and minor stroke in the stroke unit setting. Neurology 2016;87:861-869. Chatzikonstantinou A, Wolf ME, Schaefer A, et al. Risk prediction of subsequent early stroke in patients with transient ischemic attacks. Cerebrovasc Dis 2013;36:106-109. Dai Q, Sun W, Xiong Y, et al. From clinical to tissue-based dual TIA: validation and refinement of ABCD3-I score. Neurology 2015;84:1426-1432. Song B, Fang H, Zhao L, et al. Validation of the ABCD3-I score to predict stroke risk after transient ischemic attack. Stroke 2013;44:1244-1248. De Marchis GM, Weck A, Audebert H, et al. Copeptin for the prediction of recurrent cerebrovascular events after transient ischemic attack. Stroke 2014;45:2918-2923. Josephson SA, Bryant SO, Mak HK, et al. Evaluation of carotid stenosis using CT angiography in the initial evaluation of stroke and TIA. Neurology 2004;63:457-460. Blaser T, Hofmann K, Buerger T, et al. Risk of stroke, transient ischemic attack, and vessel occlusion before endarterectomy in patients with symptomatic severe carotid stenosis. Stroke 2002;33:1057-1062. Norris JW, Zhu CZ, Bornstein NM, et al. Vascular risks of asymptomatic carotid stenosis. Stroke 1991;22:1485-1490. Diener H-C, Frank B. Stroke: stroke prevention—time to say goodbye to the ABCD2 score? Nat Rev Neurol 2015;11:552-553. Sheehan OC, Kyne L, Kelly LA, et al. Population-based study of ABCD2 score, carotid stenosis, and atrial fibrillation for early stroke prediction after transient ischemic attack. Stroke 2010;41:844-850. Purroy F, Montaner J, Molina CA, et al. Patterns and predictors of early risk of recurrence after transient ischemic attack with respect to etiologic subtypes. Stroke 2007;38:3225-3229.

ARTICLE IN PRESS STROKE PREDICTION ACCURACY OF ABCD2 AND ABCD3-I 34. Holmstedt CA, Turan TN, Chimowitz MI. Atherosclerotic intracranial arterial stenosis: risk factors, diagnosis, and treatment. Lancet Neurol 2013;12:1106-1114. 35. Gorelick PB, Wong KS, Bae H-J, et al. Large artery intracranial occlusive disease. Stroke 2008;39:2396-2399. 36. Ranta A, Dovey S, Weatherall M, et al. Cluster randomized controlled trial of TIA electronic decision support in primary care. Neurology 2015;84:1545-1551.

9 37. Bradley D, Cronin S, Kinsella JA, et al. Frequent inaccuracies in ABCD2 scoring in non-stroke specialists’ referrals to a daily Rapid Access Stroke Prevention service. J Neurol Sci 2013;332:30-34. 38. Brazzelli M, Shuler K, Quayyum Z, et al. Clinical and imaging services for TIA and minor stroke: results of two surveys of practice across the UK. BMJ Open 2013;3:e003359.