- Email: [email protected]
A stroke prediction score in the elderly
Journal of Clinical Epidemiology 55 (2002) 129–136
A stroke prediction score in the elderly: validation and Web-based application Thomas Lumleya,*, Richard A. Kronmala, Mary Cushmanb, Teri A. Manolioc, Steven Goldsteind a University of Washington, CHS Coordinating Center, Plaza 600 Building, Suite 700, 600 Stewart St., Seattle, WA 98101, USA University of Vermont, Asst. Professor/Dept. of Medicine, University of Vermont-Pathology, Colchester Research Facility, #T205, 208 South Park Drive, Suite 2, Colchester, VT 05446, USA c National Heart Lung and Blood Institute, Director, Epid/Biometry Program, NIH/NHLBI/DECA, Two Rockledge Center, Rm. 8160, 6701 Rockledge Drive MSC 7934, Bethesda, MD 20892-7934, USA d University of Pittsburgh, UPMC Stroke Institute, Room C400 PUH, 200 Lothrop Street, Pittsburgh, PA 15213, USA Received 18 October 2000; received in revised form 1 August 2001; accepted 3 August 2001 b
Abstract The objective of this study was to construct a prediction model for predicting stroke in an elderly U.S. population, and to assess the accuracy in this population of other previously published prediction models. The subjects were participants in the Cardiovascular Health Study: 2,495 men and 3,393 women age 65 years and older at baseline, and followed for 6.3 years. Among 5,711 participants free of baseline stroke, 399 strokes occurred. Sex-specific prediction equations were constructed using study variables that were most importantly related to incident stroke: age, systolic blood pressure, diabetes, ECG diagnosis of atrial fibrillation or left ventricular hypertrophy, confirmed history of cardiovascular disease, diabetes, time to walk 15 ft, and serum creatinine. The prediction rule was implemented as a risk score and in a Web-based interactive Java applet. Overall, the model predicted 5-year stroke risks ranging from less than 1 to 59%. The 20% of subjects in the highest predicted risk group had a 5-year actual stroke incidence rate of 15%, while the 20% lowest risk group had a 1% incidence. Risk scores from two other studies performed well in these study participants. Effective discrimination between low and high stroke risk in the elderly was possible in this cohort with data that are easy to obtain. Evaluation of the generalizability and clinical usefulness of this prediction model requires further research. © 2002 Elsevier Science Inc. All rights reserved. Keywords: Risk; Proportional hazards models; Aged; Cerebrovascular disorders
1. Introduction Stroke is a major cause of mortality and disability in the United States, particularly in the elderly. A number of potent risk factors are known such as age, hypertension, and atrial fibrillation [1]. This makes it likely that stroke risk can be predicted relatively accurately. Such prediction would be of clinical and public health value, as stroke risk can be reduced in high-risk subgroups by treatment of hypertension, anticoagulation, carotid endarterectomy, or modification of other risk factors [2]. The Cardiovascular Health Study [3] (CHS) is a population-based cohort study of 5,888 men and women age 65 and older at study entry who, at the time of analysis, have been followed for a median of 6.3 years. The purpose was to investigate risk factors for cardiovascular diseases in the elderly populations. A previous article [1] reported the risk * Corresponding author. Fax: 206-543-8864. E-mail address: [email protected] (T. Lumley)
factors for stroke in this cohort. In this article we use these risk factors and those identified in other studies to construct a prediction model for stroke over a 5–7-year period in this older age group. The resulting model is presented both as a risk score calculation and as a Java applet (program that runs in a Web browser) available from the CHS World Wide Web site (http://chs3.chs.biostat.washington.edu/chs/ stroke.htm). The Java applet allows risk calculations to be performed interactively using any Web browser that supports Java 1.1. Using this method, patient information can be simply entered on a computer, which applies complex statistical models that yield instantaneous calculation of a risk score. Two stroke risk prediction models have been published: the Framingham Study model, and the Israeli Ischemic Heart Disease Project [5] (IIHD) model. The Framingham model was based on a middle to older aged population living in a single community. The IIHD model was based on an ethnically diverse, even younger population of male civil and municipal employees in Israel. To address generaliz-
0895-4356/02/$ – see front matter © 2002 Elsevier Science Inc. All rights reserved. PII: S0895-4356(01)00 4 3 4 - 6
130
T. Lumley et al. / Journal of Clinical Epidemiology 55 (2002) 129–136
ability of these scores, we applied the Framingham and IIHD models in the CHS cohort. 2. Methods The CHS study design, including definitions of variables, has been published [3]. The cohort was recruited in two waves—the first wave of 5,201 in 1989–1990 from random samples of the Health Care Financing Administration Medicare eligibility lists in Forsyth Country, NC; Sacramento County, CA; Washington County, MD; and Allegheny County, PA. The second wave of 687 was recruited 2 years later from the same population but restricted to African-Americans to improve the representativeness of the cohort. The major reasons for exclusion were being institutionalized or under current treatment for cancer. Table 1 shows the distribution of some important risk factors. A detailed description of the stroke ascertainment has been published [1]. Potential stroke events were identified at annual follow-up examinations and at 6-month interim phone contacts. The participant or next of kin was interviewed about the surrounding circumstances. Hospital records for all reported strokes and for all hospitalizations with ICD-9 [6] codes 430–438, identifying cerebrovascular disease, were reviewed by a neurologist at each field center. Information on reported, nonhospitalized stroke patients was obtained by physician questionnaire. This information was reviewed by a CHS neurologist and any inconsistencies reviewed with the participant’s physician. Films of CT or MRI scans were available in 70% of cases; hard copy reports in an addition 17%. These were also reviewed centrally. Potential stroke cases were adjudicated by a committee of neurologists, neuroradiologists, and internists; their
decisions were usually unanimous. Criteria for stroke were similar to those used in the Systolic Hypertension in the Elderly Program (SHEP) [7]. Although nearly 90% of strokes were able to be classified into ischemic or hemorraghic, this classification was not used in the analysis, largely because interest was in the clinical event “stroke.” In addition, as the unclassifiable strokes outnumbered those classified as hemorraghic (4%) it would be hard to separate the two in analysis. The Framingham and IIHD models also used a clinical definition of stroke, and did not distinguish ischemic from hemorrhagic strokes in their modeling. Where definitions are not explicitly given here, they follow those described for the CHS study as a whole [3]. Stroke predictors included were demographic information (sex, race, age, smoking status, and history), clinical history [diabetes, use of antihypertensive medications, history of coronary heart disease (CHD), peripheral vascular disease, congestive heart failure, transient ischemic attack, atrial fibrillation], and physical and biochemical measurements (systolic blood pressure, time to walk 15 ft, serum creatinine, ECG determinations of atrial fibrillation and left ventricular hypertrophy, ratio of HDL to total cholesterol). Ultrasound assessment of carotid wall thickness was the average of internal and common carotid maximum wall thickness each divided by its standard deviation, as previously described [8]. Blood pressure was measured in the right arm with the subject supine, after 5 min rest, using a Doppler probe. Diabetes was based on American Diabetes Association classification, with diabetes defined as fasting glucose 126 mg/dL (7.0 mmol/L) or treatment with insulin or oral hypoglycemic agents, and impaired fasting glucose defined as fasting glucose between 110 and 126 mg/dL (6.1 and 7.0 mmol/L).
Table 1 Risk factors included in the CHS, IIHD, and Framingham models and coefficients for the CHS model (the last six variables were not in the CHS model but were in one of the other models) Coefficientc
Mean
LVH by ECG Diabetes Impaired fasting glucose Creatinine 1.25 mg/dL (110 mol/L) 15 ft walk time Systolic blood pressure History of heart disease Atrial fibrillation by ECG Age Use of antihypertensive drugs Current smokers Current smoking rate History of PVD History of CHD % HDL cholesterol a
(2,495 men)
(3,393 women)
(men)
(women)
5.1% 14.9% 19.0% 39.6% 5.5 s 143 mmHg 26.5% 3.5% 73 year 44% 11.1% 21 cigs/day 4.1% 25.7% 24.4%
4.9% 12.5% 14.4% 8.1% 6.0 s 144 mmHg 16.1% 2.1% 73 year 48% 12.5% 16 cigs/day 1.5% 14.7% 27.2%
0.501 0.521 0.347 0.141 0.099b .172/10 mmHga 0.445 0.410 .382/10 year — — — — —
0.073 1.35 0.613/10 year — — — — —
Systolic blood pressure below 120 mmHg set to 120 mmHg. 15 ft walk times above 20 s were set to 20 s. c In the absence of important risk modification a single coefficient is given for men and women as described in the Methods section. b
T. Lumley et al. / Journal of Clinical Epidemiology 55 (2002) 129–136
Amount of current smoking was used in the IIHD risk model but was not measured in the CHS cohort. To apply the IIHD model to CHS, for current smokers we used the subject’s reported typical number of cigarettes per day over their entire smoking history. There were three vascular disease history variables used in these analyses. The IIHD model used history of CHD, defined as angina or myocardial infarction (MI), and history of peripheral vascular disease (PVD), defined as claudication in the legs. The Framingham model combined these two with coronary insufficiency and congestive heart failure (CHF) into one variable, here called “history of heart disease.” These variables do not correspond exactly to any classifications used in CHS. To match the IIHD definitions as closely as possible we defined “history of CHD” as definite or possible history of MI, angina, ECG evidence of past silent MI, coronary bypass surgery, or angioplasty and “history of PVD” as definite or possible history of claudication in the legs. “History of heart disease” was then defined to match the Framingham definition as history of CHD or PVD or definite or possible history of CHF. Data analysis was performed using the S-PLUS statistical package, versions 4.5 and 2000. The 5711 CHS participants without a prebaseline history of stroke were divided at random, two-thirds (n 3,799) in a fitting set and one-third (n 1,912) in a validation set. A separate validation set gives an unbiased assessment of the predictive power and goodness of fit of a model even when a complex model selection strategy is used. Participants with missing values for any of the variables were excluded. Of the 5,711 participants, only 480 (8.5%) had missing data for any of the variables in the final model. These participants tended to have worse values for all the predictors studied, but a multiple imputation analysis [9] did not give appreciably different estimates or confidence intervals. The Cox stratified proportional hazards model [10] was used as the primary model for stroke prediction, although nonproportional hazards models and survival tree (recursive partitioning) models [11] were also explored. The stratified model allows the shape of the survival curve to differ between men and women, and allows either separate or common hazard ratios by sex. Separate hazard ratios by sex were used if the difference in hazard ratios was large or statistically significant. Variables were explored in groups with routine clinical information (blood pressure, age, sex, medical history) included first and less routine measurements entered later if they provided substantial independent prediction. The functional form of the continuous variables was explored by graphing fitted regression splines [12] with knots at the quintiles, and by smoothed residual plots [13]. Both of these methods can estimate any continuous dose–response relationship. Simple functional forms were then chosen based on these graphs, with logarithmic and exponential transformations, and (possibly discontinuous) linear functions with thresholds being considered. If the graphs suggested more than one candidate transformation the partial likelihood for
131
models with different transformations was compared. This led to linear functions with thresholds being chosen for two variables, and one (creatinine) being fitted by dichotomizing. Variables were retained in the model if they were significant at the .05 level or if they had large coefficients and were significant in the Framingham risk model. Deviations from proportional hazards were assessed using the methods of Grambsch and Therneau [14], which provide both formal tests and graphical estimates of lack of fit. After the prediction model was selected the fitting and validation subsets were combined and the model was refit to provide more precise parameter estimates. The predicted survival curves given by the Java applet compute the predicted survival at 1-year intervals and interpolate linearly between these points. Predictive accuracy of the model was assessed using Receiver Operating Characteristic (ROC) curves and by comparing observed and expected survival curves. The ROC curve shows the ability of a risk score to discriminate between high- and low-risk subjects. Any specific value of predicted risk can be used to divide people into high- and low-risk groups. For any given value, the true positive rate (sensitivity) was the proportion of those with strokes who were classified as high risk, and the false positive rate (1specificity) was the proportion without strokes who were classified as high risk. These numbers were plotted against each other for each possible cut point of predicted risk. A larger area under the curve indicates better discrimination. To compare area under the curve for different cut points defining risk, the fitting set was resampled and the CHS risk model refitted, then the validation set was resampled and the ROC curves for the two models calculated. This was done 500 times, and a confidence interval for the difference in area under the ROC curve was calculated using the percentile bootstrap method [15]. The expected survival curve for a group is calculated by averaging the predicted survival for each member of the group. This is then compared to the observed survival curve computed by the Kaplan-Meier method. Inaccuracies in the model cause the expected and observed survival curves to differ; in particular, overfitting of the model will cause the expected survival curves to be more widely separated than the observed survival curves. 3. Results A total of 399 strokes occurred after a median follow-up of 6.3 years, giving a rate of approximately 1.3%/year. Fifteen (3.8%) strokes were hemorrhagic and 344 (86%) were ischemic, with the remainder of unknown type. No distinction was made in the final analysis: prediction of the clinical event “stroke” was the goal of the model, the a priori most important modifiable predictor, blood pressure, is relevant to both types of stroke, and the previous models have not distinguished between stroke types. The variables included in this and the two previous risk models profiles are shown
132
T. Lumley et al. / Journal of Clinical Epidemiology 55 (2002) 129–136
Fig. 1. Observed (solid lines) and predicted (dotted lines) survival distributions for upper, middle, and lower tertiles of risk in the validation subset.
in Table 1, with their mean levels for men and women and their multiple regression coefficients in the CHS model. Transformations were considered as described above for all the continuous variables. Three of the continuous variables were transformed for inclusion in the final model. Systolic blood pressure showed a linear association above approximately 120 mmHg, and no effect below this level, consistent with the JNC-V guidelines on hypertension [16]. The time to walk 15 ft showed a linear association below a threshold of 20 sec. Serum creatinine was dichotomized at 1.25 mg/dL (110.6 mol/L). Carotid wall thickness was statistically significant when added to the model but did not improve the predictions substantially so it was not retained in the final model. Percent carotid stenosis was also explored, but was less strongly predictive than carotid wall thickness. Interactions between carotid wall thickness or carotid stenosis and other predictors were also explored but did not give substantially better prediction. The model coefficients are listed in Table 1. Figure 1 shows the predictive accuracy of the CHS model. We divided the validation set into three groups based on the tertiles of 5-year risk and computed the observed stroke survival curve (solid line) and the predicted curves (dashed line) given by the CHS model. These curves show that the risk was accurately estimated in the middle group, but was underestimated in the high-risk group and overestimated in the low-risk group. The ability of the CHS model and the Framingham model to discriminate high and low risk subjects is summarized by Table 2 and Figures 2 and 3. Table 2 shows the in-
cidence of stroke in the validation set by quintile of 5-year risk for the two risk models profiles. There was is a 15-fold incidence gradient across these five groups, showing powerful discrimination between high- and low-risk individuals. These crude incidence rates were lower than the stroke probabilities that would be computed from 1-survival probabilities in Figure 1 because the survival probabilities were adjusted for differing lengths of followup caused by different accrual times, death, and loss to follow-up. Figure 2 is a ROC curve for 5-year stroke incidence in the validation set. The lower left portion of the curve shows discrimination in high risk subjects, where the CHS model (solid line) predicted better. The upper right portion shows low risk subjects, where the Framingham model (dotted line) predicted better. The dashed line gives the prediction based on carotid wall thickness alone. As the ROC curve shows, carotid wall thickness alone performs almost as well as the fitted risk model in higher risk subjects, although it did not discriminate as well in moderate to low risk subjects. Figure 3a and 3b shows separate ROC curves for men and women. For men these are based on the Framingham, Table 2 Five-year stroke incidence by quintile of the CHS and Framingham predicted risk Quintile
CHS model
Framingham model
1Highest 2 3 4 5lowest
54/351 (15%) 39/350 (11%) 21/351 (6%) 14/350 (4%) 4/351 (1%)
51/351 (15%) 39/350 (11%) 28/351 (8%) 12/350 (3%) 2/351 (1%)
T. Lumley et al. / Journal of Clinical Epidemiology 55 (2002) 129–136
133
Fig. 2. Receiver Operating Characteristic Curve for 5-year stroke incidence: comparing predictions from the CHS and Framingham risk models profiles and from carotid wall thickness alone in the validation subset.
CHS, and IIHD models, and for women on the Framingham and CHS models. In men, all three models were almost identical over the clinically important higher risk range, although the CHS model was less predictive at low to moderate risks. In women, the CHS model predicted much better than the Framingham model. Combining men and women, the CHS and Framingham models both gave an area under the ROC curve of 0.74 (P .56 for difference). For men the Framingham model gave an area of .69 and the CHS model .65 (P .10). For women the Framingham model gave an area of .73 and the CHS model .77 (P .044). As the Framingham model appears to predict stroke in elderly women less effectively, it is of interest to know where the model predictions differ. Women for whom the CHS risk score and Framingham risk score are very different are more likely to be current smokers, with a history of cardivascular disease and taking antihypertensive medications. The Framingham model weights these variables more strongly; they may be more important in younger populations or the differences may reflect changes over time in risks or treatments for women. The 15-ft timed walk is not available in all epidemiologic studies, and is not validated for use in general practice. In the CHS risk model, imputation of a value of 5 sec for the timed walk gave very similar rankings (ROC area .73) compared to the full model (ROC area .73), but the absolute risk of stroke was underestimated by 10–15% in some high-risk individuals. The other variable that may not always be routinely available in practice is serum creatinine. Good predictions were still be obtained by assuming that se-
Fig. 3. Receiver Operating Characteristic Curve for 5-year stroke incidence in men (a) and in women (b): comparing predictions from the CHS, Framingham and (for men) IIHD risk models profiles in the validation subset. The IIHD risk profile is defined only for men.
rum creatinine was less than 1.25 mg/dL (110.6 mol/L) (data not shown). An example of the Web-based Java applet is shown in Figure 4. After input of the patient data, the screen displays the 3-year and 5-year probabilities of stroke, and a graph showing the risk of stroke over time up to 6 years. Figure 4 shows results for an 80-year-old man with systolic blood pressure of 140 mmHg, diabetes, and no other risk factors,
134
T. Lumley et al. / Journal of Clinical Epidemiology 55 (2002) 129–136
stroke risk in men or women. The stroke risks were obtained by computing the risk using the continuous model for each CHS participant whose risk points fell into that category and taking the median of this group, with the interval from 10th to 90th percentiles given to indicate the range of variability. This variability was due to rounding the coefficients to obtain whole numbers of points, from categorizing the risk factors, and from taking five-point groupings of the resulting risk score. To illustrate the use of this table, consider the same hypothetical 80-year old man whose risk is shown in Figure 4. He would receive four risk points for blood pressure, seven points for 15-ft timed walk, six points for diabetes, and six points for age, giving a total of 23 points. The 5-year stroke risk for a man with 21–25 risk points is then estimated as 12%, with a prediction interval of 10–15%. 4. Discussion
Fig. 4. Screenshot of the CHS risk model Web-based Java applet for the CHS stroke risk model.
who takes 7 sec to walk 15 ft. His stroke risk was estimated as 5.5% at 3 years and 10.1% at 5 years. On the Web, as risk factors are entered using buttons and sliders, the risk curve and risk probabilities are immediately updated. Table 3 shows approximate risk scores for the CHS model, based on categorizing the variables in the Cox regression model, for when the Java applet is not available. To use the table one adds up the risk points for each variable and then looks at the appropriate survival table for 5-year
The major finding of this study was that a prediction rule could be derived to give accurate discrimination of stroke risk in this elderly population. The previously published risk models from the IIHD and Framingham studies gave comparable predictions. This indicates that the variables they incorporated capture a substantial amount of the variation in stroke risk, and suggests that the CHS model is also likely to generalize well. A simply used Java applet was presented that allows public use of the CHS prediction model using the Internet. The CHS prediction model included a 15-ft walk time, a variable not routinely measured, but one that had a strong association with stroke. According to our model, imputing a
Table 3 The CHS risk score Five-year stroke risk Risk points Systolic pressure 125 mmHg 125–134 135–144 145–154 155–164 165–174 175–184 185
0 2 4 6 8 10 12 14
15 ft walk time: LVH by ECG Creatinine 1.25 mg/dL Diabetes: definite Impaired fasting glucose
1/sec (max 20) 6 2 6 4 Men Women 0 0 3 5 6 10 9 15 12 20 16 5 5 1
Total risk score 1–5 6–10 11–15 15–20 21–25
Age: 70 and under 71–77 78–84 85–91 92 Atrial fibrillation by ECG History of heart disease
26–30 31–35 36–40 41–45
Men
Women
2.5% (2.1–3.0) 3.5% (2.8–4.5) 5.2% (4.2–6.4) 7.9% (6.1–10) 12% (10–15) 19% (16–23) 27% (22–32) 42% (36–46) 59% (58–61)
3.5% (1.3–2.2) 2.4% (1.8–3.3) 3.6% (2.6–4.9) 5.6% (4.1–7.5) 9.2% (7.1–11) 14% (11–17) 21% (16–26) 29% (26–36) 39% (35–46)
T. Lumley et al. / Journal of Clinical Epidemiology 55 (2002) 129–136
value of 5 sec for walk time was not acceptable, as higher risk individuals tend to have much larger than a 5-sec walk time and so have their risks underestimated. Previous CHS reports reported that a 15-ft walk time was strongly associated with white matter grade [17] and lacunar infarcts [18] on cranial magnetic resonance imaging, and may thus be an indicator of subclinical cerebrovascular disease. The timed walk is simple to perform, inexpensive, and places no significant burden on physician or patient. It may be a useful routine test in elderly patients and in future cohort studies. The CHS model gave more sensitive stroke prediction than the Framingham model in elderly women and almost identical prediction in elderly men. Our results also show, then, that the Framingham model gave good prediction of stroke risk in our population, even though the CHS cohort is older than the Framingham cohort. Compared to the CHS and Framingham models, the IIHD risk profile, despite being derived on a very different population, gave a very similar risk ranking for men. In contrast to the findings in the Framingham study [4] antihypertensive therapy directly, as an interaction with blood pressure, or as a stratifying factor, did not improve stroke prediction. There was no indication in the CHS data that the stroke risk for a given blood pressure depended on whether it was attained with medication or not. This may reflect differences in the CHS and Framingham populations, or different choices of treatment for hypertension in the two studies. Including carotid wall thickness or carotid stenosis in the model did not give substantially improved stroke prediction. It should be emphasized that this is a separate issue from whether carotid wall thickness is a useful indicator of subclinical disease, a question we did do not address. In this article we are concerned solely with predicting strokes as accurately and simply as possible, and carotid ultrasound did not substantially improve the predictions. Laupacis et al. [19] present a set of methodological standards for clinical prediction rules, an updated version of those given by Wasson et al. [20]. Of their 17 criteria, our method completely satisfied 12. Two further criteria involve testing the reproducibility of the rule and the variables. This has been done to some extent by including a validation set of data separate from that used to fit the model. Additional evidence that the variables and the risk prediction are reproducible comes from the good performance of the Framingham risk profile in these data. Another two criteria concern prospective validation of the accuracy and clinical benefits of the prediction rule. This can only be completed in future studies. Finally, Laupacis et al. [19] hypothesized that prediction rules would be more useful if they suggest a course of action rather than just giving a probability. We believe this approach would be inappropriate for a stroke model, as the available interventions and their risks and potential benefits may vary substantially among individuals. One of the criteria emphasized as desirable in the design of clinical prediction rules is ease of use [19]. The Java ap-
135
plet allows ease of application of very complicated prediction rules. It is still important that the model underlying the prediction rule be can be clearly and thoroughly explained; however, the computer can now perform the arithmetic necessary to compute risks. Experimentation with the interactive display can even improve understanding of the model by providing a clear picture of the differences in predicted risk among individuals. It should be noted that the ease of use of the Java applet may exacerbate an important danger of prediction rules: the temptation to make causal inferences. For example, there is no evidence our model would correctly predict the effect on stroke risk of an exercise program that reduced the 15-ft walk time by 20%. Randomized controlled trials are required to assess treatment benefits, and the risk reductions found in these trials should be compared with those predicted by our model. Similar comparisons have already been done in the case of systolic blood pressure as a univariate predictor [21], indicating that the association with stroke observed in observational studies was of a similar magnitude to the effect of blood pressure reduction observed in randomized trials. In conclusion, we constructed a new stroke prediction model in the CHS cohort and validated other prediction models for stroke in this cohort. The new risk model may be easily applied by calculating a risk score or using a Webbased Java applet. Further validation of this model in other prospective studies and comparison of the predictions of the model with effects of treatments/interventions in randomized trials is required.
Acknowledgments The Cardiovascular Health Study is funded by the National Heart, Lung and Blood Institute through contracts N01-HC-85079 to N01-HC-85086.
References [1] Manolio TA, Kronmal RA, Burke GL, O’Leary DH, Price TR, for the CHS Collaborative Research Group. Short-term predictors of incident stroke in older adults: the Cardiovascular Health Study. Stroke 1996;27:1479–86. [2] Sacco RL, Benjamin EJ, Broderick JP, Dyken M, Easton JD, Feinberg WM, Goldstein LB, Gorelick PB, Howard G, Kittner SJ, Manolio TA, Whisnant JP, Wolf PA. American Heart Association prevention conference. IV. Prevention and rehabilitation of stroke. Risk factors. Stroke 1997;28:1507–17. [3] Fried LP, Borhani NO, Enright P, Furberg CD, Gardin JM, Kronmal RA, Kuller LH, Manolio TA, Mittelmark MB, Newman A, O’Leary DH, Psaty B, Rautaharju P, Tracy RP, Weiler PG, for the Cardiovascular Health Study Research Group. The cardiovascular health study: design and rationale. Ann Epidemiol 1991;1:263–76. [4] D’Agostino RB, Wolf PA, Belanger AJ, Kannel WB. Probability of stroke: a risk profile from the Framingham Study. Stroke 1991;22:312–8. [5] Tanne D, Yaari S, Goldbourt U. Risk profile and prediction of longterm ischemic stroke mortality: a 21-year follow-up in the Israeli Ischemic Heart Disease (IIHD) project. Circulation 1998;98:1365–71.
136
T. Lumley et al. / Journal of Clinical Epidemiology 55 (2002) 129–136
[6] World Health Organization. Manual of the international classification of diseases, injuries, and causes of death. Vol I. 1975 Revision. Geneva: WHO, 1977. [7] Systolic hypertension in the elderly program (SHEP) cooperative research group. Rationale and design of a randomized clinical trial on prevention of stroke in isolated systolic hypertension. J Clin Epidemiol 1988;41:197–208. [8] O’Leary DH, Polak JF, Kronmal RA, Manolio TA, Burke GL, Wolfson SK, for the Cardiovascular Health Study Collaborative Research Group. Carotid artery intima and media thickness as a risk factor for myocardial infarction and stroke in older adults. N Engl J Med 1999;340(22):1762–3. [9] Schafer JL. Analysis of incomplete multivariate data. London: Chapman & Hall, 1997. [10] Klein JP, Moeschberger ML Survival analysis. New York: SpringerVerlag, 1997. [11] Breiman L, Friedman J, Olshen RA, Stone CJ. Classification and regression trees. Wadsworth, New York 1984. [12] Fleming TR, Harrington D. Statistical models based on counting processes. New York: Wiley, 1990. [13] Greenland S. Dose–response and trend analysis in epidemiology: alternatives to categorical analysis. Epidemiology 1995;6(4):356–65. [14] Grambsch P, Therneau T. Proportional hazards diagnostics based on weighted residuals. Biometrics 1994;81:515–526.
[15] Efron B, Tibshirani RJ. An introduction to the bootstrap. London: Chapman & Hall, 1993. [16] The Expert Panel. Expert Panel on detection, evaluation and treatment of high blood pressure (JNC V). Arch Intern Med 1993;153: 154–83. [17] Longstreth W, Manolio T, Arnold A, Burke G, Bryan N, Jungreis C, Enright P, O’Leary D, Poirier V, Fried L. Clinical correlates of white matter findings on cranial magnetic resonance imaging of 3301 elderly people. The Cardiovascular Health Study. Stroke 1996;27:1274–82. [18] Longstreth WT Jr, Bernick C, Manolio TA, Bryan N, Jungreis CA, Price TR. Lacunar infarcts defined by magnetic resonance imaging of 3660 elderly people: The Cardiovascular Health Study. Arch Neurol 1998;55:1217–25. [19] Laupacis A, Sekar N, Stiell IG. Clinical prediction rules: a review and suggested modifications of methodological standards. JAMA 1997; 277:488–94. [20] Wasson JH, Sox HC, Neff RK, Goldman L. Clinical prediction rules: application and methodological standards. N Engl J Med 1985;271: 827–32. [21] Collins R, Peto R, MacMahon S, et al. Blood pressure, stroke and coronary heart disease, part 2, short-term reductions in blood pressure: overview of randomized drug trials in their epidemiological context. Lancet 1990;335:827–83.
A stroke prediction score in the elderly: validation and Web-based application Thomas Lumleya,*, Richard A. Kronmala, Mary Cushmanb, Teri A. Manolioc, Steven Goldsteind a University of Washington, CHS Coordinating Center, Plaza 600 Building, Suite 700, 600 Stewart St., Seattle, WA 98101, USA University of Vermont, Asst. Professor/Dept. of Medicine, University of Vermont-Pathology, Colchester Research Facility, #T205, 208 South Park Drive, Suite 2, Colchester, VT 05446, USA c National Heart Lung and Blood Institute, Director, Epid/Biometry Program, NIH/NHLBI/DECA, Two Rockledge Center, Rm. 8160, 6701 Rockledge Drive MSC 7934, Bethesda, MD 20892-7934, USA d University of Pittsburgh, UPMC Stroke Institute, Room C400 PUH, 200 Lothrop Street, Pittsburgh, PA 15213, USA Received 18 October 2000; received in revised form 1 August 2001; accepted 3 August 2001 b
Abstract The objective of this study was to construct a prediction model for predicting stroke in an elderly U.S. population, and to assess the accuracy in this population of other previously published prediction models. The subjects were participants in the Cardiovascular Health Study: 2,495 men and 3,393 women age 65 years and older at baseline, and followed for 6.3 years. Among 5,711 participants free of baseline stroke, 399 strokes occurred. Sex-specific prediction equations were constructed using study variables that were most importantly related to incident stroke: age, systolic blood pressure, diabetes, ECG diagnosis of atrial fibrillation or left ventricular hypertrophy, confirmed history of cardiovascular disease, diabetes, time to walk 15 ft, and serum creatinine. The prediction rule was implemented as a risk score and in a Web-based interactive Java applet. Overall, the model predicted 5-year stroke risks ranging from less than 1 to 59%. The 20% of subjects in the highest predicted risk group had a 5-year actual stroke incidence rate of 15%, while the 20% lowest risk group had a 1% incidence. Risk scores from two other studies performed well in these study participants. Effective discrimination between low and high stroke risk in the elderly was possible in this cohort with data that are easy to obtain. Evaluation of the generalizability and clinical usefulness of this prediction model requires further research. © 2002 Elsevier Science Inc. All rights reserved. Keywords: Risk; Proportional hazards models; Aged; Cerebrovascular disorders
1. Introduction Stroke is a major cause of mortality and disability in the United States, particularly in the elderly. A number of potent risk factors are known such as age, hypertension, and atrial fibrillation [1]. This makes it likely that stroke risk can be predicted relatively accurately. Such prediction would be of clinical and public health value, as stroke risk can be reduced in high-risk subgroups by treatment of hypertension, anticoagulation, carotid endarterectomy, or modification of other risk factors [2]. The Cardiovascular Health Study [3] (CHS) is a population-based cohort study of 5,888 men and women age 65 and older at study entry who, at the time of analysis, have been followed for a median of 6.3 years. The purpose was to investigate risk factors for cardiovascular diseases in the elderly populations. A previous article [1] reported the risk * Corresponding author. Fax: 206-543-8864. E-mail address: [email protected] (T. Lumley)
factors for stroke in this cohort. In this article we use these risk factors and those identified in other studies to construct a prediction model for stroke over a 5–7-year period in this older age group. The resulting model is presented both as a risk score calculation and as a Java applet (program that runs in a Web browser) available from the CHS World Wide Web site (http://chs3.chs.biostat.washington.edu/chs/ stroke.htm). The Java applet allows risk calculations to be performed interactively using any Web browser that supports Java 1.1. Using this method, patient information can be simply entered on a computer, which applies complex statistical models that yield instantaneous calculation of a risk score. Two stroke risk prediction models have been published: the Framingham Study model, and the Israeli Ischemic Heart Disease Project [5] (IIHD) model. The Framingham model was based on a middle to older aged population living in a single community. The IIHD model was based on an ethnically diverse, even younger population of male civil and municipal employees in Israel. To address generaliz-
0895-4356/02/$ – see front matter © 2002 Elsevier Science Inc. All rights reserved. PII: S0895-4356(01)00 4 3 4 - 6
130
T. Lumley et al. / Journal of Clinical Epidemiology 55 (2002) 129–136
ability of these scores, we applied the Framingham and IIHD models in the CHS cohort. 2. Methods The CHS study design, including definitions of variables, has been published [3]. The cohort was recruited in two waves—the first wave of 5,201 in 1989–1990 from random samples of the Health Care Financing Administration Medicare eligibility lists in Forsyth Country, NC; Sacramento County, CA; Washington County, MD; and Allegheny County, PA. The second wave of 687 was recruited 2 years later from the same population but restricted to African-Americans to improve the representativeness of the cohort. The major reasons for exclusion were being institutionalized or under current treatment for cancer. Table 1 shows the distribution of some important risk factors. A detailed description of the stroke ascertainment has been published [1]. Potential stroke events were identified at annual follow-up examinations and at 6-month interim phone contacts. The participant or next of kin was interviewed about the surrounding circumstances. Hospital records for all reported strokes and for all hospitalizations with ICD-9 [6] codes 430–438, identifying cerebrovascular disease, were reviewed by a neurologist at each field center. Information on reported, nonhospitalized stroke patients was obtained by physician questionnaire. This information was reviewed by a CHS neurologist and any inconsistencies reviewed with the participant’s physician. Films of CT or MRI scans were available in 70% of cases; hard copy reports in an addition 17%. These were also reviewed centrally. Potential stroke cases were adjudicated by a committee of neurologists, neuroradiologists, and internists; their
decisions were usually unanimous. Criteria for stroke were similar to those used in the Systolic Hypertension in the Elderly Program (SHEP) [7]. Although nearly 90% of strokes were able to be classified into ischemic or hemorraghic, this classification was not used in the analysis, largely because interest was in the clinical event “stroke.” In addition, as the unclassifiable strokes outnumbered those classified as hemorraghic (4%) it would be hard to separate the two in analysis. The Framingham and IIHD models also used a clinical definition of stroke, and did not distinguish ischemic from hemorrhagic strokes in their modeling. Where definitions are not explicitly given here, they follow those described for the CHS study as a whole [3]. Stroke predictors included were demographic information (sex, race, age, smoking status, and history), clinical history [diabetes, use of antihypertensive medications, history of coronary heart disease (CHD), peripheral vascular disease, congestive heart failure, transient ischemic attack, atrial fibrillation], and physical and biochemical measurements (systolic blood pressure, time to walk 15 ft, serum creatinine, ECG determinations of atrial fibrillation and left ventricular hypertrophy, ratio of HDL to total cholesterol). Ultrasound assessment of carotid wall thickness was the average of internal and common carotid maximum wall thickness each divided by its standard deviation, as previously described [8]. Blood pressure was measured in the right arm with the subject supine, after 5 min rest, using a Doppler probe. Diabetes was based on American Diabetes Association classification, with diabetes defined as fasting glucose 126 mg/dL (7.0 mmol/L) or treatment with insulin or oral hypoglycemic agents, and impaired fasting glucose defined as fasting glucose between 110 and 126 mg/dL (6.1 and 7.0 mmol/L).
Table 1 Risk factors included in the CHS, IIHD, and Framingham models and coefficients for the CHS model (the last six variables were not in the CHS model but were in one of the other models) Coefficientc
Mean
LVH by ECG Diabetes Impaired fasting glucose Creatinine 1.25 mg/dL (110 mol/L) 15 ft walk time Systolic blood pressure History of heart disease Atrial fibrillation by ECG Age Use of antihypertensive drugs Current smokers Current smoking rate History of PVD History of CHD % HDL cholesterol a
(2,495 men)
(3,393 women)
(men)
(women)
5.1% 14.9% 19.0% 39.6% 5.5 s 143 mmHg 26.5% 3.5% 73 year 44% 11.1% 21 cigs/day 4.1% 25.7% 24.4%
4.9% 12.5% 14.4% 8.1% 6.0 s 144 mmHg 16.1% 2.1% 73 year 48% 12.5% 16 cigs/day 1.5% 14.7% 27.2%
0.501 0.521 0.347 0.141 0.099b .172/10 mmHga 0.445 0.410 .382/10 year — — — — —
0.073 1.35 0.613/10 year — — — — —
Systolic blood pressure below 120 mmHg set to 120 mmHg. 15 ft walk times above 20 s were set to 20 s. c In the absence of important risk modification a single coefficient is given for men and women as described in the Methods section. b
T. Lumley et al. / Journal of Clinical Epidemiology 55 (2002) 129–136
Amount of current smoking was used in the IIHD risk model but was not measured in the CHS cohort. To apply the IIHD model to CHS, for current smokers we used the subject’s reported typical number of cigarettes per day over their entire smoking history. There were three vascular disease history variables used in these analyses. The IIHD model used history of CHD, defined as angina or myocardial infarction (MI), and history of peripheral vascular disease (PVD), defined as claudication in the legs. The Framingham model combined these two with coronary insufficiency and congestive heart failure (CHF) into one variable, here called “history of heart disease.” These variables do not correspond exactly to any classifications used in CHS. To match the IIHD definitions as closely as possible we defined “history of CHD” as definite or possible history of MI, angina, ECG evidence of past silent MI, coronary bypass surgery, or angioplasty and “history of PVD” as definite or possible history of claudication in the legs. “History of heart disease” was then defined to match the Framingham definition as history of CHD or PVD or definite or possible history of CHF. Data analysis was performed using the S-PLUS statistical package, versions 4.5 and 2000. The 5711 CHS participants without a prebaseline history of stroke were divided at random, two-thirds (n 3,799) in a fitting set and one-third (n 1,912) in a validation set. A separate validation set gives an unbiased assessment of the predictive power and goodness of fit of a model even when a complex model selection strategy is used. Participants with missing values for any of the variables were excluded. Of the 5,711 participants, only 480 (8.5%) had missing data for any of the variables in the final model. These participants tended to have worse values for all the predictors studied, but a multiple imputation analysis [9] did not give appreciably different estimates or confidence intervals. The Cox stratified proportional hazards model [10] was used as the primary model for stroke prediction, although nonproportional hazards models and survival tree (recursive partitioning) models [11] were also explored. The stratified model allows the shape of the survival curve to differ between men and women, and allows either separate or common hazard ratios by sex. Separate hazard ratios by sex were used if the difference in hazard ratios was large or statistically significant. Variables were explored in groups with routine clinical information (blood pressure, age, sex, medical history) included first and less routine measurements entered later if they provided substantial independent prediction. The functional form of the continuous variables was explored by graphing fitted regression splines [12] with knots at the quintiles, and by smoothed residual plots [13]. Both of these methods can estimate any continuous dose–response relationship. Simple functional forms were then chosen based on these graphs, with logarithmic and exponential transformations, and (possibly discontinuous) linear functions with thresholds being considered. If the graphs suggested more than one candidate transformation the partial likelihood for
131
models with different transformations was compared. This led to linear functions with thresholds being chosen for two variables, and one (creatinine) being fitted by dichotomizing. Variables were retained in the model if they were significant at the .05 level or if they had large coefficients and were significant in the Framingham risk model. Deviations from proportional hazards were assessed using the methods of Grambsch and Therneau [14], which provide both formal tests and graphical estimates of lack of fit. After the prediction model was selected the fitting and validation subsets were combined and the model was refit to provide more precise parameter estimates. The predicted survival curves given by the Java applet compute the predicted survival at 1-year intervals and interpolate linearly between these points. Predictive accuracy of the model was assessed using Receiver Operating Characteristic (ROC) curves and by comparing observed and expected survival curves. The ROC curve shows the ability of a risk score to discriminate between high- and low-risk subjects. Any specific value of predicted risk can be used to divide people into high- and low-risk groups. For any given value, the true positive rate (sensitivity) was the proportion of those with strokes who were classified as high risk, and the false positive rate (1specificity) was the proportion without strokes who were classified as high risk. These numbers were plotted against each other for each possible cut point of predicted risk. A larger area under the curve indicates better discrimination. To compare area under the curve for different cut points defining risk, the fitting set was resampled and the CHS risk model refitted, then the validation set was resampled and the ROC curves for the two models calculated. This was done 500 times, and a confidence interval for the difference in area under the ROC curve was calculated using the percentile bootstrap method [15]. The expected survival curve for a group is calculated by averaging the predicted survival for each member of the group. This is then compared to the observed survival curve computed by the Kaplan-Meier method. Inaccuracies in the model cause the expected and observed survival curves to differ; in particular, overfitting of the model will cause the expected survival curves to be more widely separated than the observed survival curves. 3. Results A total of 399 strokes occurred after a median follow-up of 6.3 years, giving a rate of approximately 1.3%/year. Fifteen (3.8%) strokes were hemorrhagic and 344 (86%) were ischemic, with the remainder of unknown type. No distinction was made in the final analysis: prediction of the clinical event “stroke” was the goal of the model, the a priori most important modifiable predictor, blood pressure, is relevant to both types of stroke, and the previous models have not distinguished between stroke types. The variables included in this and the two previous risk models profiles are shown
132
T. Lumley et al. / Journal of Clinical Epidemiology 55 (2002) 129–136
Fig. 1. Observed (solid lines) and predicted (dotted lines) survival distributions for upper, middle, and lower tertiles of risk in the validation subset.
in Table 1, with their mean levels for men and women and their multiple regression coefficients in the CHS model. Transformations were considered as described above for all the continuous variables. Three of the continuous variables were transformed for inclusion in the final model. Systolic blood pressure showed a linear association above approximately 120 mmHg, and no effect below this level, consistent with the JNC-V guidelines on hypertension [16]. The time to walk 15 ft showed a linear association below a threshold of 20 sec. Serum creatinine was dichotomized at 1.25 mg/dL (110.6 mol/L). Carotid wall thickness was statistically significant when added to the model but did not improve the predictions substantially so it was not retained in the final model. Percent carotid stenosis was also explored, but was less strongly predictive than carotid wall thickness. Interactions between carotid wall thickness or carotid stenosis and other predictors were also explored but did not give substantially better prediction. The model coefficients are listed in Table 1. Figure 1 shows the predictive accuracy of the CHS model. We divided the validation set into three groups based on the tertiles of 5-year risk and computed the observed stroke survival curve (solid line) and the predicted curves (dashed line) given by the CHS model. These curves show that the risk was accurately estimated in the middle group, but was underestimated in the high-risk group and overestimated in the low-risk group. The ability of the CHS model and the Framingham model to discriminate high and low risk subjects is summarized by Table 2 and Figures 2 and 3. Table 2 shows the in-
cidence of stroke in the validation set by quintile of 5-year risk for the two risk models profiles. There was is a 15-fold incidence gradient across these five groups, showing powerful discrimination between high- and low-risk individuals. These crude incidence rates were lower than the stroke probabilities that would be computed from 1-survival probabilities in Figure 1 because the survival probabilities were adjusted for differing lengths of followup caused by different accrual times, death, and loss to follow-up. Figure 2 is a ROC curve for 5-year stroke incidence in the validation set. The lower left portion of the curve shows discrimination in high risk subjects, where the CHS model (solid line) predicted better. The upper right portion shows low risk subjects, where the Framingham model (dotted line) predicted better. The dashed line gives the prediction based on carotid wall thickness alone. As the ROC curve shows, carotid wall thickness alone performs almost as well as the fitted risk model in higher risk subjects, although it did not discriminate as well in moderate to low risk subjects. Figure 3a and 3b shows separate ROC curves for men and women. For men these are based on the Framingham, Table 2 Five-year stroke incidence by quintile of the CHS and Framingham predicted risk Quintile
CHS model
Framingham model
1Highest 2 3 4 5lowest
54/351 (15%) 39/350 (11%) 21/351 (6%) 14/350 (4%) 4/351 (1%)
51/351 (15%) 39/350 (11%) 28/351 (8%) 12/350 (3%) 2/351 (1%)
T. Lumley et al. / Journal of Clinical Epidemiology 55 (2002) 129–136
133
Fig. 2. Receiver Operating Characteristic Curve for 5-year stroke incidence: comparing predictions from the CHS and Framingham risk models profiles and from carotid wall thickness alone in the validation subset.
CHS, and IIHD models, and for women on the Framingham and CHS models. In men, all three models were almost identical over the clinically important higher risk range, although the CHS model was less predictive at low to moderate risks. In women, the CHS model predicted much better than the Framingham model. Combining men and women, the CHS and Framingham models both gave an area under the ROC curve of 0.74 (P .56 for difference). For men the Framingham model gave an area of .69 and the CHS model .65 (P .10). For women the Framingham model gave an area of .73 and the CHS model .77 (P .044). As the Framingham model appears to predict stroke in elderly women less effectively, it is of interest to know where the model predictions differ. Women for whom the CHS risk score and Framingham risk score are very different are more likely to be current smokers, with a history of cardivascular disease and taking antihypertensive medications. The Framingham model weights these variables more strongly; they may be more important in younger populations or the differences may reflect changes over time in risks or treatments for women. The 15-ft timed walk is not available in all epidemiologic studies, and is not validated for use in general practice. In the CHS risk model, imputation of a value of 5 sec for the timed walk gave very similar rankings (ROC area .73) compared to the full model (ROC area .73), but the absolute risk of stroke was underestimated by 10–15% in some high-risk individuals. The other variable that may not always be routinely available in practice is serum creatinine. Good predictions were still be obtained by assuming that se-
Fig. 3. Receiver Operating Characteristic Curve for 5-year stroke incidence in men (a) and in women (b): comparing predictions from the CHS, Framingham and (for men) IIHD risk models profiles in the validation subset. The IIHD risk profile is defined only for men.
rum creatinine was less than 1.25 mg/dL (110.6 mol/L) (data not shown). An example of the Web-based Java applet is shown in Figure 4. After input of the patient data, the screen displays the 3-year and 5-year probabilities of stroke, and a graph showing the risk of stroke over time up to 6 years. Figure 4 shows results for an 80-year-old man with systolic blood pressure of 140 mmHg, diabetes, and no other risk factors,
134
T. Lumley et al. / Journal of Clinical Epidemiology 55 (2002) 129–136
stroke risk in men or women. The stroke risks were obtained by computing the risk using the continuous model for each CHS participant whose risk points fell into that category and taking the median of this group, with the interval from 10th to 90th percentiles given to indicate the range of variability. This variability was due to rounding the coefficients to obtain whole numbers of points, from categorizing the risk factors, and from taking five-point groupings of the resulting risk score. To illustrate the use of this table, consider the same hypothetical 80-year old man whose risk is shown in Figure 4. He would receive four risk points for blood pressure, seven points for 15-ft timed walk, six points for diabetes, and six points for age, giving a total of 23 points. The 5-year stroke risk for a man with 21–25 risk points is then estimated as 12%, with a prediction interval of 10–15%. 4. Discussion
Fig. 4. Screenshot of the CHS risk model Web-based Java applet for the CHS stroke risk model.
who takes 7 sec to walk 15 ft. His stroke risk was estimated as 5.5% at 3 years and 10.1% at 5 years. On the Web, as risk factors are entered using buttons and sliders, the risk curve and risk probabilities are immediately updated. Table 3 shows approximate risk scores for the CHS model, based on categorizing the variables in the Cox regression model, for when the Java applet is not available. To use the table one adds up the risk points for each variable and then looks at the appropriate survival table for 5-year
The major finding of this study was that a prediction rule could be derived to give accurate discrimination of stroke risk in this elderly population. The previously published risk models from the IIHD and Framingham studies gave comparable predictions. This indicates that the variables they incorporated capture a substantial amount of the variation in stroke risk, and suggests that the CHS model is also likely to generalize well. A simply used Java applet was presented that allows public use of the CHS prediction model using the Internet. The CHS prediction model included a 15-ft walk time, a variable not routinely measured, but one that had a strong association with stroke. According to our model, imputing a
Table 3 The CHS risk score Five-year stroke risk Risk points Systolic pressure 125 mmHg 125–134 135–144 145–154 155–164 165–174 175–184 185
0 2 4 6 8 10 12 14
15 ft walk time: LVH by ECG Creatinine 1.25 mg/dL Diabetes: definite Impaired fasting glucose
1/sec (max 20) 6 2 6 4 Men Women 0 0 3 5 6 10 9 15 12 20 16 5 5 1
Total risk score 1–5 6–10 11–15 15–20 21–25
Age: 70 and under 71–77 78–84 85–91 92 Atrial fibrillation by ECG History of heart disease
26–30 31–35 36–40 41–45
Men
Women
2.5% (2.1–3.0) 3.5% (2.8–4.5) 5.2% (4.2–6.4) 7.9% (6.1–10) 12% (10–15) 19% (16–23) 27% (22–32) 42% (36–46) 59% (58–61)
3.5% (1.3–2.2) 2.4% (1.8–3.3) 3.6% (2.6–4.9) 5.6% (4.1–7.5) 9.2% (7.1–11) 14% (11–17) 21% (16–26) 29% (26–36) 39% (35–46)
T. Lumley et al. / Journal of Clinical Epidemiology 55 (2002) 129–136
value of 5 sec for walk time was not acceptable, as higher risk individuals tend to have much larger than a 5-sec walk time and so have their risks underestimated. Previous CHS reports reported that a 15-ft walk time was strongly associated with white matter grade [17] and lacunar infarcts [18] on cranial magnetic resonance imaging, and may thus be an indicator of subclinical cerebrovascular disease. The timed walk is simple to perform, inexpensive, and places no significant burden on physician or patient. It may be a useful routine test in elderly patients and in future cohort studies. The CHS model gave more sensitive stroke prediction than the Framingham model in elderly women and almost identical prediction in elderly men. Our results also show, then, that the Framingham model gave good prediction of stroke risk in our population, even though the CHS cohort is older than the Framingham cohort. Compared to the CHS and Framingham models, the IIHD risk profile, despite being derived on a very different population, gave a very similar risk ranking for men. In contrast to the findings in the Framingham study [4] antihypertensive therapy directly, as an interaction with blood pressure, or as a stratifying factor, did not improve stroke prediction. There was no indication in the CHS data that the stroke risk for a given blood pressure depended on whether it was attained with medication or not. This may reflect differences in the CHS and Framingham populations, or different choices of treatment for hypertension in the two studies. Including carotid wall thickness or carotid stenosis in the model did not give substantially improved stroke prediction. It should be emphasized that this is a separate issue from whether carotid wall thickness is a useful indicator of subclinical disease, a question we did do not address. In this article we are concerned solely with predicting strokes as accurately and simply as possible, and carotid ultrasound did not substantially improve the predictions. Laupacis et al. [19] present a set of methodological standards for clinical prediction rules, an updated version of those given by Wasson et al. [20]. Of their 17 criteria, our method completely satisfied 12. Two further criteria involve testing the reproducibility of the rule and the variables. This has been done to some extent by including a validation set of data separate from that used to fit the model. Additional evidence that the variables and the risk prediction are reproducible comes from the good performance of the Framingham risk profile in these data. Another two criteria concern prospective validation of the accuracy and clinical benefits of the prediction rule. This can only be completed in future studies. Finally, Laupacis et al. [19] hypothesized that prediction rules would be more useful if they suggest a course of action rather than just giving a probability. We believe this approach would be inappropriate for a stroke model, as the available interventions and their risks and potential benefits may vary substantially among individuals. One of the criteria emphasized as desirable in the design of clinical prediction rules is ease of use [19]. The Java ap-
135
plet allows ease of application of very complicated prediction rules. It is still important that the model underlying the prediction rule be can be clearly and thoroughly explained; however, the computer can now perform the arithmetic necessary to compute risks. Experimentation with the interactive display can even improve understanding of the model by providing a clear picture of the differences in predicted risk among individuals. It should be noted that the ease of use of the Java applet may exacerbate an important danger of prediction rules: the temptation to make causal inferences. For example, there is no evidence our model would correctly predict the effect on stroke risk of an exercise program that reduced the 15-ft walk time by 20%. Randomized controlled trials are required to assess treatment benefits, and the risk reductions found in these trials should be compared with those predicted by our model. Similar comparisons have already been done in the case of systolic blood pressure as a univariate predictor [21], indicating that the association with stroke observed in observational studies was of a similar magnitude to the effect of blood pressure reduction observed in randomized trials. In conclusion, we constructed a new stroke prediction model in the CHS cohort and validated other prediction models for stroke in this cohort. The new risk model may be easily applied by calculating a risk score or using a Webbased Java applet. Further validation of this model in other prospective studies and comparison of the predictions of the model with effects of treatments/interventions in randomized trials is required.
Acknowledgments The Cardiovascular Health Study is funded by the National Heart, Lung and Blood Institute through contracts N01-HC-85079 to N01-HC-85086.
References [1] Manolio TA, Kronmal RA, Burke GL, O’Leary DH, Price TR, for the CHS Collaborative Research Group. Short-term predictors of incident stroke in older adults: the Cardiovascular Health Study. Stroke 1996;27:1479–86. [2] Sacco RL, Benjamin EJ, Broderick JP, Dyken M, Easton JD, Feinberg WM, Goldstein LB, Gorelick PB, Howard G, Kittner SJ, Manolio TA, Whisnant JP, Wolf PA. American Heart Association prevention conference. IV. Prevention and rehabilitation of stroke. Risk factors. Stroke 1997;28:1507–17. [3] Fried LP, Borhani NO, Enright P, Furberg CD, Gardin JM, Kronmal RA, Kuller LH, Manolio TA, Mittelmark MB, Newman A, O’Leary DH, Psaty B, Rautaharju P, Tracy RP, Weiler PG, for the Cardiovascular Health Study Research Group. The cardiovascular health study: design and rationale. Ann Epidemiol 1991;1:263–76. [4] D’Agostino RB, Wolf PA, Belanger AJ, Kannel WB. Probability of stroke: a risk profile from the Framingham Study. Stroke 1991;22:312–8. [5] Tanne D, Yaari S, Goldbourt U. Risk profile and prediction of longterm ischemic stroke mortality: a 21-year follow-up in the Israeli Ischemic Heart Disease (IIHD) project. Circulation 1998;98:1365–71.
136
T. Lumley et al. / Journal of Clinical Epidemiology 55 (2002) 129–136
[6] World Health Organization. Manual of the international classification of diseases, injuries, and causes of death. Vol I. 1975 Revision. Geneva: WHO, 1977. [7] Systolic hypertension in the elderly program (SHEP) cooperative research group. Rationale and design of a randomized clinical trial on prevention of stroke in isolated systolic hypertension. J Clin Epidemiol 1988;41:197–208. [8] O’Leary DH, Polak JF, Kronmal RA, Manolio TA, Burke GL, Wolfson SK, for the Cardiovascular Health Study Collaborative Research Group. Carotid artery intima and media thickness as a risk factor for myocardial infarction and stroke in older adults. N Engl J Med 1999;340(22):1762–3. [9] Schafer JL. Analysis of incomplete multivariate data. London: Chapman & Hall, 1997. [10] Klein JP, Moeschberger ML Survival analysis. New York: SpringerVerlag, 1997. [11] Breiman L, Friedman J, Olshen RA, Stone CJ. Classification and regression trees. Wadsworth, New York 1984. [12] Fleming TR, Harrington D. Statistical models based on counting processes. New York: Wiley, 1990. [13] Greenland S. Dose–response and trend analysis in epidemiology: alternatives to categorical analysis. Epidemiology 1995;6(4):356–65. [14] Grambsch P, Therneau T. Proportional hazards diagnostics based on weighted residuals. Biometrics 1994;81:515–526.
[15] Efron B, Tibshirani RJ. An introduction to the bootstrap. London: Chapman & Hall, 1993. [16] The Expert Panel. Expert Panel on detection, evaluation and treatment of high blood pressure (JNC V). Arch Intern Med 1993;153: 154–83. [17] Longstreth W, Manolio T, Arnold A, Burke G, Bryan N, Jungreis C, Enright P, O’Leary D, Poirier V, Fried L. Clinical correlates of white matter findings on cranial magnetic resonance imaging of 3301 elderly people. The Cardiovascular Health Study. Stroke 1996;27:1274–82. [18] Longstreth WT Jr, Bernick C, Manolio TA, Bryan N, Jungreis CA, Price TR. Lacunar infarcts defined by magnetic resonance imaging of 3660 elderly people: The Cardiovascular Health Study. Arch Neurol 1998;55:1217–25. [19] Laupacis A, Sekar N, Stiell IG. Clinical prediction rules: a review and suggested modifications of methodological standards. JAMA 1997; 277:488–94. [20] Wasson JH, Sox HC, Neff RK, Goldman L. Clinical prediction rules: application and methodological standards. N Engl J Med 1985;271: 827–32. [21] Collins R, Peto R, MacMahon S, et al. Blood pressure, stroke and coronary heart disease, part 2, short-term reductions in blood pressure: overview of randomized drug trials in their epidemiological context. Lancet 1990;335:827–83.