Validity of the Framingham point scores in the elderly: Results from the Rotterdam study

Validity of the Framingham point scores in the elderly: Results from the Rotterdam study Michael T. Koller, MD, MSc,a,b,c Ewout W. Steyerberg, PhD,b Marcel Wolbers, PhD,a Theo Stijnen, PhD,c Heiner C. Bucher, MD, MPH,a M.G. Myriam Hunink, MD, PhD,c,d,e and Jacqueline C.M. Witteman, PhDc Basel, Switzerland; Rotterdam, the Netherlands; and Boston, MA

Background The National Cholesterol Education Program recommends assessing 10-year risk of coronary heart disease (CHD) in individuals free of established CHD with the Framingham Point Scores (FPS). Individuals with a risk N20% are classified as high risk and are candidates for preventive intervention. We aimed to validate the FPS in a European population of elderly subjects. Methods Subjects free of established CHD at baseline were selected from the Rotterdam study, a population-based cohort of subjects 55 years or older in the Netherlands. We studied calibration, discrimination (c-index), and the accuracy of high-risk classifications. Events consisted of fatal CHD and nonfatal myocardial infarction. Results

Among 6795 subjects, 463 died because of CHD and 336 had nonfatal myocardial infarction. Predicted 10-year risk of CHD was on average well calibrated for women (9.9% observed vs 10.1% predicted) but showed substantial overestimation in men (14.3% observed vs 19.8% predicted), particularly with increasing age. This resulted in substantial number of false-positive classifications (specificity 70%) in men. In women, discrimination of the FPS was better than that in men (c-index 0.73 vs 0.63, respectively). However, because of the low baseline risk of CHD and limited discriminatory power, only 33% of all CHD events occurred in women classified as high risk.

Conclusions The FPS need recalibration for elderly men with better incorporation of the effect of age. In elderly women, FPS perform reasonably well. However, maintaining the rational of the high-risk threshold requires better performing models for a population with low incidence of CHD. (Am Heart J 2007;154:87-93.) In persons without established coronary heart disease (CHD), clinical strategies for prevention of new-onset CHD are often based on the prediction of 10-year risks. Persons with a predicted risk N20% are usually classified as high risk and require various forms of preventive interventions.1 With the third report of the National Cholesterol Education Program,1 an updated risk prediction system, the Framingham Point Scores (FPS), was presented and recommended to guide, for example, the intensity of low-density lipoprotein (LDL) lowering.1 The FPS are represented in the Adult Treatment Panel (ATP) III as risk charts separate for men and women.

From the aBasel Institute for Clinical Epidemiology, Basel, Switzerland, Departments of b Public Health, cEpidemiology and Biostatistics, and dRadiology, Erasmus MC, Rotterdam, the Netherlands, and eDepartment of Health Policy and Management, Harvard School of Public Health, Boston, MA. Submitted November 25, 2006; accepted March 15, 2007. Reprint requests: Michael Koller, MD, MSc, Basel Institute for Clinical Epidemiology, University Hospital Basel, Petersgraben 4CH-4031 Basel, Switzerland. E-mail: [email protected] 0002-8703/$ - see front matter n 2007, Mosby, Inc. All rights reserved. doi:10.1016/j.ahj.2007.03.022

Points assigned to categorized risk factors are summed over all risk factors, and the summary score corresponds to a 10-year predicted risk of CHD. Prediction of CHD risk in elderly asymptomatic individuals becomes increasingly important because of the ageing of populations. This was recognized by the developers of the FPS, and they extended risk predictions for an age range up to 80 years. Compared with earlier risk scorings from the Framingham Heart Study (FHS),2,3 FPS have been additionally modified in several important aspects. Predictions of FPS apply now specifically to bhardQ CHD consisting of objectively determined fatal CHD or nonfatal myocardial infarction (MI). Previous scorings2,3 estimated a broader range of events (total CHD), which led, together with other reasons, to risk overestimation and difficulties with generalizability.4-6 Other important modifications include the introduction of interaction terms of total cholesterol (TC) and smoking with age and of systolic blood pressure with current medication for systolic blood pressure. And finally, the update of the FHS database accounts for the declining CHD incidence in industrialized nations.7,8

American Heart Journal July 2007

88 Koller et al

Primary prevention strategies are increasingly extended to older persons9-13; and therefore, risk predictions from FPS and the accuracy of the high-risk classification should be validated for the elderly population. The aim of this study was to validate FPS in the prediction of nonfatal MI and fatal CHD in elderly men and women.

Methods The Rotterdam study is a prospective, population-based study among subjects 55 years and older living in a suburb of Rotterdam, the Netherlands. The rationale and design of the Rotterdam study have been described elsewhere.14 We included follow-up information until January 1, 2006, and selected all 6795 subjects without evidence of CHD or cerebrovascular disease at baseline. Subjects with a history of MI or stroke, as verified by the general practitioner or cardiologist; subjects with signs of prior MI during electrocardiographic evaluation; and subjects with coronary procedures or endarterectomy at baseline were excluded from the analysis. Baseline examinations were conducted from 1990 to 1993. Participants were interviewed at home by trained research assistants using a computerized questionnaire. These baseline data included information on the current health status, history of cardiovascular disease, medication use, and cardiovascular risk factors. Subsequently, the participants visited the research center for several measurements, including body mass index, blood pressure, TC, high-density lipoprotein cholesterol (HDL-C), and nonfasting glucose level, and for an electrocardiogram. Clinical follow-up data were obtained from the general practitioners of the participants from 1990 onward. All subjects gave written informed consent, and the study was approved by the medical ethics committee of Erasmus MC.

Assessment of risk factors Subjects were categorized in groups of current smokers and nonsmokers (never smoked or abstinence of at least 2 years). Blood pressure was calculated as the mean of 2 consecutive measurements with a random-zero sphygmomanometer at the right brachial artery in sitting position. Serum TC level was determined by an automated enzymatic procedure using the cholesterol oxidaseperoxide aminophenazone kit (Roche Diagnostics, Basel, Switzerland), and serum HDL-C was measured with the Roche HDL-C assay using phosphotungstate-magnesium.

Clinical end points All events were classified according to the International Classification of Diseases, version 10. The primary outcome for this study was hard CHD, which consisted of the combined end point of nonfatal MI and fatal CHD. The latter consisted of fatal MI (I21, I24), sudden cardiac death (I46, I49, R96), death from chronic ischemic heart disease (I25), and death due to heart failure (I50) other than hypertensive (I11) or nonrheumatic valve disorders (I05-I09). Because CHD is the leading cause of heart failure in elderly subjects, we primarily considered heart failure death as part of the fatal CHD outcome and performed secondary analyses excluding fatal heart failure cases from fatal CHD. Information on fatal and nonfatal

cardiovascular end points was obtained from general practitioners and from letters and discharge reports from medical specialists. All events were classified independently by 2 research physicians. If the physicians disagreed, a consensus was reached in a special session. Finally, all events were verified by a cardiologist. In cases of unresolved discrepancy, the judgment by the expert was considered definite.

Statistical analysis Ten-year risk for hard CHD for each subject in the Rotterdam study was calculated using the FPS charts as given for men and women separately with the ATP III.1 The predictive performance of FPS for the Rotterdam cohort was assessed by studying calibration and discrimination. Calibration refers to agreement between predicted and observed 10-year CHD risks. The observed 10-year risk (in the population and for subgroups) was estimated with the Kaplan-Meier estimator. We first compared the mean of the predicted risks with the observed population risk for all men and women in the cohort (calibration-in-the-large). Second, we assessed calibration by plotting predicted risk against observed risk for each of the 13 risk categories specified by the FPS as described in ATP III. Because the highest possible risk category from FPS is denoted as z30% but not in terms of a single value, we used a conservative value of 35% for subjects in this category. Discrimination refers to the ability of the FPS to assign higher predicted risks to subjects who developed CHD during follow-up than to subjects who did not develop CHD. We used the c-index to study discrimination15 and report values for men and women separately. The c-index is a generalization of the area under the receiver operating characteristic curve to continuous, right-censored data and has a similar interpretation. A value of 0.5 refers to no discriminative ability; a value of 1 means that the model is able to perfectly separate events from nonevents. Furthermore, we assessed the accuracy of the high-risk classification (FPS predicted risk N20%) by calculating sensitivity and specificity. To account for censoring, we used 10-year Kaplan-Meier estimates to calculate the expected number of events within low-risk (V20%) and high-risk (N20%) categories.16 Sensitivity in this context refers to the probability of being correctly classified as high risk given that an event occurred during the 10-year follow-up. Similarly, specificity refers to the probability of being classified not as high risk given that a person did not experience an event during follow-up. For simplicity, we denote the category of predicted FPS risk V20% as low risk. We used an extensive approach to deal with missing covariate values. First, we assessed the frequency of missing data and its association with the outcome.17 For reasons of efficiency and to avoid potential bias, we used multiple imputation of missing covariates. We defined extensive imputation models for each variable with linear regression for continuous and logistic regression for binary missing data.18 We used 5 imputed data sets and pooled analyses across data sets for final reporting.19,20 All analyses were performed for both complete cases as well as for multiple imputed data. If not indicated otherwise, results are reported for multiple imputed data. Any relevant differences in results between the 2 methods are reported.

American Heart Journal Volume 154, Number 1

Koller et al 89

Table I. Baseline characteristics and summary event table according to sex with 10-year risk of hard CHD among 6795 subjects free of CHD at baseline in the Rotterdam study Men Characteristic All (n = 6791) Age (y)

Smokingy

Hypertension

TC (mmol/L)z

HDL-C (mmol/L)z

Women

Category

n

CHD events

10-y risk (%)*

n

CHD events

10-y risk (%)*

55-64 65-74 z75 No Current Missing No Yes Treated Untreated Missing Low (b5.18) Borderline (5.18-6.19) High (N6.19) Missing High (z1.6) Low (b1.6) Missing

2452 991 888 573 1643 739 70 1608 621 384 237 223 336 718 1159 239 317 1887 248

351 87 151 113 227 110 14 195 111 71 40 45 35 93 174 49 36 264 51

14.3 7.6 15.7 30.0 13.7 14.7 30.1 11.4 18.1 17.5 19.5 27.6 11.5 12.6 14.0 27.4 12.2 13.3 27.0

4343 1479 1368 1496 3364 725 254 2392 1470 1075 395 481 266 827 2647 599 1262 2461 620

448 58 126 264 338 73 37 173 204 159 45 71 18 71 254 105 86 254 108

9.9 3.0 7.2 23.1 9.6 8.7 21.7 6.2 13.3 14.2 11.0 23.1 7.6 7.7 8.4 24.6 5.8 9.5 23.9

Missing data are reported per covariate and illustrate the association with increased 10-year risk. * Kaplan-Meier estimates. yFormer smokers were coded together with nonsmokers. zTo convert millimoles per liter into milligrams per deciliter, multiply by 38.61.

All analyses were performed using R version 2.3.1 (R Foundation for Statistical Computing, Vienna, Austria).21 Multiple imputations were performed with the contributed R package mice.18 We report estimates with 95% CIs; the amount of loss to follow-up was summarized according to the method suggested by Clark et al.22

Results In the Rotterdam study, 7983 of 10 275 subjects considered for enrollment consented to participation; 6795 of these were free of established CHD at baseline and included in the present analysis. Over a median follow-up duration of 12.9 years, 799 hard CHD events occurred. The 799 events consisted of 336 nonfatal MI and 463 coronary deaths. Of the latter, 208 were due to sudden death, 180 due to heart failure, 64 due to fatal MI, and 11 due to chronic ischemic heart disease. Of the theoretically possible total follow-up duration, 98% was actually observed, indicating a low rate of dropout.22 Mean age at baseline was 70.2 years: 68.5 years for men and 71.1 years for women. Subjects with at least one missing risk factor value were older (average age 77.2 against 68.8 years, respectively) and showed substantially higher incidences of CHD than subjects with known data for smoking, hypertension, TC, and HDL-C (Table I). Of note, subjects with missing information for smoking showed a stronger univariate relation with CHD in men than in women. A complete case analysis

would lead to a loss of 13.5% (330 of 2452) of all observations in men and 18.6% (809 of 4343) in women.

Calibration The 10-year estimates from FPS for hard CHD events showed overestimation in men. The average predicted risk was 19.8% compared with the observed cumulative risk of 14.3% (95% CI 12.8%-15.8%). In women, the average predicted risk of 10.1% was similar to the observed risk of 9.9% (95% CI 9.0%-10.9%). Comparing predicted risk with observed risk according to risk categories of FPS (Figure 1) showed that estimates were systematically too high in men, especially in categories of high and low predicted risk, but predictions from women showed good calibration over the whole range of predicted risk. We further studied the deviation of predicted risk from observed risk (expressed as the absolute difference) over categories of age used in FPS (Figure 2). A strong association of the deviation with age was seen in men. In age categories of men b75 years, FPS overestimated risk, which rapidly turned into underestimation in age categories N75 years. In contrast, in women, the estimates showed good calibrated risks up to 75 years, but underestimation with increasing age N75 years. Results for calibration were very similar for both the complete case analysis and the multiple imputed analyses.

American Heart Journal July 2007

90 Koller et al

Figure 1

Observed 10-year risks of hard CHD by categories of predicted 10-year risk from FPS for 6795 subjects in the Rotterdam study. The line of identity illustrates perfect agreement between observed and predicted CHD risk. Good calibration is seen for women, whereas the risk is overestimated for men with FPS. Vertical lines at the bottom of the plots show the distribution of predicted risk. In women, most of the predicted risk is b20%, illustrating the low incidence of CHD in this population.

In secondary analyses, when fatal heart failure events were not considered as CHD death, predictions overestimated the observed risk in women on average (predicted risk of 10.1% against observed risk of 7.1% [95% CI 6.3-7.9]) and in categories of high predicted risk. However, predictions were well calibrated over the entire age range (data not shown). In men, FPS calibrated very similarly in primary and secondary analysis.

Discrimination The FPS showed reasonable discriminative performance in women (c-index 0.73) but substantially lower performance in men (c-index 0.63) (Table II). Validating FPS according to the recommended high-risk threshold showed that predictions N20% occurred in 13% and 32% of women and men, respectively (Table II, Figure 1). Only 33% of all women with a new-onset CHD event during follow-up were correctly classified as high risk (sensitivity), whereas 90% of the women who remained event free were correctly classified as low risk (specificity). Twenty-seven percent of high-risk women developed new-onset CHD during prospective follow-up (positive predictive value), and 92% of low-risk women remained event free (negative predictive value). In men, 48% of the CHD events were correctly predicted as high risk, but fewer (70%) event-free men than women were correctly classified as low risk (Table II). Twenty-two percent of men in the high-risk category developed new-onset CHD, and 89% in the low-

risk category remained event free (Table II). The performance of the FPS was slightly lower running complete case analyses compared with analyses on multiple imputed data, with the largest difference of 4 percentage points lower positive predictive value in women (23%). In secondary analyses excluding fatal heart failure cases, discrimination (c-index) and sensitivity of predictions in relation to the high-risk threshold slightly decreased in women (0.71 and 31%, respectively). The remaining accuracy parameters and all results in men did not change compared with the primary analysis.

Discussion Framingham Point Scores as presented with ATP III1 showed reasonable ability to predict hard CHD events in women aged N55 years. For elderly men, however, who are more prone to CHD, FPS overestimated CHD risk and showed poor discriminative ability. We found that miscalibration of predictions over the entire age range was an important reason for poor performance of FPS in elderly men. In contrast, CHD risk was underestimated only in women older than 75 years. The high-risk decision rule for case finding as recommended by ATP III was associated with a high false-negative rate in women and captured relatively few of the subjects who later had CHD events. However, the high negative predictive value of the high-risk

American Heart Journal Volume 154, Number 1

Figure 2

Deviation of 10-year predicted risk from observed risk according to age for 6795 subjects in the Rotterdam study. The zero line illustrates no deviation of predicted from observed risk. For younger male age categories, risk was substantially overestimated. For both older men and women, risk was underestimated with FPS. However, up to 75 years of age, predictions were well calibrated for women. Analyses with complete observations (complete cases) were similar to analyses where missing data were multiple imputed (see bMethodsQ section).

threshold makes a hard CHD event unlikely in women with a predicted risk V20%. Previous FHS-derived risk models systematically overestimated risk of CHD in different populations from Western and Northern Europe.4,23-25 These validation studies examined earlier FHS models predominantly in men. This pattern of systematic risk overestimation has changed with the FPS, at least for women where predictions were well calibrated for those aged up to 75 years. Validation of the FPS is of particular relevance because it is directly related to the treatment recommendations as statin- and diet-induced LDL-cholesterol (LDL-C) reduction, smoking cessation, lifestyle modification, and treatment of diabetes and hypertension.1 With increasing life expectancy, the aging population has become an important target of CHD risk prevention.10,26 Risk projections from FPS were thus extended up to 79 years of age compared with earlier FHS-based functions where the age range was restricted to 74 years.2,3 Furthermore, interactions of cholesterol with age and smoking with age were adopted in FPS, reflecting the decreased predictive importance of these risk factors with age.1,27,28 The Rotterdam study started in 1990,14 and events were recorded until the

Koller et al 91

end of 2005. Although ATP III was introduced in 2002 during the course of the study,1 we believe that the influence of ATP III recommendations on CHD outcomes is probably small. The present study includes subjects recruited between 1990 and 1993. However, the distribution of risk factors in later rounds of the Rotterdam cohort (between 1997 and 1999) was comparable with the recordings in the beginning, indicating that our results may also be valid in a more contemporary population.29 Within the ATP III framework, FPS are recommended for case finding, that is, the identification of subjects with a 10-year CHD risk N20%.1,30,31 When validating FPS in relation to this threshold, we have to consider 2 quantities: the predictive performance of FPS and the baseline population risk. The FPS systematically overestimated risk in men, which reflected inadequate calibration; too many subjects were classified as high risk. In women, FPS calibrated well for subjects up to 75 years; but the low 10-year incidence of CHD limited its usefulness. Only a minority of the women had predicted risks higher than the 20% threshold (Figure 2), with the consequence that only one third of the women with new-onset CHD would receive treatment. A better discriminating model would be needed to raise a relevant part of the distribution beyond 20%. The decision rule by itself implies that N20% events are expected over 10 years in the high-risk group.32 A major factor influencing cost-effectiveness of LDL-lowering therapy is absolute risk for CHD. The LDL-lowering therapy with statins is cost-effective for those with prior CHD events and those without CHD event but with comparable 10-year risk (CHD risk equivalents).1 Costeffectiveness is greater for those at highest risk and decreases progressively as risk falls. The issue of FPS is therefore to augment the population risk of CHD in highrisk subjects N20%. We found an observed risk of CHD in men classified as high risk of 22% and in women of 27% (Table II). Therefore, in our study population, costeffectiveness of LDL lowering might be lower for men than that for women. The reason is the poor discriminative ability of FPS in men, whereas fewer women would get treatment because of the low population risk. Updating and adapting CHD prediction models to the national context is important.6,32 The original model with all parameters underlying FPS has not been published is what makes it difficult to update FPS to regional or national settings, such as the Netherlands. Moreover, no clear-cut end point information beyond that FPS predict nonfatal MI and fatal CHD is available. Considering an elderly population, heart failure caused through CHD becomes increasingly prevalent; and the question rises whether heart failure death should be part of the definition of fatal CHD. In our primary and secondary analyses either including or disregarding fatal heart failure, the performance of the FPS was better on

American Heart Journal July 2007

92 Koller et al

Table II. Performance of the FPS to predict 10-year risk of hard CHD in 6795 men and women enrolled in the Rotterdam study Men (n = 2452) CHD* Predicted risk N20%4 Predicted risk V20%4 Total Percentage classified as high risk C-index Sensitivity Specificity Accuracy PPV NPV

171.8 183.4 355.2 32% (793.6 / 2452)

Women (n = 4343)

No CHD* 621.8 1475.0 2096.8 13% (553.8 / 4343)

Total

CHD*

No CHD*

Total

793.6 1658.4 2452

149.3 297.6 446.9

404.5 3491.6 3896.1

553.8 3789.2 4343

0.63 (0.52-0.74)y 48% (171.8 / 355.2) 70% (1475 / 2096.8) 67% ([171.8 + 1475.0] / 2452) 22% (171.8 / 793.6) 89% (1475.0 / 1658.4)

0.73 (0.65-0.83)z 33% (149.3 / 446.9) 90% (3491.6 / 3896.1) 84% ([149.3 + 3491.6] / 4343) 27% (149.3 / 553.8) 92% (3491.6 / 3789.2)

Analyses are based on multiple imputations for missing covariate values. PPV, Positive predictive value; NPV, negative predictive value. 4 Number of events and nonevents at 10 years was estimated with Kaplan-Meier analysis (see bMethodsQ section for details). yAccording to the high-risk threshold. z95% CI.

the primary analysis because most fatal heart failure cases had predicted risks N15%. However, sex-specific differences are important; the FPS performance was poor in men irrespective of whether fatal heart failure was considered as CHD or not.

Limitations One limitation of our study is that we use a European population–based cohort to validate a non-European prediction system. Despite the well-known limitations to transfer prediction systems between populations, the issue of inadequate age specification in men should be further investigated within US-based cohorts, especially because FPS performed adequately in European women. Adopting a European perspective, ATP III has also found wide acceptance in European countries especially with regard to LDL-C treatment targets of dyslipidemia.33-35 In the Rotterdam study, LDL-C measurements are not available. Therefore, the consequences of combinations of risk predictions (eg, the intermediate risk category) with levels of LDL-C as suggested by ATP III could not be assessed. Because evidence exists that LDL-lowering drug therapy is cost-effective in patients with CHD risk equivalents, we decided to validate FPS for the highrisk threshold. Conclusion In European elderly men, FPS perform poorly because age specification is not adequate for use in an elderly patient. Thus, updating of the age effect is needed before using FPS in an elderly population. The FPS predicted hard CHD in the female elderly population reasonably well particularly when fatal heart failure events were considered as fatal CHD. However, with the high-risk decision rule, only a small number of events are captured because of the low baseline risk.

Maintaining the rational of the high-risk threshold requires better performing models for a population with low incidence of CHD.

References 1. National Cholesterol Education Program. Third report of the National Cholesterol Education Program (NCEP) expert panel on detection, evaluation, and treatment of high blood cholesterol in adults (Adult Treatment Panel III) final report. Circulation 2002;106:3143 - 421. 2. Anderson KM, Wilson PW, Odell PM, et al. An updated coronary risk profile. A statement for health professionals. Circulation 1991;83: 356 - 62. 3. Wilson PW, D’Agostino RB, Levy D, et al. Prediction of coronary heart disease using risk factor categories. Circulation 1998;97:1837 - 47. 4. Brindle P, Emberson J, Lampe F, et al. Predictive accuracy of the Framingham coronary risk score in British men: prospective cohort study. BMJ 2003;327:1267. 5. Menotti A, Puddu PE, Lanti M. Comparison of the Framingham risk function-based coronary chart with risk function from an Italian population study. Eur Heart J 2000;21:365 - 70. 6. Hense HW, Schulte H, Lowel H, et al. Framingham risk function overestimates risk of coronary heart disease in men and women from Germany—results from the MONICA Augsburg and the PROCAM cohorts. Eur Heart J 2003;24:937 - 45. 7. Unal B, Critchley JA, Capewell S. Modelling the decline in coronary heart disease deaths in England and Wales, 1981-2000: comparing contributions from primary prevention and secondary prevention. BMJ 2005;331:614. 8. Hunink MG, Goldman L, Tosteson AN, et al. The recent decline in mortality from coronary heart disease, 1980-1990. The effect of secular trends in risk factors and treatment. JAMA 1997;277:535 - 42. 9. Grundy SM, Cleeman JI, Merz CN, et al. Implications of recent clinical trials for the National Cholesterol Education Program Adult Treatment Panel III guidelines. Circulation 2004;110:227 - 39. 10. Shepherd J, Blauw GJ, Murphy MB, et al. Pravastatin in elderly individuals at risk of vascular disease (PROSPER): a randomised controlled trial. Lancet 2002;360:1623 - 30.

American Heart Journal Volume 154, Number 1

11. Sever PS, Dahlof B, Poulter NR, et al. Prevention of coronary and stroke events with atorvastatin in hypertensive patients who have average or lower-than-average cholesterol concentrations, in the Anglo-Scandinavian Cardiac Outcomes Trial–Lipid Lowering Arm (ASCOT-LLA): a multicentre randomised controlled trial. Lancet 2003;361:1149 - 58. 12. Andrawes WF, Bussy C, Belmin J. Prevention of cardiovascular events in elderly people. Drugs Aging 2005;22:859 - 76. 13. Dornbrook-Lavender KA, Roth MT, Pieper JA. Secondary prevention of coronary heart disease in the elderly. Ann Pharmacother 2003;37:1867 - 76. 14. Hofman A, Grobbee DE, de Jong PT, et al. Determinants of disease and disability in the elderly: the Rotterdam Elderly Study. Eur J Epidemiol 1991;7:403 - 22. 15. Harrell FE . Regression modeling strategies with application to linear models, logistic regression, and survival analysis. New York, NY; Springer-Verlag; 2001. 16. Graf E, Schmoor C, Sauerbrei W, et al. Assessment and comparison of prognostic classification schemes for survival data. Stat Med 1999;18:2529 - 45. 17. Altman DG, Royston P. What do we mean by validating a prognostic model? Stat Med 2000;19:453 - 73. 18. Van Buuren S, Oudshoorn CGM. Multivariate imputation by chained equations. MICE V1.0 user’s manual, TNO Prevention and Health, Leiden, The Netherlands. http://web.inter.nl.edu/users/ S.van/Buuren/mi/hmtl/mice.htm, 2000. 19. Little RJA, Rubin DB. Statistical analysis with missing data. New York, NY: John Wiley; 1987. 20. Schafer JL. Analysis of incomplete multivariate data. New York: Chapman & Hall; 1997. 21. R Development Core Team. R: a language and environment for statistical computing. R Foundation for Statistical Computing. Vienna, Austria; R Development Core Team: 2006. 22. Clark TG, Altman DG, De Stavola BL. Quantification of the completeness of follow-up. Lancet 2002;359:1309 - 10. 23. Thomsen TF, McGee D, Davidsen M, et al. A cross-validation of risk scores for coronary heart disease mortality based on data from the Glostrup Population Studies and Framingham Heart Study. Int J Epidemiol 2002;31:817 - 22. 24. Marrugat J, D’Agostino R, Sullivan L, et al. An adaptation of the Framingham coronary heart disease risk function to European

Koller et al 93

25.

26.

27.

28.

29.

30. 31. 32. 33.

34.

35.

Mediterranean areas. J Epidemiol Community Health 2003;57:634 - 8. Empana JP, Ducimetiere P, Arveiler D, et al. Are the Framingham and PROCAM coronary heart disease risk functions applicable to different European populations? The PRIME Study. Eur Heart J 2003;24:1903 - 11. Stone NJ, Bilek S, Rosenbaum S. Recent National Cholesterol Education Program Adult Treatment Panel III update: adjustments and options. Am J Cardiol 2005;96:53E - 9E. Psaty BM, Furberg CD, Kuller LH, et al. Traditional risk factors and subclinical disease measures as predictors of first myocardial infarction in older adults: the Cardiovascular Health Study. Arch Intern Med 1999;159:1339 - 47. Law MR, Wald NJ, Thompson SG. By how much and how quickly does reduction in serum cholesterol concentration lower risk of ischaemic heart disease? BMJ 1994;308:367 - 72. Vliegenthart R, Oudkerk M, Hofman A, et al. Coronary calcification improves cardiovascular risk prediction in the elderly. Circulation 2005;112:572 - 7. Jackson R. Updated New Zealand cardiovascular disease riskbenefit prediction guide. BMJ 2000;320:709 - 10. Jackson R. Guidelines on preventing cardiovascular disease in clinical practice. BMJ 2000;320:659 - 61. Hense HW. Observations, predictions and decisions—assessing cardiovascular risk assessment. Int J Epidemiol 2004;33:235 - 9. Assmann G, Benecke H, Neiss A, et al. Gap between guidelines and practice: attainment of treatment targets in patients with primary hypercholesterolemia starting statin therapy. Results of the 4ERegistry (Efficacy Calculation and Measurement of Cardiovascular and Cerebrovascular Events including Physicians’ Experience and Evaluation). Eur J Cardiovasc Prev Rehabil 2006;13:776 - 83. Fornengo P, Bruno G, De Salvia A, et al. Low adherence of general practitioners to National Cholesterol Education Program guidelines for the management of hyperlipidaemia. Diabetes Nutr Metab 2000;13:263 - 8. De Backer G, Ambrosioni E, Borch-Johnsen K, et al. European guidelines on cardiovascular disease prevention in clinical practice: third joint task force of European and other societies on cardiovascular disease prevention in clinical practice (constituted by representatives of eight societies and by invited experts). Eur J Cardiovasc Prev Rehabil 2003;10:S1-S10.