- Email: [email protected]
Coronary risk assessment methods and cholesterol lowering
COMMENTARY
CORRESPONDENCE Coronary risk assessment methods and cholesterol lowering Sir—In their comparison of methods for assessing coronary (CHD) risk, P N Durrington and colleagues (Jan 23, p 278)1 suggest that the Sheffield table is too conservative and not backed by evidence on its accuracy. In fact the table is more generous with lipidlowering drug therapy2 than are the British Hyperlipidaemia Association guidelines,3 on which Durrington was a co-author. Furthermore, almost all those treated according to the Sheffield table have CHD risk of 3% or greater per year, because the table has a specificity of 98% compared with the full Framingham risk function.4 There is no such certainty for the British Hyperlipidaemia Association guidelines,3 which did not advocate formal CHD risk estimation. With respect to accuracy, Sheffield tables based on total cholesterol and on the total cholesterol/HDL ratio have in fact been compared with the Framingham risk function,4 and with the Münster Heart Study (PROCAM) risk function as an independent standard4 (table). The Sheffield table based on total cholesterol has high specificity but low sensitivity (table), although the sensitivity is nowhere near as low as Durrington’s evaluation suggests. The table based on the total cholesterol/HDL ratio was much more accurate, as expected, and we have recommended use of this table and made it available on request for some time.5 Our evaluation4 also confirms that the Joint European Task Force chart has high sensitivity, but shows that it has very low specificity (table). Durrington and colleagues ignore specificity entirely in their evaluation. It is no surprise that the Sheffield table fared badly in Durrington and coworkers’ evaluation, because they did not use it correctly. The notes on use of the table state clearly that it should be used only after hypertension is controlled to systolic 160 mm Hg or
less, and specifically warn about low HDL concentrations. These investigators ignore these caveats. Patients referred to a lipid clinic are not typical and likely to defeat any simple method, and the acid test of these risk assessment methods is their accuracy in the general population. The most recent Sheffield table is based on total cholesterol/HDL ratio and identifies CHD levels of 1·5% and 3% per year (available on request). It has been evaluated for accuracy in a random sample of 1000 people aged 35–64 years in the general population who might require primary prevention (unpublished results). This table has sensitivity of 97% and specificity of 95% for identifying CHD risk of 1·5% or greater per year, and sensitivity of 82% and specificity of 99% for identifying CHD risk 3% per year, even when those with uncontrolled hypertension are included. Excluding people with uncontrolled hypertension, the sensitivity is 100% and specificity 99% for identifying CHD risk 3% or greater per year. Durrington et al were wrong to suggest that the Sheffield tables are not supported by evidence for their accuracy. We believe that this is the only Framingham-based method that is validated against an independent risk function,4 and has proven accuracy in the general population. *L E Ramsay, E J Wallis, I U Haq, P R Jac k son University of Sheffield, Clinical Pharmacology and Therapeutics, Royal Hallamshire Hospital, Sheffield S10 2JF, UK 1
2
Durrington PN, Prais H, Bhatnagar D, et al. Indications for cholesterol lowering medication: comparison of risk-assessment methods. Lancet 1999; 353: 278–81. Unwin N, Thomson R, O’Byrne AM, Laker M, Armstrong H. Implications of applying widely accepted cholesterol screening and management guidelines to a British adult population; cross-sectional study of cardiovascular disease and risk factors. BMJ 1998; 317: 1125–30.
Method
CHD risk targeted (% per year)
Sensitivity (%)
Specificity (%)
CHD risk treated
CHD risk not treated
Sheffield TC Sheffield TC/HDL Joint Euro Task Force
3 3 2
52 97 98
96 82 37
4·5 3·8 2·6
1·9 1·3 0·6
From ref 4. TC=total cholesterol.
Accuracy of methods based on the Framingham risk function for predicting CHD risk calculated by the PROCAM risk function in 126 men with high risk
THE LANCET • Vol 353 • March 27, 1999
3
Betteridge DJ, Dodson PM, Durrington PN, et al. Management of hyperlipidaemia: Guidelines of the British Hyperlipidaemia Association. Post Grad Med J 1993; 69: 359–69. 4 Haq IU, Jackson PR, Yeo WW, Ramsey LE. A comparison of methods for targeting CHD risk for primary prevention. Heart 1997; 77 (suppl 1): 36. 5 Wallis EJ, Ramsay LE, Jackson PR, Yeo WW, Williamson R. Drug therapy for coronary heart disease: the Sheffield table. Lancet 1997; 350: 1852.
Sir—P N Durrington and colleagues1 correctly conclude that the Sheffield tables would miss most people with a coronary heart disease (CHD) risk that is sufficiently high to justify the use of statins to lower blood cholesterol.1 However, it is too optimistic and simplistic to suggest that the joint European guidelines risk chart is sufficiently good and that the assessment of HDL cholesterol is less critical in predicting CHD risk. One important feature of the riskassessment methods is not included in the evaluation—the false-positive error. Simplified risk-assessment methods usually use the average values for, or broad grouping of, some risk factors. Both the European risk chart and Sheffield tables, for example, use the sex-specific average concentrations to substitute for individuals’ HDL cholesterol concentrations.2,3 With this simplified approach, a person’s true risk could be underestimated, resulting in false-negative errors, or overestimated, resulting in false-positive errors. The former error will miss patients who need treatment and defeat the objective of an intervention programme, whereas the latter will lead to unnecessary treatment and waste resources. Falsepositive errors would be especially common in assessments of populations with few high-risk patients. Both aspects of a risk-assessment method should be considered in an evaluation, and are usually expressed as sensitivity and specificity, which are complementary to false-negative and false-positive errors, respectively.4 Low sensitivity, or a high rate of false-negative errors, is obviously the main drawback to the Sheffield tables. Although the European guidelines chart may not miss many in need of the treatment, its specificity or false-
1095
actual HDL cholesterol and the difference between risks when estimated from actual HDL and fixed HDL cholesterol (their figure 1) arises because the Sheffield risk tables do not take into account any association between low HDL and multiple risk factors for CHD. Their figure 2 shows the relation between HDL and the number of risk factors, particularly in women. In fact the correlation shown in figure 1 would occur even if there were no association between HDL and the number of risk factors. For any LDL cholesterol value there will be a distribution of HDL concentrations. If low HDL is an independent predictor of CHD risk then it is inevitable that for any LDL value people with low concentrations will have a high CHD risk and a high difference between the risk estimated from actual HDL cholesterol and that estimated from fixed HDL cholesterol. Conversely, those with high HDL will have a low risk and a low difference in risk. Thus risk assessment methods that do not take the HDL concentration into account (such as the original Sheffield tables) will inevitably increasingly underestimate the risk as HDL values fall.
positive error should have also been examined against the Framingham equation from which the European risk chart is derived. On the basis of the data in Durrington’s report and assuming 100% sensitivity, we estimate that the maximum possible specificity of the European guidelines risk chart is 92% in men and 97% in women.* This result might be good enough for Durrington and colleagues’ study participants who had a CHD risk well above the general adult populations. However, for the UK population aged 35–69 years in which 5·7% of men and 0·4% of women have an annual CHD risk of 3·0% or above,3 it means that at least 57% of men and 88% of women recommended for treatment would be given statin unnecessarily.* The European guidelines risk chart leads to too many unnecessary treatments when applied to general populations. Predicting risk in individuals is notoriously difficult. The simplified methods in guidelines for cholesterollowering interventions cause so many misclassification errors that they have little value for decision making. Therefore, more accurate methods, such as the computer risk program which has been made fairly easy to use in clinical settings,5 should be used wherever accessible.
Jeremy Wight
*Tables available from the authors or The Lancet, on request.
Wakefield Health Authority, W akefield W F 1 1LT, UK
*Jin-Ling Tang, James A Dickinson, Joseph L Y Liu
Authors’ reply
*Department of Community and Family Medicine, Faculty of Medicine, Chinese University of Hong Kong Shatin, NT, Hong Kong; and Centre for Clinical Trials and Epidemiological Research, Hong Kong (e-mail: [email protected])
Sir—It is not the conservatism of the Sheffield tables about which we were critical, but their inaccuracy. As we stated in our report, whatever degree of risk is recommended for the use of lipid-lowering drugs “all recommendations depend for their clinical effectiveness on the accuracy with which the clinician can estimate CHD risk in individual patients”. The Sheffield tables proved to be worryingly inaccurate on this criterion, identifying only 9% of men and less than 1% of women in a population in which they were designed to identify 22% of men and 7% of women. Their sensitivity and specificity will depend on the frequency of low HDL cholesterol. That is precisely why L E Ramsay and colleagues are wrong to suggest that the appropriate population in which to test methods of coronary risk assessment is the general population. It is not. It is in the type of patients in whom clinicians wish to know whether to prescribe lipid-lowering treatment: in the patients included in our study this was the question that concerned the referring physician. Incidentally the point raised by Ramsay et al about systolic blood
1
Durrington PN, Prais H, Bhatnagar D, et al. Indications for cholesterol lowering medication: comparison of risk-assessment methods. Lancet 1999; 353: 278–81. 2 Pyörälä K, de Backer G, Graham I, et al. Prevention of coronary heart disease in clinical practice: recommendations of the Task Force of the European Society of Cardiology, European Atherosclerosis Society and European Society of Hypertension. Eur Heart J 1994; 15: 1300–31. 3 Ramsay LE, Haq IU, Jackson PR, Yeo WW, Pickin DM, Payne JN. Targeting lipidlowering drug therapy for primary prevention of coronary heart disease: an updated Sheffield table. Lancet 1996; 348: 387–88. 4 Sackett DL, Haynes RB, Guyatt GH, Tugwell P. Clinical epidemiology: a basic science for clinical medicine. 2nd ed. Boston: Little, Brown and Company, 1991. 5 Hingorani AD, Vallance P. A simple computer programme for guiding management of cardiovascular risk factors and prescribing. BMJ 1999; 318: 101–05.
Sir—P N Durrington and colleagues1 claim that the correlation they show between CHD risk estimated from
1096
pressure does not alter our findings, and the advice not to use the Sheffield tables when HDL cholesterol is low cannot be heeded if total serum cholesterol only is measured as is intended by the tables. The new tables of Ramsay et al based on the total to HDL cholesterol ratio, providing both 1·5% and 3% annual CHD risk, are thus far published only as an asbtract.1 They seem complex and we suggest that they also be evaluated for ease of use in routine medical practice as well as for their accuracy. Ramsay and colleagues are critical of the British Hyperlipidaemia Association guidelines published in 1993. These prioritised secondary prevention for cholesterol-lowering drugs and were thus prescient because the publication of the first of the clinical trials of statins was not until 1994.2 The British Hyperlipidaemia Association has now revised its recommendations jointly with the British Cardiac Society, British Hypertension Society, and the British Diabetic Association.3 These continue to underscore secondary prevention, but also emphasise the need for primary prevention3 and acknowledge Ramsay’s own valuable contribution. For primary prevention a computer program to assess coronary risk is preferred, which does not in itself impose any arbitrary thresholds as in the case of the Sheffield tables. Jin-Ling Tang and colleagues are wrong to suggest that we advocate the use of the joint European charts in our report. Extrapolation of specificities in our high-risk population to the general population is too speculative. Tang et al may, however, have missed the most important limitation of the joint European charts: they could be applied to only 59% of our patients, which is considerably fewer than any other method. The issue raised by Jeremy Wight about our Altman and Bland plot (figure 1) is more complex than he suggests. The more general answers to his criticisms are discussed elsewhere,4 but in this case one must also consider that total and HDL cholesterol are computed as a ratio in the Framingham risk equation (rather than as HDL and LDL discretely) and that HDL is not strictly an independent risk factor, because it correlates with the presence of other risk factors. The extent to which risk is underestimated by assigning a fixed value to HDL cholesterol in practice is thus impossible to assess without doing the experiment we have done. Surely the clinical point on which we are agreed is that HDL cholesterol is an essential part of the assessment of CHD risk.
THE LANCET • Vol 353 • March 27, 1999
CORRESPONDENCE Coronary risk assessment methods and cholesterol lowering Sir—In their comparison of methods for assessing coronary (CHD) risk, P N Durrington and colleagues (Jan 23, p 278)1 suggest that the Sheffield table is too conservative and not backed by evidence on its accuracy. In fact the table is more generous with lipidlowering drug therapy2 than are the British Hyperlipidaemia Association guidelines,3 on which Durrington was a co-author. Furthermore, almost all those treated according to the Sheffield table have CHD risk of 3% or greater per year, because the table has a specificity of 98% compared with the full Framingham risk function.4 There is no such certainty for the British Hyperlipidaemia Association guidelines,3 which did not advocate formal CHD risk estimation. With respect to accuracy, Sheffield tables based on total cholesterol and on the total cholesterol/HDL ratio have in fact been compared with the Framingham risk function,4 and with the Münster Heart Study (PROCAM) risk function as an independent standard4 (table). The Sheffield table based on total cholesterol has high specificity but low sensitivity (table), although the sensitivity is nowhere near as low as Durrington’s evaluation suggests. The table based on the total cholesterol/HDL ratio was much more accurate, as expected, and we have recommended use of this table and made it available on request for some time.5 Our evaluation4 also confirms that the Joint European Task Force chart has high sensitivity, but shows that it has very low specificity (table). Durrington and colleagues ignore specificity entirely in their evaluation. It is no surprise that the Sheffield table fared badly in Durrington and coworkers’ evaluation, because they did not use it correctly. The notes on use of the table state clearly that it should be used only after hypertension is controlled to systolic 160 mm Hg or
less, and specifically warn about low HDL concentrations. These investigators ignore these caveats. Patients referred to a lipid clinic are not typical and likely to defeat any simple method, and the acid test of these risk assessment methods is their accuracy in the general population. The most recent Sheffield table is based on total cholesterol/HDL ratio and identifies CHD levels of 1·5% and 3% per year (available on request). It has been evaluated for accuracy in a random sample of 1000 people aged 35–64 years in the general population who might require primary prevention (unpublished results). This table has sensitivity of 97% and specificity of 95% for identifying CHD risk of 1·5% or greater per year, and sensitivity of 82% and specificity of 99% for identifying CHD risk 3% per year, even when those with uncontrolled hypertension are included. Excluding people with uncontrolled hypertension, the sensitivity is 100% and specificity 99% for identifying CHD risk 3% or greater per year. Durrington et al were wrong to suggest that the Sheffield tables are not supported by evidence for their accuracy. We believe that this is the only Framingham-based method that is validated against an independent risk function,4 and has proven accuracy in the general population. *L E Ramsay, E J Wallis, I U Haq, P R Jac k son University of Sheffield, Clinical Pharmacology and Therapeutics, Royal Hallamshire Hospital, Sheffield S10 2JF, UK 1
2
Durrington PN, Prais H, Bhatnagar D, et al. Indications for cholesterol lowering medication: comparison of risk-assessment methods. Lancet 1999; 353: 278–81. Unwin N, Thomson R, O’Byrne AM, Laker M, Armstrong H. Implications of applying widely accepted cholesterol screening and management guidelines to a British adult population; cross-sectional study of cardiovascular disease and risk factors. BMJ 1998; 317: 1125–30.
Method
CHD risk targeted (% per year)
Sensitivity (%)
Specificity (%)
CHD risk treated
CHD risk not treated
Sheffield TC Sheffield TC/HDL Joint Euro Task Force
3 3 2
52 97 98
96 82 37
4·5 3·8 2·6
1·9 1·3 0·6
From ref 4. TC=total cholesterol.
Accuracy of methods based on the Framingham risk function for predicting CHD risk calculated by the PROCAM risk function in 126 men with high risk
THE LANCET • Vol 353 • March 27, 1999
3
Betteridge DJ, Dodson PM, Durrington PN, et al. Management of hyperlipidaemia: Guidelines of the British Hyperlipidaemia Association. Post Grad Med J 1993; 69: 359–69. 4 Haq IU, Jackson PR, Yeo WW, Ramsey LE. A comparison of methods for targeting CHD risk for primary prevention. Heart 1997; 77 (suppl 1): 36. 5 Wallis EJ, Ramsay LE, Jackson PR, Yeo WW, Williamson R. Drug therapy for coronary heart disease: the Sheffield table. Lancet 1997; 350: 1852.
Sir—P N Durrington and colleagues1 correctly conclude that the Sheffield tables would miss most people with a coronary heart disease (CHD) risk that is sufficiently high to justify the use of statins to lower blood cholesterol.1 However, it is too optimistic and simplistic to suggest that the joint European guidelines risk chart is sufficiently good and that the assessment of HDL cholesterol is less critical in predicting CHD risk. One important feature of the riskassessment methods is not included in the evaluation—the false-positive error. Simplified risk-assessment methods usually use the average values for, or broad grouping of, some risk factors. Both the European risk chart and Sheffield tables, for example, use the sex-specific average concentrations to substitute for individuals’ HDL cholesterol concentrations.2,3 With this simplified approach, a person’s true risk could be underestimated, resulting in false-negative errors, or overestimated, resulting in false-positive errors. The former error will miss patients who need treatment and defeat the objective of an intervention programme, whereas the latter will lead to unnecessary treatment and waste resources. Falsepositive errors would be especially common in assessments of populations with few high-risk patients. Both aspects of a risk-assessment method should be considered in an evaluation, and are usually expressed as sensitivity and specificity, which are complementary to false-negative and false-positive errors, respectively.4 Low sensitivity, or a high rate of false-negative errors, is obviously the main drawback to the Sheffield tables. Although the European guidelines chart may not miss many in need of the treatment, its specificity or false-
1095
actual HDL cholesterol and the difference between risks when estimated from actual HDL and fixed HDL cholesterol (their figure 1) arises because the Sheffield risk tables do not take into account any association between low HDL and multiple risk factors for CHD. Their figure 2 shows the relation between HDL and the number of risk factors, particularly in women. In fact the correlation shown in figure 1 would occur even if there were no association between HDL and the number of risk factors. For any LDL cholesterol value there will be a distribution of HDL concentrations. If low HDL is an independent predictor of CHD risk then it is inevitable that for any LDL value people with low concentrations will have a high CHD risk and a high difference between the risk estimated from actual HDL cholesterol and that estimated from fixed HDL cholesterol. Conversely, those with high HDL will have a low risk and a low difference in risk. Thus risk assessment methods that do not take the HDL concentration into account (such as the original Sheffield tables) will inevitably increasingly underestimate the risk as HDL values fall.
positive error should have also been examined against the Framingham equation from which the European risk chart is derived. On the basis of the data in Durrington’s report and assuming 100% sensitivity, we estimate that the maximum possible specificity of the European guidelines risk chart is 92% in men and 97% in women.* This result might be good enough for Durrington and colleagues’ study participants who had a CHD risk well above the general adult populations. However, for the UK population aged 35–69 years in which 5·7% of men and 0·4% of women have an annual CHD risk of 3·0% or above,3 it means that at least 57% of men and 88% of women recommended for treatment would be given statin unnecessarily.* The European guidelines risk chart leads to too many unnecessary treatments when applied to general populations. Predicting risk in individuals is notoriously difficult. The simplified methods in guidelines for cholesterollowering interventions cause so many misclassification errors that they have little value for decision making. Therefore, more accurate methods, such as the computer risk program which has been made fairly easy to use in clinical settings,5 should be used wherever accessible.
Jeremy Wight
*Tables available from the authors or The Lancet, on request.
Wakefield Health Authority, W akefield W F 1 1LT, UK
*Jin-Ling Tang, James A Dickinson, Joseph L Y Liu
Authors’ reply
*Department of Community and Family Medicine, Faculty of Medicine, Chinese University of Hong Kong Shatin, NT, Hong Kong; and Centre for Clinical Trials and Epidemiological Research, Hong Kong (e-mail: [email protected])
Sir—It is not the conservatism of the Sheffield tables about which we were critical, but their inaccuracy. As we stated in our report, whatever degree of risk is recommended for the use of lipid-lowering drugs “all recommendations depend for their clinical effectiveness on the accuracy with which the clinician can estimate CHD risk in individual patients”. The Sheffield tables proved to be worryingly inaccurate on this criterion, identifying only 9% of men and less than 1% of women in a population in which they were designed to identify 22% of men and 7% of women. Their sensitivity and specificity will depend on the frequency of low HDL cholesterol. That is precisely why L E Ramsay and colleagues are wrong to suggest that the appropriate population in which to test methods of coronary risk assessment is the general population. It is not. It is in the type of patients in whom clinicians wish to know whether to prescribe lipid-lowering treatment: in the patients included in our study this was the question that concerned the referring physician. Incidentally the point raised by Ramsay et al about systolic blood
1
Durrington PN, Prais H, Bhatnagar D, et al. Indications for cholesterol lowering medication: comparison of risk-assessment methods. Lancet 1999; 353: 278–81. 2 Pyörälä K, de Backer G, Graham I, et al. Prevention of coronary heart disease in clinical practice: recommendations of the Task Force of the European Society of Cardiology, European Atherosclerosis Society and European Society of Hypertension. Eur Heart J 1994; 15: 1300–31. 3 Ramsay LE, Haq IU, Jackson PR, Yeo WW, Pickin DM, Payne JN. Targeting lipidlowering drug therapy for primary prevention of coronary heart disease: an updated Sheffield table. Lancet 1996; 348: 387–88. 4 Sackett DL, Haynes RB, Guyatt GH, Tugwell P. Clinical epidemiology: a basic science for clinical medicine. 2nd ed. Boston: Little, Brown and Company, 1991. 5 Hingorani AD, Vallance P. A simple computer programme for guiding management of cardiovascular risk factors and prescribing. BMJ 1999; 318: 101–05.
Sir—P N Durrington and colleagues1 claim that the correlation they show between CHD risk estimated from
1096
pressure does not alter our findings, and the advice not to use the Sheffield tables when HDL cholesterol is low cannot be heeded if total serum cholesterol only is measured as is intended by the tables. The new tables of Ramsay et al based on the total to HDL cholesterol ratio, providing both 1·5% and 3% annual CHD risk, are thus far published only as an asbtract.1 They seem complex and we suggest that they also be evaluated for ease of use in routine medical practice as well as for their accuracy. Ramsay and colleagues are critical of the British Hyperlipidaemia Association guidelines published in 1993. These prioritised secondary prevention for cholesterol-lowering drugs and were thus prescient because the publication of the first of the clinical trials of statins was not until 1994.2 The British Hyperlipidaemia Association has now revised its recommendations jointly with the British Cardiac Society, British Hypertension Society, and the British Diabetic Association.3 These continue to underscore secondary prevention, but also emphasise the need for primary prevention3 and acknowledge Ramsay’s own valuable contribution. For primary prevention a computer program to assess coronary risk is preferred, which does not in itself impose any arbitrary thresholds as in the case of the Sheffield tables. Jin-Ling Tang and colleagues are wrong to suggest that we advocate the use of the joint European charts in our report. Extrapolation of specificities in our high-risk population to the general population is too speculative. Tang et al may, however, have missed the most important limitation of the joint European charts: they could be applied to only 59% of our patients, which is considerably fewer than any other method. The issue raised by Jeremy Wight about our Altman and Bland plot (figure 1) is more complex than he suggests. The more general answers to his criticisms are discussed elsewhere,4 but in this case one must also consider that total and HDL cholesterol are computed as a ratio in the Framingham risk equation (rather than as HDL and LDL discretely) and that HDL is not strictly an independent risk factor, because it correlates with the presence of other risk factors. The extent to which risk is underestimated by assigning a fixed value to HDL cholesterol in practice is thus impossible to assess without doing the experiment we have done. Surely the clinical point on which we are agreed is that HDL cholesterol is an essential part of the assessment of CHD risk.
THE LANCET • Vol 353 • March 27, 1999