- Email: [email protected]
Epidemiological Studies Are Meaningless without Proof of Long-Term Data Stability
LETTER
Epidemiological Studies Are Meaningless without Proof of Long-Term Data Stability To the Editor: Cobain et al detail the lifetime risk of developing dyslipidemia based on data samples collected 4-yearly between 1971 and 2000 in the Framingham Offspring study.1 This might be a useful contribution but suffers from a significant flaw. In such a long-term study it is vital that the long-term stability of the lipid assays employed is proven. Most commercial analyzer platforms have a functional lifespan of about 7 years. Thus, it would be expected that over 30 years at least 5 different platforms will have been used. Every time an assay platform changes it is essential that method comparison studies assess the comparability of the results. This should include plots of absolute and relative bias across the clinical range to exclude concentration-dependent problems.2 The only method comparison cited by Cobain is for high-density lipoprotein (HDL) cholesterol and was published in 1983 when a manual colorimetric LiebermannBurchard3 was replaced by a manual enzymatic method.4 No bias evaluations were carried out and precision was only evaluated on fresh serum at HDL concentrations of 40 and 52 mg/L. No report of the precision of the older method was made. In no way does this constitute acceptable proof that results from 2000 are the same as results from 1971. Furthermore, since 1983, many changes in HDL assays have been made, including the introduction of direct (nonprecipitation) assays with considerable shifts in mean values.5 The Friedewald equation was based on an estimate from just 46 patients with type II hyperlipidemia that very lowdensity lipoprotein (LDL) cholesterol could be estimated as 20% of triglyceride concentration.6 This estimate was validated against a larger set of ultracentrifuge data, but nevertheless, calculated LDL is more imprecise because it is based on the outcome of 3 other assays. Consequently, it is essential that all biases and drifts over the assessment period
0002-9343/$ -see front matter © 2008 Elsevier Inc. All rights reserved.
for all contributing assays are accurately quantified because small variations in the input variables cumulate in large changes in the output LDL cholesterol as well as affecting risk algorithm outcomes. Standardization and quality control are essential to performing high-quality epidemiological studies. Unfortunately, unless laboratory assays are absolutely and identifiably linked, the pyramid of Framingham study evidence will be unstable because the vertex has been used as the foundation. Tim M. Reynolds, MD Department of Clinical Chemistry Queen’s Hospital Burton-on-Trent, UK
Anthony S. Wierzbicki, DM St Thomas’ Hospital London, UK
Patrick J. Twomey, MB BCh Ipswich Hospital Ipswich, UK
doi:10.1016/j.amjmed.2007.08.040
References 1. Cobain MR, Pencina MJ, A’Agostino RB, Vasan RS. Lifetime risk for developing dyslipidemia: the Framingham Offspring Study. Am J Med. 2007;120:623-630. 2. Twomey PJ. How to use difference plots in quantitative method comparison studies. Ann Clin Biochem. 2006;43:124-129. 3. Lipid Research Clinics Program. Manual of Laboratory Operation. Bethesda, MD: National Institutes of Health; 1974:75-628. 4. Warnick GR, Benderson J, Albers J. Dextran-sulphate-Mg2⫹ precipitation procedure for quantitation of high-density-lipoprotein cholesterol. Clin Chem. 1982;28:1379-1388. 5. Durrington PN. A comparison of three methods of measuring serum high density lipoprotein cholesterol in diabetics and non-diabetics. Ann Clin Biochem. 1980;17:199-204. 6. Friedewald WT, Levy RI, Fredrickson DS. Estimation of the concentration of low-density lipoprotein in plasma, without use of the preparative ultracentrifuge. Clin Chem. 1972;18:499-502.
Epidemiological Studies Are Meaningless without Proof of Long-Term Data Stability To the Editor: Cobain et al detail the lifetime risk of developing dyslipidemia based on data samples collected 4-yearly between 1971 and 2000 in the Framingham Offspring study.1 This might be a useful contribution but suffers from a significant flaw. In such a long-term study it is vital that the long-term stability of the lipid assays employed is proven. Most commercial analyzer platforms have a functional lifespan of about 7 years. Thus, it would be expected that over 30 years at least 5 different platforms will have been used. Every time an assay platform changes it is essential that method comparison studies assess the comparability of the results. This should include plots of absolute and relative bias across the clinical range to exclude concentration-dependent problems.2 The only method comparison cited by Cobain is for high-density lipoprotein (HDL) cholesterol and was published in 1983 when a manual colorimetric LiebermannBurchard3 was replaced by a manual enzymatic method.4 No bias evaluations were carried out and precision was only evaluated on fresh serum at HDL concentrations of 40 and 52 mg/L. No report of the precision of the older method was made. In no way does this constitute acceptable proof that results from 2000 are the same as results from 1971. Furthermore, since 1983, many changes in HDL assays have been made, including the introduction of direct (nonprecipitation) assays with considerable shifts in mean values.5 The Friedewald equation was based on an estimate from just 46 patients with type II hyperlipidemia that very lowdensity lipoprotein (LDL) cholesterol could be estimated as 20% of triglyceride concentration.6 This estimate was validated against a larger set of ultracentrifuge data, but nevertheless, calculated LDL is more imprecise because it is based on the outcome of 3 other assays. Consequently, it is essential that all biases and drifts over the assessment period
0002-9343/$ -see front matter © 2008 Elsevier Inc. All rights reserved.
for all contributing assays are accurately quantified because small variations in the input variables cumulate in large changes in the output LDL cholesterol as well as affecting risk algorithm outcomes. Standardization and quality control are essential to performing high-quality epidemiological studies. Unfortunately, unless laboratory assays are absolutely and identifiably linked, the pyramid of Framingham study evidence will be unstable because the vertex has been used as the foundation. Tim M. Reynolds, MD Department of Clinical Chemistry Queen’s Hospital Burton-on-Trent, UK
Anthony S. Wierzbicki, DM St Thomas’ Hospital London, UK
Patrick J. Twomey, MB BCh Ipswich Hospital Ipswich, UK
doi:10.1016/j.amjmed.2007.08.040
References 1. Cobain MR, Pencina MJ, A’Agostino RB, Vasan RS. Lifetime risk for developing dyslipidemia: the Framingham Offspring Study. Am J Med. 2007;120:623-630. 2. Twomey PJ. How to use difference plots in quantitative method comparison studies. Ann Clin Biochem. 2006;43:124-129. 3. Lipid Research Clinics Program. Manual of Laboratory Operation. Bethesda, MD: National Institutes of Health; 1974:75-628. 4. Warnick GR, Benderson J, Albers J. Dextran-sulphate-Mg2⫹ precipitation procedure for quantitation of high-density-lipoprotein cholesterol. Clin Chem. 1982;28:1379-1388. 5. Durrington PN. A comparison of three methods of measuring serum high density lipoprotein cholesterol in diabetics and non-diabetics. Ann Clin Biochem. 1980;17:199-204. 6. Friedewald WT, Levy RI, Fredrickson DS. Estimation of the concentration of low-density lipoprotein in plasma, without use of the preparative ultracentrifuge. Clin Chem. 1972;18:499-502.