- Email: [email protected]
Making positive out of negative trials
Editorial
Making positive out of negative trials David E. Kandzari, MD, and Pascal J. Goldschmidt-Clermont, MD Durham, NC
See related article on page 971.
In this issue of the Journal, Koch and colleagues describe the lack of association of a genetic polymorphism in monocyte differentiation antigen CD14 with the presence of coronary artery disease and myocardial infarction. Although the study is possibly considered by some to be a “negative trial,” these findings instead offer much insight to our understanding of inherited alterations in genes and their contribution to heart disease. First, such results further clarify the role of inherited polymorphisms of the CD14 gene in atherosclerosis and adverse cardiac events. Yet, in a much broader perspective, the publication of negative studies about genetic associations is an integral part of refining a relatively new and emerging field of cardiovascular science. Publication of negative results from well-designed, adequately powered clinical studies is positively essential to our interpretation of the role of genetics in cardiovascular disease. In this study, the results by Koch et al add to a growing body of conflicting evidence regarding the contribution of CD14 polymorphisms to inflammation, atherosclerosis, and cardiovascular events. With a population of nearly 1800 individuals with coronary artery disease (approximately 800 of whom had prior myocardial infarctions), the presence of a single nucleotide polymorphism characterized by a substitution of cytosine (C) for thymine (T) at position –159 of the CD14 gene promoter was not associated with an increased likelihood of coronary artery disease or myocardial infarction relative to a group of individuals without established cardiovascular disease. The lack of an association was maintained even after adjustment for standard cardiovascular risk factors. Moreover, the distribution of genotypes among the groups did not significantly differ, a finding that seems to further support the lack of substantial involvement of this particular polymorphism in the development of coronary artery disease. Should the study by Koch and colleagues be considered the final word on the role of the –159C/T poly-
From the Division of Cardiology and Duke Clinical Research Institute, Duke University Medical Center, Durham, NC. Submitted November 1, 2001; accepted December 5, 2001. Reprint requests: David E. Kandzari, MD, PO Box 31203, Duke University Medical Center, Durham, NC 27710. E-mail: [email protected] Am Heart J 2002;143:950-1. © 2002, Mosby, Inc. All rights reserved. 0002-8703/2002/$35.00 ⫹ 0 4/4/122513 doi:10.1067/mhj.2002.122513
morphism in cardiovascular disease? If not, how should clinicians interpret these findings in the context of conflicting results from other epidemiologic studies? As the investigators note, previous studies involving smaller numbers of subjects have reported higher levels of circulating inflammatory markers or higher frequencies of coronary events associated with the TT genotype. The mechanistic studies that show increased levels of CD14 among TT homozygotes also seem to intuitively support the concept that a proinflammatory state would contribute to atherogenesis and subsequent cardiovascular events. Perhaps the most appropriate way to interpret results from this and other contemporary genetic association studies is to accept them as part of a growing body of evidence to define the role of inherited polymorphisms in the development of coronary artery disease, in this case regarding the contribution of a specific inflammatory marker to atherosclerosis and thrombotic events. Although it is appealing to attribute divergent results from similar studies to experimental design or patient populations, it is also important to recognize that the clinical diagnosis of acute coronary syndromes includes a broad range of patient and disease characteristics. Identifying genetic variants as a single cause of cardiovascular disease would be an oversimplification. Instead, the contribution of these polymorphisms should also be considered in the context of nongenetic risk factors (eg, smoking, diabetes, hyperlipidemia) and in selected patient populations. For example, although the contribution of the inherited platelet polymorphism, PlA2, to cardiovascular events in the general population remains controversial, its association with thrombotic events in younger individuals and after catheter-based interventions is more consistent.1-4 Until larger databases involving thousands of patients are established to reconcile divergent findings, publication of well-designed studies with negative results will remain important to our understanding of the role of genetics in this disease process. Omission of studies showing no significance or a negative effect because of editorial policy or investigator indifference is a major driver of publication bias that leads to an overemphasis of the information in positive studies. Furthermore, because most contemporary studies of genetics in cardiovascular disease involve a limited number of subjects, publication bias toward positive studies undermines any effort to combine the results of individual analyses. Moreover, methods to estimate the impact of publication bias in metaanalyses (eg, funnel
American Heart Journal Volume 143, Number 6
plots) are subjective and also dependent on study size. For these reasons, metaanalyses of genetics studies in cardiovascular disease should be avoided at this point. However, encouraging publication of negative trials should not lower the threshold for acceptance of genetic studies but instead promote even greater scrutiny. Investigators performing such studies must ask themselves: were the statistical implications of multiple testing in association studies considered? Was the study end point prespecified or rather dictated by the data?5 Do the conclusions characterize a population or just the sample studied? Do allele and genotype frequencies deviate from Hardy-Weinberg equilibrium? Considering the complexities of inheritance patterns, the accuracy of genotyping, and the impact of environmental factors, the required sample size for adequate statistical power has been estimated to include several thousands of patients.6 Trials lacking proper statistical methodology may erroneously dismiss the contribution of important genetic variants. Alternatively, even with larger sample sizes, a random approach to investigating the impact of polymorphisms would lead to a significant number of false-positive associations by chance alone, further obscuring the ability to discriminate true-positive from false-positive results. Because the annual publication rate of gene polymorphism studies has nearly tripled in the past decade, these misperceptions are likely to become even more common. Until a more coordinated effort to confirm the associated risks of specific gene polymorphisms is established, contemporary trials should still be considered exploratory and hypothesis-generating. The promise of genetic association studies in cardiovascular disease is met with an unprecedented opportunity for discovery balanced by unprecedented challenges of trial design and analysis. Although no clinical advancements on the basis of genetic studies have been made to date, there are several examples where genetic variability might relate to a disease process or treatment response. At present, however, the technol-
Kandzari and Goldschmidt-Clermont 951
ogy to identify gene polymorphisms is progressing ahead of clinical research. Yet, rather than simply probe individual databases for newly identified polymorphisms, more effort must also be concentrated on defining the “cellular phenotype” through mechanistic studies that may support or refute the hypothesized genetic role in a disease process. Because little coordinated effort has been made to establish larger databases coupling genomic and clinical information, identifying the risk associated with genetic discoveries and translating this knowledge into clinical practice will likely occur gradually rather than by one revolutionary step. Only through a larger collective effort will the role of susceptibility genes for cardiovascular disease be defined. But even then, our understanding of genetic polymorphisms will rely on the presentation of data from well-designed studies—positive or not.
References 1. Weiss EJ, Bray PF, Tayback M, et al. A polymorphism of a platelet glycoprotein receptor as an inherited risk factor for coronary thrombosis. N Engl J Med 1996;334:1090-4. 2. Mikkelsson J, Perola M, Laippala P, et al. Glycoprotein IIIa/PlA polymorphism associates with progression of coronary artery disease and with myocardial infarction in an autopsy series of middleaged men who died suddenly. Arteroscler Thromb Vasc Biol 1999; 19:2573-8. 3. Mikkelsson J, Perola M, Laippala P, et al. Glycoprotein IIIa/PlA polymorphism associates with progression of coronary artery disease and with myocardial infarction in an autopsy series of middleaged men who died suddenly. Arteroscler Thromb Vasc Biol 1999; 19:2573-8. 4. Laule M, Cascorbi I, Stangl V, et al. A1/A2 polymorphism of glycoprotein IIIa and association with excess procedural risk for coronary catheter interventions: a case-controlled study. Lancet 1999;353: 708-12. 5. Moye LA. Random research. Circulation 2001;103:3150-3. 6. Winkelmann BR, Hager J, Kraus WE, et al. Genetics of coronary heart disease: current knowledge and research principles. Am Heart J 2000;140(Suppl):S11-26.
Making positive out of negative trials David E. Kandzari, MD, and Pascal J. Goldschmidt-Clermont, MD Durham, NC
See related article on page 971.
In this issue of the Journal, Koch and colleagues describe the lack of association of a genetic polymorphism in monocyte differentiation antigen CD14 with the presence of coronary artery disease and myocardial infarction. Although the study is possibly considered by some to be a “negative trial,” these findings instead offer much insight to our understanding of inherited alterations in genes and their contribution to heart disease. First, such results further clarify the role of inherited polymorphisms of the CD14 gene in atherosclerosis and adverse cardiac events. Yet, in a much broader perspective, the publication of negative studies about genetic associations is an integral part of refining a relatively new and emerging field of cardiovascular science. Publication of negative results from well-designed, adequately powered clinical studies is positively essential to our interpretation of the role of genetics in cardiovascular disease. In this study, the results by Koch et al add to a growing body of conflicting evidence regarding the contribution of CD14 polymorphisms to inflammation, atherosclerosis, and cardiovascular events. With a population of nearly 1800 individuals with coronary artery disease (approximately 800 of whom had prior myocardial infarctions), the presence of a single nucleotide polymorphism characterized by a substitution of cytosine (C) for thymine (T) at position –159 of the CD14 gene promoter was not associated with an increased likelihood of coronary artery disease or myocardial infarction relative to a group of individuals without established cardiovascular disease. The lack of an association was maintained even after adjustment for standard cardiovascular risk factors. Moreover, the distribution of genotypes among the groups did not significantly differ, a finding that seems to further support the lack of substantial involvement of this particular polymorphism in the development of coronary artery disease. Should the study by Koch and colleagues be considered the final word on the role of the –159C/T poly-
From the Division of Cardiology and Duke Clinical Research Institute, Duke University Medical Center, Durham, NC. Submitted November 1, 2001; accepted December 5, 2001. Reprint requests: David E. Kandzari, MD, PO Box 31203, Duke University Medical Center, Durham, NC 27710. E-mail: [email protected] Am Heart J 2002;143:950-1. © 2002, Mosby, Inc. All rights reserved. 0002-8703/2002/$35.00 ⫹ 0 4/4/122513 doi:10.1067/mhj.2002.122513
morphism in cardiovascular disease? If not, how should clinicians interpret these findings in the context of conflicting results from other epidemiologic studies? As the investigators note, previous studies involving smaller numbers of subjects have reported higher levels of circulating inflammatory markers or higher frequencies of coronary events associated with the TT genotype. The mechanistic studies that show increased levels of CD14 among TT homozygotes also seem to intuitively support the concept that a proinflammatory state would contribute to atherogenesis and subsequent cardiovascular events. Perhaps the most appropriate way to interpret results from this and other contemporary genetic association studies is to accept them as part of a growing body of evidence to define the role of inherited polymorphisms in the development of coronary artery disease, in this case regarding the contribution of a specific inflammatory marker to atherosclerosis and thrombotic events. Although it is appealing to attribute divergent results from similar studies to experimental design or patient populations, it is also important to recognize that the clinical diagnosis of acute coronary syndromes includes a broad range of patient and disease characteristics. Identifying genetic variants as a single cause of cardiovascular disease would be an oversimplification. Instead, the contribution of these polymorphisms should also be considered in the context of nongenetic risk factors (eg, smoking, diabetes, hyperlipidemia) and in selected patient populations. For example, although the contribution of the inherited platelet polymorphism, PlA2, to cardiovascular events in the general population remains controversial, its association with thrombotic events in younger individuals and after catheter-based interventions is more consistent.1-4 Until larger databases involving thousands of patients are established to reconcile divergent findings, publication of well-designed studies with negative results will remain important to our understanding of the role of genetics in this disease process. Omission of studies showing no significance or a negative effect because of editorial policy or investigator indifference is a major driver of publication bias that leads to an overemphasis of the information in positive studies. Furthermore, because most contemporary studies of genetics in cardiovascular disease involve a limited number of subjects, publication bias toward positive studies undermines any effort to combine the results of individual analyses. Moreover, methods to estimate the impact of publication bias in metaanalyses (eg, funnel
American Heart Journal Volume 143, Number 6
plots) are subjective and also dependent on study size. For these reasons, metaanalyses of genetics studies in cardiovascular disease should be avoided at this point. However, encouraging publication of negative trials should not lower the threshold for acceptance of genetic studies but instead promote even greater scrutiny. Investigators performing such studies must ask themselves: were the statistical implications of multiple testing in association studies considered? Was the study end point prespecified or rather dictated by the data?5 Do the conclusions characterize a population or just the sample studied? Do allele and genotype frequencies deviate from Hardy-Weinberg equilibrium? Considering the complexities of inheritance patterns, the accuracy of genotyping, and the impact of environmental factors, the required sample size for adequate statistical power has been estimated to include several thousands of patients.6 Trials lacking proper statistical methodology may erroneously dismiss the contribution of important genetic variants. Alternatively, even with larger sample sizes, a random approach to investigating the impact of polymorphisms would lead to a significant number of false-positive associations by chance alone, further obscuring the ability to discriminate true-positive from false-positive results. Because the annual publication rate of gene polymorphism studies has nearly tripled in the past decade, these misperceptions are likely to become even more common. Until a more coordinated effort to confirm the associated risks of specific gene polymorphisms is established, contemporary trials should still be considered exploratory and hypothesis-generating. The promise of genetic association studies in cardiovascular disease is met with an unprecedented opportunity for discovery balanced by unprecedented challenges of trial design and analysis. Although no clinical advancements on the basis of genetic studies have been made to date, there are several examples where genetic variability might relate to a disease process or treatment response. At present, however, the technol-
Kandzari and Goldschmidt-Clermont 951
ogy to identify gene polymorphisms is progressing ahead of clinical research. Yet, rather than simply probe individual databases for newly identified polymorphisms, more effort must also be concentrated on defining the “cellular phenotype” through mechanistic studies that may support or refute the hypothesized genetic role in a disease process. Because little coordinated effort has been made to establish larger databases coupling genomic and clinical information, identifying the risk associated with genetic discoveries and translating this knowledge into clinical practice will likely occur gradually rather than by one revolutionary step. Only through a larger collective effort will the role of susceptibility genes for cardiovascular disease be defined. But even then, our understanding of genetic polymorphisms will rely on the presentation of data from well-designed studies—positive or not.
References 1. Weiss EJ, Bray PF, Tayback M, et al. A polymorphism of a platelet glycoprotein receptor as an inherited risk factor for coronary thrombosis. N Engl J Med 1996;334:1090-4. 2. Mikkelsson J, Perola M, Laippala P, et al. Glycoprotein IIIa/PlA polymorphism associates with progression of coronary artery disease and with myocardial infarction in an autopsy series of middleaged men who died suddenly. Arteroscler Thromb Vasc Biol 1999; 19:2573-8. 3. Mikkelsson J, Perola M, Laippala P, et al. Glycoprotein IIIa/PlA polymorphism associates with progression of coronary artery disease and with myocardial infarction in an autopsy series of middleaged men who died suddenly. Arteroscler Thromb Vasc Biol 1999; 19:2573-8. 4. Laule M, Cascorbi I, Stangl V, et al. A1/A2 polymorphism of glycoprotein IIIa and association with excess procedural risk for coronary catheter interventions: a case-controlled study. Lancet 1999;353: 708-12. 5. Moye LA. Random research. Circulation 2001;103:3150-3. 6. Winkelmann BR, Hager J, Kraus WE, et al. Genetics of coronary heart disease: current knowledge and research principles. Am Heart J 2000;140(Suppl):S11-26.