- Email: [email protected]
P >0.05 ≠ Equivalent or Noninferior
Readers’ Comments
and soft tissues.2 The !Kung Bushmen of Botswana are almost entirely dependent on hunting and gathering. However, in addition to prey, what other sources of Mg exist for these extraordinary populations? Mongongo nuts, because of their reliability and abundance, provide about 1/2 of their vegetable diet by weight. The average daily consumption of about 300 of these nuts provides ⬎1,200 calories and ⬎50 g of protein (the caloric equivalent of ⬎2 lb of cooked rice and protein equivalent of about 14 oz of lean beef). Finally, these nuts are drought resistant and can be harvested for up to 1 year after falling to the ground.5 William J. Rowe, MD Keswick, Virginia 18 May 2005
1. Nash S, Westpfal M. Cardiovascular benefits of nuts. Am J Cardiol 2005;95:963–965. 2. Seelig M. Cardiovascular consequences of magnesium deficiency and loss: pathogenesis, prevalence and manifestations—magnesium and chloride loss in refractory potassium repletion. Am J Cardiol 1989;63(suppl):4G– 21G. 3. Maier J, Malpuech-Brugere C, Zimowska W, Rayssiguier Y , Mazur A. Low magnesium promotes endothelial cell dysfunction :implications for atherosclerosis, inflammation and thrombosis. Biochim Biophys Acta 2004;1689: 13–21. 4. Rowe W. Endurance exercise and injury to the heart. Guest Editorial. Sports Med 1993;16: 73–79. 5. Lee RB. What hunters do for a living, or how to make out on scarce resources. In: Lee RB, De Vore I, eds. Man the Hunter. New York: Aldine De Gruyter, 1968:30 – 48. doi:10.1016/j.amjcard.2005.05.006
P >0.05 Equivalent or Noninferior Lee and colleagues’1 assertion that their trial results indicate that cilostazol is as safe and effective as clopidogrel in preventing thrombotic complications after stenting is certainly not supported by the results they reported. A number of important methodologic and interpretive issues require clarification. First, the authors state that a computer-generated randomization list was used, but it is not clear what type of randomization was performed. For ex-
ample, in a trial of 689 patients, the probability of simple randomization yielding 344 and 345 in the 2 groups is about 3%. As such, it suggests that some form of restricted randomization was used. In the case of an open-label trial, as this was, this leads to the possibility of selection bias, which is a major threat to trial validity. Second, given their description of the sample size estimation, it is clear that this was designed as a superiority trial. It would appear that the expected rate in the clopidogrel group was 12% to 13% and that the authors postulated a 6% to 6.5% absolute reduction in major adverse cardiac events (MACEs) with cilostazol. This seems overly optimistic given the contemporary knowledge of the benefit of other antiplatelet agents. Moreover, the authors could easily have determined that the expected confidence interval (CI) width of their trial was around ⫾ 4.5%, so that if no difference was found between groups (i.e., absolute reduction equal to 0), it would be compatible with an absolute increase in MACE of 4.5%. This limit is clearly too wide to be considered clinically equivalent. Third, the authors report the rate of MACEs in the clopidogrel and cilostazol groups and a p value of 0.61, which is clearly conventionally nonsignificant. The authors, reviewers, and editors have, however, made the common error of assuming that a nonsignificant p value somehow proves the null hypothesis. Although the authors excuse their study for being underpowered to detect the small difference in the risk of subacute stent thrombosis and MACEs, in this regard they are rather lucky, because the point estimates of both event rates were higher (i.e., consistent with harm) with cilostazol. It is possible to show that the probability that cilostazol is worse than clopidogrel is 68.6%; a larger trial just may have proved that. Finally, the question really becomes whether the difference between groups is small enough to be safely ignored. A conventional p value cannot tell you this. So-called negative trials or trials showing no difference are easier to interpret if data are presented as point
1035
estimates and a measure of precision, such as a CI. It is notable that the report is devoid of the latter. The results take on a much different flavor when expressed as relative and absolute differences, along with CIs. The risk of MACEs with cilostazol is higher than with clopidogrel: (1) relative risk 1.3 times worse (95% CI 0.5 to 3.4); (2) relative risk increase 29% worse (95% CI 51% better to 242% worse), and (3) absolute risk increase 0.6% worse (95% CI 1.8% better to 3.1% worse). This trial is a failed superiority trial, plain and simple. In no way has it shown equivalence or noninferiority. On the basis of this trial, on the balance of probabilities, cilostazol is likely less effective than clopidogrel and a larger study may well confirm this. David Massel, MD London Health Sciences Centre London, Ontario, Canada 18 May 2005
1. Lee SW, Park SW, Hong MK, et al. Comparison of cilostazol and clopidogrel after successful coronary stenting. Am J Cardiol 2005; 95:859 – 862. doi:10.1016/j.amjcard.2005.05.005
Bleeding Complications Related to Aspirin Dose In the current era of evidence-based medicine, focused clinical questions may best be assessed by systematic reviews of studies of high methodologic quality, presented in transparent form to allow the reader to evaluate and replicate the authors’ work. Thus, we were surprised by publication of the analysis by Serebruany et al1 of bleeding complications related to aspirin dose. The question of a dose response for bleeding due to lowdose aspirin (75 to 325 mg daily) is an important one. However, this analysis suffers from a number of methodologic shortcomings. For example, the authors provided almost no information pertaining to their search strategy, inclusion or exclusion criteria, or method of data extraction. Also, although the title of the report states that it is an analysis of randomized controlled trials, the largest group of patients included (United States Nurses’ Study) was a nonrandomized cohort study.
and soft tissues.2 The !Kung Bushmen of Botswana are almost entirely dependent on hunting and gathering. However, in addition to prey, what other sources of Mg exist for these extraordinary populations? Mongongo nuts, because of their reliability and abundance, provide about 1/2 of their vegetable diet by weight. The average daily consumption of about 300 of these nuts provides ⬎1,200 calories and ⬎50 g of protein (the caloric equivalent of ⬎2 lb of cooked rice and protein equivalent of about 14 oz of lean beef). Finally, these nuts are drought resistant and can be harvested for up to 1 year after falling to the ground.5 William J. Rowe, MD Keswick, Virginia 18 May 2005
1. Nash S, Westpfal M. Cardiovascular benefits of nuts. Am J Cardiol 2005;95:963–965. 2. Seelig M. Cardiovascular consequences of magnesium deficiency and loss: pathogenesis, prevalence and manifestations—magnesium and chloride loss in refractory potassium repletion. Am J Cardiol 1989;63(suppl):4G– 21G. 3. Maier J, Malpuech-Brugere C, Zimowska W, Rayssiguier Y , Mazur A. Low magnesium promotes endothelial cell dysfunction :implications for atherosclerosis, inflammation and thrombosis. Biochim Biophys Acta 2004;1689: 13–21. 4. Rowe W. Endurance exercise and injury to the heart. Guest Editorial. Sports Med 1993;16: 73–79. 5. Lee RB. What hunters do for a living, or how to make out on scarce resources. In: Lee RB, De Vore I, eds. Man the Hunter. New York: Aldine De Gruyter, 1968:30 – 48. doi:10.1016/j.amjcard.2005.05.006
P >0.05 Equivalent or Noninferior Lee and colleagues’1 assertion that their trial results indicate that cilostazol is as safe and effective as clopidogrel in preventing thrombotic complications after stenting is certainly not supported by the results they reported. A number of important methodologic and interpretive issues require clarification. First, the authors state that a computer-generated randomization list was used, but it is not clear what type of randomization was performed. For ex-
ample, in a trial of 689 patients, the probability of simple randomization yielding 344 and 345 in the 2 groups is about 3%. As such, it suggests that some form of restricted randomization was used. In the case of an open-label trial, as this was, this leads to the possibility of selection bias, which is a major threat to trial validity. Second, given their description of the sample size estimation, it is clear that this was designed as a superiority trial. It would appear that the expected rate in the clopidogrel group was 12% to 13% and that the authors postulated a 6% to 6.5% absolute reduction in major adverse cardiac events (MACEs) with cilostazol. This seems overly optimistic given the contemporary knowledge of the benefit of other antiplatelet agents. Moreover, the authors could easily have determined that the expected confidence interval (CI) width of their trial was around ⫾ 4.5%, so that if no difference was found between groups (i.e., absolute reduction equal to 0), it would be compatible with an absolute increase in MACE of 4.5%. This limit is clearly too wide to be considered clinically equivalent. Third, the authors report the rate of MACEs in the clopidogrel and cilostazol groups and a p value of 0.61, which is clearly conventionally nonsignificant. The authors, reviewers, and editors have, however, made the common error of assuming that a nonsignificant p value somehow proves the null hypothesis. Although the authors excuse their study for being underpowered to detect the small difference in the risk of subacute stent thrombosis and MACEs, in this regard they are rather lucky, because the point estimates of both event rates were higher (i.e., consistent with harm) with cilostazol. It is possible to show that the probability that cilostazol is worse than clopidogrel is 68.6%; a larger trial just may have proved that. Finally, the question really becomes whether the difference between groups is small enough to be safely ignored. A conventional p value cannot tell you this. So-called negative trials or trials showing no difference are easier to interpret if data are presented as point
1035
estimates and a measure of precision, such as a CI. It is notable that the report is devoid of the latter. The results take on a much different flavor when expressed as relative and absolute differences, along with CIs. The risk of MACEs with cilostazol is higher than with clopidogrel: (1) relative risk 1.3 times worse (95% CI 0.5 to 3.4); (2) relative risk increase 29% worse (95% CI 51% better to 242% worse), and (3) absolute risk increase 0.6% worse (95% CI 1.8% better to 3.1% worse). This trial is a failed superiority trial, plain and simple. In no way has it shown equivalence or noninferiority. On the basis of this trial, on the balance of probabilities, cilostazol is likely less effective than clopidogrel and a larger study may well confirm this. David Massel, MD London Health Sciences Centre London, Ontario, Canada 18 May 2005
1. Lee SW, Park SW, Hong MK, et al. Comparison of cilostazol and clopidogrel after successful coronary stenting. Am J Cardiol 2005; 95:859 – 862. doi:10.1016/j.amjcard.2005.05.005
Bleeding Complications Related to Aspirin Dose In the current era of evidence-based medicine, focused clinical questions may best be assessed by systematic reviews of studies of high methodologic quality, presented in transparent form to allow the reader to evaluate and replicate the authors’ work. Thus, we were surprised by publication of the analysis by Serebruany et al1 of bleeding complications related to aspirin dose. The question of a dose response for bleeding due to lowdose aspirin (75 to 325 mg daily) is an important one. However, this analysis suffers from a number of methodologic shortcomings. For example, the authors provided almost no information pertaining to their search strategy, inclusion or exclusion criteria, or method of data extraction. Also, although the title of the report states that it is an analysis of randomized controlled trials, the largest group of patients included (United States Nurses’ Study) was a nonrandomized cohort study.