- Email: [email protected]
Comparability versus statistical correctness
European Journal of Radiology 82 (2013) e908
Contents lists available at ScienceDirect
European Journal of Radiology journal homepage: www.elsevier.com/locate/ejrad
Response to Letter to the Editor Comparability versus statistical correctness Dear Sirs, Your response to our paper [1] regarding tumor size estimation in high risk women with MRI is much appreciated. We can only agree that Bland–Altman plots are statistically a better way to deal with agreement between two measurements than calculation of the Pearson correlation coefficient (PCC).
The reason not to opt for the use of Bland–Altman plots, but rather to choose the less correct PCC is the simple need for comparability. As is correctly stated in the response to our paper, most of the studies evaluating tumor size estimations with MRI and pathology use the PCC. Because our intention was to compare our results to those published in literature, we were forced to use the methods that were employed in those studies. Because the PCC implies correlation but not identical values we evaluated the percentage of tumors that were measured within 1 cm from pathologic size, as we do feel that this may have impact on the clinical practice. Our results clearly show that size estimations with MRI, when using pathology as gold standard, are not accurate enough for a direct replacement of pathological tumor size, because for TTA underestimation of more than 1 cm occurred in 16% of cases and overestimation occurred in 10% of cases. Similarly for LF underestimation of more than 1 cm occurred in 9% and overestimation occurred in 11%. These numbers are all far from the desired 2.5% that would get 95% of MRI measurements within 1 cm of pathology. The 95% limits of agreement are much wider and therefore unfortunately also clinically meaningless. For TTA they are −29.4
DOIs of original articles: http://dx.doi.org/10.1016/j.ejrad.2013.03.003, http://dx.doi.org/10.1016/j.ejrad.2013.08.025 0720-048X/$ – see front matter © 2013 Elsevier Ireland Ltd. All rights reserved. http://dx.doi.org/10.1016/j.ejrad.2013.08.012
to 29.4 mm, and for LF −20.4 to 20.4 mm. Respective Bland Altman plots are depicted below. Comparable to the PCC analysis, the plots also show a clear increase in the measurement errors with increasing tumor size. In contrast to the PCC analysis, this evaluation shows that the LF is in general more accurately measured than the TTA, despite the somewhat lower PCC.
Nevertheless, introducing a new analysis method in an already existing large body of evidence may render the results of a study worthless when the aim is to compare the data to the literature. Moreover, a new analysis method may mask results that are not in line with earlier results. For example in the official paper on the COMICE trial, reported in the Lancet, no correlation coefficients are reported, but rather a linear weighted kappa is introduced. Although this is probably also a statistically more sound approach than calculating the PCC, it makes comparison of the results to earlier studies impossible. Only by reading the full HTA report (over 100 pages), the poor agreement of radiological measurements with pathology in this study is understood, because only there PCC’s of 0.4 for ILC and 0.56 for other tumors are mentioned, whereas other studies consistently report PCC’s of between 0.75 and 0.98. Consequently, we are of the opinion that comparability is sometimes more important than choosing the optimal statistical method. Of course this should not have impact on the soundness of the conclusions drawn from the results. Reference [1] Mann RM, Bult P, van Laarhoven HWM, Span PN, Schlooz M, Veltman J, et al. Breast cancer size estimation with MRI in BRCA mutation carriers and other high risk patients. Eur J Radiol 2013;82:1416–22.
R.M. Mann ∗ P.N. Span N. Hoogerbrugge ∗ Corresponding
author.
Contents lists available at ScienceDirect
European Journal of Radiology journal homepage: www.elsevier.com/locate/ejrad
Response to Letter to the Editor Comparability versus statistical correctness Dear Sirs, Your response to our paper [1] regarding tumor size estimation in high risk women with MRI is much appreciated. We can only agree that Bland–Altman plots are statistically a better way to deal with agreement between two measurements than calculation of the Pearson correlation coefficient (PCC).
The reason not to opt for the use of Bland–Altman plots, but rather to choose the less correct PCC is the simple need for comparability. As is correctly stated in the response to our paper, most of the studies evaluating tumor size estimations with MRI and pathology use the PCC. Because our intention was to compare our results to those published in literature, we were forced to use the methods that were employed in those studies. Because the PCC implies correlation but not identical values we evaluated the percentage of tumors that were measured within 1 cm from pathologic size, as we do feel that this may have impact on the clinical practice. Our results clearly show that size estimations with MRI, when using pathology as gold standard, are not accurate enough for a direct replacement of pathological tumor size, because for TTA underestimation of more than 1 cm occurred in 16% of cases and overestimation occurred in 10% of cases. Similarly for LF underestimation of more than 1 cm occurred in 9% and overestimation occurred in 11%. These numbers are all far from the desired 2.5% that would get 95% of MRI measurements within 1 cm of pathology. The 95% limits of agreement are much wider and therefore unfortunately also clinically meaningless. For TTA they are −29.4
DOIs of original articles: http://dx.doi.org/10.1016/j.ejrad.2013.03.003, http://dx.doi.org/10.1016/j.ejrad.2013.08.025 0720-048X/$ – see front matter © 2013 Elsevier Ireland Ltd. All rights reserved. http://dx.doi.org/10.1016/j.ejrad.2013.08.012
to 29.4 mm, and for LF −20.4 to 20.4 mm. Respective Bland Altman plots are depicted below. Comparable to the PCC analysis, the plots also show a clear increase in the measurement errors with increasing tumor size. In contrast to the PCC analysis, this evaluation shows that the LF is in general more accurately measured than the TTA, despite the somewhat lower PCC.
Nevertheless, introducing a new analysis method in an already existing large body of evidence may render the results of a study worthless when the aim is to compare the data to the literature. Moreover, a new analysis method may mask results that are not in line with earlier results. For example in the official paper on the COMICE trial, reported in the Lancet, no correlation coefficients are reported, but rather a linear weighted kappa is introduced. Although this is probably also a statistically more sound approach than calculating the PCC, it makes comparison of the results to earlier studies impossible. Only by reading the full HTA report (over 100 pages), the poor agreement of radiological measurements with pathology in this study is understood, because only there PCC’s of 0.4 for ILC and 0.56 for other tumors are mentioned, whereas other studies consistently report PCC’s of between 0.75 and 0.98. Consequently, we are of the opinion that comparability is sometimes more important than choosing the optimal statistical method. Of course this should not have impact on the soundness of the conclusions drawn from the results. Reference [1] Mann RM, Bult P, van Laarhoven HWM, Span PN, Schlooz M, Veltman J, et al. Breast cancer size estimation with MRI in BRCA mutation carriers and other high risk patients. Eur J Radiol 2013;82:1416–22.
R.M. Mann ∗ P.N. Span N. Hoogerbrugge ∗ Corresponding
author.