- Email: [email protected]
A letter to the editor
A LETTER T O THE EDITOR Dear Sirs: A recent letter to the editor of "Neurotoxicology" (Benignus and Muller, Neurotox. 3: 153-54, 1082) points out that many toxicology studies use the t-test repetitively, leading to errors in estimates of levels of overall significance. The authors recommended that scientists need to do the obvious: Use simpler designs and/or use ANOVA or multivariate techniques. This is logical advice for those toxicological studies that test a small n u m b e r of specific hypotheses, but much of applied toxicology is a search for possible toxic effects among a large n u m b e r of parameters. Many toxicology studies utilize an exploratory strategy which involves the measurements of as many as 30 different parameters in a control group and in subiects treated at several different dose levels of a chemical. Although some toxicologists may use the t-test, ANOVA with multiple comparisons are typically used for each parameter. The data for each parameter are, however, from the same subjects; thus, the measurements are correlated to an u n k n o w n degree and the ability to define the true level of type I or type II error is hopelessly confounded. Therefore, as with repetitive use of the t-test, the repetitive use of ANOVA is also incorrect. As an alternative approach, one may consider using multivariate methods, but this also is usually unsatisfactory. Multivariate statistics can be very powerful and at the same time coot rol the overall risk of type I error, but the variates must be carefully chosen. In the usual exploratory strategy one does not know which parameters may be affected by the chemical and, therefore, the variates cannot be sensibly chosen. If one blindly performs multivariate analyses, the cumulative result of combining unaffected variates with those that have treatment effect can be to statistically mask even dramatic toxico]ogic events. Toxicologists have long used probability statements in exploratory studies, even though they are known to have undefined levels of type I or type II error. A common sentence in toxicology reports is: "although statisti-
16/a
cally significant, these findings a~. probably not treatment related . , .'. In other words, the toxicologist routinely exercises judgment when reporting the effects of chemicals. The problem is thai ah'aough no one knows the true level of statistical signific;;:'ce, some people will assume that probability statemeM~ mean exactly what they say. At worst, incorrect probability statements can lead to invalid conclusions and, ~: best, they lead to unnecess,~ ry confusion and con t ro~. ~rsy. In an effort to minimize this stat,:;tical confusion, the editors of "'Neuroloxicology" now ,~,quire that articles published in that journal will have ay0ropriate statistical methods or will have qualifying stater~ ents when reporting results. They have also added a st aislical editor, and their manuscript review form now requires comments on the adequacy of the statistical methods. In the absence of a 'Tox Community" solution to the appropriate use of statistics in exploratory studies, our policy is to use statistical methods such as ANOVA and multiple comparisons as purely descriptive statistics, and probability values are not reported in r,he manuscript. The methods section identifies the alpha level at which the analyses were performed, along with a statement about the confounding of overall levels of risk. Because exploratory studies generate masses of data, the intent is to use statistical methods to facilitate identification of those numbers that deserve particular scrutiny. Eventually, the toxicologist must examine all of the data, use all available information, and then make a judgmental decision about the toxicologic significance of the data, It would seem that the continued use of ingot,rate probability statements in the exploratory toxicology literature can only be counterproductive, and it is hoped that this let ter will stimulate the editors of 'Fundamental and Applied Toxicology', and the editors of the sister journal 'Toxicology and Applied Phrmacology', to seek an editorial policy that will encourage use of more appropriate data analysis and discourage the overinterpretation of statistical results. Sincerely, Joel L. Maltsson, D.V.M., Ph.D. Toxicology Research Laboratory Health and Environmental Sciences Dow Chemical U.S.A. Colin N. Park, Ph.D. Biostatistician Itealth and Environmental Sciences Dow Chemical U.S.A.
Fundam. AppL Toxicol. f2)
November/December, 1982
16/a
cally significant, these findings a~. probably not treatment related . , .'. In other words, the toxicologist routinely exercises judgment when reporting the effects of chemicals. The problem is thai ah'aough no one knows the true level of statistical signific;;:'ce, some people will assume that probability statemeM~ mean exactly what they say. At worst, incorrect probability statements can lead to invalid conclusions and, ~: best, they lead to unnecess,~ ry confusion and con t ro~. ~rsy. In an effort to minimize this stat,:;tical confusion, the editors of "'Neuroloxicology" now ,~,quire that articles published in that journal will have ay0ropriate statistical methods or will have qualifying stater~ ents when reporting results. They have also added a st aislical editor, and their manuscript review form now requires comments on the adequacy of the statistical methods. In the absence of a 'Tox Community" solution to the appropriate use of statistics in exploratory studies, our policy is to use statistical methods such as ANOVA and multiple comparisons as purely descriptive statistics, and probability values are not reported in r,he manuscript. The methods section identifies the alpha level at which the analyses were performed, along with a statement about the confounding of overall levels of risk. Because exploratory studies generate masses of data, the intent is to use statistical methods to facilitate identification of those numbers that deserve particular scrutiny. Eventually, the toxicologist must examine all of the data, use all available information, and then make a judgmental decision about the toxicologic significance of the data, It would seem that the continued use of ingot,rate probability statements in the exploratory toxicology literature can only be counterproductive, and it is hoped that this let ter will stimulate the editors of 'Fundamental and Applied Toxicology', and the editors of the sister journal 'Toxicology and Applied Phrmacology', to seek an editorial policy that will encourage use of more appropriate data analysis and discourage the overinterpretation of statistical results. Sincerely, Joel L. Maltsson, D.V.M., Ph.D. Toxicology Research Laboratory Health and Environmental Sciences Dow Chemical U.S.A. Colin N. Park, Ph.D. Biostatistician Itealth and Environmental Sciences Dow Chemical U.S.A.
Fundam. AppL Toxicol. f2)
November/December, 1982