- Email: [email protected]
‘Trend’ and ‘almost significant’ in anaesthesia research
892
| Correspondence
‘Trend’ and ‘almost significant’ in anaesthesia research I. Smith* Stoke-on-Trent, North Midlands, UK *E-mail: [email protected]
authors want to highlight these results, even if ‘trend’ is not the correct word to use. Statistical tests provide a precise estimate of the likelihood that the results are attributable to random chance, or to a genuine effect, along a continuum extending from near certainty to almost infinite improbability. Yet we choose to discard almost all of this information and distil it into an arbitrary, yes/no cut-off between significant or non-significant. Perhaps the time has come to drop the concept of significance and simply let the P-values talk for themselves.
Declaration of interest None declared.
Reference 1. Gibbs NM, Gibbs SV. Misuse of ‘trend’ to describe ‘almost significant’ differences in anaesthesia research. Br J Anaesth 2015; 115: 337–9 doi:10.1093/bja/aew144
Reply N. M. Gibbs* and S. V. Gibbs Nedlands, WA, Australia *E-mail: [email protected]
Editor—We would like to thank Dr Smith for his interest and comments on our editorial.1 We agree with his remarks about medical conservatism and appreciate the desire of many authors to highlight results that are almost, but not quite, statistically significant. However, this can be achieved without using the potentially misleading term ‘trend’. Authors can legitimately describe such findings as ‘encouraging’, ‘of interest’, or ‘worthy of further investigation’, so long as the lack of statistical significance is clear and the same interpretation is given to all P-values in their study that fall within this ‘almost statistically significant’ range, and not only those that support their test hypothesis. Alternatively, authors are at liberty to accept a larger α error from the outset (e.g. using P<0.1 as statistically significant rather than P<0.05). We agree that P-values present a continuum and that actual P-values contain valuable information, whether or not they are ‘significant’. For this reason, we would encourage reporting of actual P-values in all circumstances (i.e. not only as significant vs non-significant). We consider that when interpreted correctly,
P-values already ‘speak for themselves’, although the effect size, prespecified minimal clinically important effect size, and β error must also be considered.2 We feel that it is not necessary ‘to drop the concept of significance’. It is necessary only to use it correctly and to know its limitations.
Declaration of interest None declared.
References 1. Gibbs NM, Gibbs SV. Misuse of ‘trend’ to describe ‘almost significant’ differences in anaesthesia research. Br J Anaesth 2015; 115: 337–9 2. Gibbs NM, Weightman WM. Beyond effect size: consideration of the minimum effect size of interest in anesthesia trials. Anesth Analg 2012; 114: 471–5 doi:10.1093/bja/aew145
Downloaded from http://bja.oxfordjournals.org/ at Orta Dogu Teknik University Library (ODTU) on May 22, 2016
Editor—I thank Drs Gibbs and Gibbs1 for highlighting the linguistic and statistical inaccuracies in the use of ‘trends’ in anaesthesia research literature. As a teacher and reviewer, I have argued against this practice, whereas as an author I have almost certainly fallen into the same trap myself! While I cannot disagree with a word of the editorial, I can perhaps offer a little more insight into why this trend might be developing. The authors correctly describe how a predefined P-value is used to determine whether the null hypothesis is rejected or accepted, hence whether results are statistically significant or not. As a result of medical conservatism, we are reluctant to replace a tried and tested therapy with something relatively new, and so the threshold for rejecting the null hypothesis is set high deliberately, typically at a level where the observed result would have occurred by chance only less than five times in 100. Conversely, this means that, in the area in which ‘trends’ tend to be mentioned, the observed result is likely to represent a genuine difference between treatments ∼90 to just <95% of the time. In other circumstances, these would still be very good odds indeed! It is not surprising that
| Correspondence
‘Trend’ and ‘almost significant’ in anaesthesia research I. Smith* Stoke-on-Trent, North Midlands, UK *E-mail: [email protected]
authors want to highlight these results, even if ‘trend’ is not the correct word to use. Statistical tests provide a precise estimate of the likelihood that the results are attributable to random chance, or to a genuine effect, along a continuum extending from near certainty to almost infinite improbability. Yet we choose to discard almost all of this information and distil it into an arbitrary, yes/no cut-off between significant or non-significant. Perhaps the time has come to drop the concept of significance and simply let the P-values talk for themselves.
Declaration of interest None declared.
Reference 1. Gibbs NM, Gibbs SV. Misuse of ‘trend’ to describe ‘almost significant’ differences in anaesthesia research. Br J Anaesth 2015; 115: 337–9 doi:10.1093/bja/aew144
Reply N. M. Gibbs* and S. V. Gibbs Nedlands, WA, Australia *E-mail: [email protected]
Editor—We would like to thank Dr Smith for his interest and comments on our editorial.1 We agree with his remarks about medical conservatism and appreciate the desire of many authors to highlight results that are almost, but not quite, statistically significant. However, this can be achieved without using the potentially misleading term ‘trend’. Authors can legitimately describe such findings as ‘encouraging’, ‘of interest’, or ‘worthy of further investigation’, so long as the lack of statistical significance is clear and the same interpretation is given to all P-values in their study that fall within this ‘almost statistically significant’ range, and not only those that support their test hypothesis. Alternatively, authors are at liberty to accept a larger α error from the outset (e.g. using P<0.1 as statistically significant rather than P<0.05). We agree that P-values present a continuum and that actual P-values contain valuable information, whether or not they are ‘significant’. For this reason, we would encourage reporting of actual P-values in all circumstances (i.e. not only as significant vs non-significant). We consider that when interpreted correctly,
P-values already ‘speak for themselves’, although the effect size, prespecified minimal clinically important effect size, and β error must also be considered.2 We feel that it is not necessary ‘to drop the concept of significance’. It is necessary only to use it correctly and to know its limitations.
Declaration of interest None declared.
References 1. Gibbs NM, Gibbs SV. Misuse of ‘trend’ to describe ‘almost significant’ differences in anaesthesia research. Br J Anaesth 2015; 115: 337–9 2. Gibbs NM, Weightman WM. Beyond effect size: consideration of the minimum effect size of interest in anesthesia trials. Anesth Analg 2012; 114: 471–5 doi:10.1093/bja/aew145
Downloaded from http://bja.oxfordjournals.org/ at Orta Dogu Teknik University Library (ODTU) on May 22, 2016
Editor—I thank Drs Gibbs and Gibbs1 for highlighting the linguistic and statistical inaccuracies in the use of ‘trends’ in anaesthesia research literature. As a teacher and reviewer, I have argued against this practice, whereas as an author I have almost certainly fallen into the same trap myself! While I cannot disagree with a word of the editorial, I can perhaps offer a little more insight into why this trend might be developing. The authors correctly describe how a predefined P-value is used to determine whether the null hypothesis is rejected or accepted, hence whether results are statistically significant or not. As a result of medical conservatism, we are reluctant to replace a tried and tested therapy with something relatively new, and so the threshold for rejecting the null hypothesis is set high deliberately, typically at a level where the observed result would have occurred by chance only less than five times in 100. Conversely, this means that, in the area in which ‘trends’ tend to be mentioned, the observed result is likely to represent a genuine difference between treatments ∼90 to just <95% of the time. In other circumstances, these would still be very good odds indeed! It is not surprising that