- Email: [email protected]
Author’s response to Chapman and Hayen
Available online at www.sciencedirect.com
Health Policy 88 (2008) 397–399
Letter to the Editor
Author’s response to Chapman and Hayen We agree entirely with Chapman and Hayen [1] that those looking to research for guidance on the impact of Australia’s 1996 gun laws should exercise caution. We suggest such advice be applied to all research, given that challenging preconceptions and critically analysing problems advance knowledge and policy. Transparency about competing interests is also important when readers are invited to doubt or accept the importance of national suicide prevention policies based on empirical data. Thus, it is disappointing that Professor Chapman failed to disclose his longstanding affiliation with Australia’s gun prohibition lobby [2], ongoing involvement with anti-gun campaigning [3], and corresponding history of claims about policy efficacy that have been revealed to rest upon mathematical inaccuracies [3]. When scrutinised, Chapman and Hayen’s [1] reaction to our paper, on suicide prevention and the apparent need for gender-specific prevention policy in Australia, contains numerous flaws and serious misunderstandings. For example, it appears the authors misunderstand the concept of ‘rate of change’ versus ‘absolute fall’. Chapman and Hayen [1] claim McPhedran and Baker [4] do not calculate a ‘rate of change’. This is demonstrably incorrect. McPhedran and Baker [4] calculate, and present values pertaining to, the rate at which the incidence of suicide per 100,000/population (within each gender) fell per year of data. This includes the slope of the line of best fit for each group, which shows how much, on average, the standardised incidence of suicide changed across each year for each group. Therefore, it
is inaccurate to state that McPhedran and Baker [4] did not provide such information, when that information is clearly contained within the paper. In Section 2 of their commentary, Chapman and Hayen [1] appear to compare absolute suicide rates in 1997 and absolute suicide rates in 2005 (rather than comparative trends over 9 years of data). This is a method of assessing an absolute fall in suicide per 100,000/population between two points in time—precisely the type of analysis Chapman and Hayen [1] propose should not be undertaken. It is unclear why the authors claim a log transformation of the data is necessary to avoid predicted suicide rates falling below zero by 2013. This is a significant misinterpretation of the way in which McPhedran and Baker [4] applied their analysis. McPhedran and Baker [4] did not seek to ‘predict’ suicide rates into the future, but retrospectively examined the time series 1997–2005 and fitted their model to existing data only (which were, by definition, positive values). This was undertaken to examine whether initiatives such as the National Suicide Prevention Strategy had an apparently equal influence across gender on declines in suicide, over the time period examined. Given the objectives of our work, the claim that a log transformation is required to avoid negative values lacks foundation. The fact that Chapman and Hayen’s rationale for transforming the data cannot be sustained makes their use of a highly specific form of log-linear model (Poisson analysis) all the more puzzling. We agree that Poisson analysis is commonly used in epidemiology, particularly for large, complex datasets containing multiple variables that are not linearly related (as opposed to the small, simple dataset examined by McPhedran
0168-8510/$ – see front matter © 2008 Elsevier Ireland Ltd. All rights reserved. doi:10.1016/j.healthpol.2008.10.004
398
Letter to the Editor / Health Policy 88 (2008) 397–399
and Baker [4]). However, data assumptions that should be met for a Poisson model to be credibly applied are: • Positively skewed data. • Heterogeneity of variances. • Variances equal to the means (equidispersal). In the current instance, the data do not adequately fulfil those assumptions, indicating that the application of Chapman and Hayen’s [1] method is not suitable for the figures at hand. In contrast, a linear regression model, applied by McPhedran and Baker [4], is appropriate where data show the following characteristics: • Linearity of the relationship between dependent and independent variables. • Independence of errors. • Homeoscedasticity (constant variance) of errors. • Normality of the error distribution. The suicide data satisfy those parameters, and McPhedran and Baker’s [4] goodness of fit values demonstrate that the model provided a good estimation of observed data. Chapman and Hayen [1] neglect to comment on potential implications of log transformations in public health research. Log transformations minimise differences and force data closer together. This can alter the data distribution, ‘flatten’ trends, and bias towards zero the slope of the lines of best fit (upon which rates of change are based). In the current context, bias manifests in values which suggest there has been little change in suicide trends for males or females over time, and, by extension, that male and suicides have not differed in their declines. It is apparent from Figs. 2 and 3 in McPhedran and Baker [4] that since 1997 there have been downwards trends for male suicide, female suicide, male nonfirearm, female non-firearm, and male firearm suicide (female firearm suicides are extremely low, rendering analysis problematic). The transformation touted by Chapman and Hayen [1] does not adequately reflect the nature of these trends, or the relationship they bear to one another. When transformation biases towards zero the slopes for linear trends, a large sample size and/or effect would be needed before any significant differences were detected among groups. By forcing the data to
fit a Poisson model (log transformation), information about small but important changes within groups is lost, with a corresponding decrease in the sensitivity to detect policy-relevant health outcomes. This is precisely what analysis of small, simple datasets should strive to avoid. In light of the constraints produced by transforming the data, it is unclear how Chapman and Hayen [1] conclude that one group of suicides (male firearm) declined more quickly than other groups (male nonfirearm and female non-firearm). It is a substantial concern that the outcomes of statistical analyses are not reported at any point in their Section 2, with no values from statistical tests presented to support the assertions. Without any statistical information, the reader cannot assess whether or not the claims are valid. Above all, it is extremely troubling that Chapman and Hayen [1] focus exclusively on gun laws – a relatively minor aspect of McPhedran and Baker’s [4] paper – and ignore the myriad factors that may contribute positively to reducing suicide in Australia. Had this not been the case, poor model selection and data transformation resulting in a loss of critical information, failure to adequately report statistical outcomes, and even the lack of disclosure of competing interests could perhaps be overlooked. However, the elephant in the living room is that Chapman and Hayen [1] are unwilling to mention a single word about evidence-based suicide prevention in Australia. Nor do they see fit to discuss barriers to help-seeking, the apparent success of comprehensive national prevention efforts, or the need to build upon such initiatives to further reduce the occurrence of suicide among Australian men and women. References [1] Chapman S, Hayen A. Declines in Australian suicide: a reanalysis of McPhedran and Baker (2008). Letter to the Editor. Health Policy 2008;88(1):152–4. [2] Chan, G. University research centre to back gun control lobby. The Australian, June 5, 1996. [3] Australian Broadcasting Commission (ABC). The law report—gun control, October 31, 2006. Transcript available from http://www.abc.net.au/rn/lawreport/stories/2006/ 1776336.htm.
Letter to the Editor / Health Policy 88 (2008) 397–399 [4] McPhedran S, Baker J. Recent Australian suicide trends for males and females at the national level: has the rate of decline differed? Health Policy 2008;87(3):350–8.
Jeanine Baker ∗ Samara McPhedran International Coalition for Women in Shooting and
399
Hunting (WiSH), PO Box 393, Glebe, New South Wales 2037, Australia ∗ Corresponding
author. Tel.: +61 427 186 184. E-mail address: [email protected] (J. Baker)
Health Policy 88 (2008) 397–399
Letter to the Editor
Author’s response to Chapman and Hayen We agree entirely with Chapman and Hayen [1] that those looking to research for guidance on the impact of Australia’s 1996 gun laws should exercise caution. We suggest such advice be applied to all research, given that challenging preconceptions and critically analysing problems advance knowledge and policy. Transparency about competing interests is also important when readers are invited to doubt or accept the importance of national suicide prevention policies based on empirical data. Thus, it is disappointing that Professor Chapman failed to disclose his longstanding affiliation with Australia’s gun prohibition lobby [2], ongoing involvement with anti-gun campaigning [3], and corresponding history of claims about policy efficacy that have been revealed to rest upon mathematical inaccuracies [3]. When scrutinised, Chapman and Hayen’s [1] reaction to our paper, on suicide prevention and the apparent need for gender-specific prevention policy in Australia, contains numerous flaws and serious misunderstandings. For example, it appears the authors misunderstand the concept of ‘rate of change’ versus ‘absolute fall’. Chapman and Hayen [1] claim McPhedran and Baker [4] do not calculate a ‘rate of change’. This is demonstrably incorrect. McPhedran and Baker [4] calculate, and present values pertaining to, the rate at which the incidence of suicide per 100,000/population (within each gender) fell per year of data. This includes the slope of the line of best fit for each group, which shows how much, on average, the standardised incidence of suicide changed across each year for each group. Therefore, it
is inaccurate to state that McPhedran and Baker [4] did not provide such information, when that information is clearly contained within the paper. In Section 2 of their commentary, Chapman and Hayen [1] appear to compare absolute suicide rates in 1997 and absolute suicide rates in 2005 (rather than comparative trends over 9 years of data). This is a method of assessing an absolute fall in suicide per 100,000/population between two points in time—precisely the type of analysis Chapman and Hayen [1] propose should not be undertaken. It is unclear why the authors claim a log transformation of the data is necessary to avoid predicted suicide rates falling below zero by 2013. This is a significant misinterpretation of the way in which McPhedran and Baker [4] applied their analysis. McPhedran and Baker [4] did not seek to ‘predict’ suicide rates into the future, but retrospectively examined the time series 1997–2005 and fitted their model to existing data only (which were, by definition, positive values). This was undertaken to examine whether initiatives such as the National Suicide Prevention Strategy had an apparently equal influence across gender on declines in suicide, over the time period examined. Given the objectives of our work, the claim that a log transformation is required to avoid negative values lacks foundation. The fact that Chapman and Hayen’s rationale for transforming the data cannot be sustained makes their use of a highly specific form of log-linear model (Poisson analysis) all the more puzzling. We agree that Poisson analysis is commonly used in epidemiology, particularly for large, complex datasets containing multiple variables that are not linearly related (as opposed to the small, simple dataset examined by McPhedran
0168-8510/$ – see front matter © 2008 Elsevier Ireland Ltd. All rights reserved. doi:10.1016/j.healthpol.2008.10.004
398
Letter to the Editor / Health Policy 88 (2008) 397–399
and Baker [4]). However, data assumptions that should be met for a Poisson model to be credibly applied are: • Positively skewed data. • Heterogeneity of variances. • Variances equal to the means (equidispersal). In the current instance, the data do not adequately fulfil those assumptions, indicating that the application of Chapman and Hayen’s [1] method is not suitable for the figures at hand. In contrast, a linear regression model, applied by McPhedran and Baker [4], is appropriate where data show the following characteristics: • Linearity of the relationship between dependent and independent variables. • Independence of errors. • Homeoscedasticity (constant variance) of errors. • Normality of the error distribution. The suicide data satisfy those parameters, and McPhedran and Baker’s [4] goodness of fit values demonstrate that the model provided a good estimation of observed data. Chapman and Hayen [1] neglect to comment on potential implications of log transformations in public health research. Log transformations minimise differences and force data closer together. This can alter the data distribution, ‘flatten’ trends, and bias towards zero the slope of the lines of best fit (upon which rates of change are based). In the current context, bias manifests in values which suggest there has been little change in suicide trends for males or females over time, and, by extension, that male and suicides have not differed in their declines. It is apparent from Figs. 2 and 3 in McPhedran and Baker [4] that since 1997 there have been downwards trends for male suicide, female suicide, male nonfirearm, female non-firearm, and male firearm suicide (female firearm suicides are extremely low, rendering analysis problematic). The transformation touted by Chapman and Hayen [1] does not adequately reflect the nature of these trends, or the relationship they bear to one another. When transformation biases towards zero the slopes for linear trends, a large sample size and/or effect would be needed before any significant differences were detected among groups. By forcing the data to
fit a Poisson model (log transformation), information about small but important changes within groups is lost, with a corresponding decrease in the sensitivity to detect policy-relevant health outcomes. This is precisely what analysis of small, simple datasets should strive to avoid. In light of the constraints produced by transforming the data, it is unclear how Chapman and Hayen [1] conclude that one group of suicides (male firearm) declined more quickly than other groups (male nonfirearm and female non-firearm). It is a substantial concern that the outcomes of statistical analyses are not reported at any point in their Section 2, with no values from statistical tests presented to support the assertions. Without any statistical information, the reader cannot assess whether or not the claims are valid. Above all, it is extremely troubling that Chapman and Hayen [1] focus exclusively on gun laws – a relatively minor aspect of McPhedran and Baker’s [4] paper – and ignore the myriad factors that may contribute positively to reducing suicide in Australia. Had this not been the case, poor model selection and data transformation resulting in a loss of critical information, failure to adequately report statistical outcomes, and even the lack of disclosure of competing interests could perhaps be overlooked. However, the elephant in the living room is that Chapman and Hayen [1] are unwilling to mention a single word about evidence-based suicide prevention in Australia. Nor do they see fit to discuss barriers to help-seeking, the apparent success of comprehensive national prevention efforts, or the need to build upon such initiatives to further reduce the occurrence of suicide among Australian men and women. References [1] Chapman S, Hayen A. Declines in Australian suicide: a reanalysis of McPhedran and Baker (2008). Letter to the Editor. Health Policy 2008;88(1):152–4. [2] Chan, G. University research centre to back gun control lobby. The Australian, June 5, 1996. [3] Australian Broadcasting Commission (ABC). The law report—gun control, October 31, 2006. Transcript available from http://www.abc.net.au/rn/lawreport/stories/2006/ 1776336.htm.
Letter to the Editor / Health Policy 88 (2008) 397–399 [4] McPhedran S, Baker J. Recent Australian suicide trends for males and females at the national level: has the rate of decline differed? Health Policy 2008;87(3):350–8.
Jeanine Baker ∗ Samara McPhedran International Coalition for Women in Shooting and
399
Hunting (WiSH), PO Box 393, Glebe, New South Wales 2037, Australia ∗ Corresponding
author. Tel.: +61 427 186 184. E-mail address: [email protected] (J. Baker)