Exercise for Depression: Cochrane systematic reviews are rigorous, but how subjective are the assessment of bias and the practice implications?

Exercise for Depression: Cochrane systematic reviews are rigorous, but how subjective are the assessment of bias and the practice implications?

G Model AIMED-51; No. of Pages 3 Advances in Integrative Medicine xxx (2015) xxx–xxx Contents lists available at ScienceDirect Advances in Integrat...

182KB Sizes 0 Downloads 11 Views

G Model

AIMED-51; No. of Pages 3 Advances in Integrative Medicine xxx (2015) xxx–xxx

Contents lists available at ScienceDirect

Advances in Integrative Medicine journal homepage: www.elsevier.com/locate/aimed

Exercise for Depression: Cochrane systematic reviews are rigorous, but how subjective are the assessment of bias and the practice implications? Simon Spedding * Visiting Physician and Adjunct Senior Research Fellow, Alliance for Research in Exercise, Nutrition and Activity (ARENA), Sansom Institute for Health Research, University of South Australia, Frome Road, Adelaide, SA 5000, Australia

A R T I C L E I N F O

A B S T R A C T

Article history: Available online xxx

The Cochrane review Exercise for Depression and 12 previous systematic reviews support the view that exercise is as effective as conventional therapy for reducing symptoms of depression. However concerns remain about bias leading to downgrading of practice implications. The Cochrane ‘Risk of Bias’ tool is recognised as having poor reliability. So subjective judgements made about risk of bias and practice implications are discussed and compared with the response to doubtful blinding in antidepressant trials. Clinicians can explore the holistic management of depression including exercise based on the highest level of evidence. ß 2015 Elsevier Ltd. All rights reserved.

Keywords: Depression Exercise Physical activity Mood disorder Cochrane Collaboration Meta-analysis Systematic review Bias Practice implications

One of the most popular reviews is ‘Exercise for Depression’ by Gary Cooney and colleagues [1]. Depression is common, affecting 350 million people worldwide, and the leading cause of disability and fourth-leading cause of the global disease burden [2]. However there is no single proven way that people recover from depression [3]. There are doubts about the efficacy of conventional therapies [5]. For example, claims of antidepressant medication efficacy are inflated by publication bias [6] whilst guidelines promoting their use may not discuss these limitations [7]. For these reasons health professionals explore complementary and lifestyle approaches which may be more in line with their patient’s philosophy [4]. The clinical effect was downgraded from moderate to small in earlier reviews as the risk of bias was assessed as uncertain or higher. However, some Cochrane researchers are concerned about the validity of the assessment of bias in primary studies and note that observer variation influence outcomes more than the intervention itself [8]. This occurs particularly with bias due inadequate blinding as this has the lowest inter-rater reliability [9]: for example four in five reviewers disagree about the level of ‘risk of bias’ due to blinding, as compared to one in five or less for assessments of other forms of bias (Box 1).

* Tel.: +61 439 687 886. E-mail address: [email protected]

Aim The aims of this paper are: 1. To describe the findings of this systematic review Exercise for Depression. 2. To clarify the process that led to the grading of the practice implications.

Findings of the ‘exercise for depression’ review The Exercise for Depression is a systematic review of 39 trials and meta-analysis of 35 trials with 1356 exercising participants. It shows the effectiveness of exercise was clinically and statistically significant as compared to control. The size of the effect, the standardised mean difference (SMD), was 0.62 (95% CI 0.81 to 0.42). This indicates exercise would be expected to make a moderate clinical difference as compared to no treatment and the result did not occur by chance (Box 2). Subgroup and sensitivity analysis evaluated different interventions, controls, levels of depression, and study quality and secondary outcomes. These confirmed the original findings showing a moderate clinical effect in favour of exercise. Analysis of the intensity, number, type, duration and frequency of exercise

http://dx.doi.org/10.1016/j.aimed.2015.02.005 2212-9588/ß 2015 Elsevier Ltd. All rights reserved.

Please cite this article in press as: Spedding S. Exercise for Depression: Cochrane systematic reviews are rigorous, but how subjective are the assessment of bias and the practice implications? Adv Integr Med (2015), http://dx.doi.org/10.1016/j.aimed.2015.02.005

G Model

AIMED-51; No. of Pages 3 2

S. Spedding / Advances in Integrative Medicine xxx (2015) xxx–xxx

Box 1. Practice implications and risk of bias The grade of practice implications of a review are based on the effect size moderated by the risk of bias assessment. The Cochrane ‘Risk of Bias’ tool assesses the chance of bias occurring in the primary trials. Risk of Bias is assessed as high, low or unclear. The two forms of bias involve blinding or concealment of group allocation. Performance bias may occur when the participant is not blinded. Detection bias may occur when the observer or clinician making an objective assessment of the outcome is not blinded.

sessions, showed similar SMDs to the figure for all 35 trials. No specific regime showed a clear benefit over any other. No difference in efficacy or adverse events was found between exercise and usual care with pharmaceutical, psychological, and alternative therapies except the sertraline groups in two trials had significantly more adverse events; diarrhoea, fatigue and sexual dysfunction. Subgroup analysis for risk of bias from ‘allocation concealment’ and ‘intention to treat’ confirmed the original findings. The analysis for ‘binding of the outcome assessment’ showed a smaller effect size (SMD 0.36, 95% CI 0.60 to 0.12). However, this may not signify a real difference as the confidence intervals overlapped those for all 35 trials. The methodology of the review following the Cochrane process used clearly defined processes to ensure consistency and reliability. However judgements about inclusion criteria and the assessment of bias are more subjective and may be less consistent. The inclusion criteria limited the scope of this review as the authors affirm. Trials of physical activity designed to improve psychological health were excluded and whereas interventions that improved physical fitness were included. Trials were excluded if they were uncontrolled, used combined intervention, or were in community participants if they included people with subclinical or no depression. Using the Cochrane ‘risk of bias’ tool, many studies were assessed as high or uncertain and only a third were rated as having a low risk of bias. The reviewers gave a high rating for bias due to blinding, as only observer-rated trials were considered blinded. Eleven previous systematic reviews found exercise was beneficial for depression. The results of 10 reviews in community participants did not vary substantially with the inclusion of uncontrolled trials. The authors of the remaining review concluded ‘exercise may have a small, short-term beneficial effect in relieving symptoms of depression’ [10]; however, Cooney et al. interpreted it as finding no evidence of benefit. Another review not cited by Cooney et al. showed exercise was associated ‘with significantly lower depression severity’ irrespective of whether participant eligibility was determined by clinical diagnosis or symptom checklist [11]. Results were remarkably consistent across this review Exercise for Depression and the 12 previous systematic reviews. They Box 2. Practice implications and effect size The grade of practice implications of a review are based on the effect size moderated by the risk of bias assessment. The size of the effect indicates the clinically significance of the intervention. The measure is the standardised mean difference (SMD) with 95% confidence intervals (95% CI). The SMD indicates the level of clinical significance; with SMD > 0.8 indicating a large clinical effects, 0.5–0.8 a moderate effect, and 0.2–0.5 a small effect. The 95% CI indicates statistical significance when both numbers are either above or below zero.

concluded exercise is moderately effective for reducing symptoms of depression, more effective than placebo but not more than other therapies. Furthermore outcomes did not vary substantially with the date or size of the review, age of participants, the severity of the depression or how it was determined, or even if trials were uncontrolled. However there were different assessments of risk of bias due to blinding. The process leading to practice implications The Cochrane systematic review process for estimating the effect size is rigorous, whereas the assessment of risk of bias is subjective and lacks reliability [12–15]. This may have led to studies of exercise being assessed as having a high risk of bias. This is supported by an examination of the effect of a lack of funding and assessment of bias on the practice implications in this review. Lack of resources affected the scope and depth of this review. As the authors stated, they ‘did not have the study protocols’ and were therefore ‘uncertain what effect this (inadequate blinding) would have on the risk of bias’. Thus many trials were assessed as having an unclear or high risk of bias and the reviewers decided to downgrade the practice implications. There is a risk of performance bias as participants cannot be blinded to exercise observers cannot record objective facts about depression. There is a risk of detection bias as symptoms are described by non-blinded participants. Therefore the reviewers stated ‘studies of depression are not free from detection bias even with blinded clinician rated outcomes’. Thus no study design could preclude some risk of bias with the current system for assessing risk of bias. The issue is how bias is assessed and how that affects practice implications. The current system for assessing bias, the Cochrane ‘Risk of Bias’ tool, is recognised as having poor reliability particularly for performance bias and detection bias [8,9,12–15]. Thus the decision to allocate a high, low, or unclear risk of bias is subjective, as is the subsequent decision to downgrade practice implications. By comparison, practice implications of pharmaceutical drugs are not downgraded from moderate to small on the basis of doubtful blinding, even though 80% of participants in double-blind antidepressant trials were able to identify from the side effects whether they were on drug or placebo [16]. Similarly there is a risk of funding bias as described in a previous Cochrane Corner [17] in pharmaceutical studies, the practice implications of these reviews are not downgraded from moderate to small. Conclusion The Exercise for Depression and 12 previous systematic reviews support the view that exercise is moderately effective for reducing symptoms of depression across studies with different methodologies. It is more effective than placebo and not different from conventional therapies. However concerns remain about bias due to inadequate blinding of participants and observers. Metaanalysis provides the highest level of evidence to translate research into policy and practice. Systemic reviews use rigorous Cochrane methodology to provide transparent, valid and reliable results up to the assessment of the effect size. However the final stage of moderating the practice implications based on the risk of bias may be subjective, unclear and unreliable. Research implications Reviewers recommend ‘more large scale high quality trials are performed’ as ‘substantial practice implications cannot be made due to the risk of bias’. However reviewers make choices to assess the risk of bias as uncertain or higher and downgrade

Please cite this article in press as: Spedding S. Exercise for Depression: Cochrane systematic reviews are rigorous, but how subjective are the assessment of bias and the practice implications? Adv Integr Med (2015), http://dx.doi.org/10.1016/j.aimed.2015.02.005

G Model

AIMED-51; No. of Pages 3 S. Spedding / Advances in Integrative Medicine xxx (2015) xxx–xxx

practice implications. Now it is being recognised meta-analyses are prone to observer variation influencing these outcomes more than the intervention itself. Consequently recent publications identify these limitations in the Cochrane ‘Risk of Bias’ tool and call for changes to its interpretation [8,9,12–15]. Without these changes, it may be argued future high quality trials will not alter the outcome, as practice implications for exercise will continue to be downgraded, unlike the practice implications for antidepressants drugs. Funding implications No specific funding source was identified for such a major undertaking as this review, whilst reviews of antidepressant drugs may identify five specific sources of funding including pharmaceutical companies, national bodies and academic institutions. The lack of resources limited the scope and depth of this review, thus increased funding is required to improve the quality of exercise research and systematic reviews. Clinical implications The use of exercise in the management of depression is based on the highest level of evidence from 13 concordant systematic reviews and meta-analyses. Although limitations due to the risk of bias assessment may devalue practice implications, this may be dependent on the methodology of the review more than the efficacy of exercise. As there is no single proven way to treat depression, clinicians can now explore the holistic management of depression including exercise based on the highest level of evidence. References [1] Cooney, GM, Dwan K, Greig CA, Lawlor DA, Rimer J, Waugh FR, et al. Exercise for depression. (Published online: 12 September 2013) . [2] Hyman S, Chisholm D, Kessler R, Patel V, Whiteford H. Mental disorders. In: Jamison DT, Breman JG, Measham AR, Alleyne G, Claeson M, Evans DB, et al., editors. Disease control priorities in developing countries. 2nd ed., New York: Oxford University Press; 2006. p. 605–26.

3

[3] Beyond Blue. Treatments – for-depression. Beyond Blue Web Site: . [4] Astin JA. Why patients use alternative medicine: results of a national study. JAMA 1998;279(19):1548–53. http://dx.doi.org/10.1001/jama.279.19.1548. [5] Bracken P, Thomas P, Timimi S. Psychiatry beyond the current paradigm. Br J Psychiatry 2012;201:430–4. [6] Turner EH, Matthews AM, Linardatos E, Tell RA, Rosenthal R:. Selective publication of antidepressant trials and its influence on apparent efficacy. N Engl J Med 2008;358:252–60. [7] Davidson JR. Major depressive disorder treatment guidelines in America and Europe. J Clin Psychiatry 2010;71(Suppl. E1):e04. http://dx.doi.org/10.4088/ JCP.9058se1c.04gry. [8] Tendal B, Higgins JPT, Ju¨ni P, Hro´bjartsson A, Trelle S, Nu¨esch E, et al. Disagreements in meta-analyses using outcomes measured on continuous or rating scales: observer agreement study. BMJ 2009;339:b3128. http:// dx.doi.org/10.1136/bmj.b3128. [9] Rada G, Rivera S, Capurro D. Risk of bias assessment of studies included in multiple Cochrane reviews: agreement between different reviews. The Cochrane Collaboration. Cochrane Database Syst Rev 2013;(Suppl):213–22. http://dx.doi.org/10.1002/14651858.CD201300 (published by John Wiley & Sons, Ltd.). [10] Krogh J, Nordentoft M, Sterne J, Lawlor DA. The effect of exercise in clinical depressed adults: systematic review and meta-analysis of randomised controlled trials. J Clin Psychiatry 2011;72(4):529–38. [11] Bridle C, Spanjers K, Patel S, Atherton NM, Lamb SE. Effect of exercise on depression severity in older people: systematic review and meta-analysis of randomised controlled trials. Br J Psychiatry 2012;201(3):180–5. http:// dx.doi.org/10.1192/bjp.bp.111.095174. [12] Hartling L, Ospina M, Liang Y, Dryden DM, Hooton N, Krebs Seida J, et al. Risk of bias versus quality assessment of randomised controlled trials: cross sectional study. BMJ 2009;339:1017. http://dx.doi.org/10.1136/bmj.b4012. [13] Armijo-Olivo S, Stiles CR, Hagen NA, Biondo PD, Cummings GG. Assessment of study quality for systematic reviews: a comparison of the Cochrane Collaboration Risk of Bias Tool and the Effective Public Health Practice Project Quality Assessment Tool: methodological research. J Eval Clin Pract 2012;18:12–8. http://dx.doi.org/10.1111/j.1365-2753.2010.01516.x. [14] Hartling L, Hamm MP, Milne A, Vandermeer B, Santaguida PL, et al. Testing the Risk of Bias tool showed low reliability between individual reviewers and across consensus assessments of reviewer pairs. J Clin Epidemiol 2012;66:973–81. [15] Armijo-Olivo S, Ospina M, da Costa BR, et al. Poor Reliability between Cochrane reviewers and blinded external reviewers when applying the Cochrane risk of bias tool in physical therapy trials. In: Manzoli L, editor. PLOS ONE 2014;9(5):e96920. http://dx.doi.org/10.1371/journal.pone.0096920. [16] Greenberg RP, Fisher S. Examining antidepressant effectiveness: findings ambiguities and some vexing puzzles. In: Fisher S, Greenberg RP, editors. The limits of biological treatments for psychological distress: comparisons with psychotherapy and placebo. New Jersey: Lawrence Erlbaum Associates Inc.; 1989. p. 1–37. [17] Spedding S. Cochrane corner commentary: do funding sources create bias’ Cochrane. Adv Integr Med 2014, http://www.journaltocs.ac.uk/ articleHomePage.php?id=14975447&userID=0&mainAccountID=0, >.

Please cite this article in press as: Spedding S. Exercise for Depression: Cochrane systematic reviews are rigorous, but how subjective are the assessment of bias and the practice implications? Adv Integr Med (2015), http://dx.doi.org/10.1016/j.aimed.2015.02.005