- Email: [email protected]
European clinical practice guidelines for depression in adults: Are they good enough?
Journal Pre-proof
European Clinical Practice Guidelines for Depression in Adults: Are They Good Enough? ´ Joan C. Medina , Emma Schmelefske , Catherine Hebert , Martin Drapeau PII: DOI: Reference:
S0165-0327(19)32724-7 https://doi.org/10.1016/j.jad.2019.12.005 JAD 11421
To appear in:
Journal of Affective Disorders
Received date: Revised date: Accepted date:
7 October 2019 30 November 2019 4 December 2019
´ Please cite this article as: Joan C. Medina , Emma Schmelefske , Catherine Hebert , Martin Drapeau , European Clinical Practice Guidelines for Depression in Adults: Are They Good Enough?, Journal of Affective Disorders (2019), doi: https://doi.org/10.1016/j.jad.2019.12.005
This is a PDF file of an article that has undergone enhancements after acceptance, such as the addition of a cover page and metadata, and formatting for readability, but it is not yet the definitive version of record. This version will undergo additional copyediting, typesetting and review before it is published in its final form, but we are providing this version to give early visibility of the article. Please note that, during the production process, errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain. © 2019 Published by Elsevier B.V.
Highlights
European guidelines covering psychotherapy for depression are of high quality.
Editorial independence and Applicability are areas that need further improvement.
More research is needed to analyse the quality of guidelines in mental health.
1
European Clinical Practice Guidelines for Depression in Adults: Are They Good Enough?
Joan C. Medinaa,b,*, Emma Schmelefskec, Catherine Hébertc, Martin Drapeauc,d.
a
Department of Clinical Psychology and Psychobiology, Universitat de
Barcelona, Spain. b
Institut d’Investigació Biomèdica de Bellvitge, Hospital Duran i Reynals,
L’Hospitalet de Llobregat, Barcelona, Spain. c
Department of Counselling Psychology, McGill University, Montréal,
Québec, Canada. d
Department of Psychiatry, McGill University.
* Correspondence to:
Joan C. Medina, PhD. Department of Clinical Psychology and Psychobiology Universitat de Barcelona Passeig de la Vall d'Hebron, 171 - 08035 Barcelona E-mail: [email protected] / Telf: +34 93 312 50 23
2
Abstract Background: Despite the increasing attention given to evidence-based practice, little research has focused on the quality of clinical practice guidelines (CPGs) involving psychotherapy. The goal of the present study was to evaluate the quality of national CPGs for psychological treatments for depression in European countries. Methods: A search of the Guidelines International Network’s library was conducted. Four guidelines met inclusion criteria for the study and were assessed using the Appraisal of Guidelines for Research and Evaluation II (AGREE-II) platform. Results: Overall, the guidelines included in the study were found to be of good quality, although two guidelines received low scores for applicability, and some did not provide information on competing interests or potential influences from funding bodies. Limitations: Although guidelines were retrieved from a well-known international library, some European guidelines for depression published elsewhere may have been overlooked. Conclusions: Results of this study provide support for the high quality of the guidelines reviewed, but also raise some concerns regarding editorial independence and the applicability of the guidelines, areas that should be the focus of improvement in future versions of these guidelines. Keywords Clinical practice guideline, Depression, Mood disorder, Evidence-based practice.
3
Introduction More than 264 million individuals suffer from depressive disorders worldwide (James et al., 2018). Despite such pervasiveness, it has been suggested that less than 50% of individuals with depression receive proper care, with only 10 to 25% being provided with evidence-based treatments (Voytenko et al., 2018). This is surprising given the availability of well-established interventions, including several psychological therapies (Hollon and Ponniah, 2010). A number of reasons have been put forward to explain this state of affairs, including the cost of some evidence-based treatments, the sceptical opinion of some professionals about them, or simply a lack of awareness about new research findings, which leads to them relying predominantly on their intuition or on knowledge that may be dated (Lilienfeld et al., 2013; Rye et al., 2019). In order to bridge science and practice, Clinical Practice Guidelines (CPGs) have flourished in the last decades in mental health. CPGs have evolved from expert statements (often referred to as an eminence model) that are not necessarily grounded in science, to a method that involves the systematic search and analysis of empirical findings (referred to as an evidence model). CPGs hence summarise the state-of-the-art practices in a particular domain; unsurprisingly, they have been proposed as a means to facilitate access to the best available, scienceinformed clinical knowledge (Graham et al., 2006). Despite this progress, the quality of many CPGs still appears to fall below desirable standards. For example, research has shown that that there is considerable variability in the guidelines available to mental health professionals, and that some CPGs have important methodological limitations (e.g., Middleton, Kalogeropoulos, Middleton, & Drapeau, 2019; Stamoulos, Reyes, Trepanier, & Drapeau, 2014). Moreover, the vast majority of these studies were conducted in North America. In Europe, some research has found the same heterogeneity in quality; studies also suggest that national CPGs developed by state agencies are generally better than those published by professional societies (Burgers et al., 2003), and that their implementation is likely to be cost-effective (Meeuwissen et al., 2019). Most of this knowledge has been acquired thanks to the proliferation of CPGs in the medical field in the last decades, including psychiatry (Alonso-Coello et al., 2010). However, little is still known about the quality of European guidelines that are available specifically for psychotherapists. Hence, the objective of this study was to appraise the quality of national CPGs covering psychological interventions for depression in adults currently used in European countries. Method Guideline search Guidelines were included in the study if: 1) their focus was on the management of depression in adults, with no comorbid diagnoses, 2) included psychotherapy, 3) were currently in use, 4) were published by state agencies of European countries, and 5) were available in English.
4
Guidelines were searched on the 10th of July, 2018, using the Guidelines International Network’s library (G-I-N; https://www.g-i-n.net/library/international-guidelineslibrary), with the keyword “Depression”. G-I-N is a well-known organisation which seeks to promote the creation, appraisal and implementation of high-quality CPGs worldwide, and therefore has been included as a source in several studies (Xie et al., 2016; Zafra-Tanaka et al., 2019). The search led to the identification of an initial set of 10 guidelines. Six of these documents were excluded: four were not published by European institutions, one focused on adults with a chronic physical health problem, and one was not fully available in English. Therefore, four CPGs were appraised. The order in which the guidelines were assessed was randomised by an external researcher. The final sequence was: “Non-pharmaceutical management of depression in adults: A national clinical guideline” (Scotland; Scottish Intercollegiate Guidelines Network, 2010); “The long-term efficacy of psychotherapy, alone or in combination with antidepressants, in the treatment of adult major depression” (Belgium; Karyotaki et al., 2014); “Depression in adults: recognition and management” (United Kingdom; National Collaborating Centre for Mental Health, 2010; National Institute for Health and Care Excellence, 2009); and “Clinical Practice Guideline on the Management of Major Depression in Adults” (Spain; Working Group of the Clinical Practice Guideline on the Management of Depression in Adults, 2014). Instruments The guidelines included were assessed with the Appraisal of Guidelines for Research and Evaluation II (AGREE II) scale (Brouwers et al., 2010). The AGREE II includes 23 items assigned to six domains. Scope and Purpose (3 items) assesses whether the guideline’s overall objectives, questions, and target population are specifically described. Stakeholder Involvement (3 items) appraises whether all relevant professional groups and target users were consulted. Rigour of Development (8 items) focuses on the methods used to search for evidence, analyse it, and formulate recommendations. Clarity of Presentation (3 items) assesses the specificity, clarity and representativeness of these recommendations, while Applicability (4 items) considers the provision of support and monitoring criteria to implement them. Finally, Editorial Independence (2 items) evaluates the influence that funding bodies may have had on guidelines’ content, and the treatment given to competing interests. In addition, appraisers are requested to provide an overall quality score for each guideline. Every item in AGREE II is rated using a 1 (strongly disagree) to 7 (strongly agree) Likert scale, with higher scores indicating greater quality. Assessment process The guidelines were assessed by three postgraduate-level appraisers (i.e., doctoral students) trained in the use of AGREE II. Their training consisted of the online guided modules on AGREE website plus a brief workshop led by an experienced member of the research team, followed by the supervised appraisal of a CPG. They scored each guideline independently, and then met to discuss their results and reach consensus. Intraclass correlations (ICC) were calculated to assess level of agreement before discussion.
5
In accordance with the AGREE II manual (Brouwers et al., 2010), scores were computed by summing the ratings for each item within a given domain, and then scaling the total as a percentage of the maximum possible score. Although no thresholds have been provided by the AGREE consortium, a score of 60/100 was considered to be equivalent to acceptable quality; this cut-off has been used by other researchers (e.g., Middleton et al., 2019; Xie et al., 2016). Results The level of agreement between raters was good for all included guidelines: Scotland (mean ICC = .68, SD = .20), Belgium (ICC = .82, SD = .09), United Kingdom (ICC = .80, SD = .12), and Spain (ICC = .76, SD = .03). The final score assigned to each specific domain, and the global quality scores, are shown in Table 1. All guidelines obtained the highest possible score in Scope and Purpose (100), which made this first domain the highest-rated. Stakeholder Involvement was also appraised very positively, with a mean score of 92.75 and all guidelines receiving scores above 90. For Rigour of Development, the average score was 86.25. The Spanish guideline received a score of 71, which still exceeded the 60 cut-off criterion but was inferior than in the other CPGs, which received ratings above 90. For the fourth domain, Clarity of Presentation, the mean score was again very high (94.25), and all guidelines obtained ratings over 80. The Belgian document received the lowest score, with a score of 83, while the others scored 98. Applicability was by far the domain with the lowest ratings, with an average of 59. In fact, this was the only domain that did not reach the cut-off score of 60, and in which the Spanish and British guidelines received higher scores (65 and 79, respectively) than the Scottish and the Belgian CPGs (scores of 50 and 42). In the sixth domain, Editorial Independence, all guidelines scored above the cut-off, with an average of 73. The highest-rated guideline was the one developed in Belgium, with a score of 89, while all others received scores between 60 and 70. [Insert Table 1 about here] Finally, for the overall scores, the Scottish guideline was the best according to the appraisers, with a rating of 89. The Spanish CPG, on the other hand, obtained the lowest score (78), while the Belgian and British CPGs both scored 83. Discussion Our findings shed light on the quality of European national CPGs for depression in adults that include recommendations for psychological interventions. Overall, the four guidelines were found to be of good quality, with all scoring above the cut-off of 60 in global scores, as well as in almost all specific domains. The only exceptions were the Applicability ratings of both the Scottish and Belgian documents; this domain received the lowest mean rating. Appraisers noted that resource implications of applying the recommendations, as well as monitoring and auditing criteria, were not always clearly provided. Such results are concerning given that lack of applicability has been underlined by clinicians as one of the main reasons to not use research evidence to inform their practice (Drapeau and Hunsley, 2014).
6
The Editorial Independence domain also requires improvement, as information on competing interests and potential influence from funding bodies was not always complete (e.g., interests were listed but no description of how they were managed was provided). More clarity and transparency on such matters would likely increase confidence among guideline users. Rigour of Development was in general positively rated, although in some cases it was not clear if an external review had been conducted and how it affected the recommendations (e.g., Karyotaki et al., 2014; Working Group of the Clinical Practice Guideline on the Management of Depression in Adults, 2014). Furthermore, the Spanish document did not describe the criteria employed for selecting the evidence as clearly as the other guidelines, which led to a lower score. Stakeholder Involvement was an area well covered by the four CPGs, with all of them receiving high scores. This suggests that current guidelines already involve a diverse range of professionals and other stakeholders in their development. The only concern raised by raters was that in some cases it was not specified how the feedback received was taken into consideration when making recommendations. With regard to Clarity of Presentation, in general all documents received good ratings. Only the Belgian guideline scored slightly below than the others; this is because the different management options were well described but not clearly associated with specific patient characteristics. Finally, in the Scope and Purpose domain all CPGs received the maximum score, which further facilitates a clear and efficient use of the guidelines for decision-making purposes. These results are in line with previous studies highlighting the good quality of CPGs developed by state agencies (Burgers et al., 2003). Nonetheless, the scores assigned to the four documents appraised are higher than those generally reported in the literature (see for example the assessment of the Canadian Psychiatric Association's guideline, which received significantly lower scores; Middleton et al., 2019). This suggests that more varied and inclusive development groups (i.e., Stakeholder Involvement), and more rigorous procedures (i.e., Rigour of Development), are used in European CPGs. While our findings are generally reassuring as to the quality of the guidelines reviewed, it must be acknowledged as a limitation that our search for CPGs was not systematic, what might influence the results. Although it was conducted using a well-known international resource, it is also true that there are other European guidelines for the psychological management of depression in adults that were not retrieved, which is also partly accounted by the English language inclusion criterion. However, our search retrieved the two European CPGs found by more systematic studies on the topic (ZafraTanaka et al., 2019) and indeed added two more, what provides support to the representativeness of the source consulted. Some strengths of this work may also be highlighted. For example, our study involved the use of a method generally considered to be the gold standard to appraise guidelines (Brouwers et al., 2010). This is especially important given the lack of information on the quality of European CPGs for depression covering psychological interventions, which is an under-researched area that this study seeks to advance (Hayes and Gregg, 2001; Stamoulos et al., 2014). In this regard, this study provides support for the Scottish, Belgian, British and Spanish guidelines given the generally high scores they received in both the specific domains and the overall ratings. Our results also raise some 7
concerns regarding editorial independence and applicability, areas that should be the focus of further improvement in future CPGs.
Author statement
Contributors JM coordinated the study, conducted the search and analyses, participated in the assessments and discussions, and wrote the article. ES and CH participated in the assessments and discussions, and assisted with the statistical analyses and the writing. MD supervised the study, acted as advisor, provided the training, and reviewed the manuscript. All authors gave approval of the submitted version. Role of the funding source This work was supported by the Spanish Ministry of Economy and Competitiveness [grant number BES-2015-071490]. Conflict of interest None.
Acknowledgements The authors acknowledge the contribution of Leah Beaulieu and Andrea Reyes from McGill University in the training of appraisers.
References Alonso-Coello, P., Irfan, A., Solà, I., Gich, I., Delgado-Noguera, M., Rigau, D., Tort, S., Schunemann, H., 2010. The quality of clinical practice guidelines over the last two decades: A systematic review of guideline appraisal studies. Qual. Saf. Heal. Care 19, e58. https://doi.org/10.1136/qshc.2010.042077 Brouwers, M., Kho, M.E., Browman, G.P., Cluzeau, F., Feder, G., Fervers, B., Hanna, S., Makarski, J. on behalf of the A.C.N.S., 2010. AGREE II: Advancing guideline development, reporting and evaluation in healthcare. Can. Med. Assoc. J. 182, E839-842. https://doi.org/10.1503/090449 Burgers, J.S., Cluzeau, F.A., Hanna, S.E., Hunt, C., Grol, R., 2003. Characteristics of 8
high-quality guidelines: Evaluation of 86 clinical guidelines developed in ten European countries and Canada. Int. J. ofTechnology Assess. Heal. Care 19, 148– 157. https://doi.org/10.1017/S026646230300014X Drapeau, M., Hunsley, J., 2014. Where’s the science? Introduction to a special issue of Canadian Psychology on science in psychology. Can. Psychol. Can. 55, 145–152. https://doi.org/10.1037/a0037321 Graham, I.D., Logan, J., Harrison, M.B., Straus, S.E., Tetroe, J., Caswell, W., Robinson, N., 2006. Lost in knowledge translation: Time for a map? J. Contin. Educ. Health Prof. 26, 13–24. https://doi.org/10.1002/chp.47 Hayes, S.C., Gregg, J., 2001. Factors promoting and inhibiting the development and use of clinical practice guidelines. Behav. Ther. 32, 211–217. https://doi.org/10.1016/S0005-7894(01)80002-2 Hollon, S.D., Ponniah, K., 2010. A review of empirically supported psychological therapies for mood disorders in adults. Depress. Anxiety 27, 891–932. https://doi.org/10.1002/da.20741 James, S.L., and GBD 2017 Disease and Injury Incidence and Prevalence Collaborators, 2018. Global, regional, and national incidence, prevalence, and years lived with disability for 354 diseases and injuries for 195 countries and territories, 1990– 2017: a systematic analysis for the Global Burden of Disease Study 2017. Lancet 392, 1789–1858. https://doi.org/10.1016/S0140-6736(18)32279-7 Karyotaki, E., Smit, Y., Cuijpers, P., Debauche, M., De Keyser, T., Habraken, H., Pitchot, W., Raes, F., Salomez, D., Gillain, B., Fairon, N., Paulus, D., Robays, J., Holdt Henningsen, K., 2014. The long-term efficacy of psychotherapy, alone or in combination with antidepressants, in the treatment of adult major depression. Brussels, Belgium. Lilienfeld, S.O., Ritschel, L.A., Lynn, S.J., Cautin, R.L., Latzman, R.D., 2013. Why many clinical psychologists are resistant to evidence-based practice: Root causes and constructive remedies. Clin. Psychol. Rev. 33, 883–900. https://doi.org/10.1016/j.cpr.2012.09.008 Meeuwissen, J.A.C., Feenstra, T.L., Smit, F., Blankers, M., Spijker, J., Bockting, C.L.H., van Balkom, A.J.L.M., Buskens, E., 2019. The cost-utility of stepped-care algorithms according to depression guideline recommendations – Results of a state-transition model analysis. J. Affect. Disord. 242, 244–254. https://doi.org/10.1016/j.jad.2018.08.024 Middleton, J.C., Kalogeropoulos, C., Middleton, J.A., Drapeau, M., 2019. Assessing the methodological quality of the Canadian Psychiatric Association’s anxiety and depression clinical practice guidelines. J. Eval. Clin. Pract. 25, 613–621. https://doi.org/10.1111/jep.13026 National Collaborating Centre for Mental Health, 2010. Depression: The NICE Guideline on the Treatment and Management of Depression in Adults. The British Psychological Society and The Royal College of Psychiatrists, London, UK. National Institute for Health and Care Excellence, 2009. Depression in adults: recognition and management. London, UK. 9
Rye, M., Friborg, O., Skre, I., 2019. Attitudes of mental health providers towards adoption of evidence-based interventions: relationship to workplace, staff roles and social and psychological factors at work. BMC Health Serv. Res. 19, 110. https://doi.org/10.1186/s12913-019-3933-4 Scottish Intercollegiate Guidelines Network, 2010. Non-pharmaceutical management of depression in adults: A national clinical guideline. Edinburgh, Scotland. Stamoulos, C., Reyes, A., Trepanier, L., Drapeau, M., 2014. The methodological quality of clinical practice guidelines in psychology: A pilot study of the guidelines developed by the Ordre des Psychologues du Québec. Can. Psychol. Can. 55, 177– 186. https://doi.org/10.1037/a0037287 Voytenko, V., Nykamp, L., Achtyes, E., Stoyanov, R., Anderson, K., Thomas, S., Hill Lee, S., Edwards, E., Post, D., Wyngarden, N., 2018. Evidence-based practice guideline for the treatment of adult patients with depressive disorders. Part I: Psychiatric management. Psychiatr. i Psychol. Klin. 18, 234–241. https://doi.org/10.15557/PiPK.2018.0029 Working Group of the Clinical Practice Guideline on the Management of Depression in Adults, 2014. Clinical Practice Guideline on the Management of Major Depression in Adults. Madrid, Spain. Xie, Z., Wang, X., Sun, L., Liu, J., Guo, Y., Xu, B., Zhao, L., Shen, A., 2016. Appraisal of clinical practice guidelines on community-acquired pneumonia in children with AGREE II instrument. BMC Pediatr. 16, 119. https://doi.org/10.1186/s12887-0160651-5 Zafra-Tanaka, J.H., Goicochea-Lugo, S., Villarreal-Zegarra, D., Taype-Rondan, A., 2019. Characteristics and quality of clinical practice guidelines for depression in adults: a scoping review. BMC Psychiatry 19, 76. https://doi.org/10.1186/s12888019-2057-z
10
Table 1.
Domain and Overall Scores for European Guidelines for Depression in Adults Guideline
Country
(Scottish Intercollegiate
Scotland
(Karyotaki et al., 2014)
Scope
Stakeholder
Rigour of
Clarity of
Applicability
100
93
93
98
Belgium
100
91
90
United
100
96
Spain
100
100
and
purpose Guidelines Network, 2010) (National Collaborating
Centre for Mental Health,
2010; National Institute for
Kingdom
Editorial
Global
50*
69
89
83
42*
89
83
91
98
79
67
83
91
71
98
65
67
78
92.75
86.25
94.25
59*
73
83.25
involvement development presentation
independence
Health and Care
Excellence, 2009)
(Working Group of the
Clinical Practice Guideline on the Management of Depression in Adults, 2014)
Average * Scores below the cut-off score.
11
European Clinical Practice Guidelines for Depression in Adults: Are They Good Enough? ´ Joan C. Medina , Emma Schmelefske , Catherine Hebert , Martin Drapeau PII: DOI: Reference:
S0165-0327(19)32724-7 https://doi.org/10.1016/j.jad.2019.12.005 JAD 11421
To appear in:
Journal of Affective Disorders
Received date: Revised date: Accepted date:
7 October 2019 30 November 2019 4 December 2019
´ Please cite this article as: Joan C. Medina , Emma Schmelefske , Catherine Hebert , Martin Drapeau , European Clinical Practice Guidelines for Depression in Adults: Are They Good Enough?, Journal of Affective Disorders (2019), doi: https://doi.org/10.1016/j.jad.2019.12.005
This is a PDF file of an article that has undergone enhancements after acceptance, such as the addition of a cover page and metadata, and formatting for readability, but it is not yet the definitive version of record. This version will undergo additional copyediting, typesetting and review before it is published in its final form, but we are providing this version to give early visibility of the article. Please note that, during the production process, errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain. © 2019 Published by Elsevier B.V.
Highlights
European guidelines covering psychotherapy for depression are of high quality.
Editorial independence and Applicability are areas that need further improvement.
More research is needed to analyse the quality of guidelines in mental health.
1
European Clinical Practice Guidelines for Depression in Adults: Are They Good Enough?
Joan C. Medinaa,b,*, Emma Schmelefskec, Catherine Hébertc, Martin Drapeauc,d.
a
Department of Clinical Psychology and Psychobiology, Universitat de
Barcelona, Spain. b
Institut d’Investigació Biomèdica de Bellvitge, Hospital Duran i Reynals,
L’Hospitalet de Llobregat, Barcelona, Spain. c
Department of Counselling Psychology, McGill University, Montréal,
Québec, Canada. d
Department of Psychiatry, McGill University.
* Correspondence to:
Joan C. Medina, PhD. Department of Clinical Psychology and Psychobiology Universitat de Barcelona Passeig de la Vall d'Hebron, 171 - 08035 Barcelona E-mail: [email protected] / Telf: +34 93 312 50 23
2
Abstract Background: Despite the increasing attention given to evidence-based practice, little research has focused on the quality of clinical practice guidelines (CPGs) involving psychotherapy. The goal of the present study was to evaluate the quality of national CPGs for psychological treatments for depression in European countries. Methods: A search of the Guidelines International Network’s library was conducted. Four guidelines met inclusion criteria for the study and were assessed using the Appraisal of Guidelines for Research and Evaluation II (AGREE-II) platform. Results: Overall, the guidelines included in the study were found to be of good quality, although two guidelines received low scores for applicability, and some did not provide information on competing interests or potential influences from funding bodies. Limitations: Although guidelines were retrieved from a well-known international library, some European guidelines for depression published elsewhere may have been overlooked. Conclusions: Results of this study provide support for the high quality of the guidelines reviewed, but also raise some concerns regarding editorial independence and the applicability of the guidelines, areas that should be the focus of improvement in future versions of these guidelines. Keywords Clinical practice guideline, Depression, Mood disorder, Evidence-based practice.
3
Introduction More than 264 million individuals suffer from depressive disorders worldwide (James et al., 2018). Despite such pervasiveness, it has been suggested that less than 50% of individuals with depression receive proper care, with only 10 to 25% being provided with evidence-based treatments (Voytenko et al., 2018). This is surprising given the availability of well-established interventions, including several psychological therapies (Hollon and Ponniah, 2010). A number of reasons have been put forward to explain this state of affairs, including the cost of some evidence-based treatments, the sceptical opinion of some professionals about them, or simply a lack of awareness about new research findings, which leads to them relying predominantly on their intuition or on knowledge that may be dated (Lilienfeld et al., 2013; Rye et al., 2019). In order to bridge science and practice, Clinical Practice Guidelines (CPGs) have flourished in the last decades in mental health. CPGs have evolved from expert statements (often referred to as an eminence model) that are not necessarily grounded in science, to a method that involves the systematic search and analysis of empirical findings (referred to as an evidence model). CPGs hence summarise the state-of-the-art practices in a particular domain; unsurprisingly, they have been proposed as a means to facilitate access to the best available, scienceinformed clinical knowledge (Graham et al., 2006). Despite this progress, the quality of many CPGs still appears to fall below desirable standards. For example, research has shown that that there is considerable variability in the guidelines available to mental health professionals, and that some CPGs have important methodological limitations (e.g., Middleton, Kalogeropoulos, Middleton, & Drapeau, 2019; Stamoulos, Reyes, Trepanier, & Drapeau, 2014). Moreover, the vast majority of these studies were conducted in North America. In Europe, some research has found the same heterogeneity in quality; studies also suggest that national CPGs developed by state agencies are generally better than those published by professional societies (Burgers et al., 2003), and that their implementation is likely to be cost-effective (Meeuwissen et al., 2019). Most of this knowledge has been acquired thanks to the proliferation of CPGs in the medical field in the last decades, including psychiatry (Alonso-Coello et al., 2010). However, little is still known about the quality of European guidelines that are available specifically for psychotherapists. Hence, the objective of this study was to appraise the quality of national CPGs covering psychological interventions for depression in adults currently used in European countries. Method Guideline search Guidelines were included in the study if: 1) their focus was on the management of depression in adults, with no comorbid diagnoses, 2) included psychotherapy, 3) were currently in use, 4) were published by state agencies of European countries, and 5) were available in English.
4
Guidelines were searched on the 10th of July, 2018, using the Guidelines International Network’s library (G-I-N; https://www.g-i-n.net/library/international-guidelineslibrary), with the keyword “Depression”. G-I-N is a well-known organisation which seeks to promote the creation, appraisal and implementation of high-quality CPGs worldwide, and therefore has been included as a source in several studies (Xie et al., 2016; Zafra-Tanaka et al., 2019). The search led to the identification of an initial set of 10 guidelines. Six of these documents were excluded: four were not published by European institutions, one focused on adults with a chronic physical health problem, and one was not fully available in English. Therefore, four CPGs were appraised. The order in which the guidelines were assessed was randomised by an external researcher. The final sequence was: “Non-pharmaceutical management of depression in adults: A national clinical guideline” (Scotland; Scottish Intercollegiate Guidelines Network, 2010); “The long-term efficacy of psychotherapy, alone or in combination with antidepressants, in the treatment of adult major depression” (Belgium; Karyotaki et al., 2014); “Depression in adults: recognition and management” (United Kingdom; National Collaborating Centre for Mental Health, 2010; National Institute for Health and Care Excellence, 2009); and “Clinical Practice Guideline on the Management of Major Depression in Adults” (Spain; Working Group of the Clinical Practice Guideline on the Management of Depression in Adults, 2014). Instruments The guidelines included were assessed with the Appraisal of Guidelines for Research and Evaluation II (AGREE II) scale (Brouwers et al., 2010). The AGREE II includes 23 items assigned to six domains. Scope and Purpose (3 items) assesses whether the guideline’s overall objectives, questions, and target population are specifically described. Stakeholder Involvement (3 items) appraises whether all relevant professional groups and target users were consulted. Rigour of Development (8 items) focuses on the methods used to search for evidence, analyse it, and formulate recommendations. Clarity of Presentation (3 items) assesses the specificity, clarity and representativeness of these recommendations, while Applicability (4 items) considers the provision of support and monitoring criteria to implement them. Finally, Editorial Independence (2 items) evaluates the influence that funding bodies may have had on guidelines’ content, and the treatment given to competing interests. In addition, appraisers are requested to provide an overall quality score for each guideline. Every item in AGREE II is rated using a 1 (strongly disagree) to 7 (strongly agree) Likert scale, with higher scores indicating greater quality. Assessment process The guidelines were assessed by three postgraduate-level appraisers (i.e., doctoral students) trained in the use of AGREE II. Their training consisted of the online guided modules on AGREE website plus a brief workshop led by an experienced member of the research team, followed by the supervised appraisal of a CPG. They scored each guideline independently, and then met to discuss their results and reach consensus. Intraclass correlations (ICC) were calculated to assess level of agreement before discussion.
5
In accordance with the AGREE II manual (Brouwers et al., 2010), scores were computed by summing the ratings for each item within a given domain, and then scaling the total as a percentage of the maximum possible score. Although no thresholds have been provided by the AGREE consortium, a score of 60/100 was considered to be equivalent to acceptable quality; this cut-off has been used by other researchers (e.g., Middleton et al., 2019; Xie et al., 2016). Results The level of agreement between raters was good for all included guidelines: Scotland (mean ICC = .68, SD = .20), Belgium (ICC = .82, SD = .09), United Kingdom (ICC = .80, SD = .12), and Spain (ICC = .76, SD = .03). The final score assigned to each specific domain, and the global quality scores, are shown in Table 1. All guidelines obtained the highest possible score in Scope and Purpose (100), which made this first domain the highest-rated. Stakeholder Involvement was also appraised very positively, with a mean score of 92.75 and all guidelines receiving scores above 90. For Rigour of Development, the average score was 86.25. The Spanish guideline received a score of 71, which still exceeded the 60 cut-off criterion but was inferior than in the other CPGs, which received ratings above 90. For the fourth domain, Clarity of Presentation, the mean score was again very high (94.25), and all guidelines obtained ratings over 80. The Belgian document received the lowest score, with a score of 83, while the others scored 98. Applicability was by far the domain with the lowest ratings, with an average of 59. In fact, this was the only domain that did not reach the cut-off score of 60, and in which the Spanish and British guidelines received higher scores (65 and 79, respectively) than the Scottish and the Belgian CPGs (scores of 50 and 42). In the sixth domain, Editorial Independence, all guidelines scored above the cut-off, with an average of 73. The highest-rated guideline was the one developed in Belgium, with a score of 89, while all others received scores between 60 and 70. [Insert Table 1 about here] Finally, for the overall scores, the Scottish guideline was the best according to the appraisers, with a rating of 89. The Spanish CPG, on the other hand, obtained the lowest score (78), while the Belgian and British CPGs both scored 83. Discussion Our findings shed light on the quality of European national CPGs for depression in adults that include recommendations for psychological interventions. Overall, the four guidelines were found to be of good quality, with all scoring above the cut-off of 60 in global scores, as well as in almost all specific domains. The only exceptions were the Applicability ratings of both the Scottish and Belgian documents; this domain received the lowest mean rating. Appraisers noted that resource implications of applying the recommendations, as well as monitoring and auditing criteria, were not always clearly provided. Such results are concerning given that lack of applicability has been underlined by clinicians as one of the main reasons to not use research evidence to inform their practice (Drapeau and Hunsley, 2014).
6
The Editorial Independence domain also requires improvement, as information on competing interests and potential influence from funding bodies was not always complete (e.g., interests were listed but no description of how they were managed was provided). More clarity and transparency on such matters would likely increase confidence among guideline users. Rigour of Development was in general positively rated, although in some cases it was not clear if an external review had been conducted and how it affected the recommendations (e.g., Karyotaki et al., 2014; Working Group of the Clinical Practice Guideline on the Management of Depression in Adults, 2014). Furthermore, the Spanish document did not describe the criteria employed for selecting the evidence as clearly as the other guidelines, which led to a lower score. Stakeholder Involvement was an area well covered by the four CPGs, with all of them receiving high scores. This suggests that current guidelines already involve a diverse range of professionals and other stakeholders in their development. The only concern raised by raters was that in some cases it was not specified how the feedback received was taken into consideration when making recommendations. With regard to Clarity of Presentation, in general all documents received good ratings. Only the Belgian guideline scored slightly below than the others; this is because the different management options were well described but not clearly associated with specific patient characteristics. Finally, in the Scope and Purpose domain all CPGs received the maximum score, which further facilitates a clear and efficient use of the guidelines for decision-making purposes. These results are in line with previous studies highlighting the good quality of CPGs developed by state agencies (Burgers et al., 2003). Nonetheless, the scores assigned to the four documents appraised are higher than those generally reported in the literature (see for example the assessment of the Canadian Psychiatric Association's guideline, which received significantly lower scores; Middleton et al., 2019). This suggests that more varied and inclusive development groups (i.e., Stakeholder Involvement), and more rigorous procedures (i.e., Rigour of Development), are used in European CPGs. While our findings are generally reassuring as to the quality of the guidelines reviewed, it must be acknowledged as a limitation that our search for CPGs was not systematic, what might influence the results. Although it was conducted using a well-known international resource, it is also true that there are other European guidelines for the psychological management of depression in adults that were not retrieved, which is also partly accounted by the English language inclusion criterion. However, our search retrieved the two European CPGs found by more systematic studies on the topic (ZafraTanaka et al., 2019) and indeed added two more, what provides support to the representativeness of the source consulted. Some strengths of this work may also be highlighted. For example, our study involved the use of a method generally considered to be the gold standard to appraise guidelines (Brouwers et al., 2010). This is especially important given the lack of information on the quality of European CPGs for depression covering psychological interventions, which is an under-researched area that this study seeks to advance (Hayes and Gregg, 2001; Stamoulos et al., 2014). In this regard, this study provides support for the Scottish, Belgian, British and Spanish guidelines given the generally high scores they received in both the specific domains and the overall ratings. Our results also raise some 7
concerns regarding editorial independence and applicability, areas that should be the focus of further improvement in future CPGs.
Author statement
Contributors JM coordinated the study, conducted the search and analyses, participated in the assessments and discussions, and wrote the article. ES and CH participated in the assessments and discussions, and assisted with the statistical analyses and the writing. MD supervised the study, acted as advisor, provided the training, and reviewed the manuscript. All authors gave approval of the submitted version. Role of the funding source This work was supported by the Spanish Ministry of Economy and Competitiveness [grant number BES-2015-071490]. Conflict of interest None.
Acknowledgements The authors acknowledge the contribution of Leah Beaulieu and Andrea Reyes from McGill University in the training of appraisers.
References Alonso-Coello, P., Irfan, A., Solà, I., Gich, I., Delgado-Noguera, M., Rigau, D., Tort, S., Schunemann, H., 2010. The quality of clinical practice guidelines over the last two decades: A systematic review of guideline appraisal studies. Qual. Saf. Heal. Care 19, e58. https://doi.org/10.1136/qshc.2010.042077 Brouwers, M., Kho, M.E., Browman, G.P., Cluzeau, F., Feder, G., Fervers, B., Hanna, S., Makarski, J. on behalf of the A.C.N.S., 2010. AGREE II: Advancing guideline development, reporting and evaluation in healthcare. Can. Med. Assoc. J. 182, E839-842. https://doi.org/10.1503/090449 Burgers, J.S., Cluzeau, F.A., Hanna, S.E., Hunt, C., Grol, R., 2003. Characteristics of 8
high-quality guidelines: Evaluation of 86 clinical guidelines developed in ten European countries and Canada. Int. J. ofTechnology Assess. Heal. Care 19, 148– 157. https://doi.org/10.1017/S026646230300014X Drapeau, M., Hunsley, J., 2014. Where’s the science? Introduction to a special issue of Canadian Psychology on science in psychology. Can. Psychol. Can. 55, 145–152. https://doi.org/10.1037/a0037321 Graham, I.D., Logan, J., Harrison, M.B., Straus, S.E., Tetroe, J., Caswell, W., Robinson, N., 2006. Lost in knowledge translation: Time for a map? J. Contin. Educ. Health Prof. 26, 13–24. https://doi.org/10.1002/chp.47 Hayes, S.C., Gregg, J., 2001. Factors promoting and inhibiting the development and use of clinical practice guidelines. Behav. Ther. 32, 211–217. https://doi.org/10.1016/S0005-7894(01)80002-2 Hollon, S.D., Ponniah, K., 2010. A review of empirically supported psychological therapies for mood disorders in adults. Depress. Anxiety 27, 891–932. https://doi.org/10.1002/da.20741 James, S.L., and GBD 2017 Disease and Injury Incidence and Prevalence Collaborators, 2018. Global, regional, and national incidence, prevalence, and years lived with disability for 354 diseases and injuries for 195 countries and territories, 1990– 2017: a systematic analysis for the Global Burden of Disease Study 2017. Lancet 392, 1789–1858. https://doi.org/10.1016/S0140-6736(18)32279-7 Karyotaki, E., Smit, Y., Cuijpers, P., Debauche, M., De Keyser, T., Habraken, H., Pitchot, W., Raes, F., Salomez, D., Gillain, B., Fairon, N., Paulus, D., Robays, J., Holdt Henningsen, K., 2014. The long-term efficacy of psychotherapy, alone or in combination with antidepressants, in the treatment of adult major depression. Brussels, Belgium. Lilienfeld, S.O., Ritschel, L.A., Lynn, S.J., Cautin, R.L., Latzman, R.D., 2013. Why many clinical psychologists are resistant to evidence-based practice: Root causes and constructive remedies. Clin. Psychol. Rev. 33, 883–900. https://doi.org/10.1016/j.cpr.2012.09.008 Meeuwissen, J.A.C., Feenstra, T.L., Smit, F., Blankers, M., Spijker, J., Bockting, C.L.H., van Balkom, A.J.L.M., Buskens, E., 2019. The cost-utility of stepped-care algorithms according to depression guideline recommendations – Results of a state-transition model analysis. J. Affect. Disord. 242, 244–254. https://doi.org/10.1016/j.jad.2018.08.024 Middleton, J.C., Kalogeropoulos, C., Middleton, J.A., Drapeau, M., 2019. Assessing the methodological quality of the Canadian Psychiatric Association’s anxiety and depression clinical practice guidelines. J. Eval. Clin. Pract. 25, 613–621. https://doi.org/10.1111/jep.13026 National Collaborating Centre for Mental Health, 2010. Depression: The NICE Guideline on the Treatment and Management of Depression in Adults. The British Psychological Society and The Royal College of Psychiatrists, London, UK. National Institute for Health and Care Excellence, 2009. Depression in adults: recognition and management. London, UK. 9
Rye, M., Friborg, O., Skre, I., 2019. Attitudes of mental health providers towards adoption of evidence-based interventions: relationship to workplace, staff roles and social and psychological factors at work. BMC Health Serv. Res. 19, 110. https://doi.org/10.1186/s12913-019-3933-4 Scottish Intercollegiate Guidelines Network, 2010. Non-pharmaceutical management of depression in adults: A national clinical guideline. Edinburgh, Scotland. Stamoulos, C., Reyes, A., Trepanier, L., Drapeau, M., 2014. The methodological quality of clinical practice guidelines in psychology: A pilot study of the guidelines developed by the Ordre des Psychologues du Québec. Can. Psychol. Can. 55, 177– 186. https://doi.org/10.1037/a0037287 Voytenko, V., Nykamp, L., Achtyes, E., Stoyanov, R., Anderson, K., Thomas, S., Hill Lee, S., Edwards, E., Post, D., Wyngarden, N., 2018. Evidence-based practice guideline for the treatment of adult patients with depressive disorders. Part I: Psychiatric management. Psychiatr. i Psychol. Klin. 18, 234–241. https://doi.org/10.15557/PiPK.2018.0029 Working Group of the Clinical Practice Guideline on the Management of Depression in Adults, 2014. Clinical Practice Guideline on the Management of Major Depression in Adults. Madrid, Spain. Xie, Z., Wang, X., Sun, L., Liu, J., Guo, Y., Xu, B., Zhao, L., Shen, A., 2016. Appraisal of clinical practice guidelines on community-acquired pneumonia in children with AGREE II instrument. BMC Pediatr. 16, 119. https://doi.org/10.1186/s12887-0160651-5 Zafra-Tanaka, J.H., Goicochea-Lugo, S., Villarreal-Zegarra, D., Taype-Rondan, A., 2019. Characteristics and quality of clinical practice guidelines for depression in adults: a scoping review. BMC Psychiatry 19, 76. https://doi.org/10.1186/s12888019-2057-z
10
Table 1.
Domain and Overall Scores for European Guidelines for Depression in Adults Guideline
Country
(Scottish Intercollegiate
Scotland
(Karyotaki et al., 2014)
Scope
Stakeholder
Rigour of
Clarity of
Applicability
100
93
93
98
Belgium
100
91
90
United
100
96
Spain
100
100
and
purpose Guidelines Network, 2010) (National Collaborating
Centre for Mental Health,
2010; National Institute for
Kingdom
Editorial
Global
50*
69
89
83
42*
89
83
91
98
79
67
83
91
71
98
65
67
78
92.75
86.25
94.25
59*
73
83.25
involvement development presentation
independence
Health and Care
Excellence, 2009)
(Working Group of the
Clinical Practice Guideline on the Management of Depression in Adults, 2014)
Average * Scores below the cut-off score.
11