- Email: [email protected]
Use of evidence in WHO recommendations
Correspondence
the development process of WHO guidelines includes the assessment of potential harms of a treatment along with the expected benefits might not have been mentioned by some directors for this reason. We would also like to add that Integrated Management of Childhood Illness (IMCI) guidelines are evidence-based: our group has been involved in generating evidence for the acute respiratory infection portion for more than a decade.3 Oxman and colleagues also suggest that WHO technical guidelines make little attempt to “help member states adapt the global recommendations to account for local needs”. Although this might be true in some areas, the WHO IMCI guidelines have explicitly incorporated local adaptation tools since 1997.4 Oxman and colleagues could have highlighted this approach as a model for departments rather than leaving it unreferenced. The question posed by Oxman and colleagues is important, but the methods, data, and interpretation left us unconvinced that we yet know how WHO is doing in its attempt to bring more systematic evidence to bear for the development of recommendations. A question this important is worth answering systematically. We declare that we have no conflict of interest.
Jonathon Simon, *Donald M Thea [email protected] Department of International Health, Boston University School of Public Health, 85 East Concord St, Boston, MA 02118, USA 1
2
3
4
Oxman AD, Lavis JN, Fretheim A. Use of evidence in WHO recommendations. Lancet 2007; 369: 1883–89. Global Programme on Evidence for Health Policy. Guidelines for WHO guidelines. Geneva: World Health Organization, 2003. Addo-Yobo E, Chisaka N, Hassan M, et al, for the Amoxicillin Penicillin Pneumonia International Study (APPIS) group. Oral amoxicillin versus injectable penicillin for severe pneumonia in children aged 3 to 59 months: a randomised multicentre equivalency study. Lancet 2004; 364: 1141–48. WHO Department of Child and Adolescent Health. IMCI adaptation guide, version 5. Geneva: World Health Organization, 2002. http://www.who.int/child-adolescent-health/ New_Publications/IMCI/IMCI_Adaptation_ Guige/sectiona.pdf (accessed May 24, 2007).
www.thelancet.com Vol 370 September 8, 2007
Although we fully concur with Andrew Oxman and colleagues about the importance of evidence in developing WHO recommendations,1 our experience with WHO’s Department of Reproductive Health and Research (RHR) provides strong evidence that the development process for at least some of WHO’s guidelines is truly exemplary. RHR has produced the Medical Eligibility Criteria for Contraceptive Use,2 a guideline that has been incorporated into national programme guidance in more than 50 countries. The process for creating and updating this guideline and its counterpart, the Selected Practice Recommendations for Contraceptive Use,3 has been described in detail.4,5 Briefly, systematic reviews with graded quality of evidence are used by an expert working group (clinical and research experts and those who will have to live with the guidelines, such as family planning providers in low-income countries) to develop recommendations. Evidence from these systematic reviews is cited in the guidelines, and many of the systematic reviews are published in peer-reviewed journals. Furthermore, the Continuous Identification of Research Evidence (CIRE) system has been developed to identify, assess, and synthesise new evidence as it becomes available. CIRE findings are used to determine whether an update to the guidelines is warranted. WHO and the United Nations Population Fund have created a strategic partnership programme that is implementing these guidelines at regional and country levels. Oxman and colleagues’ point about the need for resources for WHO recommendations is an important one. The system described above, while making efficient and effective use of limited existing resources, requires additional support from several donors. We declare that we have no conflict of interest. The findings and conclusions in the letter are those of the authors and do not necessarily represent the views of the Centers for Disease Control and Prevention.
Herbert B Peterson, *Kathryn M Curtis, Anna Glasier, Kerstin Hagenfeldt [email protected] Department of Maternal and Child Health, University of North Carolina School of Public Health, Chapel Hill, NC, USA (HBP); Division of Reproductive Health, Centers for Disease Control and Prevention, Atlanta, GA 30341, USA (KMC); Family Planning and Well Woman Services, University of Edinburgh, Edinburgh, UK (AG); and Department of Obstetrics and Gynecology, Karolinska Institute, Stockholm, Sweden (KH) 1
2
3
4
5
Oxman AD, Lavis JN, Fretheim A. Use of evidence in WHO recommendations. Lancet 2007; 369: 1883–89. WHO. Medical eligibility criteria for contraceptive use. Geneva: World Health Organization, 2004. http://www.who.int/ reproductive-health/publications/mec/index. htm (accessed Aug 14, 2007). WHO. Selected practice recommendations for contraceptive use. Geneva: World Health Organization, 2005. http://www.who.int/ reproductive-health/publications/spr/index. htm (accessed Aug 14, 2007). Mohllajee AP, Curtis KM, Flanagan RG, Rinehart W, Gaffield ML, Peterson HB. Keeping up with evidence: a new system for WHO’s evidence-based family planning guidance. Am J Prev Med 2005; 28: 483–90. Peterson HB, Curtis KM. The World Health Organization’s global guidance for family planning: an achievement to celebrate. Contraception 2006; 73: 113–14.
We agree wholeheartedly with Andrew Oxman and colleagues1 in their call for better use of systematic reviews of the evidence base for the development of guidelines in health care. It would be impractical, however, to suggest that WHO departments alone can do this, or that we should wait for exhaustive processes such as those of the Cochrane collaboration before making recommendations. The relationship between WHO and its collaborating centres and with academic institutions provides a good example of the way in which WHO is making progress in achieving evidence-based practice in the area of child health. A multi-institutional collaboration comprising developed and developing country partners began the process of critically appraising the evidence behind WHO guidelines for paediatric care in the Pocket book of hospital care for children.2 Specifically aimed at end users, the International Child Health Review Collaboration (ICHRC) is an attempt to
For the International Child Health Review Collaboration website see http://www.ichrc.org
825
Correspondence
WHO
provide what Oxman and colleagues highlight in as being deficient. With a method that is explicit, reproducible, transparent, and widely accessible, the collaboration aims to undertake systematic reviews of the evidence, then prepare brief reviews to be made available for those who use the guidelines.3 The collaboration invites end users and experts from WHO departments to identify areas of uncertainty or controversy and gaps in the current guidelines so that these can be given priority within the ICHRC review process. More broadly, the collaboration supports colleges, universities, and other institutions to incorporate the teaching of evidence-based medicine into their curricula, provides a strategy for implementing WHO guidelines, and finally, and most importantly, encourages broad participation in the appraisal of the evidence. The Centre for International Child Health receives funding from WHO to coordinate the ICHRC.
Julian Kelly, Harry Campbell, Naor Bar-Zeev, Giorgio Tamburlini, *Trevor Duke [email protected] Centre for International Child Health, Department of Paediatrics, University of Melbourne, Melbourne, Victoria, Australia (JK, NBZ, TD); Public Health Sciences, University of Edinburgh, Edinburgh, UK (HC); Institute of Child Health Burlo Garofolo, Trieste, Italy (GT); and WHO Collaborating Centre for Research and Training in Child and Neonatal Health, Department of Paediatrics, Royal Children’s Hospital, Parkville, Victoria 3052, Australia (TD) 1
2
3
Oxman AD, Lavis JN, Fretheim A. Use of evidence in WHO recommendations. Lancet 2007; 369: 1883–89. WHO. Pocket book of hospital care for children: guidelines for the management of common illnesses with limited resources. Geneva: World Health Organization, 2005. http://www.who.int/child-adolescent-health/ publications/CHILD_HEALTH/PB.htm (accessed Aug 14, 2007). Duke T, Kelly J, Weber M, et al. Hospital care for children in developing countries: clinical guidelines and the need for evidence. J Trop Pediatr 2006; 52: 1–2.
Andrew Oxman and colleagues1 highlight the limited systematic use of evidence in the development of WHO recommendations. In response, WHO has committed to improving the 826
quality of the current system,2 which is based on definition of norms and standards in the form of international recommendations and dissemination to member states for implementation. For well characterised domains such as those reviewed by Oxman and colleagues, the current approach is, however, poorly responsive to the needs of country-level policymakers. International recommendations are unable to respond adequately to global diversity in local conditions or reflect the complexity of current knowledge. We propose revolutionary change to the current approach and envisage two key roles for WHO. First, WHO would coordinate the construction of comprehensive and authoritative compilations of systematic reviews designed, populated, and updated by international networks with content and methodological expertise. These “evidence maps” would be structured to incorporate important contextual characteristics, be designed to help users interpret evidence,3 and be accessible as living web-based resources. Second, WHO would support the development of knowledge transfer capacity within countries, enabling norms and standards to be driven by policymakers with understanding of the evidence and local conditions. The EVIPNet network4 and the Regional East African Community Health initiative5 are important early examples of this approach. The time has come to increase the bandwidth of the connection between global knowledge and policy development in WHO member states. WHO should drive decentralisation of knowledge transfer by building global evidence maps and country-level knowledge transfer capacity. RLG is principal investigator of the Global Evidence Mapping Initiative, a not-for-profit, collaborative, university-based research programme that aims to map research evidence in broad clinical areas.
*Julian H Elliott, Russell L Gruen [email protected]
Macfarlane Burnet Institute for Medical Research and Public Health, Melbourne, Victoria 3004, Australia (JHE); and Department of Surgery, University of Melbourne, Melbourne, Victoria, Australia (RLG) 1
2 3
4
5
Oxman A, Lavis J, Fretheim A. Use of evidence in WHO recommendations. Lancet 2007; 369: 1883–89. Hill S, Pang T. Leading by example: a culture change at WHO. Lancet 2007; 369: 1842–44. Gruen RL, Morris PS, McDonald L, Bailie RS. Making systematic reviews more useful for policy-makers. Bull World Health Organ 2005; 83: 480–81. EVIPNet: evidence-informed policy network. http://www.who.int/rpc/evipnet/en/ (accessed May 23, 2007). East African Community. Regional East African Community health-policy initiative. http://www. idrc.ca/uploads/user-S/11551301781REACH_ Prospectus.pdf (accessed May 23, 2007).
Authors’ reply We agree with Justus Krabshuis and colleagues that resources are important and that evidence is not the key issue. But we disagree that resources are the key issue. The key issue is how best to use available resources to achieve health goals. It is not possible to make informed judgments about how best to use scarce resources without evidence to inform those judgments. This includes economic evidence as well as evidence of the effects of relevant options for achieving health goals.1,2 The best way to ensure that recommendations are well informed by the best available evidence is by use of systematic and transparent processes. Without this, it is difficult, if not impossible, to know how much evidence there is to support a recommendation or a cascade of options. Although Donald Thea and Jonathon Simon are correct about the timing of our study, our member checking extended well over a year later, during which time no mention was made of a fundamental shift that diminished the credibility of our findings. We asked explicitly about harms during the interviews, and no one commented on this finding during our elaborate member checking. We agree with Thea and Simon, and with Herbert Peterson and colleagues and Julian Kelly and colleagues, that there are examples of excellence at www.thelancet.com Vol 370 September 8, 2007
the development process of WHO guidelines includes the assessment of potential harms of a treatment along with the expected benefits might not have been mentioned by some directors for this reason. We would also like to add that Integrated Management of Childhood Illness (IMCI) guidelines are evidence-based: our group has been involved in generating evidence for the acute respiratory infection portion for more than a decade.3 Oxman and colleagues also suggest that WHO technical guidelines make little attempt to “help member states adapt the global recommendations to account for local needs”. Although this might be true in some areas, the WHO IMCI guidelines have explicitly incorporated local adaptation tools since 1997.4 Oxman and colleagues could have highlighted this approach as a model for departments rather than leaving it unreferenced. The question posed by Oxman and colleagues is important, but the methods, data, and interpretation left us unconvinced that we yet know how WHO is doing in its attempt to bring more systematic evidence to bear for the development of recommendations. A question this important is worth answering systematically. We declare that we have no conflict of interest.
Jonathon Simon, *Donald M Thea [email protected] Department of International Health, Boston University School of Public Health, 85 East Concord St, Boston, MA 02118, USA 1
2
3
4
Oxman AD, Lavis JN, Fretheim A. Use of evidence in WHO recommendations. Lancet 2007; 369: 1883–89. Global Programme on Evidence for Health Policy. Guidelines for WHO guidelines. Geneva: World Health Organization, 2003. Addo-Yobo E, Chisaka N, Hassan M, et al, for the Amoxicillin Penicillin Pneumonia International Study (APPIS) group. Oral amoxicillin versus injectable penicillin for severe pneumonia in children aged 3 to 59 months: a randomised multicentre equivalency study. Lancet 2004; 364: 1141–48. WHO Department of Child and Adolescent Health. IMCI adaptation guide, version 5. Geneva: World Health Organization, 2002. http://www.who.int/child-adolescent-health/ New_Publications/IMCI/IMCI_Adaptation_ Guige/sectiona.pdf (accessed May 24, 2007).
www.thelancet.com Vol 370 September 8, 2007
Although we fully concur with Andrew Oxman and colleagues about the importance of evidence in developing WHO recommendations,1 our experience with WHO’s Department of Reproductive Health and Research (RHR) provides strong evidence that the development process for at least some of WHO’s guidelines is truly exemplary. RHR has produced the Medical Eligibility Criteria for Contraceptive Use,2 a guideline that has been incorporated into national programme guidance in more than 50 countries. The process for creating and updating this guideline and its counterpart, the Selected Practice Recommendations for Contraceptive Use,3 has been described in detail.4,5 Briefly, systematic reviews with graded quality of evidence are used by an expert working group (clinical and research experts and those who will have to live with the guidelines, such as family planning providers in low-income countries) to develop recommendations. Evidence from these systematic reviews is cited in the guidelines, and many of the systematic reviews are published in peer-reviewed journals. Furthermore, the Continuous Identification of Research Evidence (CIRE) system has been developed to identify, assess, and synthesise new evidence as it becomes available. CIRE findings are used to determine whether an update to the guidelines is warranted. WHO and the United Nations Population Fund have created a strategic partnership programme that is implementing these guidelines at regional and country levels. Oxman and colleagues’ point about the need for resources for WHO recommendations is an important one. The system described above, while making efficient and effective use of limited existing resources, requires additional support from several donors. We declare that we have no conflict of interest. The findings and conclusions in the letter are those of the authors and do not necessarily represent the views of the Centers for Disease Control and Prevention.
Herbert B Peterson, *Kathryn M Curtis, Anna Glasier, Kerstin Hagenfeldt [email protected] Department of Maternal and Child Health, University of North Carolina School of Public Health, Chapel Hill, NC, USA (HBP); Division of Reproductive Health, Centers for Disease Control and Prevention, Atlanta, GA 30341, USA (KMC); Family Planning and Well Woman Services, University of Edinburgh, Edinburgh, UK (AG); and Department of Obstetrics and Gynecology, Karolinska Institute, Stockholm, Sweden (KH) 1
2
3
4
5
Oxman AD, Lavis JN, Fretheim A. Use of evidence in WHO recommendations. Lancet 2007; 369: 1883–89. WHO. Medical eligibility criteria for contraceptive use. Geneva: World Health Organization, 2004. http://www.who.int/ reproductive-health/publications/mec/index. htm (accessed Aug 14, 2007). WHO. Selected practice recommendations for contraceptive use. Geneva: World Health Organization, 2005. http://www.who.int/ reproductive-health/publications/spr/index. htm (accessed Aug 14, 2007). Mohllajee AP, Curtis KM, Flanagan RG, Rinehart W, Gaffield ML, Peterson HB. Keeping up with evidence: a new system for WHO’s evidence-based family planning guidance. Am J Prev Med 2005; 28: 483–90. Peterson HB, Curtis KM. The World Health Organization’s global guidance for family planning: an achievement to celebrate. Contraception 2006; 73: 113–14.
We agree wholeheartedly with Andrew Oxman and colleagues1 in their call for better use of systematic reviews of the evidence base for the development of guidelines in health care. It would be impractical, however, to suggest that WHO departments alone can do this, or that we should wait for exhaustive processes such as those of the Cochrane collaboration before making recommendations. The relationship between WHO and its collaborating centres and with academic institutions provides a good example of the way in which WHO is making progress in achieving evidence-based practice in the area of child health. A multi-institutional collaboration comprising developed and developing country partners began the process of critically appraising the evidence behind WHO guidelines for paediatric care in the Pocket book of hospital care for children.2 Specifically aimed at end users, the International Child Health Review Collaboration (ICHRC) is an attempt to
For the International Child Health Review Collaboration website see http://www.ichrc.org
825
Correspondence
WHO
provide what Oxman and colleagues highlight in as being deficient. With a method that is explicit, reproducible, transparent, and widely accessible, the collaboration aims to undertake systematic reviews of the evidence, then prepare brief reviews to be made available for those who use the guidelines.3 The collaboration invites end users and experts from WHO departments to identify areas of uncertainty or controversy and gaps in the current guidelines so that these can be given priority within the ICHRC review process. More broadly, the collaboration supports colleges, universities, and other institutions to incorporate the teaching of evidence-based medicine into their curricula, provides a strategy for implementing WHO guidelines, and finally, and most importantly, encourages broad participation in the appraisal of the evidence. The Centre for International Child Health receives funding from WHO to coordinate the ICHRC.
Julian Kelly, Harry Campbell, Naor Bar-Zeev, Giorgio Tamburlini, *Trevor Duke [email protected] Centre for International Child Health, Department of Paediatrics, University of Melbourne, Melbourne, Victoria, Australia (JK, NBZ, TD); Public Health Sciences, University of Edinburgh, Edinburgh, UK (HC); Institute of Child Health Burlo Garofolo, Trieste, Italy (GT); and WHO Collaborating Centre for Research and Training in Child and Neonatal Health, Department of Paediatrics, Royal Children’s Hospital, Parkville, Victoria 3052, Australia (TD) 1
2
3
Oxman AD, Lavis JN, Fretheim A. Use of evidence in WHO recommendations. Lancet 2007; 369: 1883–89. WHO. Pocket book of hospital care for children: guidelines for the management of common illnesses with limited resources. Geneva: World Health Organization, 2005. http://www.who.int/child-adolescent-health/ publications/CHILD_HEALTH/PB.htm (accessed Aug 14, 2007). Duke T, Kelly J, Weber M, et al. Hospital care for children in developing countries: clinical guidelines and the need for evidence. J Trop Pediatr 2006; 52: 1–2.
Andrew Oxman and colleagues1 highlight the limited systematic use of evidence in the development of WHO recommendations. In response, WHO has committed to improving the 826
quality of the current system,2 which is based on definition of norms and standards in the form of international recommendations and dissemination to member states for implementation. For well characterised domains such as those reviewed by Oxman and colleagues, the current approach is, however, poorly responsive to the needs of country-level policymakers. International recommendations are unable to respond adequately to global diversity in local conditions or reflect the complexity of current knowledge. We propose revolutionary change to the current approach and envisage two key roles for WHO. First, WHO would coordinate the construction of comprehensive and authoritative compilations of systematic reviews designed, populated, and updated by international networks with content and methodological expertise. These “evidence maps” would be structured to incorporate important contextual characteristics, be designed to help users interpret evidence,3 and be accessible as living web-based resources. Second, WHO would support the development of knowledge transfer capacity within countries, enabling norms and standards to be driven by policymakers with understanding of the evidence and local conditions. The EVIPNet network4 and the Regional East African Community Health initiative5 are important early examples of this approach. The time has come to increase the bandwidth of the connection between global knowledge and policy development in WHO member states. WHO should drive decentralisation of knowledge transfer by building global evidence maps and country-level knowledge transfer capacity. RLG is principal investigator of the Global Evidence Mapping Initiative, a not-for-profit, collaborative, university-based research programme that aims to map research evidence in broad clinical areas.
*Julian H Elliott, Russell L Gruen [email protected]
Macfarlane Burnet Institute for Medical Research and Public Health, Melbourne, Victoria 3004, Australia (JHE); and Department of Surgery, University of Melbourne, Melbourne, Victoria, Australia (RLG) 1
2 3
4
5
Oxman A, Lavis J, Fretheim A. Use of evidence in WHO recommendations. Lancet 2007; 369: 1883–89. Hill S, Pang T. Leading by example: a culture change at WHO. Lancet 2007; 369: 1842–44. Gruen RL, Morris PS, McDonald L, Bailie RS. Making systematic reviews more useful for policy-makers. Bull World Health Organ 2005; 83: 480–81. EVIPNet: evidence-informed policy network. http://www.who.int/rpc/evipnet/en/ (accessed May 23, 2007). East African Community. Regional East African Community health-policy initiative. http://www. idrc.ca/uploads/user-S/11551301781REACH_ Prospectus.pdf (accessed May 23, 2007).
Authors’ reply We agree with Justus Krabshuis and colleagues that resources are important and that evidence is not the key issue. But we disagree that resources are the key issue. The key issue is how best to use available resources to achieve health goals. It is not possible to make informed judgments about how best to use scarce resources without evidence to inform those judgments. This includes economic evidence as well as evidence of the effects of relevant options for achieving health goals.1,2 The best way to ensure that recommendations are well informed by the best available evidence is by use of systematic and transparent processes. Without this, it is difficult, if not impossible, to know how much evidence there is to support a recommendation or a cascade of options. Although Donald Thea and Jonathon Simon are correct about the timing of our study, our member checking extended well over a year later, during which time no mention was made of a fundamental shift that diminished the credibility of our findings. We asked explicitly about harms during the interviews, and no one commented on this finding during our elaborate member checking. We agree with Thea and Simon, and with Herbert Peterson and colleagues and Julian Kelly and colleagues, that there are examples of excellence at www.thelancet.com Vol 370 September 8, 2007