- Email: [email protected]
An equity lens can ensure an equity-oriented approach to agenda setting and priority setting of Cochrane Reviews
Journal of Clinical Epidemiology 66 (2013) 511e521
An equity lens can ensure an equity-oriented approach to agenda setting and priority setting of Cochrane Reviews Mona Nassera,*, Erin Ueffingb, Vivian Welchc,d, Peter Tugwellc,d,e a
Cochrane Agenda and Priority Setting Methods Group, Peninsula School of Medicine and Dentistry, Plymouth University, Plymouth PL6 8BU, UK b The Canadian Cochrane Centre, Ottawa Hospital Research Institute, Ottawa, Canada c Campbell and Cochrane Equity Methods Group, Institute of Population Health, University of Ottawa, Ottawa, Canada d Ottawa Hospital Research Institute, Ottawa, Canada e Department of Medicine, University of Ottawa, Ottawa, Canada Accepted 25 November 2012; Published online 9 March 2013
Abstract Objectives: This study aimed to develop and pilot an equity lens that could help researchers in developing a more equity-oriented approach toward priority setting and agenda setting in systematic reviews. Study Design and Setting: We developed an equity lens to guide the development and evaluation of a prioritization process and evaluate its outcomes based on the information derived from a discussion workshop and a comparison with the existing literature on the topic. We piloted the process section of the equity lens across the 13 structured priority-setting approaches in the Cochrane Collaboration. Results: We devised an equity lens with two checklists: one to guide the process of priority setting (nine questions) and the other to evaluate the outcomes of priority setting (eight questions). Of the nine questions, seven questions were partially addressed by at least one of the prioritization projects. Two questions were not considered in any of them. The prioritization projects did not report sufficient outcome data, thus we could not explore the eight question on evaluating outcomes. Conclusion: Currently, there are few strategies in the Cochrane Collaboration that explicitly address the research priorities of individuals from different sociodemographic groups. The equity lens for priority setting and agenda setting can help project teams to develop a more equity-oriented approach to set a research agenda and/or prioritize research topics. However, further studies are needed to evaluate its impact on the prioritization process. Ó 2013 Elsevier Inc. All rights reserved. Keywords: Systematic reviews; Priority setting; The Cochrane Collaboration; Equity; Social determinants of health; Research priorities
1. Introduction Inequity in health has emerged as a leading issue in light of the findings of the Commission on Social Determinants of Health [1]. The Child Health and Nutrition Research Initiative (CHNRI) expresses concerns that some approaches in research investment, for example, allocation of investment in certain research topics, can increase health inequity [2]. The same concern has been raised regarding the topic selection process of systematic reviews, which provide a systematic synthesis of primary studies. In 2005, it was recognized that systematic reviews more often address health problems Conflict of Interest/Financial Disclosure: The project was partially funded by the Cochrane Collaboration Prioritization Fund. All of the authors are active members of the Cochrane Collaboration. The author team was involved in one of the prioritization projects described. There is no other conflict of interest to report. * Corresponding author. Tel. +00441752437384. E-mail address: [email protected] (M. Nasser). 0895-4356/$ - see front matter Ó 2013 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.jclinepi.2012.11.013
that are priorities in high-income countries (HICs), to the neglect of problems in low- and middle-income countries (LMICs) [3,4]. Although strategies have been taken to change this [5,6], there is still a considerable room for improvement, particularly considering the limited resources available in LMICs. A recent review of priority-setting strategies used by the World Health Organization has identified several uncertainties regarding the best methods for setting health research priorities and the need for more guidance on the process of setting research priorities in health care [7]. The Commission on Social Determinants of Health [1] defined health equity as the ‘‘absence of systematic differences in health, both between and within countries that are judged to be avoidable by reasonable action’’ and recommended measuring and understanding health inequity and assessing the results of action on addressing health inequities and inequitable conditions. Some priority-setting initiatives have addressed equity using economic or social approaches and identified effectiveness, ethics, equity, and
512
M. Nasser et al. / Journal of Clinical Epidemiology 66 (2013) 511e521
What is new? Key findings Few strategies in the Cochrane Collaboration explicitly address the research priorities of disadvantaged populations. The equity lens can help project teams to develop a more equity-oriented approach to set a research agenda and/or prioritize research topics. What this adds to what was known? This article introduces an equity lens for priority setting and agenda setting to guide the development of priority-setting processes and evaluate the outcomes of priority setting. What is the implication and what should change now? Innovative approaches are needed to ensure that the research priorities of diverse stakeholders are considered.
answerability as specific criteria that can identify differences between research questions. The Cochrane Collaboration (www.cochrane.org) is an international organization bringing together more than 28,000 clinicians, health care researchers, and consumers from 100 countries to prepare and maintain systematic reviews of the effects of interventions on health and related outcomes and promote the dissemination and accessibility of these reviews. In a survey and subsequent workshop at the 2008 Cochrane Colloquium (an annual gathering of Cochrane contributors), we recognized the need to improve the process of agenda setting and priority setting for Cochrane review topics to ensure more transparency, inclusiveness, and consider health equity. To achieve this goal, it is important to develop a process to identify and, if needed, prioritize interventions that could potentially reduce inequity or health problems that are more relevant to certain disadvantaged groups. This article explores how groups can consider equity in the research priorityesetting process, especially regarding prioritization of systematic reviews. We describe the development of an equity lens that could help researchers to develop a more equity-oriented approach toward priority setting and agenda setting. The ‘‘lens’’ provides a possibility to select, interpret, and use data from a specific position (in this case, to ensure an equity-oriented approach) [10]. We used the equity lens to evaluate the research priorityesetting processes in the Collaboration to identify potential gaps to be addressed in the future. This lens fills a need by introducing a systematic approach to prioritizing topics with a potential impact on reducing inequity in health. Furthermore, it may
increase the proportion of prioritized research questions that address health care interventions that are potentially effective in disadvantaged groups and interventions that can reduce the health equity gap. We describe the advantages and disadvantages of the equity lens and how it can be improved.
2. Methods 2.1. Developing an equity lens for priority setting and agenda setting of systematic reviews The development of the equity lens for priority setting took place in three parts: (1) a workshop presenting survey results from a previous project, (2) a review of the literature, and (3) a final workshop for refinement of the equity lens. In the first workshop (2008), we hosted an interactive workshop during the Cochrane Colloquium. About 15 people participated in the workshop. We presented a survey of research priorityesetting exercises [11], and the group discussed important issues around research priority setting. Participants identified equity in research priority setting as a significant challenge faced by researchers and systematic reviewers. During the second part of the project, we identified key literature on the methods to incorporate equity in research priorityesetting processes through literature searches and contacting experts [8,9]. We also identified tools to inform the development of the lens: (1) an equity tool to evaluate clinical guidelines [13] and (2) the PROGRESS-Plus mnemonic, which is a helpful tool to define different sociodemographic dimensions across which inequities can be assessed. PROGRESS-Plus is an extension of Evans and Brown’s framework PROGRESS (Place of residence, Race/ethnicity, Occupation, Gender, Religion, Education, Socioeconomic status, and Social capital), with ‘‘PLUS’’ representing additional dimensions such as age, sexual orientation, and disability [14e17]. To ensure that the equity lens incorporated aspects of a good research priority-setting process, we first considered the nine unique steps that appeared in research prioritye setting projects in a previous study [11] (Table 1). We selected the conceptual framework by Sibbald et al. [12] for successful health priority setting to cover the major issues to be considered in a good priority-setting process. This framework was developed in a systematic approach for health priority setting. However, it incorporated issues that are generalizable to research priority setting; throughout the remainder of this article, we refer to ‘‘priority setting’’ in the context of research. The framework included process elements (stakeholder engagement, explicit process, information management, consideration of context and values, and revisions or appeal mechanism) and outcome elements (improved stakeholder understanding, shifted priorities and/or reallocated resources, improved decision-making quality, stakeholder acceptance and satisfaction, and positive externalities) [12]. The steps of the priority-setting processes and the elements of the successful priority
Table 1. Process evaluation of prioritization for Cochrane reviewsdsteps, framework for successful priority setting (process), and proposed questions for the equity lens to guide the process of prioritization
Conceptual framework by Sibbald for successful priority setting (process)
Involving different stakeholders in the process of priority setting (in the following steps)
Process element: stakeholder engagement
Defining objectives/scope of the prioritization process Selecting appropriate methods and tools for the project, for example, Delphi consensus
Process element: using explicit process Process element: using explicit process Process element: consideration of values and context
Defining criteria to rank and differentiate between research topics
Process element: using explicit process Process element: consideration of values and context
Situation analysis (needs assessment/scoping review/evidence mapping/environmental scan)
Process element: information management
Reaching consensus on a list of prioritized topics
Process element: using explicit process Process element: consideration of values and context
Dissemination and communication strategy for the results
Process element: using explicit process Process element: consideration of values and context
Proposed questions for the equity lens 1. Are different stakeholders who might be affected by the choice of research (review) topics involved in the prioritization process (different age, sex, sexual orientation, disability, ethnicity, religion, place of residence, occupation, education, socioeconomic status, and social capital groups)? In which steps are they involved? 2. Does the prioritization project consider reducing inequity as part of its objectives? 3. Are the selected methods and tools to identify prioritize, implement, disseminate, and communicate research topics understandable, transparent, and relevant for different stakeholders (different age, sex, sexual orientation, disability, ethnicity, religion, place of residence, occupation, education, socioeconomic status, and social capital groups)? 4. Are specific strategies considered to minimize the barriers to reach disadvantaged or less accessible populations? 6. Do the criteria for prioritization consider the potential differences in the severity and urgency of health problems in disadvantaged populations or less accessible groups as opposed to the health problems in privileged populations? 7. Do the criteria for prioritization consider the potential differences in the impact of a health care intervention in disadvantaged populations as opposed to the health problems in privileged populations? 8. Do the criteria for prioritization consider that different population groups might have different values and preferences? 5. In the stage of situation analysis (evaluating the current health research coverage, identifying gaps, evaluating health care needs, etc), does the analysis consider the differences in the prevalence, severity, and urgency of health problems along with potential differences in the impact or value of the health care interventions intervention assessed across different subgroups (age, sex, sexual orientation, disability, ethnicity, religion, place of residence, occupation, education, socioeconomic status, and social capital)? The following questions which were mentioned previously addressed this step: 2. Does the prioritization project consider reducing inequity as part of its objectives? 3. Are the selected methods and tools to identify prioritize, implement, disseminate, and communicate research topics understandable, transparent, and relevant for different stakeholders (different age, sex, sexual orientation, disability, ethnicity, religion, place of residence, occupation, education, socioeconomic status, and social capital groups)? 4. Are specific strategies considered to minimize the barriers to reach disadvantaged or less accessible populations? Questions 3 and 4 relate to this step The following questions which were mentioned previously addressed this step: 3. Are the selected methods and tools to identify prioritize, implement, disseminate, and communicate research topics understandable, transparent, and relevant for different stakeholders (different age, sex, sexual orientation, disability, ethnicity, religion, place of residence, occupation, education, socioeconomic status, and social capital groups)? 4. Are specific strategies considered to minimize the barriers to reach disadvantaged or less accessible populations?
513
(Continued )
M. Nasser et al. / Journal of Clinical Epidemiology 66 (2013) 511e521
Steps of a prioritization process for Cochrane reviews (Prioritization strategy may include all or some of these steps, but they do not necessarily need to follow exactly the following order)
Process element: using explicit process Process element: consideration of values and context
Process element: using explicit process Process element: consideration of values and context
Implementation, monitoring, and evaluation process
Providing feedback and an opportunity for appeal
Steps of a prioritization process for Cochrane reviews (Prioritization strategy may include all or some of these steps, but they do not necessarily need to follow exactly the following order)
Table 1. Continued
Conceptual framework by Sibbald for successful priority setting (process)
Proposed questions for the equity lens
M. Nasser et al. / Journal of Clinical Epidemiology 66 (2013) 511e521 Questions 3 and 4 relate to this step The following questions which were mentioned previously addressed this step: 3. Are the selected methods and tools to identify prioritize, implement, disseminate, and communicate research topics understandable, transparent, and relevant for different stakeholders (different age, sex, sexual orientation, disability, ethnicity, religion, place of residence, occupation, education, socioeconomic status, and social capital groups)? 4. Are specific strategies considered to minimize the barriers to reach disadvantaged or less accessible populations? 9. Are different stakeholders groups (representing age, sex, sexual orientation, disability, ethnicity, and religion, place of residence, occupation, education, socioeconomic status, and social capital groups) provided with an opportunity to provide feedback and appeal the process and results of the prioritization process?
514
setting were cross-referenced (Figs. 1 and 2). We considered the literature on equity in priority setting as a base for developing questions across these steps. In the last stage of the project, we held a second workshop; this workshop was held at the 2009 Cochrane Colloquium. Twelve participants discussed the first draft of the equity lens; their comments were reflected in the final version. For example, the first version of the equity only considered the process of priority setting. However, we later decided to incorporate outcome evaluation as a part of it. 2.2. Exploring the role of equity in research priorityesetting process in the Cochrane Collaboration using the equity lensda pilot study In our 2008 survey of Cochrane entities, we identified 29 groups that had processes to inform their topic selection. Fifteen groups had a more structured approach; Bone, Joint and Muscle Trauma Group, Consumer Network, Effective Practice and Organisation of Care Group, Ear, Nose and Throat Disorders Group, Eyes and Vision Group, Gynecologic Cancer Group, Incontinence Group, Neonatal Group, Skin Group, Cancer Network, Musculoskeletal Group, Infectious Diseases Group, Public Health Review Group, Equity Methods Group (at time of the survey, it was a field), Health care of Older People Field. As some of the groups worked together, we had 13 unique structured approaches for ranking priority titles for new Cochrane Reviews and for updating existing reviews and/or for review dissemination. We attempted to answer the questions of the equity lens using the available data for each process [11].
3. Results 3.1. Developing the equity lens for priority setting and agenda setting The equity lens helps researchers to approach the priority-setting project with an equity perspective. It could be used in developing a new priority-setting project, prospectively/retrospectively evaluating those processes or comparing different priority-setting methods. It includes two checklists: one focusing on the process of priority setting and the other on the outcome evaluation of the priority setting. The first checklist has nine questions about different steps of priority setting, from identifying the questions and stakeholders to the evaluation strategy. The questions consider the different process elements of a successful priority-setting project. Table 1 and Fig. 2 demonstrate how these different elements are related. The second checklist focuses on the outcome evaluation and has eight questions. These questions address the final steps in the priority-setting cycle, after defining the ranked research topics. To ensure that these questions incorporate the components of a successful priority-setting exercise, they were mapped across the different elements of outcome
M. Nasser et al. / Journal of Clinical Epidemiology 66 (2013) 511e521
515
Using explicit process Q.2
Defining Objectives/ scope Involving stakeholders
Selecting appropriate methods and tools
Implementation, monitoring, evaluation
Q.3,4
Q.3,4
Providing feedback and appeal
Defining criteria to rank and differentiate between topics
Dissemination and communication strategy
Reaching consensus on the prioritized topics
Situation analysis (scoping /mapping/ need assessment)
Fig. 1. Process evaluation of prioritization for Cochrane reviewsdsteps, framework for successful priority setting (process), and proposed questions for the equity lens to guide the process of prioritization.
evaluation as identified in the framework by Sibbald et al. [12] for priority setting (Table 2 and Fig. 2). 3.2. Exploring the role of equity in research priorityesetting processes in the Cochrane Collaboration using the equity lensda pilot study We answered the questions in the equity lens for each of the 13 priority-setting approaches identified in our survey [11]. Most of the processes did not collect outcome data; therefore, we could not assess outcomes. Thus, we concentrated on process evaluation, that is, the first nine questions of the equity lens. We report each of these in the following. 1. Are different stakeholders who might be affected by the choice of review topics involved in the prioritization process (different age, sex, sexual orientation, disability, ethnicity, religion, place of residence, occupation, education, socioeconomic status, and social capital groups)? Four of the prioritization projects targeted specific stakeholder groups: one targeted consumers, two targeted policy makers, and one targeted both clinicians and patients
(through involving patient organizations and clinical associations). The remaining projects aimed to identify and prioritize important questions in health care and did not aim specifically to identify the priorities of stakeholder groups. One group tried to involve stakeholders from different countries, different ethnic backgrounds, and both genders, whereas another tried to involve diverse stakeholders by providing the survey in English and Spanish and contacting consumer advocates in both LMICs and HICs. One group collaborated with a group focused on a particular demographic to ensure that the demographic was appropriately represented and easily identified. The other groups did not recruit strategically to increase stakeholder diversity (as defined by the PROGRESS-Plus mnemonic). However, two projects had an open online survey, which provides the opportunity for different stakeholders to be involved if it is disseminated widely. One of these collected data on age, sex, country, occupation, and the participants’ role in health care. 2. Does the prioritization project consider reducing inequity as part of its objectives? Only one project specifically aimed to introduce health equity and the social determinants of health into the
516
M. Nasser et al. / Journal of Clinical Epidemiology 66 (2013) 511e521 ance
cept
r ac olde
eh
Stak
Defining Objectives/ scope
-
Providing feedback and appeal
Involving stakeholders
Selecting appropriate methods and tools
Implementation, monitoring, evaluation
Defining criteria to rank and differentiate between topics
Dissemination and communication strategy
Reaching consensus on the prioritized topics
Situation analysis (scoping /mapping/ need assessment)
Fig. 2. Outcome evaluation of prioritization for Cochrane reviewsdsteps, framework for successful priority setting (process), and proposed questions for the equity lens to guide the evaluation of the outcomes of an equity-oriented prioritization project.
prioritization process. Two groups emphasized that they would try to include the priorities of individuals based in LMICs. However, these two groups had a broader focus; stakeholders from HICs were also included. 3 and 4. Are the selected methods and tools to identify, prioritize, implement, disseminate, and communicate research topics understandable and transparent for different stakeholders (groups from different age, sex, sexual orientation, disability, ethnicity, religion, place of residence, occupation, education, socioeconomic status, and social capital)? Are specific strategies considered to minimize the barriers to reach disadvantaged or less accessible populations? Two projects used online surveys. In one, the survey was provided only in English, whereas the other survey was available in both English and Spanish. Another project, which aimed to involve patients and clinicians, made specific attempts to ensure that the provided information was understandable for both groups; patients and clinicians could share their opinions during the prioritization meetings.
5. In the stage of situation analysis (evaluating the current health research coverage, identifying gaps, evaluating health care needs, etc.), does the analysis consider the differences in the prevalence, severity, and urgency of health problems along with potential differences in the impact or value of the health care interventions assessed across different subgroups (age, sex, sexual orientation, disability, ethnicity, religion, place of residence, occupation, education, socioeconomic status, and social capital)? Two projects considered the potential differences between subgroups in mapping the evidence as basis information for setting a research agenda and prioritization. In one project, they searched for evidence and categorized the identified studies from the search according to stages of the condition, consequences of ill health, and social determinants potentially affecting the condition. These social determinants were daily living conditions (healthy behaviors, diet, and nutrition), physical and social environment, working conditions, health care, social protection, structural drivers (gender, social inequities, and socialepolitical
M. Nasser et al. / Journal of Clinical Epidemiology 66 (2013) 511e521
517
context), and macroeconomic conditions. Another project used the WHO Health Report 2002 [18,19] as a basis for categorizing the identified literature that can demonstrate the differences in severity and prevalence between LMICs and HICs.
9. Are different stakeholder groups (from different age, sex, sexual orientation, disability, ethnicity, religion, place of residence, occupation, education, socioeconomic status, and social capital) provided with an opportunity to provide feedback and appeal the results?
6. Do the criteria for prioritization consider the potential differences in the severity and urgency of health problems in disadvantaged populations or less accessible groups as opposed to the health problems in privileged populations?
None of the processes ensured that different sociodemographic groups had the opportunity to provide feedback and appeal the results.
In one group, the priority-setting group considered five dimensions in setting priority research topics. Two of the dimensions were relevant to this question: 1. Equity: Would you say that the question addresses the health needs of the underprivileged or disadvantaged groups? 2. Social determinants: Would you say the question addresses the health needs of populations across different socioeconomic gradients? The questions needed to be ranked as ‘‘yes’’ for equity or social determinants (along with other criteria) to be considered a high priority question. One group considered burden of disease, magnitude of problem, and urgency as criteria. These criteria do not directly emphasize the differences between disadvantaged populations and others. However, the project aimed to include the public health priorities of developing countries along with those from developed countries. Depending on how the panel interpreted these criteria, this might have led to the inclusion of priority questions that addressed differences across countries in severity and urgency of the problem. Two groups considered the priorities of a health care system as one of the criteria. Depending on how these priorities were developed, these health system priorities may have considered differences in severity and urgency for disadvantaged populations. The latter argument also applied for questions 6 and 7. 7. Do the criteria for prioritization consider the potential differences in the impact of a health care intervention in disadvantaged populations or less accessible groups as opposed to the health problems in privileged populations? One of the groups considered a criterion that partially addresses this question: potential impact of the intervention on changing policy or treatment especially in areas of the United Nations Millennium Development Goals. 8. Do the criteria for prioritization consider that different population groups might have different values and preferences? None of preferences.
the
processes
considered
values
and
4. Discussion We developed an equity lens with 17 questions to guide the prioritization of topics for systematic reviews of health care interventions. These questions could potentially increase the proportion of prioritized research topics that aim to reduce the health equity gap and that include health care interventions targeted for disadvantaged groups. We can more easily assess the impact of the equity lens if project teams incorporate process and outcome evaluations in their priority-setting projects. Addressing all the issues raised might not be possible in every prioritization process as it might not be feasible, the resources may not be available, or the project has a very specific aim/objective. However, explicit reporting of decisions about each of these steps can potentially help research teams in improving their process. The importance of equity in priority-setting processes was emphasized in several articles. However, the current literature focuses on incorporating equity in certain priority-setting methods and processes. Our equity lens is unique as it provides a broader framework that can be adapted for any priority-setting method. 4.1. Equity in the process of setting research topics and prioritization in the Cochrane Collaboration To judge this, we considered two questions: (1) did the prioritization project take a systematic approach to prioritizing topics with a potential impact on reducing inequity in health? and (2) did the project report describe the extent to which they achieved this? Two of the projects used an open online survey that was accessible and understandable for any (English-speaking) person who had access to a computer. One project used translation to include non-English speakers. However, their process would be strengthened if they had reported demographic data (preferably the sociodemographic variables of the PROGRESS-Plus mnemonic) so that the users of the research priorityesetting exercise could judge the diversity of the respondents involved in setting priorities. Prioritization projects need to be reported more transparently and comprehensively. Cochrane Collaboration groups can support Cochrane entities considering different social dimensions in their prioritization processes; these groups have expertise in methods and issues related to specific social groups. These include the Cochrane Child Health Field, Cochrane Health
Relevant steps of a prioritization process for Cochrane reviews
Conceptual framework by Sibbald for successful priority setting (outcomes)
Reaching consensus on a list of prioritized topics Dissemination and communication strategy for the results Implementation, monitoring, and evaluation process
Shifted priorities and/or reallocated resources Shifted priorities and/or reallocated resources
Improved decision-making quality
Providing feedback and an opportunity for appeal
Improved decision-making quality Positive externalities Stakeholder acceptance and satisfaction
Proposed questions for the equity lens 1. Did the prioritization result in more research topics (in this case Cochrane reviews) that are relevant to disadvantaged groups? 2. Did the dissemination and implementation strategy increase the likelihood that funders and research institutes become aware of the prioritized research topics and consider them as part of their research agenda or strategic planning? 3. Did the dissemination and implementation strategy increase the likelihood that the prioritized research topics that are relevant to disadvantaged groups get funded and conducted? 4. Did the dissemination and implementation strategy increase the likelihood that researchers who work with disadvantaged groups conduct or get involved in the prioritized research projects (in this case, the research project is a review of the Cochrane systematic review) 5. Did the dissemination and implementation strategy increase the likelihood that disadvantaged groups or decision makers or practitioners who work with disadvantaged groups get involved in the prioritized research topics? 6. Does the dissemination and implementation strategy increase the likelihood that policy makers and decisions makers who work with disadvantaged groups use the result of the prioritized research topics? 7. Did the results of the prioritized research topics changed policies, legislation, or clinical practice in favor of disadvantaged groups? 8. Did the appeal and enforcement strategy increase the likelihood that disadvantaged groups or decision makers, researchers, and practitioners who work with disadvantaged group had provided feedback and comments on the prioritization process or results?
Table 3. Equity lens for agenda setting and research priority setting (Part 1 e process) Equity lens 1. Are different stakeholders who might be affected by the choice of research (review) topics involved in the prioritization process (different age, sex, sexual orientation, disability, ethnicity, and religion, place of residence, occupation, education, socioeconomic status, and social capital groups)? In which steps are they involved? 2. Does the prioritization project consider reducing inequity as part of its objectives? 3. Are the selected methods and tools to identify prioritize, implement, disseminate, and communicate research topics understandable, transparent and relevant for different stakeholders (different age, sex, sexual orientation, disability, ethnicity, religion, place of residence, occupation, education, socioeconomic status, and social capital groups)?
The strategies taken in the Cochrane Collaboration to address the questiona Role of the Stakeholders recruited to be involved
1 group targeted consumers, 2 groups targeted policy makers, 1 group targeted both clinicians and patients Context and Background in recruiting 1 group involved stakeholders from different countries, different Stakeholders ethnic backgrounds, and both genders, 1 group involved diverse stakeholders by providing the survey in English and Spanish 1 group collaborated with a group focussed on a particular demographic to ensure that they are appropriately represented. 1 group (only) specifically aimed to introduce health equity and the social determinants of health into the prioritisation process. 2 groups emphasized that they would try to include the priorities of individuals based in LMICs (among others). 2 projects used online surveys a) one survey was provided only in English; b) the second one was available in both English and Spanish. 1 project, which aimed to involve patients and clinicians, made specific attempts to ensure that the provided information was understandable for both groups;
M. Nasser et al. / Journal of Clinical Epidemiology 66 (2013) 511e521
Shifted priorities and/or reallocated resources Improved stakeholder understanding Shifted priorities and/or reallocated resources Improved stakeholder understanding
518
Table 2. Outcome evaluation of prioritization for Cochrane reviewsdsteps, framework for successful priority setting (process), and proposed questions for the equity lens to guide the evaluation of the outcomes of an equity-oriented prioritization project
1 project searched for evidence and categorized the identified studies from the search according to stages of the condition, consequences of ill health, and social determinants potentially affecting the condition. 1 project used the WHO Health Report 2002 (18, 19) as a basis for categorizing the identified literature that can demonstrate the differences in severity and prevalence between Low and Middle Income countries (LMIC) and High Income countries (HICs).
1 group, the priority-setting group considered five dimensions in setting priority research topics. Two of them were relevant to this question. In the below ones, depending on how the panel interpreted these criteria, this might have led to the inclusion of priority questions that addressed question 6. 1 group considered burden of disease, magnitude of problem, and urgency as criteria. 2 groups considered the priorities of a health care system as one of the criteria. 1 group considered a criterion that partially addresses this question: potential impact of the intervention on changing policy or treatment especially in areas of the United Nations Millennium Development Goals (MDGs).
None of the processes considered values and preferences.
None of the processes ensured that different socio-demographic groups had the opportunity to provide feedback and appeal the results
M. Nasser et al. / Journal of Clinical Epidemiology 66 (2013) 511e521
4. Are specific strategies considered to minimize the barriers to reach disadvantaged or less accessible populations? 5. In the stage of situation analysis (evaluating the current health research coverage, identifying gaps, evaluating healthcare needs, etc.), does the analysis consider the differences in the prevalence, severity and urgency of health problems along with potential differences in the impact or value of the health care interventions assessed across different subgroups (age, sex, sexual orientation, disability, ethnicity, religion, place of residence, occupation, education, socioeconomic status, and social capital)? 6. Do the criteria for prioritization consider the potential differences in the severity and urgency of health problems in disadvantaged populations or less accessible groups as opposed to the health problems in privileged populations? 7. Do the criteria for prioritization consider the potential differences in the impact of a health care intervention in disadvantaged populations as opposed to the health problems in privileged populations? 8. Do the criteria for prioritisation consider that different population groups might have different values and preferences? 9. Are different stakeholders groups (representing age, sex, sexual orientation, disability, ethnicity, and religion, place of residence, occupation, education, socioeconomic status, and social capital groups) provided with an opportunity to provide feedback and appeal the process and results of the prioritisation process?
a We contacted 66 Cochrane entities (Cochrane Review groups, Fields and Network). Fifty two of them responded. Of these, 29 had an existing priority-setting process, however, 15 had a more structured approach. Two of these were joint initiatives, therefore, there were a total of 13 unique structured prioritization projects. The priority setting approaches used by these 13 groups included in the table can be categorized as follows:
(1) Approaches to identify priority research topics: (a) Collecting suggestions from different stakeholders (b) Priority topics in health care; using health status data e.g. incidence. Content analysis of policy reports (c) Recommendation from clinical guidelines and systematic reviews (d) Evaluation of existing evidence coverage (2) Approaches to rank priority research topics (a) online survey - Delphi consensus (b) face to face groups - one of them used a systematic approach (nominated group approach)
519
520
M. Nasser et al. / Journal of Clinical Epidemiology 66 (2013) 511e521
Table 4. Equity lens for agenda setting and research priority setting (Part 2 e Outcomes) (none of the research priority setting exercises were in a stage to collect data on outcome measurements so we couldn’t evaluate them based on this perspective) 1. Did the prioritisation result in more research topics (in this case Cochrane reviews) that are relevant to disadvantaged groups? 2. Did the dissemination and implementation strategy increase the likelihood that funders and research institutes become aware of the prioritised research topics and consider them as part of their research agenda or strategic planning? 3. Did the dissemination and implementation strategy increase the likelihood that the prioritised research topics that are relevant to disadvantaged groups get funded and conducted? 4. Did the dissemination and implementation strategy increase the likelihood that researchers who work with disadvantaged groups conduct or get involved in the prioritised research projects (in this case the research project is a Cochrane systematic review review) 5. Did the dissemination and implementation strategy increase the likelihood that disadvantaged groups or decision makers or practitioners who work with disadvantaged groups get involved in the prioritised research topics? 6. Does the dissemination and implementation strategy increase the likelihood that policy makers and decisions makers who work with disadvantaged groups use the result of the prioritised research topics? 7. Did the results of the prioritised research topics changed policies, legislation or clinical practice in favour of disadvantaged groups? 8. Did the appeal and enforcement strategy increase the likelihood that disadvantaged groups or decision makers, researchers and practitioners who work with disadvantaged group had provided feedback and comments on the prioritisation process or results?
Care of Older People Field, and Campbell and Cochrane Equity Methods Group. Only two of the priority-setting processes involved these groups. 4.2. Strengths The current equity lens for priority setting and agenda setting helps project teams to focus on health equity considerations in agenda setting; it encourages consideration of the needs of and impact for disadvantaged groups at each step of priority setting. It was based on a review of the prioritization processes of 66 Cochrane entities and a review of priority-setting methods available to set agendas for research topics [11]. 4.3. Limitations We did not demonstrate whether the prioritization projects achieved the goal of prioritizing topics likely to reduce health inequity because we did not assess outcomes. Evaluation of outcomes using the items in Tables 3 and 4 is needed to answer this question. The tool also does not consider the impact of contextual factors, for example, considering the transferability of the intervention to different contexts. 4.4. Future research We will work with Cochrane entities to collect data on the outcomes of their prioritization process and will pilot the outcome section of the equity lens for priority setting and agenda setting (Tables 3 and 4) to evaluate whether the methods and process can have an impact on reducing in equity in health.
5. Conclusion The equity lens for priority setting and agenda setting can help project teams to develop a more equity-oriented approach to set a research agenda and/or prioritize research topics. It could be used in developing a new priority-setting
project, prospectively/retrospectively evaluating those processes, or comparing different priority-setting methods. Currently, few strategies in the Cochrane Collaboration explicitly address the research priorities of different sociodemographic groups. We need innovative approaches to ensure that the priorities of these diverse groups and stakeholders are considered. Moreover, project teams must establish mechanisms for continuity of the priority setting and determine the process and timeline for updating priorities. In future projects, we will revise the equity lens for priority setting and agenda setting to incorporate the impact of different contexts and environments on deciding on research priorities and will develop tools that will guide researchers whether they achieved the objectives of the equity lens for priority setting and agenda setting. We will also work with Cochrane entities to collect the necessary data to evaluate the outcomes of the prioritization strategy (Table 3). There is a need for guidelines for conducting and reporting prioritization strategies. References [1] Commission on Social Determinants of Health. Closing the gap in a generation: health equity through action on the social determinants of health. Final Report of the Commission on Social Determinants of Health. Geneva: World Health Organization; 2008. [2] Rudan I, Gibson J, Kapiriri L, Lansang MA, Hyder AA, Lawn J, et al, Child Health and Nutrition Research Initiative (CHNRI). Setting priorities in global child health research investments: assessment of principles and practice. Croat Med J 2007;48:595e604. [3] Chinnock P, Siegfried N, Clarke M. Is evidence-based medicine relevant to the developing world?: systematic reviews have yet to achieve their potential as a resource for practitioners in developing countries. Evid Based Complement Alternat Med 2005;2:321e4. [4] Swingler GH, Volmink J, Ioannidis JPA. Number of published systematic reviews and global burden of disease: database analysis. BMJ 2003;327:1083e4. [5] Ortiz Z, Nasser M, Wang Li, Li Y, Pardo J, Bonfill X. (2007) The Cochrane Developing Countries Network: an initiative to improve equitable access to knowledge and information and to its management, Forum 11, Global Forum of Health Research. Beijing 29 October-2 November 2007. [6] Waters E, Doyle J. Systematic reviews of public health in developing countries are in train. BMJ 2004;328:585.
M. Nasser et al. / Journal of Clinical Epidemiology 66 (2013) 511e521 [7] Viergever RF. Health Research prioritisation at WHO. An overview of methodology and high level analysis of WHO led health research priority setting exercises. World Health Organization; 2010. Available at http://www.who.int/rpc/publications/Health_research_prioritisation_ at_WHO.pdf. Accessed February 13, 2013. [8] Tomlinson M, Rudan I, Saxena S, Swartz L, Tsai AC, Patel V. Setting priorities for global mental health research. Bull World Health Organ 2009;87:438e46. [9] Rudan I, Chopra M, Kapiriri L, Gibson J, Ann Lansang M, Carneiro I, et al. Setting priorities in global child health research investments: universal challenges and conceptual framework. Croat Med J 2008a;49:307e17. [10] Doucet A, Mauthner N. Knowing responsibly: linking ethics, research practice and epistemology. In: Mauthner M, Birch M, Jessop J, Iller T, editors. Ethics in qualitative research. London: Sage Publications; 2002. [11] Nasser M, Welch V, Tugwell P, Ueffing E, Doyle J, Waters E. Ensuring relevance for Cochrane reviews: evaluating processes and methods for prioritizing topics for Cochrane reviews. J Clin Epidemiol 2013;66:474e82 [in this issue]. [12] Sibbald SL, Singer PA, Upshur R, Martin DK. Priority setting: what constitutes success? A conceptual framework for successful priority setting. BMC Health Serv Res 2009;9:43. [13] Dans AM, Dans L, Oxman AD, Robinson V, Acuin J, Tugwell P, et al. Assessing equity in clinical practice guidelines. J Clin Epidemiol 2007;60:540e6.
521
[14] Evans T, Brown H. Road traffic crashes: operationalizing equity in the context of health sector reform. Inj Control Saf Promot 2003;10:11e2. [15] Tugwell P, Maxwell L, Welch V, Kristjansson E, Petticrew M, Wells G, et al. Is health equity considered in systematic reviews of the Cochrane Musculoskeletal Group? Arthritis Rheum 2008;59:1603e10. [16] National Institute for Health and Clinical Excellence (NICE). Methods for the development of NICE public health guidance (second edition). Issue date: April 2009. Available at http://www.nice. org.uk/media/2FB/53/PHMethodsManual110509.pdf. Accessed September 14, 2009. [17] Oliver S, Kavanagh J, Caird J, Lorenc T, Oliver K, Harden A, et al. Health promotion, inequalities and young people’s health: a systematic review of research. London: EPPI-Centre, Social Science Research Unit, Institute of Education, University of London; 2008:Available at http://eppi.ioe.ac.uk/cms/LinkClick.aspx?filetic ket5lsYdLJP8gBI%3d&tabid52410&mid54471&language5en-US: Accessed September 14, 2009. [18] World Health Organization (WHO). The World Health Report. Reducing risk, promoting healthy life; 2002. Available at http://www. who.int/whr/2002/en/whr02_en.pdf. Accessed August 24, 2010. [19] Doyle J, Waters E, Yach D, McQueen D, De Francisco A, Stewart T, et al. Global priority setting for Cochrane systematic reviews of health promotion and public health research. J Epidemiol Community Health 2005;59:193e7.
An equity lens can ensure an equity-oriented approach to agenda setting and priority setting of Cochrane Reviews Mona Nassera,*, Erin Ueffingb, Vivian Welchc,d, Peter Tugwellc,d,e a
Cochrane Agenda and Priority Setting Methods Group, Peninsula School of Medicine and Dentistry, Plymouth University, Plymouth PL6 8BU, UK b The Canadian Cochrane Centre, Ottawa Hospital Research Institute, Ottawa, Canada c Campbell and Cochrane Equity Methods Group, Institute of Population Health, University of Ottawa, Ottawa, Canada d Ottawa Hospital Research Institute, Ottawa, Canada e Department of Medicine, University of Ottawa, Ottawa, Canada Accepted 25 November 2012; Published online 9 March 2013
Abstract Objectives: This study aimed to develop and pilot an equity lens that could help researchers in developing a more equity-oriented approach toward priority setting and agenda setting in systematic reviews. Study Design and Setting: We developed an equity lens to guide the development and evaluation of a prioritization process and evaluate its outcomes based on the information derived from a discussion workshop and a comparison with the existing literature on the topic. We piloted the process section of the equity lens across the 13 structured priority-setting approaches in the Cochrane Collaboration. Results: We devised an equity lens with two checklists: one to guide the process of priority setting (nine questions) and the other to evaluate the outcomes of priority setting (eight questions). Of the nine questions, seven questions were partially addressed by at least one of the prioritization projects. Two questions were not considered in any of them. The prioritization projects did not report sufficient outcome data, thus we could not explore the eight question on evaluating outcomes. Conclusion: Currently, there are few strategies in the Cochrane Collaboration that explicitly address the research priorities of individuals from different sociodemographic groups. The equity lens for priority setting and agenda setting can help project teams to develop a more equity-oriented approach to set a research agenda and/or prioritize research topics. However, further studies are needed to evaluate its impact on the prioritization process. Ó 2013 Elsevier Inc. All rights reserved. Keywords: Systematic reviews; Priority setting; The Cochrane Collaboration; Equity; Social determinants of health; Research priorities
1. Introduction Inequity in health has emerged as a leading issue in light of the findings of the Commission on Social Determinants of Health [1]. The Child Health and Nutrition Research Initiative (CHNRI) expresses concerns that some approaches in research investment, for example, allocation of investment in certain research topics, can increase health inequity [2]. The same concern has been raised regarding the topic selection process of systematic reviews, which provide a systematic synthesis of primary studies. In 2005, it was recognized that systematic reviews more often address health problems Conflict of Interest/Financial Disclosure: The project was partially funded by the Cochrane Collaboration Prioritization Fund. All of the authors are active members of the Cochrane Collaboration. The author team was involved in one of the prioritization projects described. There is no other conflict of interest to report. * Corresponding author. Tel. +00441752437384. E-mail address: [email protected] (M. Nasser). 0895-4356/$ - see front matter Ó 2013 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.jclinepi.2012.11.013
that are priorities in high-income countries (HICs), to the neglect of problems in low- and middle-income countries (LMICs) [3,4]. Although strategies have been taken to change this [5,6], there is still a considerable room for improvement, particularly considering the limited resources available in LMICs. A recent review of priority-setting strategies used by the World Health Organization has identified several uncertainties regarding the best methods for setting health research priorities and the need for more guidance on the process of setting research priorities in health care [7]. The Commission on Social Determinants of Health [1] defined health equity as the ‘‘absence of systematic differences in health, both between and within countries that are judged to be avoidable by reasonable action’’ and recommended measuring and understanding health inequity and assessing the results of action on addressing health inequities and inequitable conditions. Some priority-setting initiatives have addressed equity using economic or social approaches and identified effectiveness, ethics, equity, and
512
M. Nasser et al. / Journal of Clinical Epidemiology 66 (2013) 511e521
What is new? Key findings Few strategies in the Cochrane Collaboration explicitly address the research priorities of disadvantaged populations. The equity lens can help project teams to develop a more equity-oriented approach to set a research agenda and/or prioritize research topics. What this adds to what was known? This article introduces an equity lens for priority setting and agenda setting to guide the development of priority-setting processes and evaluate the outcomes of priority setting. What is the implication and what should change now? Innovative approaches are needed to ensure that the research priorities of diverse stakeholders are considered.
answerability as specific criteria that can identify differences between research questions. The Cochrane Collaboration (www.cochrane.org) is an international organization bringing together more than 28,000 clinicians, health care researchers, and consumers from 100 countries to prepare and maintain systematic reviews of the effects of interventions on health and related outcomes and promote the dissemination and accessibility of these reviews. In a survey and subsequent workshop at the 2008 Cochrane Colloquium (an annual gathering of Cochrane contributors), we recognized the need to improve the process of agenda setting and priority setting for Cochrane review topics to ensure more transparency, inclusiveness, and consider health equity. To achieve this goal, it is important to develop a process to identify and, if needed, prioritize interventions that could potentially reduce inequity or health problems that are more relevant to certain disadvantaged groups. This article explores how groups can consider equity in the research priorityesetting process, especially regarding prioritization of systematic reviews. We describe the development of an equity lens that could help researchers to develop a more equity-oriented approach toward priority setting and agenda setting. The ‘‘lens’’ provides a possibility to select, interpret, and use data from a specific position (in this case, to ensure an equity-oriented approach) [10]. We used the equity lens to evaluate the research priorityesetting processes in the Collaboration to identify potential gaps to be addressed in the future. This lens fills a need by introducing a systematic approach to prioritizing topics with a potential impact on reducing inequity in health. Furthermore, it may
increase the proportion of prioritized research questions that address health care interventions that are potentially effective in disadvantaged groups and interventions that can reduce the health equity gap. We describe the advantages and disadvantages of the equity lens and how it can be improved.
2. Methods 2.1. Developing an equity lens for priority setting and agenda setting of systematic reviews The development of the equity lens for priority setting took place in three parts: (1) a workshop presenting survey results from a previous project, (2) a review of the literature, and (3) a final workshop for refinement of the equity lens. In the first workshop (2008), we hosted an interactive workshop during the Cochrane Colloquium. About 15 people participated in the workshop. We presented a survey of research priorityesetting exercises [11], and the group discussed important issues around research priority setting. Participants identified equity in research priority setting as a significant challenge faced by researchers and systematic reviewers. During the second part of the project, we identified key literature on the methods to incorporate equity in research priorityesetting processes through literature searches and contacting experts [8,9]. We also identified tools to inform the development of the lens: (1) an equity tool to evaluate clinical guidelines [13] and (2) the PROGRESS-Plus mnemonic, which is a helpful tool to define different sociodemographic dimensions across which inequities can be assessed. PROGRESS-Plus is an extension of Evans and Brown’s framework PROGRESS (Place of residence, Race/ethnicity, Occupation, Gender, Religion, Education, Socioeconomic status, and Social capital), with ‘‘PLUS’’ representing additional dimensions such as age, sexual orientation, and disability [14e17]. To ensure that the equity lens incorporated aspects of a good research priority-setting process, we first considered the nine unique steps that appeared in research prioritye setting projects in a previous study [11] (Table 1). We selected the conceptual framework by Sibbald et al. [12] for successful health priority setting to cover the major issues to be considered in a good priority-setting process. This framework was developed in a systematic approach for health priority setting. However, it incorporated issues that are generalizable to research priority setting; throughout the remainder of this article, we refer to ‘‘priority setting’’ in the context of research. The framework included process elements (stakeholder engagement, explicit process, information management, consideration of context and values, and revisions or appeal mechanism) and outcome elements (improved stakeholder understanding, shifted priorities and/or reallocated resources, improved decision-making quality, stakeholder acceptance and satisfaction, and positive externalities) [12]. The steps of the priority-setting processes and the elements of the successful priority
Table 1. Process evaluation of prioritization for Cochrane reviewsdsteps, framework for successful priority setting (process), and proposed questions for the equity lens to guide the process of prioritization
Conceptual framework by Sibbald for successful priority setting (process)
Involving different stakeholders in the process of priority setting (in the following steps)
Process element: stakeholder engagement
Defining objectives/scope of the prioritization process Selecting appropriate methods and tools for the project, for example, Delphi consensus
Process element: using explicit process Process element: using explicit process Process element: consideration of values and context
Defining criteria to rank and differentiate between research topics
Process element: using explicit process Process element: consideration of values and context
Situation analysis (needs assessment/scoping review/evidence mapping/environmental scan)
Process element: information management
Reaching consensus on a list of prioritized topics
Process element: using explicit process Process element: consideration of values and context
Dissemination and communication strategy for the results
Process element: using explicit process Process element: consideration of values and context
Proposed questions for the equity lens 1. Are different stakeholders who might be affected by the choice of research (review) topics involved in the prioritization process (different age, sex, sexual orientation, disability, ethnicity, religion, place of residence, occupation, education, socioeconomic status, and social capital groups)? In which steps are they involved? 2. Does the prioritization project consider reducing inequity as part of its objectives? 3. Are the selected methods and tools to identify prioritize, implement, disseminate, and communicate research topics understandable, transparent, and relevant for different stakeholders (different age, sex, sexual orientation, disability, ethnicity, religion, place of residence, occupation, education, socioeconomic status, and social capital groups)? 4. Are specific strategies considered to minimize the barriers to reach disadvantaged or less accessible populations? 6. Do the criteria for prioritization consider the potential differences in the severity and urgency of health problems in disadvantaged populations or less accessible groups as opposed to the health problems in privileged populations? 7. Do the criteria for prioritization consider the potential differences in the impact of a health care intervention in disadvantaged populations as opposed to the health problems in privileged populations? 8. Do the criteria for prioritization consider that different population groups might have different values and preferences? 5. In the stage of situation analysis (evaluating the current health research coverage, identifying gaps, evaluating health care needs, etc), does the analysis consider the differences in the prevalence, severity, and urgency of health problems along with potential differences in the impact or value of the health care interventions intervention assessed across different subgroups (age, sex, sexual orientation, disability, ethnicity, religion, place of residence, occupation, education, socioeconomic status, and social capital)? The following questions which were mentioned previously addressed this step: 2. Does the prioritization project consider reducing inequity as part of its objectives? 3. Are the selected methods and tools to identify prioritize, implement, disseminate, and communicate research topics understandable, transparent, and relevant for different stakeholders (different age, sex, sexual orientation, disability, ethnicity, religion, place of residence, occupation, education, socioeconomic status, and social capital groups)? 4. Are specific strategies considered to minimize the barriers to reach disadvantaged or less accessible populations? Questions 3 and 4 relate to this step The following questions which were mentioned previously addressed this step: 3. Are the selected methods and tools to identify prioritize, implement, disseminate, and communicate research topics understandable, transparent, and relevant for different stakeholders (different age, sex, sexual orientation, disability, ethnicity, religion, place of residence, occupation, education, socioeconomic status, and social capital groups)? 4. Are specific strategies considered to minimize the barriers to reach disadvantaged or less accessible populations?
513
(Continued )
M. Nasser et al. / Journal of Clinical Epidemiology 66 (2013) 511e521
Steps of a prioritization process for Cochrane reviews (Prioritization strategy may include all or some of these steps, but they do not necessarily need to follow exactly the following order)
Process element: using explicit process Process element: consideration of values and context
Process element: using explicit process Process element: consideration of values and context
Implementation, monitoring, and evaluation process
Providing feedback and an opportunity for appeal
Steps of a prioritization process for Cochrane reviews (Prioritization strategy may include all or some of these steps, but they do not necessarily need to follow exactly the following order)
Table 1. Continued
Conceptual framework by Sibbald for successful priority setting (process)
Proposed questions for the equity lens
M. Nasser et al. / Journal of Clinical Epidemiology 66 (2013) 511e521 Questions 3 and 4 relate to this step The following questions which were mentioned previously addressed this step: 3. Are the selected methods and tools to identify prioritize, implement, disseminate, and communicate research topics understandable, transparent, and relevant for different stakeholders (different age, sex, sexual orientation, disability, ethnicity, religion, place of residence, occupation, education, socioeconomic status, and social capital groups)? 4. Are specific strategies considered to minimize the barriers to reach disadvantaged or less accessible populations? 9. Are different stakeholders groups (representing age, sex, sexual orientation, disability, ethnicity, and religion, place of residence, occupation, education, socioeconomic status, and social capital groups) provided with an opportunity to provide feedback and appeal the process and results of the prioritization process?
514
setting were cross-referenced (Figs. 1 and 2). We considered the literature on equity in priority setting as a base for developing questions across these steps. In the last stage of the project, we held a second workshop; this workshop was held at the 2009 Cochrane Colloquium. Twelve participants discussed the first draft of the equity lens; their comments were reflected in the final version. For example, the first version of the equity only considered the process of priority setting. However, we later decided to incorporate outcome evaluation as a part of it. 2.2. Exploring the role of equity in research priorityesetting process in the Cochrane Collaboration using the equity lensda pilot study In our 2008 survey of Cochrane entities, we identified 29 groups that had processes to inform their topic selection. Fifteen groups had a more structured approach; Bone, Joint and Muscle Trauma Group, Consumer Network, Effective Practice and Organisation of Care Group, Ear, Nose and Throat Disorders Group, Eyes and Vision Group, Gynecologic Cancer Group, Incontinence Group, Neonatal Group, Skin Group, Cancer Network, Musculoskeletal Group, Infectious Diseases Group, Public Health Review Group, Equity Methods Group (at time of the survey, it was a field), Health care of Older People Field. As some of the groups worked together, we had 13 unique structured approaches for ranking priority titles for new Cochrane Reviews and for updating existing reviews and/or for review dissemination. We attempted to answer the questions of the equity lens using the available data for each process [11].
3. Results 3.1. Developing the equity lens for priority setting and agenda setting The equity lens helps researchers to approach the priority-setting project with an equity perspective. It could be used in developing a new priority-setting project, prospectively/retrospectively evaluating those processes or comparing different priority-setting methods. It includes two checklists: one focusing on the process of priority setting and the other on the outcome evaluation of the priority setting. The first checklist has nine questions about different steps of priority setting, from identifying the questions and stakeholders to the evaluation strategy. The questions consider the different process elements of a successful priority-setting project. Table 1 and Fig. 2 demonstrate how these different elements are related. The second checklist focuses on the outcome evaluation and has eight questions. These questions address the final steps in the priority-setting cycle, after defining the ranked research topics. To ensure that these questions incorporate the components of a successful priority-setting exercise, they were mapped across the different elements of outcome
M. Nasser et al. / Journal of Clinical Epidemiology 66 (2013) 511e521
515
Using explicit process Q.2
Defining Objectives/ scope Involving stakeholders
Selecting appropriate methods and tools
Implementation, monitoring, evaluation
Q.3,4
Q.3,4
Providing feedback and appeal
Defining criteria to rank and differentiate between topics
Dissemination and communication strategy
Reaching consensus on the prioritized topics
Situation analysis (scoping /mapping/ need assessment)
Fig. 1. Process evaluation of prioritization for Cochrane reviewsdsteps, framework for successful priority setting (process), and proposed questions for the equity lens to guide the process of prioritization.
evaluation as identified in the framework by Sibbald et al. [12] for priority setting (Table 2 and Fig. 2). 3.2. Exploring the role of equity in research priorityesetting processes in the Cochrane Collaboration using the equity lensda pilot study We answered the questions in the equity lens for each of the 13 priority-setting approaches identified in our survey [11]. Most of the processes did not collect outcome data; therefore, we could not assess outcomes. Thus, we concentrated on process evaluation, that is, the first nine questions of the equity lens. We report each of these in the following. 1. Are different stakeholders who might be affected by the choice of review topics involved in the prioritization process (different age, sex, sexual orientation, disability, ethnicity, religion, place of residence, occupation, education, socioeconomic status, and social capital groups)? Four of the prioritization projects targeted specific stakeholder groups: one targeted consumers, two targeted policy makers, and one targeted both clinicians and patients
(through involving patient organizations and clinical associations). The remaining projects aimed to identify and prioritize important questions in health care and did not aim specifically to identify the priorities of stakeholder groups. One group tried to involve stakeholders from different countries, different ethnic backgrounds, and both genders, whereas another tried to involve diverse stakeholders by providing the survey in English and Spanish and contacting consumer advocates in both LMICs and HICs. One group collaborated with a group focused on a particular demographic to ensure that the demographic was appropriately represented and easily identified. The other groups did not recruit strategically to increase stakeholder diversity (as defined by the PROGRESS-Plus mnemonic). However, two projects had an open online survey, which provides the opportunity for different stakeholders to be involved if it is disseminated widely. One of these collected data on age, sex, country, occupation, and the participants’ role in health care. 2. Does the prioritization project consider reducing inequity as part of its objectives? Only one project specifically aimed to introduce health equity and the social determinants of health into the
516
M. Nasser et al. / Journal of Clinical Epidemiology 66 (2013) 511e521 ance
cept
r ac olde
eh
Stak
Defining Objectives/ scope
-
Providing feedback and appeal
Involving stakeholders
Selecting appropriate methods and tools
Implementation, monitoring, evaluation
Defining criteria to rank and differentiate between topics
Dissemination and communication strategy
Reaching consensus on the prioritized topics
Situation analysis (scoping /mapping/ need assessment)
Fig. 2. Outcome evaluation of prioritization for Cochrane reviewsdsteps, framework for successful priority setting (process), and proposed questions for the equity lens to guide the evaluation of the outcomes of an equity-oriented prioritization project.
prioritization process. Two groups emphasized that they would try to include the priorities of individuals based in LMICs. However, these two groups had a broader focus; stakeholders from HICs were also included. 3 and 4. Are the selected methods and tools to identify, prioritize, implement, disseminate, and communicate research topics understandable and transparent for different stakeholders (groups from different age, sex, sexual orientation, disability, ethnicity, religion, place of residence, occupation, education, socioeconomic status, and social capital)? Are specific strategies considered to minimize the barriers to reach disadvantaged or less accessible populations? Two projects used online surveys. In one, the survey was provided only in English, whereas the other survey was available in both English and Spanish. Another project, which aimed to involve patients and clinicians, made specific attempts to ensure that the provided information was understandable for both groups; patients and clinicians could share their opinions during the prioritization meetings.
5. In the stage of situation analysis (evaluating the current health research coverage, identifying gaps, evaluating health care needs, etc.), does the analysis consider the differences in the prevalence, severity, and urgency of health problems along with potential differences in the impact or value of the health care interventions assessed across different subgroups (age, sex, sexual orientation, disability, ethnicity, religion, place of residence, occupation, education, socioeconomic status, and social capital)? Two projects considered the potential differences between subgroups in mapping the evidence as basis information for setting a research agenda and prioritization. In one project, they searched for evidence and categorized the identified studies from the search according to stages of the condition, consequences of ill health, and social determinants potentially affecting the condition. These social determinants were daily living conditions (healthy behaviors, diet, and nutrition), physical and social environment, working conditions, health care, social protection, structural drivers (gender, social inequities, and socialepolitical
M. Nasser et al. / Journal of Clinical Epidemiology 66 (2013) 511e521
517
context), and macroeconomic conditions. Another project used the WHO Health Report 2002 [18,19] as a basis for categorizing the identified literature that can demonstrate the differences in severity and prevalence between LMICs and HICs.
9. Are different stakeholder groups (from different age, sex, sexual orientation, disability, ethnicity, religion, place of residence, occupation, education, socioeconomic status, and social capital) provided with an opportunity to provide feedback and appeal the results?
6. Do the criteria for prioritization consider the potential differences in the severity and urgency of health problems in disadvantaged populations or less accessible groups as opposed to the health problems in privileged populations?
None of the processes ensured that different sociodemographic groups had the opportunity to provide feedback and appeal the results.
In one group, the priority-setting group considered five dimensions in setting priority research topics. Two of the dimensions were relevant to this question: 1. Equity: Would you say that the question addresses the health needs of the underprivileged or disadvantaged groups? 2. Social determinants: Would you say the question addresses the health needs of populations across different socioeconomic gradients? The questions needed to be ranked as ‘‘yes’’ for equity or social determinants (along with other criteria) to be considered a high priority question. One group considered burden of disease, magnitude of problem, and urgency as criteria. These criteria do not directly emphasize the differences between disadvantaged populations and others. However, the project aimed to include the public health priorities of developing countries along with those from developed countries. Depending on how the panel interpreted these criteria, this might have led to the inclusion of priority questions that addressed differences across countries in severity and urgency of the problem. Two groups considered the priorities of a health care system as one of the criteria. Depending on how these priorities were developed, these health system priorities may have considered differences in severity and urgency for disadvantaged populations. The latter argument also applied for questions 6 and 7. 7. Do the criteria for prioritization consider the potential differences in the impact of a health care intervention in disadvantaged populations or less accessible groups as opposed to the health problems in privileged populations? One of the groups considered a criterion that partially addresses this question: potential impact of the intervention on changing policy or treatment especially in areas of the United Nations Millennium Development Goals. 8. Do the criteria for prioritization consider that different population groups might have different values and preferences? None of preferences.
the
processes
considered
values
and
4. Discussion We developed an equity lens with 17 questions to guide the prioritization of topics for systematic reviews of health care interventions. These questions could potentially increase the proportion of prioritized research topics that aim to reduce the health equity gap and that include health care interventions targeted for disadvantaged groups. We can more easily assess the impact of the equity lens if project teams incorporate process and outcome evaluations in their priority-setting projects. Addressing all the issues raised might not be possible in every prioritization process as it might not be feasible, the resources may not be available, or the project has a very specific aim/objective. However, explicit reporting of decisions about each of these steps can potentially help research teams in improving their process. The importance of equity in priority-setting processes was emphasized in several articles. However, the current literature focuses on incorporating equity in certain priority-setting methods and processes. Our equity lens is unique as it provides a broader framework that can be adapted for any priority-setting method. 4.1. Equity in the process of setting research topics and prioritization in the Cochrane Collaboration To judge this, we considered two questions: (1) did the prioritization project take a systematic approach to prioritizing topics with a potential impact on reducing inequity in health? and (2) did the project report describe the extent to which they achieved this? Two of the projects used an open online survey that was accessible and understandable for any (English-speaking) person who had access to a computer. One project used translation to include non-English speakers. However, their process would be strengthened if they had reported demographic data (preferably the sociodemographic variables of the PROGRESS-Plus mnemonic) so that the users of the research priorityesetting exercise could judge the diversity of the respondents involved in setting priorities. Prioritization projects need to be reported more transparently and comprehensively. Cochrane Collaboration groups can support Cochrane entities considering different social dimensions in their prioritization processes; these groups have expertise in methods and issues related to specific social groups. These include the Cochrane Child Health Field, Cochrane Health
Relevant steps of a prioritization process for Cochrane reviews
Conceptual framework by Sibbald for successful priority setting (outcomes)
Reaching consensus on a list of prioritized topics Dissemination and communication strategy for the results Implementation, monitoring, and evaluation process
Shifted priorities and/or reallocated resources Shifted priorities and/or reallocated resources
Improved decision-making quality
Providing feedback and an opportunity for appeal
Improved decision-making quality Positive externalities Stakeholder acceptance and satisfaction
Proposed questions for the equity lens 1. Did the prioritization result in more research topics (in this case Cochrane reviews) that are relevant to disadvantaged groups? 2. Did the dissemination and implementation strategy increase the likelihood that funders and research institutes become aware of the prioritized research topics and consider them as part of their research agenda or strategic planning? 3. Did the dissemination and implementation strategy increase the likelihood that the prioritized research topics that are relevant to disadvantaged groups get funded and conducted? 4. Did the dissemination and implementation strategy increase the likelihood that researchers who work with disadvantaged groups conduct or get involved in the prioritized research projects (in this case, the research project is a review of the Cochrane systematic review) 5. Did the dissemination and implementation strategy increase the likelihood that disadvantaged groups or decision makers or practitioners who work with disadvantaged groups get involved in the prioritized research topics? 6. Does the dissemination and implementation strategy increase the likelihood that policy makers and decisions makers who work with disadvantaged groups use the result of the prioritized research topics? 7. Did the results of the prioritized research topics changed policies, legislation, or clinical practice in favor of disadvantaged groups? 8. Did the appeal and enforcement strategy increase the likelihood that disadvantaged groups or decision makers, researchers, and practitioners who work with disadvantaged group had provided feedback and comments on the prioritization process or results?
Table 3. Equity lens for agenda setting and research priority setting (Part 1 e process) Equity lens 1. Are different stakeholders who might be affected by the choice of research (review) topics involved in the prioritization process (different age, sex, sexual orientation, disability, ethnicity, and religion, place of residence, occupation, education, socioeconomic status, and social capital groups)? In which steps are they involved? 2. Does the prioritization project consider reducing inequity as part of its objectives? 3. Are the selected methods and tools to identify prioritize, implement, disseminate, and communicate research topics understandable, transparent and relevant for different stakeholders (different age, sex, sexual orientation, disability, ethnicity, religion, place of residence, occupation, education, socioeconomic status, and social capital groups)?
The strategies taken in the Cochrane Collaboration to address the questiona Role of the Stakeholders recruited to be involved
1 group targeted consumers, 2 groups targeted policy makers, 1 group targeted both clinicians and patients Context and Background in recruiting 1 group involved stakeholders from different countries, different Stakeholders ethnic backgrounds, and both genders, 1 group involved diverse stakeholders by providing the survey in English and Spanish 1 group collaborated with a group focussed on a particular demographic to ensure that they are appropriately represented. 1 group (only) specifically aimed to introduce health equity and the social determinants of health into the prioritisation process. 2 groups emphasized that they would try to include the priorities of individuals based in LMICs (among others). 2 projects used online surveys a) one survey was provided only in English; b) the second one was available in both English and Spanish. 1 project, which aimed to involve patients and clinicians, made specific attempts to ensure that the provided information was understandable for both groups;
M. Nasser et al. / Journal of Clinical Epidemiology 66 (2013) 511e521
Shifted priorities and/or reallocated resources Improved stakeholder understanding Shifted priorities and/or reallocated resources Improved stakeholder understanding
518
Table 2. Outcome evaluation of prioritization for Cochrane reviewsdsteps, framework for successful priority setting (process), and proposed questions for the equity lens to guide the evaluation of the outcomes of an equity-oriented prioritization project
1 project searched for evidence and categorized the identified studies from the search according to stages of the condition, consequences of ill health, and social determinants potentially affecting the condition. 1 project used the WHO Health Report 2002 (18, 19) as a basis for categorizing the identified literature that can demonstrate the differences in severity and prevalence between Low and Middle Income countries (LMIC) and High Income countries (HICs).
1 group, the priority-setting group considered five dimensions in setting priority research topics. Two of them were relevant to this question. In the below ones, depending on how the panel interpreted these criteria, this might have led to the inclusion of priority questions that addressed question 6. 1 group considered burden of disease, magnitude of problem, and urgency as criteria. 2 groups considered the priorities of a health care system as one of the criteria. 1 group considered a criterion that partially addresses this question: potential impact of the intervention on changing policy or treatment especially in areas of the United Nations Millennium Development Goals (MDGs).
None of the processes considered values and preferences.
None of the processes ensured that different socio-demographic groups had the opportunity to provide feedback and appeal the results
M. Nasser et al. / Journal of Clinical Epidemiology 66 (2013) 511e521
4. Are specific strategies considered to minimize the barriers to reach disadvantaged or less accessible populations? 5. In the stage of situation analysis (evaluating the current health research coverage, identifying gaps, evaluating healthcare needs, etc.), does the analysis consider the differences in the prevalence, severity and urgency of health problems along with potential differences in the impact or value of the health care interventions assessed across different subgroups (age, sex, sexual orientation, disability, ethnicity, religion, place of residence, occupation, education, socioeconomic status, and social capital)? 6. Do the criteria for prioritization consider the potential differences in the severity and urgency of health problems in disadvantaged populations or less accessible groups as opposed to the health problems in privileged populations? 7. Do the criteria for prioritization consider the potential differences in the impact of a health care intervention in disadvantaged populations as opposed to the health problems in privileged populations? 8. Do the criteria for prioritisation consider that different population groups might have different values and preferences? 9. Are different stakeholders groups (representing age, sex, sexual orientation, disability, ethnicity, and religion, place of residence, occupation, education, socioeconomic status, and social capital groups) provided with an opportunity to provide feedback and appeal the process and results of the prioritisation process?
a We contacted 66 Cochrane entities (Cochrane Review groups, Fields and Network). Fifty two of them responded. Of these, 29 had an existing priority-setting process, however, 15 had a more structured approach. Two of these were joint initiatives, therefore, there were a total of 13 unique structured prioritization projects. The priority setting approaches used by these 13 groups included in the table can be categorized as follows:
(1) Approaches to identify priority research topics: (a) Collecting suggestions from different stakeholders (b) Priority topics in health care; using health status data e.g. incidence. Content analysis of policy reports (c) Recommendation from clinical guidelines and systematic reviews (d) Evaluation of existing evidence coverage (2) Approaches to rank priority research topics (a) online survey - Delphi consensus (b) face to face groups - one of them used a systematic approach (nominated group approach)
519
520
M. Nasser et al. / Journal of Clinical Epidemiology 66 (2013) 511e521
Table 4. Equity lens for agenda setting and research priority setting (Part 2 e Outcomes) (none of the research priority setting exercises were in a stage to collect data on outcome measurements so we couldn’t evaluate them based on this perspective) 1. Did the prioritisation result in more research topics (in this case Cochrane reviews) that are relevant to disadvantaged groups? 2. Did the dissemination and implementation strategy increase the likelihood that funders and research institutes become aware of the prioritised research topics and consider them as part of their research agenda or strategic planning? 3. Did the dissemination and implementation strategy increase the likelihood that the prioritised research topics that are relevant to disadvantaged groups get funded and conducted? 4. Did the dissemination and implementation strategy increase the likelihood that researchers who work with disadvantaged groups conduct or get involved in the prioritised research projects (in this case the research project is a Cochrane systematic review review) 5. Did the dissemination and implementation strategy increase the likelihood that disadvantaged groups or decision makers or practitioners who work with disadvantaged groups get involved in the prioritised research topics? 6. Does the dissemination and implementation strategy increase the likelihood that policy makers and decisions makers who work with disadvantaged groups use the result of the prioritised research topics? 7. Did the results of the prioritised research topics changed policies, legislation or clinical practice in favour of disadvantaged groups? 8. Did the appeal and enforcement strategy increase the likelihood that disadvantaged groups or decision makers, researchers and practitioners who work with disadvantaged group had provided feedback and comments on the prioritisation process or results?
Care of Older People Field, and Campbell and Cochrane Equity Methods Group. Only two of the priority-setting processes involved these groups. 4.2. Strengths The current equity lens for priority setting and agenda setting helps project teams to focus on health equity considerations in agenda setting; it encourages consideration of the needs of and impact for disadvantaged groups at each step of priority setting. It was based on a review of the prioritization processes of 66 Cochrane entities and a review of priority-setting methods available to set agendas for research topics [11]. 4.3. Limitations We did not demonstrate whether the prioritization projects achieved the goal of prioritizing topics likely to reduce health inequity because we did not assess outcomes. Evaluation of outcomes using the items in Tables 3 and 4 is needed to answer this question. The tool also does not consider the impact of contextual factors, for example, considering the transferability of the intervention to different contexts. 4.4. Future research We will work with Cochrane entities to collect data on the outcomes of their prioritization process and will pilot the outcome section of the equity lens for priority setting and agenda setting (Tables 3 and 4) to evaluate whether the methods and process can have an impact on reducing in equity in health.
5. Conclusion The equity lens for priority setting and agenda setting can help project teams to develop a more equity-oriented approach to set a research agenda and/or prioritize research topics. It could be used in developing a new priority-setting
project, prospectively/retrospectively evaluating those processes, or comparing different priority-setting methods. Currently, few strategies in the Cochrane Collaboration explicitly address the research priorities of different sociodemographic groups. We need innovative approaches to ensure that the priorities of these diverse groups and stakeholders are considered. Moreover, project teams must establish mechanisms for continuity of the priority setting and determine the process and timeline for updating priorities. In future projects, we will revise the equity lens for priority setting and agenda setting to incorporate the impact of different contexts and environments on deciding on research priorities and will develop tools that will guide researchers whether they achieved the objectives of the equity lens for priority setting and agenda setting. We will also work with Cochrane entities to collect the necessary data to evaluate the outcomes of the prioritization strategy (Table 3). There is a need for guidelines for conducting and reporting prioritization strategies. References [1] Commission on Social Determinants of Health. Closing the gap in a generation: health equity through action on the social determinants of health. Final Report of the Commission on Social Determinants of Health. Geneva: World Health Organization; 2008. [2] Rudan I, Gibson J, Kapiriri L, Lansang MA, Hyder AA, Lawn J, et al, Child Health and Nutrition Research Initiative (CHNRI). Setting priorities in global child health research investments: assessment of principles and practice. Croat Med J 2007;48:595e604. [3] Chinnock P, Siegfried N, Clarke M. Is evidence-based medicine relevant to the developing world?: systematic reviews have yet to achieve their potential as a resource for practitioners in developing countries. Evid Based Complement Alternat Med 2005;2:321e4. [4] Swingler GH, Volmink J, Ioannidis JPA. Number of published systematic reviews and global burden of disease: database analysis. BMJ 2003;327:1083e4. [5] Ortiz Z, Nasser M, Wang Li, Li Y, Pardo J, Bonfill X. (2007) The Cochrane Developing Countries Network: an initiative to improve equitable access to knowledge and information and to its management, Forum 11, Global Forum of Health Research. Beijing 29 October-2 November 2007. [6] Waters E, Doyle J. Systematic reviews of public health in developing countries are in train. BMJ 2004;328:585.
M. Nasser et al. / Journal of Clinical Epidemiology 66 (2013) 511e521 [7] Viergever RF. Health Research prioritisation at WHO. An overview of methodology and high level analysis of WHO led health research priority setting exercises. World Health Organization; 2010. Available at http://www.who.int/rpc/publications/Health_research_prioritisation_ at_WHO.pdf. Accessed February 13, 2013. [8] Tomlinson M, Rudan I, Saxena S, Swartz L, Tsai AC, Patel V. Setting priorities for global mental health research. Bull World Health Organ 2009;87:438e46. [9] Rudan I, Chopra M, Kapiriri L, Gibson J, Ann Lansang M, Carneiro I, et al. Setting priorities in global child health research investments: universal challenges and conceptual framework. Croat Med J 2008a;49:307e17. [10] Doucet A, Mauthner N. Knowing responsibly: linking ethics, research practice and epistemology. In: Mauthner M, Birch M, Jessop J, Iller T, editors. Ethics in qualitative research. London: Sage Publications; 2002. [11] Nasser M, Welch V, Tugwell P, Ueffing E, Doyle J, Waters E. Ensuring relevance for Cochrane reviews: evaluating processes and methods for prioritizing topics for Cochrane reviews. J Clin Epidemiol 2013;66:474e82 [in this issue]. [12] Sibbald SL, Singer PA, Upshur R, Martin DK. Priority setting: what constitutes success? A conceptual framework for successful priority setting. BMC Health Serv Res 2009;9:43. [13] Dans AM, Dans L, Oxman AD, Robinson V, Acuin J, Tugwell P, et al. Assessing equity in clinical practice guidelines. J Clin Epidemiol 2007;60:540e6.
521
[14] Evans T, Brown H. Road traffic crashes: operationalizing equity in the context of health sector reform. Inj Control Saf Promot 2003;10:11e2. [15] Tugwell P, Maxwell L, Welch V, Kristjansson E, Petticrew M, Wells G, et al. Is health equity considered in systematic reviews of the Cochrane Musculoskeletal Group? Arthritis Rheum 2008;59:1603e10. [16] National Institute for Health and Clinical Excellence (NICE). Methods for the development of NICE public health guidance (second edition). Issue date: April 2009. Available at http://www.nice. org.uk/media/2FB/53/PHMethodsManual110509.pdf. Accessed September 14, 2009. [17] Oliver S, Kavanagh J, Caird J, Lorenc T, Oliver K, Harden A, et al. Health promotion, inequalities and young people’s health: a systematic review of research. London: EPPI-Centre, Social Science Research Unit, Institute of Education, University of London; 2008:Available at http://eppi.ioe.ac.uk/cms/LinkClick.aspx?filetic ket5lsYdLJP8gBI%3d&tabid52410&mid54471&language5en-US: Accessed September 14, 2009. [18] World Health Organization (WHO). The World Health Report. Reducing risk, promoting healthy life; 2002. Available at http://www. who.int/whr/2002/en/whr02_en.pdf. Accessed August 24, 2010. [19] Doyle J, Waters E, Yach D, McQueen D, De Francisco A, Stewart T, et al. Global priority setting for Cochrane systematic reviews of health promotion and public health research. J Epidemiol Community Health 2005;59:193e7.