- Email: [email protected]
Transparency in Canadian public drug advisory committees
Health Policy 118 (2014) 255–263
Contents lists available at ScienceDirect
Health Policy journal homepage: www.elsevier.com/locate/healthpol
Transparency in Canadian public drug advisory committees Zahava R.S. Rosenberg-Yunger a,∗ , Ahmed M. Bayoumi a,b,c,d a School of Health Services Management, Ted Rogers School of Management, Ryerson University, 350 Victoria St., Toronto, ON, Canada M5B 2K3 b Department of Medicine, University of Toronto, Toronto, ON, Canada c Institute of Health Policy, Management, and Evaluation, University of Toronto, Toronto, ON, Canada d Division of General Internal Medicine, St. Michael’s Hospital, Toronto, ON, Canada
a r t i c l e
i n f o
Article history: Received 12 November 2013 Received in revised form 7 July 2014 Accepted 18 August 2014 Keywords: Qualitative research Resources allocation Transparency Drug policy
a b s t r a c t Background: Transparency in health care resource allocation decisions is a criterion of a fair process. We used qualitative methods to explore transparency across 11 Canadian drug advisory committees. Methods: We developed seven criteria to assess transparency (disclosure of members’ names, disclosure of membership selection criteria, disclosure of conflict of interest guidelines and members’ conflicts, public posting of decisions not to fund drugs, public posting of rationales for decisions, stakeholder input, and presence of an appeals mechanism) and two sub-criteria for when rationales were posted (direct website link and readability). We interviewed a purposeful sample of key informants who were conversant in English and a current or past member of either a committee or a stakeholder group. We analyzed data using a thematic approach. Interviewing continued until saturation was reached. Results: We examined documents from 10 committees and conducted 27 interviews. The median number of criteria addressed by committees was 2 (range 0–6). Major interview themes included addressing: (1) accessibility issues, including stakeholders’ degree of access to the decision making process and appeal mechanisms; (2) communication issues, including improving internal and external communication and public access to information; and (3) confidentiality issues, including the use of proprietary evidence. Conclusion: Most committees have some mechanisms to address transparency but none had a fully transparent process. The most important ways to improve transparency include creating formal appeal mechanisms, improving communication, and establishing consistent rules about the use of, and public access to, proprietary evidence. © 2014 Elsevier Ireland Ltd. All rights reserved.
1. Introduction Decisions regarding public drug formulary listings can be controversial [1]. In Canada, such decisions are generally made by provincial or territorial Ministries of Health based on recommendations from drug advisory
∗ Corresponding author. Tel.: +416 979 5000x4213; fax: +416 979 5209. E-mail addresses: [email protected], [email protected] (Z.R.S. Rosenberg-Yunger). http://dx.doi.org/10.1016/j.healthpol.2014.08.010 0168-8510/© 2014 Elsevier Ireland Ltd. All rights reserved.
committees. Because these decisions can be contentious, the recommendation process should be legitimate and fair; transparency is an important component for judging fairness [2,3]. The American College of Physicians defines transparency within the health care context as “making available to the public, in a reliable and understandable manner, information on the health care system’s quality, efficiency and consumer experience with care” [4]. Accountability for Reasonableness, a popular framework for judging fairness in health care, asserts that transparency in health resource allocation decision making requires
256
Z.R.S. Rosenberg-Yunger, A.M. Bayoumi / Health Policy 118 (2014) 255–263
that both the recommendation process and the rationale for making recommendations are publicly available [3]. Transparency is valued because it may result in better informed recommendations, decrease appeals, increase public acceptance of recommendations, and enhance trust in the overall process [1,5]. In Canada, public drug plans are administered at a subnational (provincial or territorial) level; a few national plans also exist for specialized populations, including First Nations and Inuit Canadians, inmates in federal penitentiaries, refugee claimants, military personnel, members of the Royal Canadian Mounted Police, and veterans [6]. Drug plans vary in their coverage but all insure people who are receiving social assistance [7,8]. Some drug plans insure the elderly or have established catastrophic coverage [9]. Coverage decisions are typically made at the ministerial or senior bureaucratic level, based on the recommendation of a sub-national drug advisory committee. These committees, in turn, receive recommendations from two national committees—one assessing cancer drugs and another assessing all other drugs. These review processes are known as the pan-Canadian Oncology Drug Review and the Common Drug Review, respectively, and their expert committees. The province of Quebec has opted not to participate in the national review committee and has its own processes and committees which operate separately [10]. The number of people served by each public plan varies widely, reflecting large variations in the populations of each region and coverage policies. The review process typically starts with a submission from a manufacturer for consideration of listing on a public drug formulary. The review includes assessments by staff members of the drug’s effectiveness, safety, and cost-effectiveness compared to current treatments. At the national level, review staff solicit input from patient groups as well as from the manufacturer. All of these materials are provided to the expert committees as inputs into their decision making process. In summary, drug coverage in Canada integrates provincial and federal payors, multiple advisory committees, and a federal system in which accountability for most funding decisions is at the sub-national level. Because this system is complex and regionalized, there is the potential for considerable heterogeneity in processes [1,2,11–13]. Furthermore, small committees might not have the same resources or pressures as their larger counterparts. Transparency, as a component of fairness, has not been formally assessed within this context. We used qualitative methods to evaluate transparency across 11 Canadian drug advisory committees. 2. Methods We used a literature review, key informant interviews, and document reviews to assess transparency. 2.1. Definition of transparency We based our definition of transparency on Daniels and Sabin’s “Accountability for Reasonableness” priority setting framework, which considers decisions to be
legitimate and fair if they satisfy four conditions: publicity, relevance, revision and appeals, and regulation (also referred to as enforcement or leadership) [3]. Gibson and colleagues have proposed adding a fifth “empowerment” condition, which focuses on addressing power differences between groups within the decision-making context and to optimising opportunities for participation [14]. We focused on the conditions that we considered most relevant to issues of transparency (publicity, relevance, revisions appeal, and empowerment). Accordingly, transparency requires that reports, appraisals and decision processes are readily available and that the decision process includes definitions of the roles and responsibilities of each stakeholder. We defined four criteria related to the condition of publicity: public disclosure of committee members’ names; publicly available criteria by which committee members are selected; public disclosure of conflict of interest guidelines and committee members’ conflicts; and public posting of decisions not to fund drugs. We included a criterion related to relevance, public posting of rationales for decisions (including the evidence upon which decisions were made) and a criterion related to empowerment, assessing whether there were adequate opportunities for stakeholder input during the review process. Finally, we included a seventh criterion to assess the presence or absence of a mechanism for appeal. When rationales were posted on the internet, we also assessed: (1) whether they were accessible through direct links on committee websites; and (2) the readability of the five most recent documents posted on the websites, using the Flesch–Kincaid or the Kandel–Moles instrument for English and French rationales, respectively [15,16]. We scored rationales as readable if they scored at a grade 12 level or lower which should make them accessible to 87% of Canadians who have at least a high school education [17]. The rationales reviewed for readability are listed in the Supplementary Appendix.
2.2. Interviews and documents We identified potentially eligible key informants from websites listing drug advisory committee members, suggestions by staff in Ministries of Health, and other informants [18,19]. Our respondents represented a purposeful sample with a range of views, selected to develop a rich understanding of the topic. We used a combination of maximum variation sampling, in which we selected respondents that were representative of different stakeholder groups, and snowball sampling, in which we asked respondents to identify other participants. Our inclusion criteria were as follows: individuals who were conversant in English, who were either current or former members of provincial or federal drug advisory committees (including clinical and public or patient experts), employees of a Ministry of Health, members of a patient advocacy group, or employees of a pharmaceutical manufacturer. Interviews were semi-structured and based on a pre-established interview guide (Box 1, Interview Guide). All interviews were digitally recorded and professionally transcribed.
Z.R.S. Rosenberg-Yunger, A.M. Bayoumi / Health Policy 118 (2014) 255–263
Box 1: Interview guide. Acceptance criteria: 1. Do you think the public members involved are representative? [Probes: Do they represent patient interests? Do they represent broader public interests? Do they provide their own expertise?] 2. What do you think the purpose of public involvement should be? 3. In your estimation, do you think that the selection process is conducted in a clear, appropriate and transparent manner? 4. At which point in the process do you think the public should be involved? [Probes: Consultation vs. Deliberation vs. Appeals] 5. To what degree do you think the public should be involved? [Probes: Opinion vs. Deliberation] 6. To what extent do you think that public input affects the final decisions? [Probe: Can you give me examples of where you think the public did and did not affect a decision?] 7. Do you think the drug deliberation and recommendation process is transparent? Process criteria: 1. Do you think that public participants have access to the appropriate resources to enable them to successfully fulfill their role? 2. How would you define the role of public participation? 3. Do you think that the nature and scope of the participation task is clearly defined? 4. Do you think it is efficient to involve public in decisions? Influence of public members (power issues): Committee members: 1. What was the extent of your involvement in the committee? 2. Were social values incorporated into discussions? 3. What values are discussed? Were they discussed implicitly or explicitly? 4. How are these values reflected in the final recommendation? 5. Were the goals of public participation clearly defined? What were these goals? Do you think any other goals should be considered and why? 6. How would you have liked to be involved? 7. Whom do you think should be involved in these decisions? 8. Were any decisions altered because of public input? [Only ask if not answered above in q 3] Others: 1. Do you think that the values of the public are sufficiently incorporated into the decisions? 2. What do you think is the optimal way to involve the public? Communication/Knowledge: Committee members: 1. Did you receive any training to prepare you for your membership on the committee? [Probes: Please elaborate on the training you received? Do all committee members (i.e., including public members) receive training? Do you think the training is appropriate and useful?] 2. To what extent was the information regarding the drug communicated to you? 3. To what extent was the information about the drug understood by you? 4. Did you feel that you had enough information to take part in the process? 5. What weight was given to public input? [Probe: Are their comments respected? Do they actively take part in discussions?] 6. What do you consider to be optimal/successful public involvement? How do you define successful involvement? Others: 1. Do you think that the public sufficiently understands the issues presented to them in order to participate in the deliberative process? 2. What do you consider to be optimal/successful public involvement? How do you define successful involvement? Is there anything else you would like to add?
257
2.3. Analysis We analyzed the interviews and documents using a qualitative thematic approach consisting of line-by-line coding to develop categories. We used constant comparison to examine relationships within and across codes and categories. The first five transcripts were coded by both investigators and we discussed discrepancies and established consensus for the preliminary coding scheme. All subsequent transcripts were coded by one investigator. When discrepancies occurred both investigators met to established consensus. Interviewing continued until saturation was reached. We used NVivo 8 software to code, store, and organize the data [20]. 3. Results We considered 17 committees (Supplementary material). We excluded six committees that make recommendations about relative small populations: five federal committees and the committee advising Nunavut’s Extended Health Benefits program for non-aboriginal individuals (Supplementary material). The remaining 11 committees represented the two national committees, all Canadian provinces and the Yukon Territory, and the Drugs and Therapeutics Advisory Committee for the Non-Insured Health Benefits program, which provides coverage for a limited range of goods and services for First Nations and Inuit Canadians who are not insured elsewhere. These 11 committees have 125 members. We approached 32 individuals from nine committees (the committees from the Atlantic provinces and Manitoba declined to participate), of whom 27 (84%) agreed to participate. We also collected documents from 10 committee websites (documents from the Drugs and Therapeutics Advisory Committee were not available online). 3.1. Interviews The median age of participants was 55 (range 39–68), 18 (67%) were women, and 20 (74%) had a professional or graduate degree. We interviewed four industry representatives; six patient group representatives; one past government employee and committee member; five current government employees; and 11 current committee members. Each patient group that participated in our study had disclosed on their website that they had received industry funding. The major themes which emerged included issues regarding accessibility, communication, and confidentiality. 3.2. Accessibility issues 3.2.1. Appeal mechanisms Many participants discussed the need for formal mechanisms to appeal committee recommendations. A pharmaceutical manufacturer employee respondent highlighted how appeals differ from reconsiderations: The reconsideration [process] . . . is the same number of people looking at the . . . information . . . [An] appeal
258
Z.R.S. Rosenberg-Yunger, A.M. Bayoumi / Health Policy 118 (2014) 255–263
[process] would be . . . a separate and different group of people. (INT8) Another pharmaceutical manufacturer employee respondent noted that an appeal mechanism might facilitate greater discourse about controversial issues. However, some committee members felt that more opportunities for input might also represent opportunities for manufacturers to sway recommendations. Respondents also noted that the present mechanisms for reconsideration are often limited to manufacturers. A patient group representative respondent commented that there is no “Formal process for members of the public to raise concerns or ask any questions, or even appeal” (INT24). 3.2.2. Open processes Open processes permit stakeholders (including the public) to access part of the decision making process. One government employee explained that: The future is potentially having opened . . . committee deliberation . . . where [everyone] sits and listens to evidence and it’s done in an open format for anyone to hear what evidence is being presented, and . . . then the [committee] . . . is sequestered and can debate the merits of the evidence. (INT10) However, respondents noted that open processes could result in logistical challenges. “You have 200 people show up to the meeting, you have to have a venue that’s going to support that” (INT5). Additionally, a few respondents believed that an open process would result in an unwanted increase in manufacturer involvement and increase the time required for making recommendations. As one committee member noted: “I just don’t know if we can afford the reduction in productivity given our small scale” (INT21). Finally, an open process “Might inhibit some committee members in being frank in their discussions” (INT3). 3.3. Communication issues 3.3.1. Access to recommendation deliberations Although most committees post reports online (Table 1), few post meeting deliberations or supporting documents. A patient group respondent believed that the deliberation process has the “appearance of transparency” (INT14) while an industry respondent thought “there still is a little bit of a black box around how decisions are made” (INT15). One committee member suggested communicating to the public the “kind of discussion that goes [on] back and forth” (INT13). A government employee noted that committees “struggle” with making recommendations in situations when drugs are challenging to evaluate using existing criteria, such as expensive orphan drugs that are unlikely to be cost-effectives (INT3). In such situations, it is important that committees clearly communicate the reasons for their recommendations. As one pharmaceutical respondent noted: Every drug is different and every disease is different, and there are really different factors that drive it. But these evidence evaluation bodies need to do a better job at explaining that. (INT11)
Respondents remarked that committee reports often inadequately reflect the rationale and deliberation behind recommendations, that postings should be done in a timely manner, and that effective communication might involve direct correspondence with patient groups and the use of media including email alerts and meetings. Nonetheless, one committee member noted the need to balance transparency with members’ ability to function, “Not everything we say is published . . . I know that that limits transparency . . . but . . . it also enables people to say what’s on their mind” (INT26). 3.3.2. Access to conflict of interest guidelines and disclosures Only three committees (Yukon Formulary Working Group, NIHB’s Drugs and Therapeutics Advisory Committee, and Manitoba’s Drug Standards and Therapeutics Committee) did not post their conflict of interest guidelines online. Two committees (p-ERC and CDEC) also posted members declarations on their websites (Table 1). As one government respondent noted: . [Members] have to declare any funding sources that may lead to a real or perceived conflict-of-interest, and we’re transparent about that, so their conflict of interest gets posted . . . It at least raises the awareness . . . Because . . . often the first response [is] . . . I don’t have any conflicts. (IN10) 3.3.3. Clarity regarding use of evidence Committees consider multiple types of evidence, including clinical appraisals, economic, evaluations, and patient input. Stakeholders often inquired about the relative weighting of these types of evidence, in particular how patient input from formal submissions, website feedback or informal discussions was used. A government employee respondent noted: Is the patient input evidence weighted in the same way as the clinical and the cost evidence? . . . We don’t have a formula . . . for weighting. All of the . . . evidence is part of this deliberative process. (INT3) Many respondents wanted better understanding of this process. As one patient respondent noted: It needs to be a lot clearer in terms of . . . the balance of criteria . . . it would be good to know . . . what is the process that the committee uses . . . Is there a consideration of the data first or patient submissions? . . . How do you debate those? (INT6) 3.3.4. Understanding how recommendations lead to decisions Respondents noted that recommendations might not match the final decisions taken by the health plan or Minister of Health. One committee member noted that “the government was overturning a lot of things that we were doing, and people were getting quite frustrated” (INT17). Furthermore, the transparency of the decision making process varies across health plans. As one government employee noted:“[Even if] something’s approved doesn’t mean it gets on
Z.R.S. Rosenberg-Yunger, A.M. Bayoumi / Health Policy 118 (2014) 255–263
259
Table 1 Comparison of Canadian drug advisory committees.
Members Clinical experts (N) Patient or public expert members (N) Transparency criteria Are member names available online? Are member selection criteria available online? Are “no” decision available online? Is there stakeholder input prior to recommendation? Registered Patient Group Individuals Is there an appeals process? Are rationales available online? Are members’ conflict of interest disclosures available online?
YT
BC
AB
SK
MB
ON
QC
ATL
NIHB
CDEC
pERC
3 0
9 3
8 0
12 2
6 0
15 2
11 2
10 0
12 0
12 2
14 2
N
Y
Y
Y
N
Y
Y
Y
N
Y
Y
N
Y
N
N
N
Y
N
N
N
Y
Y
N
Y
N
Y
N
Y
N
Y
N
Y
Y
NN/A
YY
YNot Specified
N/A
NN/A
YY
YY
NN/A
NN/A
YY
YY
N/A N/A N N N
Y Y N Y N
Not Specified Not Specified N N N
N/A N/A N N N
N/A N/A N N N
Y N N Y N
Y N N Y N
N/A N/A N N N
N/A N/A N N N
Y N N Y Y
Y N N Y Y
N denotes No, Y denotes Yes, N/A denotes not applicable, YT denotes Yukon Territory, BC denotes British Columbia, AB denotes Alberta, SK denotes Saskatchewan, MB denotes Manitoba, ON denotes Ontario, QC denotes Quebec, NIHB denotes Drugs and therapeutics advisory committee for the noninsured health benefits, CDEC denotes Canadian Drug Expert Committee, and pERC denotes p-CODR Expert Review Committee.
the formulary and then you’ve got to go digging in individual provinces. . .and each province has its own slightly different system” (INT2). 3.4. Confidentiality issues 3.4.1. Proprietary evidence Some respondents thought that there is a need to challenge the current administrative norms regarding what data is considered to be proprietary. For instance, some respondents thought that proprietary evidence should not be permitted in the recommendation process. A former committee member commented, “We shouldn’t be accepting any information that isn’t releasable, because it reduces our ability to defend the decision” (INT21). Similarly an industry respondent noted the pan-Canadian Oncology Drug Review’s approach to proprietary evidence, “When they start a process, one of their first points of engagement with industry is to say, is there data here that absolutely cannot be disclosed? . . .It’s known by industry that they may say if we can’t disclose it because it’s germane to our decision, then we won’t use it (INT 15). Another committee member discussed frustration with receiving incomplete data, particularly related to drug costs: [Manufacturers] submitted the information with . . . cost information . . . blacked out, well what the hell is that? . . . [I] understand the commercial need for . . . drug companies to protect proprietary data and their proprietary information. So how do we find the balance between those pieces [i.e., transparency and proprietary information]? (INT26) A few members mentioned that achieving full transparency was difficult given the current administrative norms because
“We are not able . . .to make all the information provided from the manufacturers public because obviously that becomes available to all their competitors” (INT13). 3.4.2. Members’ names While most committees make members’ names publicly available, three did not (Table 1), ostensibly to protect members from litigation and from lobbying by stakeholder groups. One committee member noted, “When I look at the transparency of the process, I think our names do need to be published. And I know that there’s risk in doing that” (INT26). 4. Document review We obtained data from eight provincial and two national Canadian public drug expert advisory committees (Table 2). Of our seven transparency criteria, the median number addressed by committees was 2 (range 0–6). Both national committees, British Columbia, Quebec and Ontario each met six criteria; the Yukon Territory and Manitoba did not meet any criteria. Of the five committees that posted rationales, each posted a direct link on the website to rationales that were at a grade 12 reading level or less. No committee had mechanisms for appeal but most had processes for reconsideration (Table 3). The criterion that was least likely to be consistently addressed was making member selection criteria available (addressed by four committees). The criterion that was most likely to be addressed was posting of members’ names online (met by eight committees). 5. Discussion We used qualitative methods to explore transparency across 11 Canadian drug advisory committees. Our most striking finding was the large degree of heterogeneity regarding mechanisms to address transparency across the
260
Z.R.S. Rosenberg-Yunger, A.M. Bayoumi / Health Policy 118 (2014) 255–263
Table 2 Documents reviewed. Committee
Document/web title
Yukon Formulary Working Group
Yukon Drug Programs Formulary http://www.hss.gov.yk.ca/pdf/yukon drug programs formulary.pdf Drug Benefit Council membership, remuneration to drug benefit members, terms of reference and conflict of interest http://www.health.gov.bc.ca/pharmacare/formulary/dbc info.html http://www.health.gov.bc.ca/navigation/pdf/DBC remuneration.pdf http://www.health.gov.bc.ca/pharmacare/formulary/pdf/DBC-TermsOfRef.pdf Resubmission for the Alberta Drug Benefit List, Alberta Pharmaceutical Strategy, Alberta Health and Wellness Benefits List, https://www.ab.bluecross.ca/dbl/pdfs/ahwdbl sec1 intro.pdf Terms of Reference http://formulary.drugplan.health.gov.sk.ca/PDFs/DACSTOR.pdf Email correspondence with ministry and Manitoba Drug Review process http://www.gov.mb.ca/health/mdbif/review.html Membership, term, remuneration, http://www.pas.gov.on.ca/scripts/en/boardDetails.asp?boardID=817 Mandate, members, operating ruleshttp://www.inesss.qc.ca/index.php?id=26&L=1
British Columbia Drug Benefit Council
Alberta’s Expert Committee on Drug Evaluation and Therapeutics Drug Advisory Committee of Saskatchewan Manitoba Drug Standards and Therapeutics Committee Ontario Committee to Evaluate Drug L’Institut national d’excellence en santé et en services sociaux (INESSS) Comité scientifique permanent en santé et en services sociaux, Quebec Atlantic Expert Advisory Committee
Canadian Drug Expert Committee p-CODR
Committee members, Atlantic Common Drug Review http://www.gov.ns.ca/health/pharmacare/committees/EAC members.pdf http://www.gov.ns.ca/health/Pharmacare/committees/acdr.asp Process, conflict of interest, terms of reference http://www.cadth.ca/en/products/cdr/committees/canadian-drug-expert-committee http://www.pcodr.ca/wcpc/portal/Home/General PC/Opportunities? afrLoop=209898743908000& afrWindowMode=0& adf.ctrl-state=dcdh3kmuj 4#perc http://www.pcodr.ca/portal/server.pt/community/about pcodr/546/review committee
country. Using liberal criteria, we found that committees fell short of having a fully transparent process. Respondents suggested that the most important ways to improve transparency are to create formal stakeholder inclusive appeal mechanisms, improve communication, and establish consistent rules about how proprietary evidence is used and made available. No committee had an appeals mechanism. Given that manufacturers have an interest in appealing all negative recommendations and that an appeals process can be resource-intensive, clear guidelines describing the circumstances under which an appeal would be considered need to be established. Notably, the House of Common’s Standing Committee in Health in 2007 recommended that the federal non-oncology committee establish an appeal process which would include a separate group of reviewers and permit the public to appeal [21]. Such a process does not exist yet in Canada. Both manufacturers and patient groups also use mechanisms outside of the committee process to influence and sometimes alter decisions, such as media and letter writing campaigns to the ministers of health [22]. Because the decisions made after such lobbying are frequently opaque, formal appeals process will result in more transparent and consistent recommendations and decisions. In contrast to appeals, which entail a review of the recommendation by a different body, reconsiderations entail review by the same body that made the initial decision. Many committees have well-established reconsideration processes to review new information, such as new clinical trial or safety data, or to re-examine the interpretation of data. Most reconsiderations processes for specific drugs in
Canada are only open to manufacturers (Table 2), although some committees also have mechanisms for reconsideration of classes of drugs as evidence accumulates [2]. Manufacturers and patients believed that reconsiderations for specific drugs that were adjudicated by the same committee that made the original decision were unlikely to result in a decision being changed. There is also considerable heterogeneity regarding whether a submission for reconsideration would be allowed for any decision or only when processes were not properly followed. The committees varied considerably regarding processes for patients and public input. Some committees only permit registered patient advocacy groups to submit patient evidence (such as Ontario’s committee and both national committees) while others permit individual patients and caregivers to submit input (such as British Columbia) and some have no process for input (such as the Yukon Territory). Requiring patient groups to register increases transparency regarding disclosure of conflict of interest but limits the range of patients who participate. Issues of empowerment among and between the general public, individual patients, and patient groups is an important topic for future research, including the role of industry funding for patient groups [23]. The committees are also varied regarding the information that patient groups are provided to assist them in assembling their input. For example, in British Columbia, patients are asked to read a Drug Information Sheet, which includes general information and cost comparators for the drug under review but no clinical benefit data. Patients then complete a questionnaire that focuses on their anecdotal experience. In contrast, the national committee
Z.R.S. Rosenberg-Yunger, A.M. Bayoumi / Health Policy 118 (2014) 255–263
261
Table 3 Characteristics of Revision Mechanisms. Jurisdiction
Circumstance
Stakeholder who can appeal
Type of mechanism
Adjudication
YT
Any decision of the Yukon Formulary Working Group Recommendation is not supported by the evidence Process of the submission reviewed was not properly followed New information Request a change in the current special authorization or restricted benefit status Request a change in current criteria for coverage Recommendation is not supported by the evidence New information New information Change in pricing Process of the submission reviewed was not properly followed Recommendation is not supported by the evidence The Minister asks committee to revise position New information New information Change in pricing Process of the submission reviewed was not properly followed Recommendation is not supported by the evidence Feedback was received on the initial recommendationConsensus was not reached by stakeholders The recommendation was negative
Not specified
Reconsideration
Director
Manufacturers
Reconsideration
Ministry DBC Secretariat in consultation with committee Chair
Manufacturer
Reconsideration
Committee
Physicians or manufacturer
Reconsideration
Committee
Manufacturer
Reconsideration
Committee
Manufacturers
Reconsideration
Committee
Minister
Reconsideration
Minister of Health
Manufacturers Manufacturers, patient groups Manufacturers
Resubmission Reconsideration
Committee Committee
Reconsideration
Committee
Stakeholders (including but not limited to manufacturers or their representative organizations, tumor groups, and patient advocacy groups)
Reconsideration
Committee
BC
AB
SKa
MB ON
QC ATL NIHBa CDEC
pERC
YT denotes Yukon Territory, BC denotes British Columbia, DBC denotes Drug Benefit Council, AB denotes Alberta, SK denotes Saskatchewan, MB denotes Manitoba, ON denotes Ontario, QC denotes Quebec, NIHB denotes Drugs and therapeutics advisory committee for the non-insured health benefits, CDEC denotes Canadian Drug Expert Committee, and pERC denotes p-CODR Expert Review Committee. a These are informal processes.
that considers cancer drugs invites patient input both before the manufacturer’s submission is considered and after the initial recommendation is made; at that point, patient groups have access to the initial recommendation, including information on clinical benefit, patient values, economic evaluation, and adoption feasibility. Some recommendations from respondents will be very challenging to implement. For example, introducing open deliberations would require considerable resources and place additional burdens on committee members. Furthermore, there has been a vigorous debate in the literature regarding the relative merits of fair processes, formal decisions frameworks, or both for making fair decisions [24,25]. The lack of transparency regarding which types of evidence were most influential in a decision has contributed to the public’s perception of unfairness and lack of reproducibility when some drugs are approved and others denied. Conversely, committees need flexibility to make decisions that are sensitive to specific contexts and needs. We anticipate
that committees will continue to grapple with the tension between transparency and flexibility. There is concern among respondents that mechanisms to increase transparency will privilege some stakeholders over others and thus influence power relations, another important consideration in fair decision making [14,26]. For example, mechanisms for appeal or reconsideration of negative decisions or increased openness (with its attendant risk of lobbying of committee members) could result in granting greater power to manufacturers. Alternatively, mechanisms such as increased transparency with regards to proprietary information could decrease manufacturers’ influence. Interestingly, our participants did not specifically discuss transparency regarding clinical trial data. Notably, since 2010, the European Medicines Agency releases clinical trial reports upon request with the goal of publishing this data proactively for all drugs that are reviewed [27]. The United States Food and Drug Administration also has initiatives to increase transparency
262
Z.R.S. Rosenberg-Yunger, A.M. Bayoumi / Health Policy 118 (2014) 255–263
regarding drug approval and has increased the amount of information accessible to the public [28]. The question of transparency in the context of drug cost is particularly challenging. The list price of a medication may differ from the manufacturer’s confidential submitted price; both of these may vary from the final negotiated price by drug plans. These differences might explain some of the reluctance to make proprietary information or committee deliberations public and also why decisions sometimes differ from recommendations. Although some jurisdictions list “official drug prices” on their website, the real price paid by provincial plans might be very different after confidential product listing agreements or price negotiations [29]. Because such agreements and negotiations can have international consequences, it is unlikely that fully transparent price data will ever be publicly available. Our study has some limitations. We studied only drug approval mechanisms while recognizing that device and technology advisory committees face similar issues [30,31]. We analyzed whether rationales were accessible but did not analyse the quality or comprehensiveness of rationale documents beyond readability. Finally, we did not include six small committees. 6. Conclusions Canadian drug advisory committees have lower recommendation for listing rates than their counterparts in other countries for example, CDR has a recommended drug listing rate of 53.4% compared with NICE’s rate of 87.4% [32,33]. Accordingly, such decisions will remain both challenging and contentious, particularly when very expensive drugs are introduced [34]. Some steps to increase transparency are straightforward and can be implemented rapidly, such as publicising member selection criteria and decision rationales. We believe that increased transparency will lead to better decisions and will increase public confidence in formulary listing decisions. Acknowledgments Dr. Bayoumi is supported by the Canadian Institutes of Health Research/Ontario Ministry of Health and Long-Term Care Applied Chair in Health Services and Policy Research. Conflict of interest The authors have no conflict of interest. References [1] Morgan S, McMahon M, Mitton C, et al. Centralized drug review processes in Australia, New Zealand, the United Kingdom, and Canada. Health Affairs 2006;25:337–47. [2] Mitton C, Morgan S, McMahon M, Gibson J. Centralized drug review processes: are they fair? Social Science and Medicine 2006;63:200–11. [3] Daniels N, Sabin J. Setting limits fairly. Oxford: Oxford University Press; 2002. [4] American College of Physicians. Healthcare transparency—focus on price and clinical performance information: policy paper. American College of Physicians; 2010 www.acponline.org/advocacy/ current policy papers/. . ./transparency.pdf [accessed 30.04.14, 2014]. [5] Bruni R, Laupacis A, Martin DK. Public engagement in setting priorities in health care. CMAJ 2008:179.
[6] Report of the Standing Committee on Health Prescription drugs part 1—Common Drug Review: An f/p/t Process. House of Commons 39th Parliament 2nd session. Communication Canada—Publishing; 2007 http://www.parl.gc.ca/HousePublications/Publication.aspx? DocId=3162492&Mode=1&Parl=39&Ses=2&Language=E [accessed 07.06.13]. [7] Provincial and Private Drug Insurance Plans in Canada. Parliamentary research branch economics division; 1996 http://publications. gc.ca/Collection-R/LoPBdP/MR/mr142-e.htm [accessed 07.06.13]. [8] About the pan-Canadian Oncology Drug Review (pCODR); 2011 http://www.pcodr.ca/wcpc/portal/Home/AboutpCODR? afrLoop= 202027728867000& afrWindowMode=0& adf.ctrlstate=n9bjllmhl 47 [accessed 15.10.12]. [9] Daw JR, Morgan SG. Stitching the gaps in the Canadian public drug coverage patchwork. A review of provincial pharmacare policy changes from 2000 to 2010. Health Policy 2012;104:19–26. [10] Bate A, Mitton C. The application of economic principles in health care priority setting. Expert Review of Pharmacoeconomics and Outcomes Research 2006;6:275–84. [11] Tierney M, Manns B. Optimizing the use of prescription drugs in Canada through the Common Drug Review. CMAJ 2008;178: 432–5. [12] McMohon M, Morgan S, Mitton C. The Common Drug Review: a NICE start for Canada? Health Policy 2006;77:339–51. [13] Laupacis A. On bias and transparency in the development of influential recommendations. CMAJ 2006;174:335–6. [14] Gibson JL, Martin DK, Singer PA. Priority setting in hospitals: fairness, inclusiveness, and the problem of institutional power differences. Social Science and Medicine 2005;61:2355–62. [15] Ley P, Florio T. The use of readability formulas in health care. Psychology, Health & Medicine 1996;1:7–28. [16] Bamberger R, Rabin AT. New approaches to readability: Austrian research. The Reading Teacher 1984;37:512–9. [17] A Profile of Education in Canada. Canadian council on social development; 2010 http://www.ccsd.ca/factsheets/education/ [accessed 22.01.13]. [18] Denzin NK, Lincoln YS. In: The SAGE handbook of qualitative research, 3rd Thousand Oaks: California: Sage Publications; 2005. [19] Maxwell J. Qualitative research design London. London, UK: Sage Publications; 2005. [20] NVivo 8 [software]. Cambridge Qualitative Solutions Research International; 2008, Melbourne, Australia: QSR International. [21] Government Response to the Second Report of the Standing Committee on Health Prescription Drugs Part 1—Common Drug Review: An F/P/T Process. Parliament of Canada; 2007 http://www.parl.gc.ca/ HousePublications/Publication.aspx?DocId=3396385&Language=E& Mode=1&Parl=39&Ses=2 [accessed 19.12.12]. [22] Rosenberg-Yunger Z, Thorsteinsdóttirb H, Daar A, Martin D. Stakeholder involvement in expensive drug recommendation decisions: an international perspective. Health Policy 2012;105: 226–35. [23] Cohen D, Raftery J. Paying twice: questions over high cost of cystic fibrosis drug developed with charitable funding. BMJ 2014; 12:348. [24] Culyer AJ. Hic sunt dracones: the future of health technology assessment—one economist’s perspective. Medical Decision Making 2012;32:E25–32. [25] Gruskin S, Daniels N. Process is the point: justice and human rights: priority setting and fair deliberative process. American Journal of Public Health 2008;98:1573–7. [26] Abelson J, Forest P, Eyles J, Smith P, Martin E, Gauvin F. Deliberations about deliberative methods: issues in the design and evaluation of public participation processes. Social Science & Medicine 2003;5:239–51. [27] The European Medicines Agency Release of data from clinical trials. The European medicines agency http://www.ema.europa.eu/ema/ index.jsp?curl=pages/special topics/general/general content 000555.jsp&mid=WC0b01ac0580607bfa [accessed 14.03.14]. [28] Wechsler J. Transparency to alter FDA and industry practices. Pharmaceutical Technology 2010;34:28–34. [29] BC PharmaCare wants to hear Your Voice. Which drugs are being reviewed? Government of British Columbia 2013, http://www. health.gov.bc.ca/pharmacare/yourvoice/#druglist [accessed 14. 03.14]. [30] Hulstaert F, Neyt M, Vinck I, et al. Pre-market clinical evaluations of innovative high-risk medical devices in Europe. International Journal of Technology Assessment in Health Care 2012;28: 278–84.
Z.R.S. Rosenberg-Yunger, A.M. Bayoumi / Health Policy 118 (2014) 255–263 [31] Hailey D. Toward transparency in health technology assessment: a checklist for HTA reports. International Journal of Technology Assessment in Health Care 2003;19:1–7. [32] Clement FM, Harris A, Li JJ, Yong K, Lee KM, Manns BJ. Using effectiveness and cost-effectiveness to make drug coverage decisions: a comparison of Britain, Australia, and Canada. JAMA 2009;302:1437–43.
263
[33] Rocchi A, Miller E, Hopkins RB, Goeree R. Common Drug Review recommendations: an evidence base for expectations. Pharmacoeconomics 2012;30:229–46. [34] In cancer care, cost matters. New York Times; 2012 http://www. nytimes.com/2012/10/15/opinion/a-hospital-says-no-to-an11000-a-month-cancer-drug.html [accessed 07.06.13].
Contents lists available at ScienceDirect
Health Policy journal homepage: www.elsevier.com/locate/healthpol
Transparency in Canadian public drug advisory committees Zahava R.S. Rosenberg-Yunger a,∗ , Ahmed M. Bayoumi a,b,c,d a School of Health Services Management, Ted Rogers School of Management, Ryerson University, 350 Victoria St., Toronto, ON, Canada M5B 2K3 b Department of Medicine, University of Toronto, Toronto, ON, Canada c Institute of Health Policy, Management, and Evaluation, University of Toronto, Toronto, ON, Canada d Division of General Internal Medicine, St. Michael’s Hospital, Toronto, ON, Canada
a r t i c l e
i n f o
Article history: Received 12 November 2013 Received in revised form 7 July 2014 Accepted 18 August 2014 Keywords: Qualitative research Resources allocation Transparency Drug policy
a b s t r a c t Background: Transparency in health care resource allocation decisions is a criterion of a fair process. We used qualitative methods to explore transparency across 11 Canadian drug advisory committees. Methods: We developed seven criteria to assess transparency (disclosure of members’ names, disclosure of membership selection criteria, disclosure of conflict of interest guidelines and members’ conflicts, public posting of decisions not to fund drugs, public posting of rationales for decisions, stakeholder input, and presence of an appeals mechanism) and two sub-criteria for when rationales were posted (direct website link and readability). We interviewed a purposeful sample of key informants who were conversant in English and a current or past member of either a committee or a stakeholder group. We analyzed data using a thematic approach. Interviewing continued until saturation was reached. Results: We examined documents from 10 committees and conducted 27 interviews. The median number of criteria addressed by committees was 2 (range 0–6). Major interview themes included addressing: (1) accessibility issues, including stakeholders’ degree of access to the decision making process and appeal mechanisms; (2) communication issues, including improving internal and external communication and public access to information; and (3) confidentiality issues, including the use of proprietary evidence. Conclusion: Most committees have some mechanisms to address transparency but none had a fully transparent process. The most important ways to improve transparency include creating formal appeal mechanisms, improving communication, and establishing consistent rules about the use of, and public access to, proprietary evidence. © 2014 Elsevier Ireland Ltd. All rights reserved.
1. Introduction Decisions regarding public drug formulary listings can be controversial [1]. In Canada, such decisions are generally made by provincial or territorial Ministries of Health based on recommendations from drug advisory
∗ Corresponding author. Tel.: +416 979 5000x4213; fax: +416 979 5209. E-mail addresses: [email protected], [email protected] (Z.R.S. Rosenberg-Yunger). http://dx.doi.org/10.1016/j.healthpol.2014.08.010 0168-8510/© 2014 Elsevier Ireland Ltd. All rights reserved.
committees. Because these decisions can be contentious, the recommendation process should be legitimate and fair; transparency is an important component for judging fairness [2,3]. The American College of Physicians defines transparency within the health care context as “making available to the public, in a reliable and understandable manner, information on the health care system’s quality, efficiency and consumer experience with care” [4]. Accountability for Reasonableness, a popular framework for judging fairness in health care, asserts that transparency in health resource allocation decision making requires
256
Z.R.S. Rosenberg-Yunger, A.M. Bayoumi / Health Policy 118 (2014) 255–263
that both the recommendation process and the rationale for making recommendations are publicly available [3]. Transparency is valued because it may result in better informed recommendations, decrease appeals, increase public acceptance of recommendations, and enhance trust in the overall process [1,5]. In Canada, public drug plans are administered at a subnational (provincial or territorial) level; a few national plans also exist for specialized populations, including First Nations and Inuit Canadians, inmates in federal penitentiaries, refugee claimants, military personnel, members of the Royal Canadian Mounted Police, and veterans [6]. Drug plans vary in their coverage but all insure people who are receiving social assistance [7,8]. Some drug plans insure the elderly or have established catastrophic coverage [9]. Coverage decisions are typically made at the ministerial or senior bureaucratic level, based on the recommendation of a sub-national drug advisory committee. These committees, in turn, receive recommendations from two national committees—one assessing cancer drugs and another assessing all other drugs. These review processes are known as the pan-Canadian Oncology Drug Review and the Common Drug Review, respectively, and their expert committees. The province of Quebec has opted not to participate in the national review committee and has its own processes and committees which operate separately [10]. The number of people served by each public plan varies widely, reflecting large variations in the populations of each region and coverage policies. The review process typically starts with a submission from a manufacturer for consideration of listing on a public drug formulary. The review includes assessments by staff members of the drug’s effectiveness, safety, and cost-effectiveness compared to current treatments. At the national level, review staff solicit input from patient groups as well as from the manufacturer. All of these materials are provided to the expert committees as inputs into their decision making process. In summary, drug coverage in Canada integrates provincial and federal payors, multiple advisory committees, and a federal system in which accountability for most funding decisions is at the sub-national level. Because this system is complex and regionalized, there is the potential for considerable heterogeneity in processes [1,2,11–13]. Furthermore, small committees might not have the same resources or pressures as their larger counterparts. Transparency, as a component of fairness, has not been formally assessed within this context. We used qualitative methods to evaluate transparency across 11 Canadian drug advisory committees. 2. Methods We used a literature review, key informant interviews, and document reviews to assess transparency. 2.1. Definition of transparency We based our definition of transparency on Daniels and Sabin’s “Accountability for Reasonableness” priority setting framework, which considers decisions to be
legitimate and fair if they satisfy four conditions: publicity, relevance, revision and appeals, and regulation (also referred to as enforcement or leadership) [3]. Gibson and colleagues have proposed adding a fifth “empowerment” condition, which focuses on addressing power differences between groups within the decision-making context and to optimising opportunities for participation [14]. We focused on the conditions that we considered most relevant to issues of transparency (publicity, relevance, revisions appeal, and empowerment). Accordingly, transparency requires that reports, appraisals and decision processes are readily available and that the decision process includes definitions of the roles and responsibilities of each stakeholder. We defined four criteria related to the condition of publicity: public disclosure of committee members’ names; publicly available criteria by which committee members are selected; public disclosure of conflict of interest guidelines and committee members’ conflicts; and public posting of decisions not to fund drugs. We included a criterion related to relevance, public posting of rationales for decisions (including the evidence upon which decisions were made) and a criterion related to empowerment, assessing whether there were adequate opportunities for stakeholder input during the review process. Finally, we included a seventh criterion to assess the presence or absence of a mechanism for appeal. When rationales were posted on the internet, we also assessed: (1) whether they were accessible through direct links on committee websites; and (2) the readability of the five most recent documents posted on the websites, using the Flesch–Kincaid or the Kandel–Moles instrument for English and French rationales, respectively [15,16]. We scored rationales as readable if they scored at a grade 12 level or lower which should make them accessible to 87% of Canadians who have at least a high school education [17]. The rationales reviewed for readability are listed in the Supplementary Appendix.
2.2. Interviews and documents We identified potentially eligible key informants from websites listing drug advisory committee members, suggestions by staff in Ministries of Health, and other informants [18,19]. Our respondents represented a purposeful sample with a range of views, selected to develop a rich understanding of the topic. We used a combination of maximum variation sampling, in which we selected respondents that were representative of different stakeholder groups, and snowball sampling, in which we asked respondents to identify other participants. Our inclusion criteria were as follows: individuals who were conversant in English, who were either current or former members of provincial or federal drug advisory committees (including clinical and public or patient experts), employees of a Ministry of Health, members of a patient advocacy group, or employees of a pharmaceutical manufacturer. Interviews were semi-structured and based on a pre-established interview guide (Box 1, Interview Guide). All interviews were digitally recorded and professionally transcribed.
Z.R.S. Rosenberg-Yunger, A.M. Bayoumi / Health Policy 118 (2014) 255–263
Box 1: Interview guide. Acceptance criteria: 1. Do you think the public members involved are representative? [Probes: Do they represent patient interests? Do they represent broader public interests? Do they provide their own expertise?] 2. What do you think the purpose of public involvement should be? 3. In your estimation, do you think that the selection process is conducted in a clear, appropriate and transparent manner? 4. At which point in the process do you think the public should be involved? [Probes: Consultation vs. Deliberation vs. Appeals] 5. To what degree do you think the public should be involved? [Probes: Opinion vs. Deliberation] 6. To what extent do you think that public input affects the final decisions? [Probe: Can you give me examples of where you think the public did and did not affect a decision?] 7. Do you think the drug deliberation and recommendation process is transparent? Process criteria: 1. Do you think that public participants have access to the appropriate resources to enable them to successfully fulfill their role? 2. How would you define the role of public participation? 3. Do you think that the nature and scope of the participation task is clearly defined? 4. Do you think it is efficient to involve public in decisions? Influence of public members (power issues): Committee members: 1. What was the extent of your involvement in the committee? 2. Were social values incorporated into discussions? 3. What values are discussed? Were they discussed implicitly or explicitly? 4. How are these values reflected in the final recommendation? 5. Were the goals of public participation clearly defined? What were these goals? Do you think any other goals should be considered and why? 6. How would you have liked to be involved? 7. Whom do you think should be involved in these decisions? 8. Were any decisions altered because of public input? [Only ask if not answered above in q 3] Others: 1. Do you think that the values of the public are sufficiently incorporated into the decisions? 2. What do you think is the optimal way to involve the public? Communication/Knowledge: Committee members: 1. Did you receive any training to prepare you for your membership on the committee? [Probes: Please elaborate on the training you received? Do all committee members (i.e., including public members) receive training? Do you think the training is appropriate and useful?] 2. To what extent was the information regarding the drug communicated to you? 3. To what extent was the information about the drug understood by you? 4. Did you feel that you had enough information to take part in the process? 5. What weight was given to public input? [Probe: Are their comments respected? Do they actively take part in discussions?] 6. What do you consider to be optimal/successful public involvement? How do you define successful involvement? Others: 1. Do you think that the public sufficiently understands the issues presented to them in order to participate in the deliberative process? 2. What do you consider to be optimal/successful public involvement? How do you define successful involvement? Is there anything else you would like to add?
257
2.3. Analysis We analyzed the interviews and documents using a qualitative thematic approach consisting of line-by-line coding to develop categories. We used constant comparison to examine relationships within and across codes and categories. The first five transcripts were coded by both investigators and we discussed discrepancies and established consensus for the preliminary coding scheme. All subsequent transcripts were coded by one investigator. When discrepancies occurred both investigators met to established consensus. Interviewing continued until saturation was reached. We used NVivo 8 software to code, store, and organize the data [20]. 3. Results We considered 17 committees (Supplementary material). We excluded six committees that make recommendations about relative small populations: five federal committees and the committee advising Nunavut’s Extended Health Benefits program for non-aboriginal individuals (Supplementary material). The remaining 11 committees represented the two national committees, all Canadian provinces and the Yukon Territory, and the Drugs and Therapeutics Advisory Committee for the Non-Insured Health Benefits program, which provides coverage for a limited range of goods and services for First Nations and Inuit Canadians who are not insured elsewhere. These 11 committees have 125 members. We approached 32 individuals from nine committees (the committees from the Atlantic provinces and Manitoba declined to participate), of whom 27 (84%) agreed to participate. We also collected documents from 10 committee websites (documents from the Drugs and Therapeutics Advisory Committee were not available online). 3.1. Interviews The median age of participants was 55 (range 39–68), 18 (67%) were women, and 20 (74%) had a professional or graduate degree. We interviewed four industry representatives; six patient group representatives; one past government employee and committee member; five current government employees; and 11 current committee members. Each patient group that participated in our study had disclosed on their website that they had received industry funding. The major themes which emerged included issues regarding accessibility, communication, and confidentiality. 3.2. Accessibility issues 3.2.1. Appeal mechanisms Many participants discussed the need for formal mechanisms to appeal committee recommendations. A pharmaceutical manufacturer employee respondent highlighted how appeals differ from reconsiderations: The reconsideration [process] . . . is the same number of people looking at the . . . information . . . [An] appeal
258
Z.R.S. Rosenberg-Yunger, A.M. Bayoumi / Health Policy 118 (2014) 255–263
[process] would be . . . a separate and different group of people. (INT8) Another pharmaceutical manufacturer employee respondent noted that an appeal mechanism might facilitate greater discourse about controversial issues. However, some committee members felt that more opportunities for input might also represent opportunities for manufacturers to sway recommendations. Respondents also noted that the present mechanisms for reconsideration are often limited to manufacturers. A patient group representative respondent commented that there is no “Formal process for members of the public to raise concerns or ask any questions, or even appeal” (INT24). 3.2.2. Open processes Open processes permit stakeholders (including the public) to access part of the decision making process. One government employee explained that: The future is potentially having opened . . . committee deliberation . . . where [everyone] sits and listens to evidence and it’s done in an open format for anyone to hear what evidence is being presented, and . . . then the [committee] . . . is sequestered and can debate the merits of the evidence. (INT10) However, respondents noted that open processes could result in logistical challenges. “You have 200 people show up to the meeting, you have to have a venue that’s going to support that” (INT5). Additionally, a few respondents believed that an open process would result in an unwanted increase in manufacturer involvement and increase the time required for making recommendations. As one committee member noted: “I just don’t know if we can afford the reduction in productivity given our small scale” (INT21). Finally, an open process “Might inhibit some committee members in being frank in their discussions” (INT3). 3.3. Communication issues 3.3.1. Access to recommendation deliberations Although most committees post reports online (Table 1), few post meeting deliberations or supporting documents. A patient group respondent believed that the deliberation process has the “appearance of transparency” (INT14) while an industry respondent thought “there still is a little bit of a black box around how decisions are made” (INT15). One committee member suggested communicating to the public the “kind of discussion that goes [on] back and forth” (INT13). A government employee noted that committees “struggle” with making recommendations in situations when drugs are challenging to evaluate using existing criteria, such as expensive orphan drugs that are unlikely to be cost-effectives (INT3). In such situations, it is important that committees clearly communicate the reasons for their recommendations. As one pharmaceutical respondent noted: Every drug is different and every disease is different, and there are really different factors that drive it. But these evidence evaluation bodies need to do a better job at explaining that. (INT11)
Respondents remarked that committee reports often inadequately reflect the rationale and deliberation behind recommendations, that postings should be done in a timely manner, and that effective communication might involve direct correspondence with patient groups and the use of media including email alerts and meetings. Nonetheless, one committee member noted the need to balance transparency with members’ ability to function, “Not everything we say is published . . . I know that that limits transparency . . . but . . . it also enables people to say what’s on their mind” (INT26). 3.3.2. Access to conflict of interest guidelines and disclosures Only three committees (Yukon Formulary Working Group, NIHB’s Drugs and Therapeutics Advisory Committee, and Manitoba’s Drug Standards and Therapeutics Committee) did not post their conflict of interest guidelines online. Two committees (p-ERC and CDEC) also posted members declarations on their websites (Table 1). As one government respondent noted: . [Members] have to declare any funding sources that may lead to a real or perceived conflict-of-interest, and we’re transparent about that, so their conflict of interest gets posted . . . It at least raises the awareness . . . Because . . . often the first response [is] . . . I don’t have any conflicts. (IN10) 3.3.3. Clarity regarding use of evidence Committees consider multiple types of evidence, including clinical appraisals, economic, evaluations, and patient input. Stakeholders often inquired about the relative weighting of these types of evidence, in particular how patient input from formal submissions, website feedback or informal discussions was used. A government employee respondent noted: Is the patient input evidence weighted in the same way as the clinical and the cost evidence? . . . We don’t have a formula . . . for weighting. All of the . . . evidence is part of this deliberative process. (INT3) Many respondents wanted better understanding of this process. As one patient respondent noted: It needs to be a lot clearer in terms of . . . the balance of criteria . . . it would be good to know . . . what is the process that the committee uses . . . Is there a consideration of the data first or patient submissions? . . . How do you debate those? (INT6) 3.3.4. Understanding how recommendations lead to decisions Respondents noted that recommendations might not match the final decisions taken by the health plan or Minister of Health. One committee member noted that “the government was overturning a lot of things that we were doing, and people were getting quite frustrated” (INT17). Furthermore, the transparency of the decision making process varies across health plans. As one government employee noted:“[Even if] something’s approved doesn’t mean it gets on
Z.R.S. Rosenberg-Yunger, A.M. Bayoumi / Health Policy 118 (2014) 255–263
259
Table 1 Comparison of Canadian drug advisory committees.
Members Clinical experts (N) Patient or public expert members (N) Transparency criteria Are member names available online? Are member selection criteria available online? Are “no” decision available online? Is there stakeholder input prior to recommendation? Registered Patient Group Individuals Is there an appeals process? Are rationales available online? Are members’ conflict of interest disclosures available online?
YT
BC
AB
SK
MB
ON
QC
ATL
NIHB
CDEC
pERC
3 0
9 3
8 0
12 2
6 0
15 2
11 2
10 0
12 0
12 2
14 2
N
Y
Y
Y
N
Y
Y
Y
N
Y
Y
N
Y
N
N
N
Y
N
N
N
Y
Y
N
Y
N
Y
N
Y
N
Y
N
Y
Y
NN/A
YY
YNot Specified
N/A
NN/A
YY
YY
NN/A
NN/A
YY
YY
N/A N/A N N N
Y Y N Y N
Not Specified Not Specified N N N
N/A N/A N N N
N/A N/A N N N
Y N N Y N
Y N N Y N
N/A N/A N N N
N/A N/A N N N
Y N N Y Y
Y N N Y Y
N denotes No, Y denotes Yes, N/A denotes not applicable, YT denotes Yukon Territory, BC denotes British Columbia, AB denotes Alberta, SK denotes Saskatchewan, MB denotes Manitoba, ON denotes Ontario, QC denotes Quebec, NIHB denotes Drugs and therapeutics advisory committee for the noninsured health benefits, CDEC denotes Canadian Drug Expert Committee, and pERC denotes p-CODR Expert Review Committee.
the formulary and then you’ve got to go digging in individual provinces. . .and each province has its own slightly different system” (INT2). 3.4. Confidentiality issues 3.4.1. Proprietary evidence Some respondents thought that there is a need to challenge the current administrative norms regarding what data is considered to be proprietary. For instance, some respondents thought that proprietary evidence should not be permitted in the recommendation process. A former committee member commented, “We shouldn’t be accepting any information that isn’t releasable, because it reduces our ability to defend the decision” (INT21). Similarly an industry respondent noted the pan-Canadian Oncology Drug Review’s approach to proprietary evidence, “When they start a process, one of their first points of engagement with industry is to say, is there data here that absolutely cannot be disclosed? . . .It’s known by industry that they may say if we can’t disclose it because it’s germane to our decision, then we won’t use it (INT 15). Another committee member discussed frustration with receiving incomplete data, particularly related to drug costs: [Manufacturers] submitted the information with . . . cost information . . . blacked out, well what the hell is that? . . . [I] understand the commercial need for . . . drug companies to protect proprietary data and their proprietary information. So how do we find the balance between those pieces [i.e., transparency and proprietary information]? (INT26) A few members mentioned that achieving full transparency was difficult given the current administrative norms because
“We are not able . . .to make all the information provided from the manufacturers public because obviously that becomes available to all their competitors” (INT13). 3.4.2. Members’ names While most committees make members’ names publicly available, three did not (Table 1), ostensibly to protect members from litigation and from lobbying by stakeholder groups. One committee member noted, “When I look at the transparency of the process, I think our names do need to be published. And I know that there’s risk in doing that” (INT26). 4. Document review We obtained data from eight provincial and two national Canadian public drug expert advisory committees (Table 2). Of our seven transparency criteria, the median number addressed by committees was 2 (range 0–6). Both national committees, British Columbia, Quebec and Ontario each met six criteria; the Yukon Territory and Manitoba did not meet any criteria. Of the five committees that posted rationales, each posted a direct link on the website to rationales that were at a grade 12 reading level or less. No committee had mechanisms for appeal but most had processes for reconsideration (Table 3). The criterion that was least likely to be consistently addressed was making member selection criteria available (addressed by four committees). The criterion that was most likely to be addressed was posting of members’ names online (met by eight committees). 5. Discussion We used qualitative methods to explore transparency across 11 Canadian drug advisory committees. Our most striking finding was the large degree of heterogeneity regarding mechanisms to address transparency across the
260
Z.R.S. Rosenberg-Yunger, A.M. Bayoumi / Health Policy 118 (2014) 255–263
Table 2 Documents reviewed. Committee
Document/web title
Yukon Formulary Working Group
Yukon Drug Programs Formulary http://www.hss.gov.yk.ca/pdf/yukon drug programs formulary.pdf Drug Benefit Council membership, remuneration to drug benefit members, terms of reference and conflict of interest http://www.health.gov.bc.ca/pharmacare/formulary/dbc info.html http://www.health.gov.bc.ca/navigation/pdf/DBC remuneration.pdf http://www.health.gov.bc.ca/pharmacare/formulary/pdf/DBC-TermsOfRef.pdf Resubmission for the Alberta Drug Benefit List, Alberta Pharmaceutical Strategy, Alberta Health and Wellness Benefits List, https://www.ab.bluecross.ca/dbl/pdfs/ahwdbl sec1 intro.pdf Terms of Reference http://formulary.drugplan.health.gov.sk.ca/PDFs/DACSTOR.pdf Email correspondence with ministry and Manitoba Drug Review process http://www.gov.mb.ca/health/mdbif/review.html Membership, term, remuneration, http://www.pas.gov.on.ca/scripts/en/boardDetails.asp?boardID=817 Mandate, members, operating ruleshttp://www.inesss.qc.ca/index.php?id=26&L=1
British Columbia Drug Benefit Council
Alberta’s Expert Committee on Drug Evaluation and Therapeutics Drug Advisory Committee of Saskatchewan Manitoba Drug Standards and Therapeutics Committee Ontario Committee to Evaluate Drug L’Institut national d’excellence en santé et en services sociaux (INESSS) Comité scientifique permanent en santé et en services sociaux, Quebec Atlantic Expert Advisory Committee
Canadian Drug Expert Committee p-CODR
Committee members, Atlantic Common Drug Review http://www.gov.ns.ca/health/pharmacare/committees/EAC members.pdf http://www.gov.ns.ca/health/Pharmacare/committees/acdr.asp Process, conflict of interest, terms of reference http://www.cadth.ca/en/products/cdr/committees/canadian-drug-expert-committee http://www.pcodr.ca/wcpc/portal/Home/General PC/Opportunities? afrLoop=209898743908000& afrWindowMode=0& adf.ctrl-state=dcdh3kmuj 4#perc http://www.pcodr.ca/portal/server.pt/community/about pcodr/546/review committee
country. Using liberal criteria, we found that committees fell short of having a fully transparent process. Respondents suggested that the most important ways to improve transparency are to create formal stakeholder inclusive appeal mechanisms, improve communication, and establish consistent rules about how proprietary evidence is used and made available. No committee had an appeals mechanism. Given that manufacturers have an interest in appealing all negative recommendations and that an appeals process can be resource-intensive, clear guidelines describing the circumstances under which an appeal would be considered need to be established. Notably, the House of Common’s Standing Committee in Health in 2007 recommended that the federal non-oncology committee establish an appeal process which would include a separate group of reviewers and permit the public to appeal [21]. Such a process does not exist yet in Canada. Both manufacturers and patient groups also use mechanisms outside of the committee process to influence and sometimes alter decisions, such as media and letter writing campaigns to the ministers of health [22]. Because the decisions made after such lobbying are frequently opaque, formal appeals process will result in more transparent and consistent recommendations and decisions. In contrast to appeals, which entail a review of the recommendation by a different body, reconsiderations entail review by the same body that made the initial decision. Many committees have well-established reconsideration processes to review new information, such as new clinical trial or safety data, or to re-examine the interpretation of data. Most reconsiderations processes for specific drugs in
Canada are only open to manufacturers (Table 2), although some committees also have mechanisms for reconsideration of classes of drugs as evidence accumulates [2]. Manufacturers and patients believed that reconsiderations for specific drugs that were adjudicated by the same committee that made the original decision were unlikely to result in a decision being changed. There is also considerable heterogeneity regarding whether a submission for reconsideration would be allowed for any decision or only when processes were not properly followed. The committees varied considerably regarding processes for patients and public input. Some committees only permit registered patient advocacy groups to submit patient evidence (such as Ontario’s committee and both national committees) while others permit individual patients and caregivers to submit input (such as British Columbia) and some have no process for input (such as the Yukon Territory). Requiring patient groups to register increases transparency regarding disclosure of conflict of interest but limits the range of patients who participate. Issues of empowerment among and between the general public, individual patients, and patient groups is an important topic for future research, including the role of industry funding for patient groups [23]. The committees are also varied regarding the information that patient groups are provided to assist them in assembling their input. For example, in British Columbia, patients are asked to read a Drug Information Sheet, which includes general information and cost comparators for the drug under review but no clinical benefit data. Patients then complete a questionnaire that focuses on their anecdotal experience. In contrast, the national committee
Z.R.S. Rosenberg-Yunger, A.M. Bayoumi / Health Policy 118 (2014) 255–263
261
Table 3 Characteristics of Revision Mechanisms. Jurisdiction
Circumstance
Stakeholder who can appeal
Type of mechanism
Adjudication
YT
Any decision of the Yukon Formulary Working Group Recommendation is not supported by the evidence Process of the submission reviewed was not properly followed New information Request a change in the current special authorization or restricted benefit status Request a change in current criteria for coverage Recommendation is not supported by the evidence New information New information Change in pricing Process of the submission reviewed was not properly followed Recommendation is not supported by the evidence The Minister asks committee to revise position New information New information Change in pricing Process of the submission reviewed was not properly followed Recommendation is not supported by the evidence Feedback was received on the initial recommendationConsensus was not reached by stakeholders The recommendation was negative
Not specified
Reconsideration
Director
Manufacturers
Reconsideration
Ministry DBC Secretariat in consultation with committee Chair
Manufacturer
Reconsideration
Committee
Physicians or manufacturer
Reconsideration
Committee
Manufacturer
Reconsideration
Committee
Manufacturers
Reconsideration
Committee
Minister
Reconsideration
Minister of Health
Manufacturers Manufacturers, patient groups Manufacturers
Resubmission Reconsideration
Committee Committee
Reconsideration
Committee
Stakeholders (including but not limited to manufacturers or their representative organizations, tumor groups, and patient advocacy groups)
Reconsideration
Committee
BC
AB
SKa
MB ON
QC ATL NIHBa CDEC
pERC
YT denotes Yukon Territory, BC denotes British Columbia, DBC denotes Drug Benefit Council, AB denotes Alberta, SK denotes Saskatchewan, MB denotes Manitoba, ON denotes Ontario, QC denotes Quebec, NIHB denotes Drugs and therapeutics advisory committee for the non-insured health benefits, CDEC denotes Canadian Drug Expert Committee, and pERC denotes p-CODR Expert Review Committee. a These are informal processes.
that considers cancer drugs invites patient input both before the manufacturer’s submission is considered and after the initial recommendation is made; at that point, patient groups have access to the initial recommendation, including information on clinical benefit, patient values, economic evaluation, and adoption feasibility. Some recommendations from respondents will be very challenging to implement. For example, introducing open deliberations would require considerable resources and place additional burdens on committee members. Furthermore, there has been a vigorous debate in the literature regarding the relative merits of fair processes, formal decisions frameworks, or both for making fair decisions [24,25]. The lack of transparency regarding which types of evidence were most influential in a decision has contributed to the public’s perception of unfairness and lack of reproducibility when some drugs are approved and others denied. Conversely, committees need flexibility to make decisions that are sensitive to specific contexts and needs. We anticipate
that committees will continue to grapple with the tension between transparency and flexibility. There is concern among respondents that mechanisms to increase transparency will privilege some stakeholders over others and thus influence power relations, another important consideration in fair decision making [14,26]. For example, mechanisms for appeal or reconsideration of negative decisions or increased openness (with its attendant risk of lobbying of committee members) could result in granting greater power to manufacturers. Alternatively, mechanisms such as increased transparency with regards to proprietary information could decrease manufacturers’ influence. Interestingly, our participants did not specifically discuss transparency regarding clinical trial data. Notably, since 2010, the European Medicines Agency releases clinical trial reports upon request with the goal of publishing this data proactively for all drugs that are reviewed [27]. The United States Food and Drug Administration also has initiatives to increase transparency
262
Z.R.S. Rosenberg-Yunger, A.M. Bayoumi / Health Policy 118 (2014) 255–263
regarding drug approval and has increased the amount of information accessible to the public [28]. The question of transparency in the context of drug cost is particularly challenging. The list price of a medication may differ from the manufacturer’s confidential submitted price; both of these may vary from the final negotiated price by drug plans. These differences might explain some of the reluctance to make proprietary information or committee deliberations public and also why decisions sometimes differ from recommendations. Although some jurisdictions list “official drug prices” on their website, the real price paid by provincial plans might be very different after confidential product listing agreements or price negotiations [29]. Because such agreements and negotiations can have international consequences, it is unlikely that fully transparent price data will ever be publicly available. Our study has some limitations. We studied only drug approval mechanisms while recognizing that device and technology advisory committees face similar issues [30,31]. We analyzed whether rationales were accessible but did not analyse the quality or comprehensiveness of rationale documents beyond readability. Finally, we did not include six small committees. 6. Conclusions Canadian drug advisory committees have lower recommendation for listing rates than their counterparts in other countries for example, CDR has a recommended drug listing rate of 53.4% compared with NICE’s rate of 87.4% [32,33]. Accordingly, such decisions will remain both challenging and contentious, particularly when very expensive drugs are introduced [34]. Some steps to increase transparency are straightforward and can be implemented rapidly, such as publicising member selection criteria and decision rationales. We believe that increased transparency will lead to better decisions and will increase public confidence in formulary listing decisions. Acknowledgments Dr. Bayoumi is supported by the Canadian Institutes of Health Research/Ontario Ministry of Health and Long-Term Care Applied Chair in Health Services and Policy Research. Conflict of interest The authors have no conflict of interest. References [1] Morgan S, McMahon M, Mitton C, et al. Centralized drug review processes in Australia, New Zealand, the United Kingdom, and Canada. Health Affairs 2006;25:337–47. [2] Mitton C, Morgan S, McMahon M, Gibson J. Centralized drug review processes: are they fair? Social Science and Medicine 2006;63:200–11. [3] Daniels N, Sabin J. Setting limits fairly. Oxford: Oxford University Press; 2002. [4] American College of Physicians. Healthcare transparency—focus on price and clinical performance information: policy paper. American College of Physicians; 2010 www.acponline.org/advocacy/ current policy papers/. . ./transparency.pdf [accessed 30.04.14, 2014]. [5] Bruni R, Laupacis A, Martin DK. Public engagement in setting priorities in health care. CMAJ 2008:179.
[6] Report of the Standing Committee on Health Prescription drugs part 1—Common Drug Review: An f/p/t Process. House of Commons 39th Parliament 2nd session. Communication Canada—Publishing; 2007 http://www.parl.gc.ca/HousePublications/Publication.aspx? DocId=3162492&Mode=1&Parl=39&Ses=2&Language=E [accessed 07.06.13]. [7] Provincial and Private Drug Insurance Plans in Canada. Parliamentary research branch economics division; 1996 http://publications. gc.ca/Collection-R/LoPBdP/MR/mr142-e.htm [accessed 07.06.13]. [8] About the pan-Canadian Oncology Drug Review (pCODR); 2011 http://www.pcodr.ca/wcpc/portal/Home/AboutpCODR? afrLoop= 202027728867000& afrWindowMode=0& adf.ctrlstate=n9bjllmhl 47 [accessed 15.10.12]. [9] Daw JR, Morgan SG. Stitching the gaps in the Canadian public drug coverage patchwork. A review of provincial pharmacare policy changes from 2000 to 2010. Health Policy 2012;104:19–26. [10] Bate A, Mitton C. The application of economic principles in health care priority setting. Expert Review of Pharmacoeconomics and Outcomes Research 2006;6:275–84. [11] Tierney M, Manns B. Optimizing the use of prescription drugs in Canada through the Common Drug Review. CMAJ 2008;178: 432–5. [12] McMohon M, Morgan S, Mitton C. The Common Drug Review: a NICE start for Canada? Health Policy 2006;77:339–51. [13] Laupacis A. On bias and transparency in the development of influential recommendations. CMAJ 2006;174:335–6. [14] Gibson JL, Martin DK, Singer PA. Priority setting in hospitals: fairness, inclusiveness, and the problem of institutional power differences. Social Science and Medicine 2005;61:2355–62. [15] Ley P, Florio T. The use of readability formulas in health care. Psychology, Health & Medicine 1996;1:7–28. [16] Bamberger R, Rabin AT. New approaches to readability: Austrian research. The Reading Teacher 1984;37:512–9. [17] A Profile of Education in Canada. Canadian council on social development; 2010 http://www.ccsd.ca/factsheets/education/ [accessed 22.01.13]. [18] Denzin NK, Lincoln YS. In: The SAGE handbook of qualitative research, 3rd Thousand Oaks: California: Sage Publications; 2005. [19] Maxwell J. Qualitative research design London. London, UK: Sage Publications; 2005. [20] NVivo 8 [software]. Cambridge Qualitative Solutions Research International; 2008, Melbourne, Australia: QSR International. [21] Government Response to the Second Report of the Standing Committee on Health Prescription Drugs Part 1—Common Drug Review: An F/P/T Process. Parliament of Canada; 2007 http://www.parl.gc.ca/ HousePublications/Publication.aspx?DocId=3396385&Language=E& Mode=1&Parl=39&Ses=2 [accessed 19.12.12]. [22] Rosenberg-Yunger Z, Thorsteinsdóttirb H, Daar A, Martin D. Stakeholder involvement in expensive drug recommendation decisions: an international perspective. Health Policy 2012;105: 226–35. [23] Cohen D, Raftery J. Paying twice: questions over high cost of cystic fibrosis drug developed with charitable funding. BMJ 2014; 12:348. [24] Culyer AJ. Hic sunt dracones: the future of health technology assessment—one economist’s perspective. Medical Decision Making 2012;32:E25–32. [25] Gruskin S, Daniels N. Process is the point: justice and human rights: priority setting and fair deliberative process. American Journal of Public Health 2008;98:1573–7. [26] Abelson J, Forest P, Eyles J, Smith P, Martin E, Gauvin F. Deliberations about deliberative methods: issues in the design and evaluation of public participation processes. Social Science & Medicine 2003;5:239–51. [27] The European Medicines Agency Release of data from clinical trials. The European medicines agency http://www.ema.europa.eu/ema/ index.jsp?curl=pages/special topics/general/general content 000555.jsp&mid=WC0b01ac0580607bfa [accessed 14.03.14]. [28] Wechsler J. Transparency to alter FDA and industry practices. Pharmaceutical Technology 2010;34:28–34. [29] BC PharmaCare wants to hear Your Voice. Which drugs are being reviewed? Government of British Columbia 2013, http://www. health.gov.bc.ca/pharmacare/yourvoice/#druglist [accessed 14. 03.14]. [30] Hulstaert F, Neyt M, Vinck I, et al. Pre-market clinical evaluations of innovative high-risk medical devices in Europe. International Journal of Technology Assessment in Health Care 2012;28: 278–84.
Z.R.S. Rosenberg-Yunger, A.M. Bayoumi / Health Policy 118 (2014) 255–263 [31] Hailey D. Toward transparency in health technology assessment: a checklist for HTA reports. International Journal of Technology Assessment in Health Care 2003;19:1–7. [32] Clement FM, Harris A, Li JJ, Yong K, Lee KM, Manns BJ. Using effectiveness and cost-effectiveness to make drug coverage decisions: a comparison of Britain, Australia, and Canada. JAMA 2009;302:1437–43.
263
[33] Rocchi A, Miller E, Hopkins RB, Goeree R. Common Drug Review recommendations: an evidence base for expectations. Pharmacoeconomics 2012;30:229–46. [34] In cancer care, cost matters. New York Times; 2012 http://www. nytimes.com/2012/10/15/opinion/a-hospital-says-no-to-an11000-a-month-cancer-drug.html [accessed 07.06.13].