HTA of medical devices: Challenges and ideas for the future from a European perspective

Accepted Manuscript Title: HTA of medical devices: Challenges and ideas for the future from a European perspective Author: Sabine Fuchs Britta Olberg Dimitra Panteli Matthias Perleth Reinhard Busse PII: DOI: Reference:

S0168-8510(16)30215-9 http://dx.doi.org/doi:10.1016/j.healthpol.2016.08.010 HEAP 3611

To appear in:

Health Policy

Received date: Revised date: Accepted date:

2-3-2016 13-8-2016 28-8-2016

Please cite this article as: Fuchs S, Olberg B, Panteli D, Perleth M, Busse R, HTA of medical devices: Challenges and ideas for the future from a European perspective, Health Policy (2016), http://dx.doi.org/10.1016/j.healthpol.2016.08.010 This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

HTA of medical devices: Challenges and ideas for the future from a European perspective Sabine Fuchs1, Britta Olberg1,2, Dimitra Panteli1, Matthias Perleth1,2, Reinhard Busse1 Department of Health Care Management, Berlin University of Technology, Germany

2

Medical Consultancy Department, Federal Joint Committee (G-BA), Germany

ip t

1

cr

Corresponding author: Sabine Fuchs

us

Berlin University of Technology

an

Department of Health Care Management Straße des 17. Juni 135, H 80

Highlights

• • • •

Unique overview of European practices and views on HTA of medical devices (MD)

Ac ce pt e

•

d

[email protected]

M

10623 Berlin, Germany

Further regulatory changes at European level are required to generate adequate clinical data for HTA of MDs Approaches such as coverage with evidence development should be considered at national level More alignment of HTA and regulatory processes is needed to improve the current situation More methodological discussion and collaboration between different stakeholders is indicated

1. Introduction

Health technology assessment (HTA) is a support tool for evidence-based decision-making on the safe, effective and efficient introduction and use of technologies in the health care system. Despite the wide adoption of HTA in many contexts, the majority of evidence-based coverage Page 1 of 29

decisions focus on pharmaceuticals. Nevertheless, medical devices (MDs), as a heterogeneous group of products intended for different purposes, are also addressed by many HTA institutions. Europe is one of the biggest markets for MDs, encompassing over 500 000 registered products from wound dressings to PET/CT scanners with an increasing number of European MD patent applications since 2004 (1).

ip t

Valid data on clinical effectiveness and safety of a technology are essential for producing robust HTAs. However, in the case of MDs, high quality data are not always available. On the one hand this has been attributed to existing regulatory requirements for market entry at

cr

European level (2) based on directives 90/385EEC, 93/42/EEC and 98/79/EC. For introduction in the European market, MDs need a European Conformity (CE) mark from a

us

Notified Body. However, this CE mark does not presuppose a profound demonstration of clinical data relating to effectiveness of MDs. While this regulation has been revised pending

an

approval by the European Parliament (3), evidentiary requirements have not been changed substantially for all device types. On the other hand the unique features of MDs (e.g. short life

M

cycle) in comparison to other technologies pose challenges in conducting clinical trials (2, 4). Thus, performing HTA for MDs entails hurdles such as accounting for learning curve effect or organisational changes (5, 6).

d

Despite the fact that research on the perspectives of HTA agencies regarding MD assessment

Ac ce pt e

exists (7), no comprehensive study addressing this issue in the European context has been undertaken yet. Given the common regulatory framework for licensing of devices in the EU, it is important to consider countries’ experience to draw useful conclusions for future practice. As a first step towards fully understanding existing practices around MD assessment by European HTA institutions, a systematic review was carried out (8). Building on the results of this descriptive overview, the aim of the presented work was to 1) clarify, supplement and expand findings on the structural, procedural and methodological characteristics of leading European HTA institutions with regard to the assessment of MDs and 2) capture the institutions’ perceptions regarding challenges and trends as well as explore potential areas for future developments. For this purpose, interviews were conducted. While institutions were also asked to comment on a taxonomic model for MDs developed in the same project (9), those results are outside the scope of this article and will be presented separately. 2. Material and methods

2.1 Study design and period

Page 2 of 29

Semi-structured telephone interviews with representatives of European HTA institutions were performed between April and July 2015. 2.2 Institutions and participant selection Efforts were undertaken to identify all European institutions involved in HTA of MDs, using a systematic approach described elsewhere (8). Of the institutions identified, potential

ip t

interview partners were selected based on the number of publicly available HTA reports of MDs. This approach was adopted assuming that the amount of available reports is associated with the extent of an institution’s experience with MD assessment so far. Institutions were

cr

categorised according to the number of HTA reports on MDs produced between 2004 and 2014:

Institutions with fewer than 10 available reports (e.g. Agency for Quality and Accreditation in

us

-

Health Care and Social Welfare)

Institutions with between 10 and 60 available reports (e.g. Belgian Health Care Knowledge

an

-

Centre)

Institutions with more than 60 available reports (e.g. National Institute for Health and Care Excellence)

M

-

d

Six institutions per category were chosen. Once these had been selected, we consulted

Ac ce pt e

relevant experts from the authors’ and project partners’ networks to identify the appropriate person responsible for MD assessment within each institution. We sent an email invitation and three reminders in two-week intervals. Participants who confirmed their participation received the interview guide and additional material in advance. 2.3 Interview guide

Each participant was interviewed according to a semi-structured interview guide consisting of two parts (see Box 1; for more details see Appendix). The guide was based on the Drummond et al.’s framework of key principles for national HTA programmes (10) and was structured according to its general categories: (i) Structure of HTA programmes, (ii) Methods of HTA, (iii) Processes for Conduct of HTA and (iv) Use of HTAs in Decision Making. >>Box 1<< 2.4 Data collection and analysis The interviews were carried out by members of the research team either by telephone or in person depending on proximity. Interviewees provided oral consent to information from part I being reported in an institution-specific manner, information from part II being reported

Page 3 of 29

anonymously and the interview being recorded before any questions were asked. The interviews were transcribed verbatim by external translation agencies and transcripts were sent to interviewees for validation. The validated transcripts incorporating participants’ comments and/or corrections formed the basis for the analysis. For the analysis of part I we created a tabular overview for each institution (see Appendix) summarising the main information given. The compiled profiles built the basis for the

ip t

comparative analysis in the corresponding sections below. For the analysis of part II we used directed content analysis. According to Hsieh and Shannon (p. 1283), this approach aims to

cr

‘efficiently extend or refine existing theory’ (11). For this purpose, a coding framework was developed based on major themes taken from existing seminal work in the field of HTA (10)

us

and from own previous research findings (8). Themes that emerged during the data analysis but that did not fit into the initial framework we coded as new ones. Thus, the initial coding

an

framework was revised and refined iteratively during the analysis (see Appendix for the coding framework) and the overall approach combined deductive and inductive themes. Transcripts were coded by one researcher (SF) and results were checked and discussed with

M

another researcher (BO). Discrepancies were resolved by involving a third person (DP). The software Atlas.ti was used to facilitate the coding process.

d

To illustrate frequencies of relevant results we used - where appropriate - a categorisation of

Ac ce pt e

themes using the terms some (10-30%), many (31-65%) and most (66-100%) to depict how many out of all interviewed institutions mentioned each theme (12). Where indicated, we also took the institutions’ level of experience into consideration. The COREQ-checklist (13) was used to ensure that methods, results and discussion were reported appropriately. All interviewees had access to the completed analysis for final comment.

3. Results

3.1 Participating institutions

Of the 18 institutions invited to this study, 16 participated in the survey (in three instances 2 individuals from the same institution were interviewed simultaneously). All interviews were conducted by telephone (SF and DP/BO), with the exception of two (Agenas, IQWiG), which were conducted in person. The language used was English except in two cases (LBI-HTA, IQWiG). The duration of the interviews ranged from 34 to 100 minutes. Table 1 gives an overview of all participating institutions including their role in their HTA system. >>Table 1<<

Page 4 of 29

3.2 Part I: Institutional characteristics This section describes the most relevant results of part I of the interviews (see Box 1). A detailed profile for each institution can be found in the Appendix.

Separate department/unit for assessment of MDs

ip t

Many institutions (10 out of 16) have no separate department for MD assessment, mostly due to the size of the institution or staff constraints. Some other institutions focus mainly (Avalia-

cr

t, LBI-HTA) or solely (Agenas) on MD assessment.

Definition of MDs used

us

Given how heterogeneous MDs are, we intended to find out if the institutions themselves used a specific definition for or understanding of MDs. Six institutions claimed using no specific

an

definition, whereas four institutions referred to the relevant EU Directives (AAZ, Agenas, AOTMiT, NICE MTEP). Six interviewees stated that they use either existing definitions

M

(CFK, FinOHTA, OSTEBA; e.g. from the INAHTA glossary) or their own definitions which can be based also on national regulations (Avalia-t, HAS; OGYÉI TEI). Beside the variable definitions given, differences in the scope of interviewed institutions (i.e.

d

the type of devices assessed) could also be observed. For example, AOTMiT assesses new

Ac ce pt e

devices and procedures incorporating them, whereas ZiN assesses either MDs used in the outpatient setting (e.g. hearing aids) or in the course of medical specialist care (e.g. surgical implants). TLV distinguishes between ‘consumables’ (no HTA report, included in the benefit package) and MDs that are assessed on a trial basis commissioned by the government (renewed mandate that runs until December 2016). Some agencies assess single MDs from specific manufacturers, but most assessments in Europe focus on groups of similar MDs to be used as alternatives for the same intervention.

The process of MD assessment in comparison to other technologies Two institutions clearly indicated that there is a different process for MD assessment in comparison, for example, to the evaluation of drugs. Eight institutions stated that the general process of evaluation is nearly the same but slight differences exist. The main reported reasons for these differences rest with the information retrieval process, which is more standardised for drugs and based on formal manufacturer submissions. In contrast, search for information on MDs requires more time and often the institutions’ own initiative to contact the manufacturer. Among these eight institutions, three stated that differences in the approach

Page 5 of 29

may rather be attributed to different types of MDs (e.g. diagnostic vs. therapeutic devices). Only two institutions noted that there is no difference at all between devices and other types of technologies. The remaining four institutions’ scope focused mainly or solely on MDs and interviewees consider this question as not applicable. Prioritisation process used for MDs In seven cases no standardised prioritisation process exists to select topics for assessment,

ip t

because institutions are commissioned mainly by the relevant decision-makers to perform specific evaluations. In six institutions the prioritisation process and criteria for MDs are

cr

different from that used for other technologies. One of the six institutions emphasized that the

three cases no differences in processes were reported.

us

main difference is between new and established technologies rather than technology type. In

an

Methodological guidelines and specific methods for MDs

Most institutions have their own general guidance for carrying out assessments or adopting guidance from other HTA institutions. Some respondents indicated that existing guidelines

M

were under review at the time (e.g. SHTG/HIS HTA manual) or recent work on new documents had been carried out (e.g. unifying HTA document in Spain). Only four

d

institutions stated that they have a methodological document specifically for MDs or a

Ac ce pt e

separate part for MDs within their general document. However, eleven institutions answered the question on whether they use specific methods for different types of MDs with ‘Yes’. The interviewees stated that they most commonly use the EUnetHTA frameworks (e.g. Core Model™ for diagnostics) or existing guidelines from other institutions as reference.

Consideration of device-specific aspects

Fourteen interviewees stated that they consider aspects that are relevant for and specific to the assessment of MDs, such as device-operator interaction or minimum requirements (e.g. for skills of professionals). In most cases these were touched upon within the recommendation part of their assessment reports. However, the interviewees often added that there is no predetermined intention or framework for such considerations but they rather depend on the type of MD under evaluation and exact research question. Some institutions use checklists to guide the consideration of organisational aspects (e.g. OSTEBA) or to clarify specific questions, for example about how MDs work in a particular setting within a technical report (NICE MTEP).

Page 6 of 29

Link between MD assessment and coverage decision-making Nine out of sixteen interviewed institutions produce assessments that are not binding for any kind of decision-making (i.e. are just recommendations), as opposed to those of three institutions which are. In four cases, the binding legal effect of recommendations depends on either the specific programme (e.g. MTEP within NICE) or on who has commissioned the report (national or regional health service). Nevertheless, many institutions whose

ip t

assessments are not binding, expected that their work will be considered by the decision making body (e.g. MoH) nonetheless. Finally, most institutions reported trying to measure

cr

their assessments’ impact in various ways.

us

3.2 Part II: Challenges and future ideas

The second part of the interviews focused on interviewees’ perceptions regarding current

an

challenges in MD assessment and the way forward. Answers are presented by question below

M

and further illustrated in Tables, where appropriate.

Challenges specific to MD assessment: (i) from a structural perspective

d

Most institutions raised points regarding structural challenges (see Table 2). The theme ‘Transparency’ was touched upon by many interviewees. They indicated that there is a lack of

Ac ce pt e

information about which MDs are entering or are currently on the market. There are no registers available that could be helpful within the evaluation process. Detailed data about MDs with a CE mark that are in the possession of Notified Bodies are not available to HTA institutions. One interviewee emphasized one of the points as follows: ‘They (who evaluate drugs) know in advance for the next year which drugs they are looking at and the year after that, they’re usually able to plan a year in advance, and fill up all their slots. With (…) medical devices, we rely on the company notifying (us).’ Overall, challenges from a structural perspective were mostly raised by institutions with less experience regarding MD assessment. However, all institutions with extensive experience addressed the theme ‘Transparency’ as well.

>>Table 2<<

Challenges specific to MD assessment: (ii) from a procedural perspective

Page 7 of 29

Challenges from a procedural perspective were raised by most institutions. The most frequently cited challenge referred to ‘Information retrieval’ within the evaluation process of MDs, particularly requests to manufacturers for additional data. Compared to the assessment of drugs, which encompasses the standardised submission of data to the institution responsible for evaluation, MD assessment lacks a similar process. Within the information retrieval process another difficulty is that of identifying and focusing

ip t

on the most relevant literature (e.g. framing the research question) due to the frequent existence of incremental innovations resulting from the rapid pace of MD development:

cr

‘Then we discover there is a whole class of products or even one product but multiple generations of devices and that also brings with it some specific aspects in terms of knowing

us

what to focus on and which literature to focus on, because there could be quite big differences between the different generations or iterations of the devices.’

an

Overall, almost all institutions with a high level of experience addressed these issues, mainly the theme ‘Information retrieval’.

M

Challenges specific to MD assessment: (iii) methodological perspective The methodological challenges raised by all institutions fell under the theme ‘Evidence base’,

d

encompassing on one hand the fact that fewer studies on MDs are conducted or published and

Ac ce pt e

on the other hand that available studies are often not well reported. The weak evidence level was emphasized by all institutions as one interviewee expressed: ‘[…] Very often it is so, that if somebody asks us about a new device, there is hardly any published studies, which is usually not the case for drugs.’ One further theme also raised by many interviewees is the ‘Broader perspective of the assessment’, including the increased necessity to consider elements that go beyond effectiveness and safety, such as organisational issues that depend on operator, team and setting (e.g. technical specifications of different versions) but also the varying types of information required for different MDs (e.g. different outcome measures for diagnostics). The evaluation of complex interventions, wherein the use of different health technologies needs to be combined to achieve the intended clinical effect, poses additional challenges as one interviewee expressed for the example of co-dependency of diagnostics and drugs: ‘[…] we have faced some complications, especially when you think about public procurement and reimbursement processes. So most of the time what is reimbursed is the drug, but there is a need to establish an indication, which is most commonly established based on a positive or negative diagnosis performed with a medical device.’

Page 8 of 29

With regard to the mentioned themes, institutions with less experience addressed the theme ‘Broader perspective of the assessment’ more frequently while experienced institutions mainly mentioned the ‘Methodological tools’ and ‘Transferability’ (see Table 2).

Challenges specific to MD assessment: (iv) regulatory perspective During the interviews, themes encompassing challenges from a regulatory perspective

ip t

emerged inductively from the material. The theme ‘Weak EU regulation regarding licensing of MDs’ in particular was touched upon by many interviewees. They mentioned that the

cr

existing regulation on licensing of MDs in Europe is not as strict and structured as for drugs and does not require (well-established) proof of effectiveness of the device, as one

us

interviewee pointed out:

‘It's not demonstrating clinical effectiveness, it’s only a demonstration of quality from an

an

engineering perspective, and not that it [the MD] will work for users.’

Moreover, the interviewees added that the Notified Bodies use different processes that vary across and within countries.

d

experience.

M

As most of the interviewed institutions touched upon these themes, independently of level of

Ac ce pt e

Consequences of existing MD regulation for assessment and decision-making Insights relevant to both the assessment and subsequent decision-making were derived inductively form answers on the differences in assessing MDs and drugs. On one hand the use of MDs before sufficient evidence is available was flagged as a negative consequence by more than one interviewees, particularly as a threat to patient safety. In the same respect, the difficulty in restricting the use of MDs once they have entered market was seen as problematic. On the other hand, one interviewee mentioned that the high rejection rate of MD reimbursement based on HTA institutions’ recommendations could lead to the exclusion of promising technologies.

Approaches towards overcoming challenges One further theme that emerged from the interviews concerns how institutions try to deal with the aforementioned challenges. One subtheme mentioned by many interviewees was ‘Stakeholder input’, which encompassed the inclusion of expert groups, clinicians, manufacturers and also scientific societies in the assessment process with the aim of gathering more information due to a lack of (high quality) evidence. Thus, stakeholder involvement

Page 9 of 29

seems to be important but requires more time and resources and is mostly done on an ad-hoc basis. Further approaches towards solving challenges, which were however only mentioned by one interviewee each included: Use of Horizon scanning programmes,

-

Set up of post-interaction observation/post-introduction monitoring,

-

Set up of patient safety programmes,

-

Implementation of new approaches such as coverage with evidence development,

-

Introduction of a national project to address missing data (no further details given),

-

Facilitating the conduct of research studies/set up of a register.

cr

us

Role of device-specific aspects

ip t

-

Due to the broad variety of MDs - ranging from medical aids directly used by patients to big

an

ticket technologies - device specific aspects are an important component within the assessment process. We defined ‘device-specific’ as all aspects that are relevant to the assessment of MDs specifically and do not apply to other technologies (e.g. drugs). Four

M

themes emerged from the interviews (Table 3).

The most frequently cited theme was the ‘Scope of consideration’. Most interviewees stated

d

that it is crucial to address specific aspects (e.g. patient skills and acceptance) and go beyond

Ac ce pt e

clinical effectiveness. To do so, experts and manufacturers can be contacted to gain more information about technical specificities. Other interviewees pointed out that the consideration of such aspects constitutes part of the assessment but clinical effectiveness and safety as well as economic aspects remain the main focus. Whether and how device-specific aspects are considered depends on the type of the device, leading to more or less variable approaches, as one interviewee emphasised:

‘Yes, but it’s a case-by-case strategy. Probably, if you look at the reports, you see that these aspects are being considered but without “a priori” intention.’ The ‘Purpose of considering device-specific aspects’ was mentioned by many interviewees within the general context of informed decision-making (e.g. indications for use), for reimbursement questions or more specifically for formulating minimum requirements (e.g. minimum of patients to treat per year). One interviewee expressed this as follows: ‘[…] Is it feasible […] to have a service in a particular area and what is the volume-outcome relationship for this device or procedure?’

>>Table 3<<

Page 10 of 29

Separate methodological guidance for MD assessment Overall, most interviewees, including those working at institutions with a separate document for the assessment of MDs, see a separate methodological guidance addressing particularities of MDs as ‘helpful’ either for the institution itself or as a training instrument for other parties (e.g. manufacturers). Of those, some interviewees also stated that separate documents could

ip t

be helpful but only to some extent:

‘[…] it is also a matter of how specific we should be, because while the learning curve might

cr

be extremely important for some medical devices, for others it might not be.’

We clustered information on the preferred ‘Type’ of document as it emerged inductively from

us

the answers given to the question by 13 interviewees (see Table 4). Many interviewees stated that an ‘Overall document with additional sections on MDs or specific aspects to consider’

an

would be sufficient.

M

>>Table 4<<

Perceived Impact of institutions’ assessments

d

In contrast to the corresponding question in the first part of the interview, this question

Ac ce pt e

focused on the interviewee’s personal opinion regarding the actual impact their assessment have in the health system. This was primarily interesting for those institutions where decisionmakers are not required to consider assessment results. Given that impact can be defined and measured in many ways, we categorised the given views based on verbatim expressions used by the interviewees.

Overall, many of the interviewees thought that their assessments had an impact, albeit to varying degrees.

‘Ways for assessing impact’ emerged inductively from given answers. In general, these are institution-specific and include for example following the final reimbursement decision regarding implementation of recommendations and conducting own analyses based on download rate, media presentation or percentage of assessments implemented by decisionmakers.

Future developments The final questions in the second part of the interview aimed to capture interviewees‘ personal hopes and suggestions for future developments in the assessment of MDs. Three main themes Page 11 of 29

emerged from the answers and were emphasized by nearly all interviewees. The corresponding subthemes will be further described below, depicting in more detail what interviewees indicated as desired changes.

Legal requirements This theme includes the subtheme ‘Re-regulation of MD licensing process in the EU’, calling

ip t

for new, stricter regulation at EU level particularly regarding evidentiary requirements on effectiveness and safety for receiving CE mark. Detailed key points raised by the interviewees

-

cr

encompass:

A singular, structured and harmonised regulation system across European countries as it

us

already is in place for drugs:

‘[…] especially there should be a singular system as for drugs, that when a device is

-

an

being accepted in one country, it also could be transferred to another country […]’ Data Requirements including high quality studies such as RCTs but also studies addressing organizational aspects such as learning curves:

M

‘[…] in some areas you get better quality evidence, because in some clinical areas, people are much more used to conduct studies and trials and the evidence is of a better quality than in other

A transparent and publicly available system/database of licensed MDs at the European level

Ac ce pt e

-

d

areas – it varies a lot.’

including also CE mark documents, prices, instructions and indications for use as well as other information:

‘I think it will be very important to have some publicly accessible register of medical devices, including CE-mark documents at the European level, also with the prices of these medical devices. For me this publicly available register at the European level would be very important.’ -

Mandatory trial registration and publication: Obligation for studies on MDs to be documented in a central register and to be published once results are available, regardless of the nature of these results and study design, as one interviewee expressed: ‚It would be good […] to make the registration of all medical device studies compulsory, which would also include an obligation to publish the results.’

Two further subthemes were anticipated but mentioned only by some interviewees: ‘Changes regarding national frameworks’ (including stricter requirements for the introduction of MDs into the market at national level) and ‘Agreement on evidence in the post-marketing stage’

Page 12 of 29

that is legally anchored at European level (including agreements among stakeholders about which evidence to accept, how to set up evidence generation and when to re-evaluate after licensing has been granted).

Methodological requirements This theme includes ‘Common understanding’ at individual agency and/or network level

ip t

concerning general/minimum standards for methodological approaches, as one interviewee emphasized:

cr

‘So I think that it would be important to create some kind of minimum standard for the types of devices, but that would be a huge piece of work.’

-

us

According to the interviewees these should encompass in more detail:

PICO aspects, i.e. agreement of predefined comparators, outcome measures to use, the

an

(minimal) clinically meaningful change as well as clearly defined patient groups: ‘One thing that you might consider, is discussing the C a little bit as well and say what you compare this device against and how you plan to do that […] Then another issue is choosing the

M

outcomes […] it would be very important to standardise the outcomes you are measuring when you are treating a certain type of disease.’

Evidence level, i.e. agreement of an appropriate evidence level/study design depending on the

d

-

risk level of the MD in question, mostly RCTs were mentioned:

Ac ce pt e

‘[…] classification around about what would be an acceptable type of study for a technology at each level with a broad understanding and agreement on that.’ -

Learning curves, i.e. more clarity on how to consider learning curves (e.g. reporting guidelines), where applicable:

‘And I haven’t actually seen good kind of methodological description that is very sharp on the learning curve’: -

Existing tools, i.e. increasing the use of existing tools for standardised approaches such as ‘EUnetHTA submission templates’ and ‘IDEAL framework’: ‘We see many advantages from the cooperation [EUnetHTA] and the corresponding method paper, we stick to it, we want standardised European HTA products, we want a joint assessment…‘

-

New support tools to distinguish between minor and major device modifications, i.e. developing guidance on how to determine whether new evidence (or assessment) is required for incremental innovations: ‘There are lots of things that we often think about, both in terms of the ways that medical devices change and develop over time and how we can know whether the evidence is generalisable […] –

Page 13 of 29

is it still relevant for the version that you’re still looking at. That’s something we often wonder about.’

Another

theme

emphasized

the

need

for

‘Further

research

on

methodological

approaches/tools’, in particular regarding the following three areas: -

Approaches towards economic evaluation such as cost-consequence modelling:

ip t

‘And then I don’t know – I suppose there are a lot of research possibilities to develop health economics approaches for devices […]. Also more research in the use of cost-

-

cr

consequences modelling, because it is not widely used.’

Separate tools for specific device types, that are lacking for example an iteration of the GRADE

us

framework for prognostic tests:

‘Also we know that the grading system is not yet accessible for the prognostic test.’ Stakeholder involvement, that is figuring out the best way of consulting experts and/or patients

an

-

(e.g. preference elicitation):

‘[…] it’s difficult because you want to get a broad range of opinions, but then you also want to

M

understand that people who have used the device often, understand more clearly how it’s going to fit into their current treatment pathways. And so often there are questions about how is the

d

best way of expert elicitations to try and fill the gaps in the evidence that we wonder about.’

Ac ce pt e

Awareness and interaction between stakeholders in the system Two subthemes emerged from the material within this theme: ‘Awareness and focus on MDs as a policy issue’ was envisaged by most interviewees encompassing more attention and focus on different types of devices on behalf of decisionmakers, including a more structured and systematic planning for their introduction to the market (prioritisation). For this purpose, more literacy among decision-makers is required. One interviewee expressed this subtheme as follows: ‘I would say that I would definitely appreciate it if we had the same discussions and same coherent thinking [as in pharmaceuticals] about the introduction of medical devices.’ ‘Dialogue and interaction between different stakeholders’ including manufacturers and patients but also between regulatory and reimbursement bodies was also raised by some interviewees, for example sharing of existing data and early advice to companies: ‘So we need much more dialogue between the different stakeholders, including patients. Especially in terms of establishing outcomes relevant to patients […]’ 4. Discussion

Page 14 of 29

Main findings on institutional practices and comparison to the literature The presented study examined practices on MD assessment as well as challenges and future implications for the assessment of MDs as perceived by European HTA institutions. Findings from the analysis of the first part of the interviews did indeed clarify, supplement and update already captured information from a systematic overview on institutional practices (8). Therefore this step could be seen as a validation of information available in the public

ip t

domain for the interviewed institutions. In most cases, deviations in the information gleaned through the interviews concerned the prioritisation process and criteria used to select MDs for

cr

assessment (e.g. Agenas). The question concerning the use of a specific definition of MDs led to some corrections (e.g. for OSTEBA) but also to the addition of new details (e.g. for

us

OGYÉI TEI). The online search had captured almost all relevant methodological guidelines. However, more recent developments (e.g. update of NICE DAP guide) and information on

an

work in progress within the institutions (e.g. TLV’s trial to assess MDs) could be additionally obtained. It seems that research in online sources can provide a general overview of institutional practices that is representative, although more detailed analyses may require

M

participatory elements (e.g. direct link to institutions) to be complete and valid. Beyond validating existing knowledge, findings from part I also add valuable insights

d

regarding institutional practices characterising related work.

Ac ce pt e

Mainly high-risk MDs were within the institutions’ scope for assessment. In contrast, medical aids used directly by patients were less frequently considered either because other institutions (not included in our sample) were responsible for this device category or due to the regulatory background. It is intuitive that the importance of careful evaluation increases in line with the risk devices entail for patients and that this would be reflected in related regulation (9). Two examples for much stricter requirements regarding the evaluation of high(er) risk MDs are the regulations by the FDA (USA) and the recently introduced early benefit assessment for high risk MDs (§137h Social Code Book V) in Germany. The interviews showed that there is an agreement on the usefulness of considering particularities of MDs in methodological documents. However, additional sections or highlighting specific aspects in the institution’s general document was seen as sufficient and perhaps more appropriate than a fully separate document only on MDs. In addition, the consideration of specific types of MDs (e.g. diagnostics) in methodological guides was indicated as helpful. Recently, EUnetHTA also developed a document specifically for therapeutic MDs (14).

Page 15 of 29

Most interviewees thought that their institutions’ assessments have impact on the decisionmaking process. Despite the fact that they consider measurement difficult, many institutions try to capture their impact using different approaches. Impact monitoring is particularly challenging because assessments produced by the majority of institutions are not binding in the decision-making process; furthermore their impact may depend on the level they are

ip t

formulated for (e.g. national, regional).

Main findings regarding perceptions on MD evaluation and comparison to the literature

cr

The analysis of the second part of the interviews yielded rich findings, part of which confirmed what has been previously noted in the literature by individual experts. However,

us

additional ideas emerged that can be important for future considerations by HTA institutions and policy-makers alike (see Figure 1).

an

Overall, challenges in the assessment of MDs raised by interviewees resulted more from the various types of devices with their different functionalities and purposes (e.g. devices used by patients) and were not so much linked to the juxtaposition with other technologies such as

M

drugs. Most interviewees wished for a change in EU regulation regarding evidence requirements for the licensing of MDs. Previous publications have also discussed the

d

weaknesses of the regulatory process at EU level and emphasized the need for change and

Ac ce pt e

new evidence requirements (2, 15-18). Challenges from a regulatory perspective are strongly linked to the structural, procedural and - particularly - methodological challenges, as the majority of issues mentioned are induced in some way by the regulatory framework. Our interviews also confirmed that the weak evidence base available to HTA institutions (including lack of economic data), is one of the main difficulties in assessing MDs (2, 5, 6, 19, 20). Moreover, the rapid pace of MD development leads to more incremental innovations instead of breakthrough technologies; these pose specific hurdles for the institutions, for example regarding the identification of appropriate studies (2, 20). While these were not explicitly mentioned by our interviewees, possible solutions include carrying out HTA intermittently (19) and using iterative modelling approaches (5). The challenge of codependent technologies, which was brought up in the interviews, has also been addressed by Merlin et al. (21), who developed a framework for evaluating evidence on the clinical benefit of co-dependent technologies for reimbursement decisions. Further ideas provided by our interviewees and also supported by further research, encompass the need of a common understanding on methodological requirements, including specific tools such as the adaption of GRADE for prognostic studies (proposed by Huguet et al. (22)),

Page 16 of 29

an MD-specific look into economic evaluation (23) as well as the use of existing tools such as the IDEAL framework for interventional therapeutic innovations (20), which describes the stages through which such technologies pass, but also the study designs indicated for each (24). The latter was recently adapted specifically for MDs (25). Furthermore, the need for a tool on handling device modifications with regard to evidence generation emphasized by our interviewees has also been outlined in literature (2).

ip t

With regard to the level of experiences of institutions with MD assessment taken into account at some point in the analysis, it was not surprising that the answers depend on the maturity of

cr

the individual institutions’ HTA programme. Therefore, institutions with less experience addressed more themes from a structural perspective such as ‘Capacity’, whereas institutions

us

with medium experience addressed themes from a procedural perspective such as ‘Coordination of assessment’. However, it should be noted, that all institutions irrespective of

an

their experience touched upon the themes ‘Evidence base’ and ‘Weak EU regulation regrading licensing of MDs’.

M

Comparison to previous surveys

There are several surveys of institutional practices on HTA which have already been

d

compared to the findings of the review carried out ahead of the interviews (8). However, up to

Ac ce pt e

now, Ciani et al. (7) provided the only work also including interviews with a focus on MD evaluation. Despite the fact that their focus was international, their methods and sample size were quite similar to ours. From a structural perspective ‘Capacity’ (e.g. lack of resources) was raised as a challenge by many interviewees in Ciani’s survey. This theme was mentioned by only one interviewee in our study. This reflects the fact that our work mostly included countries with established HTA systems, whereas more participants from countries considered as HTA ‘emerging settings’ were interviewed by Ciani and colleagues. Regarding the procedural perspective ‘Coordination’ was identified as a one further challenge in both studies. In more detail, inclusion of experts to contextualize evidence on MDs was emphasized by many of our interviewees, compared to only some in Ciani’s work. The issue of a diverse, not standardised introduction of MDs in the health care system was highlighted by many interviewees in both studies. Moreover, in both studies the theme ‘Complex interventions/co-dependency’ emerged as an important topic, as did the weak evidence base from a methodological perspective.

Implications for policy and research

Page 17 of 29

After the analysis presented in this work was concluded, new rules on MDs and in vitro diagnostic MDs at European level were agreed on between the Netherlands presidency of the Council and representatives of the European Parliament, following a lengthy consultation process which had been initiated in 2012 (3). The ‘Medical Device Regulation (MDR)’ is to replace Directives 90/385EEC and 93/42/EEC. It will become applicable after its publication, following final approval by the European Parliament. The ‘Medical Device Coordination

ip t

Group’ foreseen in the MDR could be seen as a first step towards a more centralised system for licensing, as was wished by our interviewees. The Group’s competences will include the

cr

development of device standards and product-specific guidelines. The detailed key points on evidentiary and methodological requirements highlighted by our interviewees could inform

us

these processes. Transparency was one of the key points emphasized in our survey as well as in the literature (2, 15). With regard to the MDR, the European Database on Medical Devices

an

(EUDAMED), as a central repository for information exchange (e.g. certain aspects of conformity assessment) between national competent authorities and the Commission, will be publicly accessible and thus hopefully enhance transparency. Public access to further (either

M

existing or new) databases as well as the obligation for studies on MDs to be registered and also published is key for facilitating MD assessment.

d

Furthermore, the new regulation sets out stricter requirements for the appointment and

Ac ce pt e

monitoring of Notified Bodies, including for example the employment of well-qualified staff and the liability to both scheduled and ad-hoc controls by responsible national authorities. While this may serve to remedy the varying rigor behind awarding CE marks highlighted by our interviewees, it also constitutes a first step towards tackling corruption in this context, an issue which – while not explicitly brought up in the interviews – has posed reason for concern in the MD community (26). These authorities will also be entitled to carry out unannounced industry inspections. Post- licensing regulatory measures called for in the interviews are addressed by the implementation and maintenance of a systematic post-market surveillance system as an integral part of the manufacturers’ quality management systems. This will include periodic safety update reports on devices with moderate to high risk level. In contrast to the aforementioned elements, the new EU regulation does not mandate stricter requirements for high quality evidence on effectiveness and safety at market entry, which were unequivocally considered of vital importance in our interviews. Action at national level was raised within our study as a possible solution to overcome hurdles if relevant changes at EU level do not or only partially take place (see also relevant work by KCE (27)). One possible solution to overcome the issue of scarce or low quality data on

Page 18 of 29

effectiveness at national level might be coverage with evidence development (CED) initiatives, as suggested by both our interviewees and current literature (2, 5, 19, 28, 29). Within this approach, access to promising technologies is allowed only under the precondition that additional evidence for a later decision on full coverage is generated (30). Member states such as Germany, France, the Netherlands, Spain and the UK have already implemented CED for MDs to some extent and using different models. At the same time, it is important to

ip t

mention that the evaluation of such initiatives is pending (28). EUnetHTA recently developed a template on additional evidence generation, which could be helpful for HTA institutions and

cr

manufacturers at national level by facilitating the standardised reporting of data with regard to key methodological elements (31). Moreover, financing the clinical trials is still problematic.

us

Therefore, an agreement could be that payers pre-finance the trials and manufacturers deliver their products at a cheaper price following the completion of the trial.

an

Adaptive approaches, which could accelerate market access for innovative technologies for certain patients with the aim of expanding coverage as new evidence emerges (32) should be

required for MDs.

M

considered with caution, particularly given the already weak evidence based currently

A tool to help decision makers and HTA doers in Europe decide when a device modification

d

is substantial enough to require new evidence or a new assessment would be important and

Ac ce pt e

helpful (2, 20), as highlighted by our interviewees. A starting point to develop such a tool might be relevant FDA guidances (33, 34), which includes flowcharts helping manufacturers analyse how changes in devices may affect safety or effectiveness. Based on suggestions from our interviewees, further research to be endorsed in the field of HTA for MDs could include topics such as the evaluation of co-dependent technologies (e.g. adaption of existing framework (21) to European context) but also the development of validated tools (e.g. GRADE for prognostic studies) where these are lacking. Those suggestions on methodological requirements should be taken further by national HTA institutions and supranational institutions.

The literature generally supports more collaboration among HTA institutions and also between different stakeholders to enable better evaluations, but the opinions expressed by interviewees from 16 HTA institutions in this study show that there is still work to be done before this goal is fully reached. For more harmonisation and a more common approach between the different stakeholders, especially manufacturers and HTA agencies, existing tools such as the aforementioned EUnetHTA submission template (31), the EUnetHTA guideline for economic evaluation (35) and existing guidelines from HTA institutions (8) should be

Page 19 of 29

used more systematically. Moreover, existing tools could be used to train manufacturers before and after licensing. In the light of more collaboration between the different stakeholders, novel models to enhance early dialogue, such as the Canadian EXCITE programme, have also been suggested (7, 36). In addition, a permanent collaboration on EU level is being prepared within EUnetHTA Joint Action 3 (work package ‘joint evidence generation’) (37).

ip t

Based on the interviewees’ perception on MD evaluation and the existing literature presented in this article, we formulated recommendations for improvements on HTA of MDs beyond

cr

the modified EU regulation (see Figure 1).

us

>>Figure 1<<

an

Strengths and limitations

Sixteen institutions from 14 European countries took part in the study, with an experience in the field of HTA and MDs ranging from limited to extensive. Despite existing discussions on

M

generalisability of qualitative studies, we consider our work as a first in-depth look at current practices and opinions from a broad sample of European HTA institutions.

d

Besides the comprehensive institutional pool, the presented research has important strengths.

Ac ce pt e

The three-step process of validating obtained information included feedback by the interviewees at different stages to ensure content quality and up-to-dateness. Two interviewers were responsible for conducting and analysing the interviews. Diverse cases were discussed and a consensus was reached by involving a third person. Finally, the conception of the interview guide allowed for challenges as well as ideas for change to be captured from the perspective of those directly involved in the evaluation of MDs. Limitations arose from the difficulty to compose an interview guide that would be applicable for all types and scopes of institutions. Despite our best efforts to take differences between the health systems and remit of institutions into account (e.g. adaptation of interview guide depending on already available information), we cannot rule out the possibility that some information about institutional practices was not fully captured. Due to the fact that we chose institutions with different levels of experience, questions asked may have been too general in nature. We have tried to minimise bias in our results and their interpretation by making our methodological approach as transparent as possible (e.g. questionnaire, coding framework). Nevertheless, we are aware that due to the chosen approach of directed content analysis some

Page 20 of 29

contextual features (11), such as national reimbursement modalities that can influence MD assessment, might have been overseen. Despite verifications through the interviewees, language barriers were an issue for both interviewers and interviewees and may have led to misunderstandings and/or distortions of questions as well as answers.

ip t

5. Conclusion

This work provides a unique overview of institutional practices and opinions on MD assessment in European countries. It highlights differences in evaluation practices between

cr

different types of MDs and reaffirms that mainly regulatory changes are required to

us

strengthen and facilitate the assessment of MDs. The new MD regulation at EU level takes first steps towards addressing a number of challenges faced by HTA institutions in Europe. However, to ensure that adequate clinical data is available for the evaluation of MDs

an

additional action is required. Promoting transparency of clinical data by making the publication of study results on MDs mandatory is one issue that should be considered at EU

M

level. On the other side possible approaches to enhance the evidence base for decisionmaking, such as CED, should be taken into account at national level. More alignment of the HTA and regulatory process for example by the use of early dialogues is perceived as a

d

potential way to improve the current situation. It needs to be further explored whether new

Ac ce pt e

approaches for MD assessment are needed or adaptions of existing tools could be sufficient. Overall, further identified challenges as well as suggestions for future changes (see Figure 1) should be taken further by the corresponding stakeholders.

Conflict of interest

BO and MP work for the Federal Joint Committee (G-BA), which is the highest decisionmaking body of the joint self-government of physicians, dentists, hospitals and health insurance funds in Germany. One of its tasks is issuing directives determining the benefit basket of the statutory health insurance funds (GKV). BO is also a PhD candidate and MP a lecturer at Berlin University of Technology.

Acknowledgements

Page 21 of 29

The research was funded by the European Union`s Seventh Framework Programme (EU-FP7) and undertaken under the auspices of the ADVANCE_HTA project (Grant number 305983; www.advance-hta.eu). The overall aim of Work package 5 was the advancement of understanding of HTA for medical devices (MDs) in a multitude of ways. Another EU-FP7 project MedTecHTA (www.medtechta.eu) published a parallel survey on MD activity in nonEU countries. In the run-up to the work we distinguished our surveys geographically and

ip t

harmonized the content to be extracted. The European Commission had no role in the study design, collection and analysis of data, writing of the report or submission of the paper for

cr

publication. The authors would like to thank all interviewees for their valuable time and input; the ADVANCE_HTA consortium and our students Lisa Becker, Helene Eckhardt and Julian

an

us

Ramirez for their support.

References

M

1. MedTechEurope. The European Medical Technolgy Industry in figures; 2014. Available from: URL: http://archive.eucomed.org/uploads/Modules/Publications/20141003-medtech-brochure-digital1.pdf.

Ac ce pt e

d

2. Tarricone R, Torbica A, Ferré F, Drummond M. Generating appropriate clinical data for value assessment of medical devices: what role does regulation play? Expert review of pharmacoeconomics & outcomes research 2014; 14(5):707–18. 3. European Comission: Modernising EU rules for medical devices; 2016 [cited 2016 Jul 24]. Available from: URL: http://www.consilium.europa.eu/en/policies/new-rules-medical-in-vitro-diagnosticdevices/ 4. Santos, Isa C T, Tavares, João Manuel R S. Additional peculiarities of medical devices that should be considered in their development process. Expert review of medical devices 2013; 10(3):411–20. 5. Craig JA, Carr L, Hutton J, Glanville J, Iglesias CP, Sims AJ. A review of the economic tools for assessing new medical devices. Applied health economics and health policy 2015; 13(1):15–27. 6. Taylor RS, Iglesias CP. Assessing the clinical and cost-effectiveness of medical devices and drugs: are they that different? Value in health: the journal of the International Society for Pharmacoeconomics and Outcomes Research 2009; 12(4):404–6. 7. Ciani O, Wilcher B, Blankart CR, Hatz M, Rupel VP, Erker RS et al. Health technology assessment of medical devices: a survey of non-European union agencies. International journal of technology assessment in health care 2015; 31(3):154–65. 8. Fuchs S, Olberg B, Panteli D, Busse R. HTA of Medical Devices in Europe: processes, practices and methods. International journal of technology assessment in health care 2016. 2016 (eproof) 9. Henschke C, Panteli D, Perleth M, Busse R. Taxonomy of medical devices in the logic of health technology assessment. International journal of technology assessment in health care 2015; 31(5):324–30.

Page 22 of 29

10. Drummond MF, Schwartz JS, Jönsson B, Luce BR, Neumann PJ, Siebert U et al. Key principles for the improved conduct of health technology assessments for resource allocation decisions. International journal of technology assessment in health care 2008; 24(3):244-58. 11. Hsieh H, Shannon SE. Three approaches to qualitative content analysis. Qualitative health research 2005; 15(9):1277–88. 12. Siegel K, Schrimshaw EW. Coping with Negative Emotions: The Cognitive Strategies of HIVinfected Gay/Bisexual Men. Journal of health psychology 2000; 5(4):517–30.

ip t

13. Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. International journal for quality in health care: journal of the International Society for Quality in Health Care 2007; 19(6):349–57.

cr

14. EUnetHTA. Therapeutic medical devices; 2015 [cited 2016 Feb 15]. Available from: URL: https://eunethta.fedimbo.belgium.be/sites/5026.fedimbo.belgium.be/files/WP7-SG3-JA2-GLtherap_medical_devices.pdf.

us

15. Krüger LJ, Evers SM, Hiligsmann M, Wild C. Divergent evidence requirements for authorization and reimbursement of high-risk medical devices – The European situation. Health Policy and Technology 2014; 3(4):253–63.

an

16. Eikermann M, Gluud C, Perleth M, Wild C, Sauerland S, Gutierrez-Ibarluzea I et al. Commentary: Europe needs a central, transparent, and evidence based regulation process for devices. BMJ (Clinical research ed.) 2013; 346:f2771.

M

17. Storz-Pfennig P, Schmedders M, Dettloff M. Trials are needed before new devices are used in routine practice in Europe. BMJ (Clinical research ed.) 2013; 346:f1646. 18. Campillo-Artero C. A full-fledged overhaul is needed for a risk and value-based regulation of medical devices in Europe. Health policy 2013; 113(1-2):38–44.

Ac ce pt e

d

19. Ferguson M. Medical devices are different to pharmaceuticals in the Health Technology Assessment process. Journal of comparative effectiveness research 2014; 3(3):217–9. 20. Schnell-Inderst P, Mayer J, Lauterberg J, Hunger T, Arvandi M, Conrads-Frank A et al. Health technology assessment of medical devices: What is different? An overview of three European projects. Zeitschrift fur Evidenz, Fortbildung und Qualitat im Gesundheitswesen 2015; 109(4-5):309– 18. 21. Merlin T, Farah C, Schubert C, Mitchell A, Hiller JE, Ryan P. Assessing personalized medicines in Australia: a national framework for reviewing codependent technologies. Medical decision making: an international journal of the Society for Medical Decision Making 2013; 33(3):333–42. 22. Huguet A, Hayden JA, Stinson J, McGrath PJ, Chambers CT, Tougas ME et al. Judging the quality of evidence in reviews of prognostic factor research: adapting the GRADE framework. Systematic reviews 2013; 2:71. 23. Martelli N, Devaux C, van den Brink, Hélène, Pineau J, Prognon P, Borget I. A Systematic Review of the Level of Evidence in Economic Evaluations of Medical Devices: The Example of Vertebroplasty and Kyphoplasty. PloS one 2015; 10(12):e0144892. 24. McCulloch P, Altman DG, Campbell WB, Flum DR, Glasziou P, Marshall JC et al. No surgical innovation without evaluation: the IDEAL recommendations. The Lancet 2009; 374(9695):1105–12. 25. Sedrakyan A, Campbell B, Merino JG, Kuntz R, Hirst A, McCulloch P. IDEAL-D: a rational framework for evaluating and regulating the use of medical devices. BMJ (Clinical research ed.) 2016; 353:i2372. 26. European Commission. Study on corruption in the healthcare sector: HOME/2011/ISEC/PR/047A2. Luxembourg: Publications Office of the European Union; 2013.

Page 23 of 29

27. Baeyens H, Pouppez C, Slegers P, Vinck I, Hulstaert F, Neyt M. Towards a guided and phased introduction of high-risk medical devices in Belgium. Brussels: Belgian Health Care Knowledge Centre (KCE) 2015. KCE Reports 249. 28. Sauerland S, Brockhaus AC, Fujita-Rohwerder N, Saad S. Approaches to assessing the benefits and harms of medical devices for application in surgery. Langenbeck's archives of surgery / Deutsche Gesellschaft für Chirurgie 2014; 399(3):279–85.

ip t

29. Olberg B, Perleth M, Busse R. The new regulation to investigate potentially beneficial diagnostic and therapeutic methods in Germany: up to international standard? Health policy 2014; 117(2):135– 45.

cr

30. Hutton J, Trueman P, Henshall C. Coverage with evidence development: an examination of conceptual and policy issues. International journal of technology assessment in health care 2007; 23(4):425–32.

us

31. EUnetHTA. Core protocol Pilot for Additional Evidence Generation; 2015 [cited 2016 Jan 12]. Available from: URL: http://www.eunethta.eu/sites/5026.fedimbo.belgium.be/files/EUnetHTA_Core%20protocol%20Pilot %20for%20AEG.pdf.

an

32. Husereau D, Henshall C, Jivraj J. Adaptive approaches to licensing, health technology assessment, and introduction of drugs and devices. International journal of technology assessment in health care 2014; 30(3):241-9.

M

33. U.S. Food and Drug Administration. Deciding When to Submit a 510(k) for a Change to an Existing Device (K97-1). Last Updated: 08/10/2015. Available from: URL: http://www.fda.gov/MedicalDevices/DeviceRegulationandGuidance/GuidanceDocuments/ucm0802 35.htm#page1.

Ac ce pt e

d

34. U.S. Food and Drug Administration. Modifications to Devices Subject to Premarket Approval (PMA) - The PMA Supplement Decision; 2008. Available from: URL: http://www.fda.gov/MedicalDevices/DeviceRegulationandGuidance/GuidanceDocuments/ucm0892 74.htm#at. 35. EUnetHTA. Guideline. Methods for health economic evaluations - A guideline based on current practices in Europe. Final Version; 2015 [cited 2016 Jan 15]. Available from: URL: http://www.eunethta.eu/sites/5026.fedimbo.belgium.be/files/2015-04-29-ECOGL_Final%20version_0.pdf. 36. Tsoi B, O'Reilly D, Masucci L, Drummond M, Goeree R. Harmonization of HTA--based reimbursement and regulatory approval activities: a qualitative study. Journal of population therapeutics and clinical pharmacology = Journal de la thérapeutique des populations et de la pharamcologie clinique 2015; 22(1):e78-89. 37. EUnetHTA. Work Package 4: Joint productions; 2016 [cited 2016 Aug 09]. Available from: URL: http://www.eunethta.eu/news/work-package-4-joint-productions

Table 1: Overview of institutions included in the survey and their role regarding HTA in their jurisdiction a

Institution

Jurisdiction

AAZ

HR

Type of b Institution 2a

Agenas

IT

2a

Role of institution in the HTA system, especially regarding Medical Devices Provides requested evidence-based advice and information to decision-makers (MoH, HZZO and management board of hospitals) on drugs, MDs and procedures (appraisal and reimbursement decision done by other institutions at national level) Provides requested evidence-based advice and information to decision-maker (MoH) on a yearly predefined number of MDs (assessment of drugs done by AIFA); coordinates prioritisation operated by Italian regions to assess MDs

Page 24 of 29

within a national Horizon Scanning system

DK

2b

FinOHTA

FI

2a

HAS

FR

4

IQWiG

DE

4

KCE

BE

4

LBI-HTA

AT

1

NICE

UK/ ENG & WAL

5

OGYÉI TEI

HU

2a

OSTEBA

ES

3

SHTG /HIS

UK/ SCT

5

TLV

SE

2a

NL

4

ZiN

a

ip t

CFK

cr

3

us

ES

an

AVALIA-t

Provides requested evidence-based advice and information to decision-maker (MoH) on public funding of drugs, MDs and specific food supplements, particularly those that are included in the basic benefits package Provides requested evidence-based advice and information to decision-maker (MoH at regional and national level) on public funding of mainly MDs and procedures; performs Horizon Scanning at regional level Coordinates HTA activities across 5 Danish regions; provides evidence-based advice and information on the utilization of already implemented technologies (broader disease areas where MDs and also drugs can be one technology) Provides requested evidence-based advice and information on MDs and surgical interventions as well as evaluations of drugs versus another type of technology (assessment of solely drugs done by FIMEA) to support the decision-making process Provides requested evidence-based advice and information to decision-maker (MoH) on public funding of drugs and four categories of MDs (STAs, MTAs) Provides requested evidence-based advice and information to decision-maker (G-BA, MoH) on public funding of drugs and non-drug interventions; provides health information for the general public; has a mandate to work also on important issues independently Provides requested evidence-based advice and information to decision-maker (MoH, federal public services, NIHDI) and other stakeholders on drugs and MDs; also involved in clinical practice guidelines and health services research Provides requested evidence-based advice and information to decision-maker (MoH, Main Association of the Austrian Social Security Institutions and health funds of 9 regions) on public funding of procedures in which one or more MDs are involved; performs Horizon Scanning of oncological products Provides requested evidence-based guidance to NHS: Technology appraisal guidance, usually on pharmaceuticals has a funding direction (has to be adopted within three months); Diagnostics and MD guidance’s are recommendations for different stakeholders (e.g. clinicians, procurement managers); Actual decisions are made at local level Provides requested evidence-based advice and information to decision-maker (HIF) on public funding of drugs and two categories of MDs Provides requested advice and information to decision-makers (MoH at regional and national level) on public funding of drugs and non-drug interventions; performs Horizon Scanning at regional level SHTG provides requested evidence-based advice and information to NHS Scotland on non-drug interventions in order to support the decision-making process Makes decisions on public funding of drugs and dental care procedures; conducts HTAs of MDs on a trial basis (commissioned by the government in 2012; renewed mandate runs until December 2016) Provides requested evidence-based advice and information to decision-maker (MoH) on public funding of drugs and non-drugs

M

2a

d

PL

Ac ce pt e

AOTMiT

b

Notes: alphabetic order (abbreviations are listed in the Online Appendix); Type of institution (own categorisation): 1. Independent academic research entity, 2. Governmental institutions (a. national, b. regional), 3. Regional Ministries of Health/Social Affairs including a related department, 4. Independent research entities with function as governmental institution, 5. Non-departmental public body with legislative function; Abbreviations- AIFA: Italian Medicines Agency; FIMEA: Finnish Medicines Agency; G-BA: Federal Joint Committee, HIF: Health Insurance Fund, HZZO: Croatian Health Insurance Fund, MDs: medical devices, MoH: Ministry of Health, NHS: National Health Services, NIHDI: National Institute for Health and Disability Insurance Table 2: Challenges specific to MD assessment from a structural, procedural, methodological and regulatory perspective Major Level of Freque themes and Summary of theme Quote institutions’ a ncy b themes experiences (i) Challenges from a structural perspective Transparenc Lack of information about ‘[It is] legally forbidden to get data from Many 1-3 y which MDs are entering or notification/certification body.’ are currently in the market No central register/database

Page 25 of 29

Difficulties regarding topic selection due to the broad spectrum of MDs (iii) Challenges from a methodological perspective Evidence Weak evidence level base Lack of publicly available information (e.g. CE mark documents, fewer studies conducted/published) Weak reporting quality of available studies (e.g. no reporting of learning curves) Broader Evaluation of complex perspective interventions (e.g. treatment of the and companion diagnostic) assessment Increased necessity of considering elements beyond effectiveness and safety (e.g. ethical aspects) Diagnostic (incl. prognostic and screening) and therapeutic MDs require different types of information Methodologi Lack of specific quality cal tools assessment tools for available evidence (e.g. GRADE for prognostic studies) No existing support tool for framing research questions No existing tools for evaluation of learning curves No gold-standard of PICO aspects (e.g. comparator) Transferabili Difficulties to transfer results ty across product modification and settings

1

Some

1

‘[…] sometimes devices go through a very short life cycle before they undergo an improvement and we sort of have to start the assessment all over again for the updated versions of devices, how do you handle assessments of that, when there is a minor technological change [...]’

Most

3

cr

ip t

Some

Many

2, 3

Some

2

‘For me it’s the evidence threshold. There are quite often lower levels of evidence, and that gives us challenges in terms of its credibility, but also the assessment of it in terms of the methods of the assessment.’

Most

1-3

‘Medical devices are also commonly connected with procedures, so you should also include this organizational issue, staff issue, this learning curve not applied to pharmaceuticals. So I can say that the assessment of medical devices is much broader than the assessment of pharmaceuticals.’

Many

1, 2

‘And I haven’t actually seen good methodological descriptions that are very sharp of the learning curve. But we try to be sharp in that but I think it’s one of our main issues in the assessment.’

Some

2, 3

‘There are lots of things that we often think about, both in terms of the ways that medical devices change and develop over time and how we can know whether the evidence is generalisable […] –is it still relevant for the version that you’re still

Some

2, 3

us

‘[…] we tend to create ad-hoc groups, depending on the technology […] sometimes we can resolve [an issue] ourselves and sometimes this can lead to a working group where we involve the different parties […]’ ‘[…] we do not really know what things might be dangerous and should be evaluated better than others.’

Ac ce pt e

d

-

‘We have regulatory criteria for reimbursement. This criteria are mandatory – we have to assess medical devices with this regulatory criteria.’ ‘I hope we get new staff for the HTA department […]’

an

Prioritisation

-

M

for MDs with CE mark No flexibility in the process of evaluation due to predefined criteria by national legal framework Capacity Lack of resources (e.g. staff) for conducting HTA (ii) Challenges from a procedural perspective Information Lack of standardised retrieval submission process of data: (ad-hoc) requests to manufacturers Difficulties in the identification process of the most relevant literature due to rapid pace of innovations (i.e. incremental vs breakthrough innovations) Coordination Stakeholder input at different of points during the assessment assessment to expand limited information National legal framework

Page 26 of 29

looking at. That’s something we often wonder about.’ (iv) Challenges from a regulatory perspective Weak EU Proof of effectiveness not Regulation required for licensing regarding Decentralized licensing licensing of process MDs

‚I think for us the major aspect to change is to strenghthen the level of proof of the CE mark. It’s a weakness in the process. Some medical devices are accessing the market without any clinical data’

Many

a

1-3

b

Ac ce pt e

d

M

an

us

cr

Table 3: Role of device specific aspects assessments performed by interviewed institutions a Themes Summary of Quote Frequency theme Scope of Crucial part of ‘I don’t see any reason to Most consideration evaluation exclude specific aspects from a Variability HTA-report from the depending on beginning, whether it’s the type of MD ethical or legal, but depending Contact to on the topic, of course, certain experts and aspects will be more manufacturer important, yes or no […] So most of the times to be honest, the largest weight is given to safety, efficacy, and if relevant cost-effectiveness.’ Many Purpose of Informed ‘[…] it maybe plays a role when it comes to recommendationconsidering decision devicemaking/ making, because we specific feasibility sometimes do specific aspects Reimbursement recommendations of what questions aspects are to be considered Minimum when this technology is requirements implemented. We consider — like I was saying— the learning curve, maybe the minimum number of patients to assist. We can do estimations of patients that can be treated when it comes to very specific types of devices.’ Some Barriers of Lack of existing ‘Well, we would like to have addressing data/ studies access to good quality data on deviceUncertainties this, we could give decision specific lead to makers a more detailed aspects assumptions analysis on all the aspects of the device in question. And yeah, for example learning curves, you mentioned those, they can play an important role when we discuss efficacy or effectiveness or costeffectiveness, even. But we always go back to this, I wish we had good quality data regarding this. Certainly we can only make assumptions on that.’ No Device-specific [...] if I want to get some Some consideration aspects cannot information, regulatory be considered information for medical

ip t

Notes: Categorisation of themes in some (10-30%), many (31-65%) and most (66-100%) (12); most frequently cited theme: 1= Institutions with fewer than 10 available reports on MDs, 2= Institutions with between 10 and 60 available reports on MDs, 3= Institutions with more than 60 available reports on MDs (numbers referred to the institutions reports produced in the national health care system)

Page 27 of 29

due to regulation or due to time or/and resource constraints

devices, I can find some data on websites of […] agency […]. But if I need more detailed data I could not get it from the agency […] because it’s legally forbidden.’

a

cr

Ac ce pt e

d

M

an

us

Table 4: Usefulness of separate methodological guidance for the assessment MD Major Summary of theme Quote Frequ a themes and ency themes Usefulness of separate methodological guidance Helpful A guidance dealing ‘Companies have little Most with particularities knowledge about health seen as more or less economic analysis […] it would helpful, either for be a good training possibility institution itself, for for them […]’ other parties. No/Not A separate document ‘HTA has been very strictly Some sure would not be defined in the law, our role in advantageous (e.g. the system. However if you due to regulatory have some documents published, it maybe starts criteria) for the some discussion for changing institution or they were unsure about its things.’ potential usefulness. Type of separate methodological guidance Overall Overall document for ‘I think it might be useful to Many assessments with have sort of a basic document with additional sections or effectiveness document and additional specific aspects to then separate additions to be sections on consider would be considered for the evaluation MDs or sufficient. of drugs, vaccines, point of specific care, point of care diagnostic, aspects imaging, etc. But I think it’s too heavy if you do a separate document for everything.’ Documents Useful to have a ‘It also – it makes you think Some for specific document for specific about certain aspects and to types of types of MDs such as come to conclusions about it. MDs diagnostics. For that reason it’s okay to have such documents.’ Document A document dealing ‘I don’t believe in 15 different Some dealing with the full spectrum types of documents...So we with full of MDs was seen as think of effectiveness studies spectrum difficult to realize. and it doesn’t matter whether of MDs it’s a drug or a device…’ Country/Eu A document for the ‘I believe it should be made at Some rope-wide whole country seen as least country-wide. Well document useful, not only for hopefully it could be Europethe agency. wide. So yeah, we try to aim for standardizing our methods with all the other European agencies regarding this but of course also keeping an eye on the local special regulations and other recommendations’ EUnetHTA Existing EUnetHTA ‘…and we are trying to follow Some documents documents/tools (e.g. the Cochrane “instructions” for /other Core Model) are the systematic reviews and

ip t

Notes: see Table 2

Page 28 of 29

guidelines

important and would be/are sufficient for the institution’s work and seen as advantage towards a common methodological approach.

then we are within EUnetHTA, so we try to use the EUnetHTA frameworks.’

Some

Not specified :a

ip t

Notes see Table 2

cr

Box 1: Issues covered by the interview guide (overview)

institutional structures, processes and methods specific to MD assessment

an

•

us

Part I Aim: fill information gaps and verify institutional practices mapped during a review of publicly available information (8) Key points:

M

Part II Aim: capture perceptions regarding challenges and trends, explore potential areas for future developments Key points: challenges characterising the assessment of MDs

•

influence of the broad variety of MDs on assessment practices

•

device-specific aspects relevant to evaluation

•

Ac ce pt e

•

d

•

usefulness of separate methodological guidance for the assessment MDs

desirable future developments in the assessment of MDs

Page 29 of 29