- Email: [email protected]
Multicriteria decision aiding for early health technology assessment of medical devices
Chapter 113
Multicriteria decision aiding for early health technology assessment of medical devices J.M. Hummela,b, Simone Borscic,d,e, G. Ficoa,f a
IFMBE, HTA Division, Eindhoven, The Netherlands, bPhilips Research, Royal Philips, Eindhoven, The Netherlands, cDepartment of cognitive Psychology and Ergonomics, Faculty of Behavioural Management and Social Sciences, University of Twente, Enschede, The Netherlands, dNational Institute for Health Research, London IVD Co-operative, Faculty of Medicine, Department of Surgery & Cancer, Imperial College, London, United Kingdom, eSchools of Creative Arts, University of Hertfordshire, Hertfordshire, United Kingdom, f Department of Photonics and Biomedical Engineering, Life Supporting Technologies Research Group, Universidad Politécnica de Madrid, Madrid, Spain
Patients and healthcare pose high and diverse demands on healthcare innovations. The value of the innovation they perceive is not dependent on clinical and economic considerations alone. This may relate to health-related as well as economic, social, legal, ethical, and organizational criteria. Early health technology assessment is increasingly advocated in the development stage of new medical technologies, that is, from the initial idea up to phase III clinical trials anticipating market access and reimbursement. In this chapter, we provide guidelines on how developers can assess the potential multifaceted value of their innovation and increase the likelihood of entrance to the healthcare market. We describe how multicriteria decision analysis (MCDA) can integrate the diverse demands into the early assessment of biomedical innovations, and provide the developers with recommendations on how to optimize the value of their innovation.
Early health technology assessment The potential value of healthcare innovations can be assessed according to multiple decision criteria. Despite the commonly used synonym of ‘health economics’, the process of healthcare decision-making is not dependent on clinical and economic considerations alone. Health technology assessment is a complex and multifactorial process involving many stakeholders and allowing for different opinions. These criteria may cover health-related and economical, as well as social, legal, ethical, organizational impacts of the innovation on health care (Polisena et al., 2018). Clinical Engineering Handbook. https://doi.org/10.1016/B978-0-12-813467-2.00114-0 Copyright © 2020 Elsevier Inc. All rights reserved.
Early health technology assessment is increasingly advocated to support healthcare decision-making during the development stage of new medical technologies, that is, from the initial idea up to phase III-like trials anticipating market access and reimbursement (Vallejo-Torres et al., 2011). The rationale behind early decision-analytic modeling is to inform internal investment decisions to select potential products or prototypes to take forward and to avoid investments in new technologies that are less likely to become successful. In a later stage, these analyses can support coverage and reimbursement decisions. Ultimately, this should lead to effective and affordable technology to become available to patients more rapidly. Also in early health technology assessment, suppliers of medical technologies need to anticipate upon health-related, economic, social, organizational, legal, and ethical impacts of their technology on health care.
Multicriteria decision analysis in early health technology assessment MCDA can be used to support these complex and multifaceted assessments of medical devices. They help decision makers to evaluate a finite number of alternative healthcare devices under a finite number of performance criteria. One validated technique for MCDA is Saaty’s Analytic Hierarchy Process (AHP) (Saaty, 1989). Other commonly used tools for multicriteria or multiattribute decision analysis in health care are the elimination and choice translating 807
808 SECTION | 12 Health technology assessment
TABLE 1 Steps in multicriteria decision analysis. Step
Description
Decision problem
Problem structuring, that is, identify objectives, alternatives, decision makers, and output required
Identify criteria
Identify value criteria relevant to the decision problem
Measure performance
Gather evidence on the performance of the alternatives on the criteria
Scoring
Convert performance measures into scores that describe the desirability of achieving different levels of performance for each criterion
Weighting
Elicit the opinions of the stakeholders on the relative importance of different criteria or their preferences for criteria
Aggregation
Combine or ‘aggregate’ criteria scores and weights to estimate the overall value of an option
Sensitivity and uncertainty analysis
Assess the impact of uncertainty or changes in weights and priorities on the overall outcomes
Supporting decision making
Use the outputs from the MCDA exercise to support decision-making
reality (ELECTRE), the simple multiattribute rating technique (SMART), and multiattribute utility theory (MAUT). Experimental comparisons have been made and concluded that each of the MCDA methods has its own advantages and disadvantages. Despite these differences, some common steps can be distinguished in multicriteria decision analysis (Task force ISPOR MCDA) (Table 1). We will elaborate upon these steps, in particular, we focus on the definition of the decision structure with the decision problem and the alternative to compare and the decision criteria to include, as well as the collection of data to underpin the multicriteria analysis.
Defining the decision problem In defining the decision problem, developers of healthcare innovations are asked to answer the following questions (see PICOS Framework, n.d.). What is the specific group of patients you aim to target with your healthcare intervention, and what is the unmet need of these patients you aim to address? Where in their clinical pathway are you to intervene, in which healthcare setting? Do you support prevention, screening, diagnosis, primary treatment, follow-up treatment, or other care interventions? And is this intervention to take place at the hospital or GP, or at another healthcare setting?
Another type of question focuses on the selection of alternatives to include in your assessment. Generally, the main alternative to compare your innovation is the established current practice you aim to replace or complement. However, other newly developed innovations are relevant to consider as well, when such an emerging innovation is likely to become established clinical practice in future.
Identifying decision criteria Only when you have identified the problem you aim to address for a specific patient group in a specific setting within the clinical trajectory of these patients, the outcomes to improve, or decision criteria can be identified. These criteria relate to outcomes including safety, effectiveness, and costs. These criteria are part of the core outcomes to include in health technology assessment (Eunetha). Besides these core criteria, other criteria are relevant to include as well.
Safety Safety is a key characteristic of healthcare technologies. The safe use of health technology requires identifying all possible risk factors that may cause safety problems by technologies’ use and by the implementation of the technology into clinical settings. Among the other aspects assessed during an health technology assessment (HTA) process, a formal evaluation of the safety is usually performed to identify all possible harms associated with the use of health technology (Borsci et al., 2018).
Clinical effectiveness Besides safety, other core criteria to be included in health technology assessments include the health-related and economic impacts by new technology on health care. Efficacy is the extent to which technology does more good than harm under ideal clinical trial circumstances. Effectiveness assesses whether technology does more good than harm, in terms of mortality and morbidity when provided under usual circumstances of healthcare practice (Eunetha).
Costs and economic impacts Economic evaluations, such as cost-effectiveness or costutility analyses, incorporate an additional perspective to the assessment, the financial perspective. Here the willingness to pay for the potential health gains and costs and investment in additional resources required for that particular intervention are taken into consideration as well.
User experience Technology to treat or diagnose a patient may impact (positively or negatively) on the service workflow affecting secondary stakeholder tasks and attitude toward the device (Borsci et al., 2016). Concurrently, the usage of technology
Multicriteria decision aiding Chapter | 113 809
at the hospital or at home may affect the daily and social life of caregivers and patients (beneficiaries) by adding tasks to perform, constraints and can also generate stress, or reaction to the use of a technology (Borsci et al., 2016). To effectively assess the social aspects associated with the use of technology, in parallel with the UX evaluation of a device with primary stakeholders, it is important to map how the technology affects beneficiaries and secondary stakeholders in terms of workflow and quality of life.
Fit with the organizational setting and scaling up Considering organizational impacts, the types of resources are considered to apply and implement the healthcare innovation. These resources include, for example, material artifacts, human skills and knowledge, money, attitudes, work culture. In addition, the consequences of the implemented healthcare innovation on the organization and the healthcare system as a whole are assessed. Organizational issues include, for example, work processes and patient/ participant flow, quality, and sustainability assurance, centralization, communication, and cooperation, managerial structure, and acceptance of a technology (Eunetha core model). Finally, the scaling-up potential of the innovation, once the organizational impact is assessed, allows decision makers to understand how the innovation can be replicated within or across organization at a micro-, meso-, and macrolevel (WHO (2009) Practical guidance for scaling-up health service innovations).
Ethical and legal impacts The Ethical Analysis (ETH) domain considers prevalent social and moral norms and values relevant to the technology in question. It involves an understanding of the consequences of implementing or not implementing a healthcare technology in two respects: with regard to the prevailing societal values and with regard to the norms and values that the technology itself constructs when it is put into use. The moral values within societies are effected sociopolitical, cultural, legal, religious, and economic circumstances (Eunetha core model). Furthermore, the latest rules and regulation for medical devices concerning safety, health economics, and broader societal impacts need to be conformed to in gaining market access.
Data collection on the potential performance of alternatives In the ISPOR task force, a versatile set of MCDA applications in HTA is described, using different techniques for MCDA and including different sets of decision criteria. Most of these MCDA applications focus on healthcare innovations that were already applied in clinical practice. In this
stage, evidence in healthcare practice on the performance of the healthcare innovation on the criteria can be available as stemming from clinical trials and cost-effectiveness studies. In early health technology assessment, evidence is generally available on the comparator, that is, established clinical practice, yet not on the innovation under development. Therefore we focus on now on approaches for data collection in the early stages of new product development. A range of methods has value in evaluating new product concepts, including survey methods, clinical pathway mapping, social network analysis, interviews, and focus groups, simulation and action research. A mixed-method approach is necessary to capture and assess the complexity of new product innovations. Mixed-methods research produces more evidence than either qualitative or quantitative approaches could by themselves. Here we will describe the basic methods for data collection in early HTA.
Elicitation of expert opinion Expert opinions can be elicited already in early stages of new product development. Estimations can be collected on potential health gains and costs consequences of replacing or complementing established clinical interventions with the healthcare innovations. These expert judgments can be either collected from a series of interviews with individual experts, or by means of expert panels.
Patient-reported outcomes Standardized tools for patient and person reported outcomes (PROs) can be applied to estimate the health and social-related impact of innovations on beneficiaries. The use of PROs is a consolidated way to gather the point of view of patients and their proxies about the impact of a service on their quality of life, satisfaction, and experience (Borsci et al., 2016). Questionnaire such as the World Health Organization Quality of Life (WHOQOL, 1998, see http://www.who.int/mental_health/publications/whoqol/ en/) and the EQ-5D, (n.d.) (https://euroqol.org/) may enable the assessment of changes in quality of life of people due to a treatment or to the use of a new technology (Roset et al., 1999). Concurrently, these information could be associated to data of physical status, and technology side effects to model the impact of the technology use on quality of life (Borsci et al., 2016, 2018).
Questionnaires on user experience and clinical pathway mapping The use of qualitative methods to gather insights about the perspective of secondary stakeholders and beneficiaries about the (negative and positive) impact of the technologies on aspects such as workflow (for professionals and
810 SECTION | 12 Health technology assessment
c aregivers) and patients’ social life and psychological factors, for example, stigma, stress, etc. For instance, the understanding of the process of the technology use and the mapping of the pathway and the user journey could help to understand weakness and strength of a treatment and of a technology use, map the stakeholders and understand potential problems generated by the clinical use of the technology. For this objective, questionnaires can be used, for example, with items as derived from the technology acceptance model.
Modeling and simulation of health economic impacts, work- and patient flows Using the data collected, further simulations can be conducted on the costs consequences, health economic consequences or work- and patient flows. An example of an economic modeling is headroom analysis, and an example of an approach for health economic modeling is MAFEIP. See for more information on methods for early HTA the chapter of Pecchia et al. (2019).
Data analysis on weights of the criteria and priorities of the alternatives Prioritizing alternatives and weighting criteria Techniques to prioritize alternatives include the direct use of performance data to score the alternatives (e.g., TOPSIS), valuation of the performance of alternatives by means of pairwise comparisons (AHP), or value functions to prioritize the performance of alternatives (e.g., Promethee, MAUT). There is a variety of weighting techniques to judge the relevance of the decision criteria, including direct rating, rating with pairwise comparisons (AHP), and swing weighting.
Aggregation Most commonly applied is the additive value model. This approach estimates the simple weighted average of the value of the alternatives on all criteria, using the weights of the decision criteria. An alternative is the multiplicative value model, see for an example: Peacock et al. (2007).
Uncertainty analysis An uncertainty analysis can be performed to evaluate the robustness of the decision outcomes, the prioritized healthcare interventions. Sensitivity analysis can be applied to the weight of each criterion (find threshold for rank reversal of alternatives), or the priorities of the alternatives on the criteria. See for more advanced methods of uncertainty analysis in MCDA: Broekhuizen et al. (2015).
See for MCDA methodologies and applications the ISPOR task force papers on MCDA.
Recommendations In HTA during the development stages of new technologies, developers need to anticipate on the health-related impacts and the economic consequences on the complete healthcare system. In order to assess the actual adoption of the intervention in the envisaged healthcare setting, the fit with user experience, organizational setting as well as societal settings needs to be explored. This chapter discussed methods to face the challenge to assess the impacts of healthcare innovations and its envisaged users and its context of use and the needs of primary and secondary stakeholders and beneficiaries. This kind of evaluation may help experts of HTA to identify the effect on stakeholders and beneficiaries of the usage of technology and to map negative and positive impact of a new technology. This evaluation allows experts to identify a space of opportunity by mapping how a health service can be improved in terms of technology used and process and constraints for stakeholders and beneficiaries. In fact, the identification of emerging and unmet needs which are not currently served by a technology may lead to develop new (service and technological) solutions which could improve the experience of stakeholders and to reduce the burden in the usage of healthcare technology. The MCDA analysis can provide the developers of healthcare innovations with recommendations on the h ealthcare innovation to further develop and on how to optimize the value of their innovation. Note that the various MCA methods have different mathematical algorithms. Different decision problems and decision makers ask for different MCDA techniques, there is no generic ‘gold standard’. See for further information report 2 of the MCDA task force.
References Borsci, S., Buckle, P., Hanna, G.B., 2016. Why you need to include human factors in clinical and empirical studies of in vitro point of care devices? Review and future perspectives. Expert Rev. Med. Devices 13 (4), 405–416. Borsci, S., et al., 2018. Integrating human factors and health economics to inform the design of medical device: a conceptual framework. In: Eskola, H., et al. (Eds.), EMBEC & NBC 2017: Joint Conference of the European Medical and Biological Engineering Conference (EMBEC) and the Nordic-Baltic Conference on Biomedical Engineering and Medical Physics (NBC), Tampere, Finland, June 2017. Springer Singapore, Singapore, pp. 49–52. Broekhuizen, H., Groothuis-Oudshoorn, C.G.M., van Til, J.A., Hummel, J.M., IJzerman, M.J., 2015. A review and classification of approaches for dealing with uncertainty in multi-criteria decision analysis for healthcare decisions. Pharmacoeconomics 33 (5), 445–455.
Multicriteria decision aiding Chapter | 113 811
EQ-5D, n.d.: see https://euroqol.org/. Peacock, S.J., Richardson, J.R.J., Carter, R., Edwards, D., 2007. Priority setting in health care using multi-attribute utility theory and programme budgeting and marginal analysis (PBMA). Soc. Sci. Med. 64, 897–910. Pecchia L., R Castaldo, P Melillo, U Bracale, M Craven, M Bracale, 2019. Early stage Healthcare Technology Assessment, in Miniati, Iadanza, Dori, Clinical Engineering: From Devices to Systems, Academic Press, ISBN 0128038241. PICOS Framework, n.d. PICOS Framework: see https://researchguides. uic.edu/c.php?g=252338&p=3954402 Accessed 11.05.18. Polisena, J., Castaldo, R., Ciani, O., Federici, C., Borsci, S., Ritrovato, M., Clark, D., Pecchia, L., 2018. HTA guidelines for medical devices: how can we address the gaps? The International Federation of Medical and Biological Engineering Perspective. Int. J. Technol. Assess. Health Care https://doi.org/10.1017/S0266462318000314. Accepted 5.04.2018, online 19.06.2018. Roset, M., Badia, X., Mayo, N.E., 1999. Sample size calculations in studies using the EuroQol 5D. Qual Life Res 8, 539. https://doi.org/10.10 23/A:1008973731515. Saaty, T.L., 1989. Group decision making and the AHP. In: Golden, B.L., Wasil, E.A., Harker, P.T. (Eds.), The Analytic Hierarchy Process. Springer, Berlin, Heidelberg.
Vallejo-Torres, L., Steuten, L., Parkinson, B., Girling, A.J., Buxton, M.J., 2011. Integrating health economics into the product development cycle: a case study of absorbable pins for treating hallux valgus. Med Decis Making. 31 (4), 596–610. WHOQOL, 1998: see http://www.who.int/mental_health/publications/ whoqol/en/. World Health Organization, 2009. Practical Guidance for Scaling up Health Service Innovations. WHO, Geneva. http://whqlibdoc.who.int/ publications/2009/9789241598521_eng.pdf.
Further reading Hummel, J.M., Bridges, J.F.P., IJzerman, M.J., 2014. Group decision making with the analytic hierarchy process in benefit-risk assessment: a tutorial. Patient. https://doi.org/10.1007/s40271-014-0050-7. Lampe, K., Anttila, H., Pasternack, I., 2008. HTA Core Model Handbook. Marsh, K., IJzerman, M., Thokala, P., et al., 2016. Multiple criteria decision analysis for health care decision making—emerging good practices: report 2 of the ISPOR MCDA emerging good practices task force. Value Health 19, 125–137. Thokala, P., Devlin, N., Marsh, K., et al., 2016. Multiple criteria decision analysis for health care decision making—an introduction: report 1 of the ISPOR MCDA emerging good practices task force. Value Health 19, 1–13.
Multicriteria decision aiding for early health technology assessment of medical devices J.M. Hummela,b, Simone Borscic,d,e, G. Ficoa,f a
IFMBE, HTA Division, Eindhoven, The Netherlands, bPhilips Research, Royal Philips, Eindhoven, The Netherlands, cDepartment of cognitive Psychology and Ergonomics, Faculty of Behavioural Management and Social Sciences, University of Twente, Enschede, The Netherlands, dNational Institute for Health Research, London IVD Co-operative, Faculty of Medicine, Department of Surgery & Cancer, Imperial College, London, United Kingdom, eSchools of Creative Arts, University of Hertfordshire, Hertfordshire, United Kingdom, f Department of Photonics and Biomedical Engineering, Life Supporting Technologies Research Group, Universidad Politécnica de Madrid, Madrid, Spain
Patients and healthcare pose high and diverse demands on healthcare innovations. The value of the innovation they perceive is not dependent on clinical and economic considerations alone. This may relate to health-related as well as economic, social, legal, ethical, and organizational criteria. Early health technology assessment is increasingly advocated in the development stage of new medical technologies, that is, from the initial idea up to phase III clinical trials anticipating market access and reimbursement. In this chapter, we provide guidelines on how developers can assess the potential multifaceted value of their innovation and increase the likelihood of entrance to the healthcare market. We describe how multicriteria decision analysis (MCDA) can integrate the diverse demands into the early assessment of biomedical innovations, and provide the developers with recommendations on how to optimize the value of their innovation.
Early health technology assessment The potential value of healthcare innovations can be assessed according to multiple decision criteria. Despite the commonly used synonym of ‘health economics’, the process of healthcare decision-making is not dependent on clinical and economic considerations alone. Health technology assessment is a complex and multifactorial process involving many stakeholders and allowing for different opinions. These criteria may cover health-related and economical, as well as social, legal, ethical, organizational impacts of the innovation on health care (Polisena et al., 2018). Clinical Engineering Handbook. https://doi.org/10.1016/B978-0-12-813467-2.00114-0 Copyright © 2020 Elsevier Inc. All rights reserved.
Early health technology assessment is increasingly advocated to support healthcare decision-making during the development stage of new medical technologies, that is, from the initial idea up to phase III-like trials anticipating market access and reimbursement (Vallejo-Torres et al., 2011). The rationale behind early decision-analytic modeling is to inform internal investment decisions to select potential products or prototypes to take forward and to avoid investments in new technologies that are less likely to become successful. In a later stage, these analyses can support coverage and reimbursement decisions. Ultimately, this should lead to effective and affordable technology to become available to patients more rapidly. Also in early health technology assessment, suppliers of medical technologies need to anticipate upon health-related, economic, social, organizational, legal, and ethical impacts of their technology on health care.
Multicriteria decision analysis in early health technology assessment MCDA can be used to support these complex and multifaceted assessments of medical devices. They help decision makers to evaluate a finite number of alternative healthcare devices under a finite number of performance criteria. One validated technique for MCDA is Saaty’s Analytic Hierarchy Process (AHP) (Saaty, 1989). Other commonly used tools for multicriteria or multiattribute decision analysis in health care are the elimination and choice translating 807
808 SECTION | 12 Health technology assessment
TABLE 1 Steps in multicriteria decision analysis. Step
Description
Decision problem
Problem structuring, that is, identify objectives, alternatives, decision makers, and output required
Identify criteria
Identify value criteria relevant to the decision problem
Measure performance
Gather evidence on the performance of the alternatives on the criteria
Scoring
Convert performance measures into scores that describe the desirability of achieving different levels of performance for each criterion
Weighting
Elicit the opinions of the stakeholders on the relative importance of different criteria or their preferences for criteria
Aggregation
Combine or ‘aggregate’ criteria scores and weights to estimate the overall value of an option
Sensitivity and uncertainty analysis
Assess the impact of uncertainty or changes in weights and priorities on the overall outcomes
Supporting decision making
Use the outputs from the MCDA exercise to support decision-making
reality (ELECTRE), the simple multiattribute rating technique (SMART), and multiattribute utility theory (MAUT). Experimental comparisons have been made and concluded that each of the MCDA methods has its own advantages and disadvantages. Despite these differences, some common steps can be distinguished in multicriteria decision analysis (Task force ISPOR MCDA) (Table 1). We will elaborate upon these steps, in particular, we focus on the definition of the decision structure with the decision problem and the alternative to compare and the decision criteria to include, as well as the collection of data to underpin the multicriteria analysis.
Defining the decision problem In defining the decision problem, developers of healthcare innovations are asked to answer the following questions (see PICOS Framework, n.d.). What is the specific group of patients you aim to target with your healthcare intervention, and what is the unmet need of these patients you aim to address? Where in their clinical pathway are you to intervene, in which healthcare setting? Do you support prevention, screening, diagnosis, primary treatment, follow-up treatment, or other care interventions? And is this intervention to take place at the hospital or GP, or at another healthcare setting?
Another type of question focuses on the selection of alternatives to include in your assessment. Generally, the main alternative to compare your innovation is the established current practice you aim to replace or complement. However, other newly developed innovations are relevant to consider as well, when such an emerging innovation is likely to become established clinical practice in future.
Identifying decision criteria Only when you have identified the problem you aim to address for a specific patient group in a specific setting within the clinical trajectory of these patients, the outcomes to improve, or decision criteria can be identified. These criteria relate to outcomes including safety, effectiveness, and costs. These criteria are part of the core outcomes to include in health technology assessment (Eunetha). Besides these core criteria, other criteria are relevant to include as well.
Safety Safety is a key characteristic of healthcare technologies. The safe use of health technology requires identifying all possible risk factors that may cause safety problems by technologies’ use and by the implementation of the technology into clinical settings. Among the other aspects assessed during an health technology assessment (HTA) process, a formal evaluation of the safety is usually performed to identify all possible harms associated with the use of health technology (Borsci et al., 2018).
Clinical effectiveness Besides safety, other core criteria to be included in health technology assessments include the health-related and economic impacts by new technology on health care. Efficacy is the extent to which technology does more good than harm under ideal clinical trial circumstances. Effectiveness assesses whether technology does more good than harm, in terms of mortality and morbidity when provided under usual circumstances of healthcare practice (Eunetha).
Costs and economic impacts Economic evaluations, such as cost-effectiveness or costutility analyses, incorporate an additional perspective to the assessment, the financial perspective. Here the willingness to pay for the potential health gains and costs and investment in additional resources required for that particular intervention are taken into consideration as well.
User experience Technology to treat or diagnose a patient may impact (positively or negatively) on the service workflow affecting secondary stakeholder tasks and attitude toward the device (Borsci et al., 2016). Concurrently, the usage of technology
Multicriteria decision aiding Chapter | 113 809
at the hospital or at home may affect the daily and social life of caregivers and patients (beneficiaries) by adding tasks to perform, constraints and can also generate stress, or reaction to the use of a technology (Borsci et al., 2016). To effectively assess the social aspects associated with the use of technology, in parallel with the UX evaluation of a device with primary stakeholders, it is important to map how the technology affects beneficiaries and secondary stakeholders in terms of workflow and quality of life.
Fit with the organizational setting and scaling up Considering organizational impacts, the types of resources are considered to apply and implement the healthcare innovation. These resources include, for example, material artifacts, human skills and knowledge, money, attitudes, work culture. In addition, the consequences of the implemented healthcare innovation on the organization and the healthcare system as a whole are assessed. Organizational issues include, for example, work processes and patient/ participant flow, quality, and sustainability assurance, centralization, communication, and cooperation, managerial structure, and acceptance of a technology (Eunetha core model). Finally, the scaling-up potential of the innovation, once the organizational impact is assessed, allows decision makers to understand how the innovation can be replicated within or across organization at a micro-, meso-, and macrolevel (WHO (2009) Practical guidance for scaling-up health service innovations).
Ethical and legal impacts The Ethical Analysis (ETH) domain considers prevalent social and moral norms and values relevant to the technology in question. It involves an understanding of the consequences of implementing or not implementing a healthcare technology in two respects: with regard to the prevailing societal values and with regard to the norms and values that the technology itself constructs when it is put into use. The moral values within societies are effected sociopolitical, cultural, legal, religious, and economic circumstances (Eunetha core model). Furthermore, the latest rules and regulation for medical devices concerning safety, health economics, and broader societal impacts need to be conformed to in gaining market access.
Data collection on the potential performance of alternatives In the ISPOR task force, a versatile set of MCDA applications in HTA is described, using different techniques for MCDA and including different sets of decision criteria. Most of these MCDA applications focus on healthcare innovations that were already applied in clinical practice. In this
stage, evidence in healthcare practice on the performance of the healthcare innovation on the criteria can be available as stemming from clinical trials and cost-effectiveness studies. In early health technology assessment, evidence is generally available on the comparator, that is, established clinical practice, yet not on the innovation under development. Therefore we focus on now on approaches for data collection in the early stages of new product development. A range of methods has value in evaluating new product concepts, including survey methods, clinical pathway mapping, social network analysis, interviews, and focus groups, simulation and action research. A mixed-method approach is necessary to capture and assess the complexity of new product innovations. Mixed-methods research produces more evidence than either qualitative or quantitative approaches could by themselves. Here we will describe the basic methods for data collection in early HTA.
Elicitation of expert opinion Expert opinions can be elicited already in early stages of new product development. Estimations can be collected on potential health gains and costs consequences of replacing or complementing established clinical interventions with the healthcare innovations. These expert judgments can be either collected from a series of interviews with individual experts, or by means of expert panels.
Patient-reported outcomes Standardized tools for patient and person reported outcomes (PROs) can be applied to estimate the health and social-related impact of innovations on beneficiaries. The use of PROs is a consolidated way to gather the point of view of patients and their proxies about the impact of a service on their quality of life, satisfaction, and experience (Borsci et al., 2016). Questionnaire such as the World Health Organization Quality of Life (WHOQOL, 1998, see http://www.who.int/mental_health/publications/whoqol/ en/) and the EQ-5D, (n.d.) (https://euroqol.org/) may enable the assessment of changes in quality of life of people due to a treatment or to the use of a new technology (Roset et al., 1999). Concurrently, these information could be associated to data of physical status, and technology side effects to model the impact of the technology use on quality of life (Borsci et al., 2016, 2018).
Questionnaires on user experience and clinical pathway mapping The use of qualitative methods to gather insights about the perspective of secondary stakeholders and beneficiaries about the (negative and positive) impact of the technologies on aspects such as workflow (for professionals and
810 SECTION | 12 Health technology assessment
c aregivers) and patients’ social life and psychological factors, for example, stigma, stress, etc. For instance, the understanding of the process of the technology use and the mapping of the pathway and the user journey could help to understand weakness and strength of a treatment and of a technology use, map the stakeholders and understand potential problems generated by the clinical use of the technology. For this objective, questionnaires can be used, for example, with items as derived from the technology acceptance model.
Modeling and simulation of health economic impacts, work- and patient flows Using the data collected, further simulations can be conducted on the costs consequences, health economic consequences or work- and patient flows. An example of an economic modeling is headroom analysis, and an example of an approach for health economic modeling is MAFEIP. See for more information on methods for early HTA the chapter of Pecchia et al. (2019).
Data analysis on weights of the criteria and priorities of the alternatives Prioritizing alternatives and weighting criteria Techniques to prioritize alternatives include the direct use of performance data to score the alternatives (e.g., TOPSIS), valuation of the performance of alternatives by means of pairwise comparisons (AHP), or value functions to prioritize the performance of alternatives (e.g., Promethee, MAUT). There is a variety of weighting techniques to judge the relevance of the decision criteria, including direct rating, rating with pairwise comparisons (AHP), and swing weighting.
Aggregation Most commonly applied is the additive value model. This approach estimates the simple weighted average of the value of the alternatives on all criteria, using the weights of the decision criteria. An alternative is the multiplicative value model, see for an example: Peacock et al. (2007).
Uncertainty analysis An uncertainty analysis can be performed to evaluate the robustness of the decision outcomes, the prioritized healthcare interventions. Sensitivity analysis can be applied to the weight of each criterion (find threshold for rank reversal of alternatives), or the priorities of the alternatives on the criteria. See for more advanced methods of uncertainty analysis in MCDA: Broekhuizen et al. (2015).
See for MCDA methodologies and applications the ISPOR task force papers on MCDA.
Recommendations In HTA during the development stages of new technologies, developers need to anticipate on the health-related impacts and the economic consequences on the complete healthcare system. In order to assess the actual adoption of the intervention in the envisaged healthcare setting, the fit with user experience, organizational setting as well as societal settings needs to be explored. This chapter discussed methods to face the challenge to assess the impacts of healthcare innovations and its envisaged users and its context of use and the needs of primary and secondary stakeholders and beneficiaries. This kind of evaluation may help experts of HTA to identify the effect on stakeholders and beneficiaries of the usage of technology and to map negative and positive impact of a new technology. This evaluation allows experts to identify a space of opportunity by mapping how a health service can be improved in terms of technology used and process and constraints for stakeholders and beneficiaries. In fact, the identification of emerging and unmet needs which are not currently served by a technology may lead to develop new (service and technological) solutions which could improve the experience of stakeholders and to reduce the burden in the usage of healthcare technology. The MCDA analysis can provide the developers of healthcare innovations with recommendations on the h ealthcare innovation to further develop and on how to optimize the value of their innovation. Note that the various MCA methods have different mathematical algorithms. Different decision problems and decision makers ask for different MCDA techniques, there is no generic ‘gold standard’. See for further information report 2 of the MCDA task force.
References Borsci, S., Buckle, P., Hanna, G.B., 2016. Why you need to include human factors in clinical and empirical studies of in vitro point of care devices? Review and future perspectives. Expert Rev. Med. Devices 13 (4), 405–416. Borsci, S., et al., 2018. Integrating human factors and health economics to inform the design of medical device: a conceptual framework. In: Eskola, H., et al. (Eds.), EMBEC & NBC 2017: Joint Conference of the European Medical and Biological Engineering Conference (EMBEC) and the Nordic-Baltic Conference on Biomedical Engineering and Medical Physics (NBC), Tampere, Finland, June 2017. Springer Singapore, Singapore, pp. 49–52. Broekhuizen, H., Groothuis-Oudshoorn, C.G.M., van Til, J.A., Hummel, J.M., IJzerman, M.J., 2015. A review and classification of approaches for dealing with uncertainty in multi-criteria decision analysis for healthcare decisions. Pharmacoeconomics 33 (5), 445–455.
Multicriteria decision aiding Chapter | 113 811
EQ-5D, n.d.: see https://euroqol.org/. Peacock, S.J., Richardson, J.R.J., Carter, R., Edwards, D., 2007. Priority setting in health care using multi-attribute utility theory and programme budgeting and marginal analysis (PBMA). Soc. Sci. Med. 64, 897–910. Pecchia L., R Castaldo, P Melillo, U Bracale, M Craven, M Bracale, 2019. Early stage Healthcare Technology Assessment, in Miniati, Iadanza, Dori, Clinical Engineering: From Devices to Systems, Academic Press, ISBN 0128038241. PICOS Framework, n.d. PICOS Framework: see https://researchguides. uic.edu/c.php?g=252338&p=3954402 Accessed 11.05.18. Polisena, J., Castaldo, R., Ciani, O., Federici, C., Borsci, S., Ritrovato, M., Clark, D., Pecchia, L., 2018. HTA guidelines for medical devices: how can we address the gaps? The International Federation of Medical and Biological Engineering Perspective. Int. J. Technol. Assess. Health Care https://doi.org/10.1017/S0266462318000314. Accepted 5.04.2018, online 19.06.2018. Roset, M., Badia, X., Mayo, N.E., 1999. Sample size calculations in studies using the EuroQol 5D. Qual Life Res 8, 539. https://doi.org/10.10 23/A:1008973731515. Saaty, T.L., 1989. Group decision making and the AHP. In: Golden, B.L., Wasil, E.A., Harker, P.T. (Eds.), The Analytic Hierarchy Process. Springer, Berlin, Heidelberg.
Vallejo-Torres, L., Steuten, L., Parkinson, B., Girling, A.J., Buxton, M.J., 2011. Integrating health economics into the product development cycle: a case study of absorbable pins for treating hallux valgus. Med Decis Making. 31 (4), 596–610. WHOQOL, 1998: see http://www.who.int/mental_health/publications/ whoqol/en/. World Health Organization, 2009. Practical Guidance for Scaling up Health Service Innovations. WHO, Geneva. http://whqlibdoc.who.int/ publications/2009/9789241598521_eng.pdf.
Further reading Hummel, J.M., Bridges, J.F.P., IJzerman, M.J., 2014. Group decision making with the analytic hierarchy process in benefit-risk assessment: a tutorial. Patient. https://doi.org/10.1007/s40271-014-0050-7. Lampe, K., Anttila, H., Pasternack, I., 2008. HTA Core Model Handbook. Marsh, K., IJzerman, M., Thokala, P., et al., 2016. Multiple criteria decision analysis for health care decision making—emerging good practices: report 2 of the ISPOR MCDA emerging good practices task force. Value Health 19, 125–137. Thokala, P., Devlin, N., Marsh, K., et al., 2016. Multiple criteria decision analysis for health care decision making—an introduction: report 1 of the ISPOR MCDA emerging good practices task force. Value Health 19, 1–13.