How to measure the net benefit of treatment?

Therapie (2017) 72, 51—61

Available online at

ScienceDirect www.sciencedirect.com

GIENS WORKSHOPS 2016 /Clinical pharmacology and methodology

How to measure the net benefit of treatment?夽 Franc ¸ois Gueyffier a,∗, Pascal Piedbois b, Jean-Franc ¸ois Bergmann c , the participants of Giens XXXII Round Table No. 2, Bernard Avouac d , Thomas Borel e , Rémy Boussageon f , Catherine Brun-Strang g , David Braunstein h , Béatrice Cazeneuve i , Marine Diviné j , Patrick Dufour k , Nicolas Girerd l , Valérie Laigle m , ¸ois Liard o , Amélie Marsot p , Claire Le Jeunne n , Franc Jean-Louis Montastruc q , Albert Trinh-Duc r , Eric Vicaut s a

UMR5558, service de pharmacologie et toxicologie, faculté Laennec, 7, rue Guillaume-Paradin, 69008 Lyon, France b Laboratoire Boehringer-Ingelheim, 75013 Paris, France c Hôpital Lariboisière, AP—HP, 75010 Paris, France d 75012 Paris, France e Les entreprises du médicament (LEEM), 75017 Paris, France f Cabinet médical, 79100 Oiron, France g Laboratoire Sanofi, 91385 Chilly-Mazarin, France h Service de pharmacologie clinique, hôpital de la Timone, 13005 Marseille, France i Lilly France, 92500 Neuilly, France j Laboratoire AMGEN, 92650 Boulogne-Billancourt, France k Centre Paul-Strauss, 67085 Strasbourg, France l Centre d’investigation clinique, hôpital Brabois, CHU de Nancy, 54500 Vandœuvre-Les-Nancy, France m Laboratoire MSD, 92418 Courbevoie, France n Service de médecine interne, AP—HP, 75014 Paris, France o Cabinet médical, 37800 Saint-Epain, France p Service de pharmacologie clinique, AP—HM, 13005 Marseille, France q Service de pharmacologie, CHU de Toulouse, 31000 Toulouse, France r Association AREMA, 47000 Agen, France s Unité de recherché clinique, AP—HP, 75010 Paris, France Available online 3 January 2017 DOI of original article: http://dx.doi.org/10.1016/j.therap.2016.11.057. Articles, analyses and proposals from Giens workshops are those of the authors and do not prejudic the proposition of their parent organisation. ∗ Corresponding author. E-mail address: francois.gueyffi[email protected] (F. Gueyffier). 夽

http://dx.doi.org/10.1016/j.therap.2016.12.004 0040-5957/© 2016 Soci´ et´ e franc ¸aise de pharmacologie et de th´ erapeutique. Published by Elsevier Masson SAS. All rights reserved.

52

F. Gueyffier et al.

KEYWORDS Net clinical benefit; Risk-to-benefit ratio; Personnalised medicine; Shared medical decision-making; Decision analysis

Summary Estimating net benefit makes possible to clarify the basis for therapeutic decisions on an individual and collective level. This clarification is a must in shared medical decision-making and evidence-based medicine. Numerous methods are available, although none outweigh the others. The complex specifications of net benefit estimation should be tailored to the expectations of the central stakeholder, patient or society, and the unlimited range of potential contexts. The challenges, limitations, constraints and skills to be acquired by all stakeholders were discussed by the participants of the round table. They are described in this article, enabling key messages and guidelines to be presented. The essential priority is to ensure that all stakeholders receive the required training. © 2016 Soci´ et´ e franc ¸aise de pharmacologie et de th´ erapeutique. Published by Elsevier Masson SAS. All rights reserved.

Abbreviations

• the context of use for the net benefit, whether on an individual or population-based level.

APEX

The type of drug is defined by the active substance; this article does not discuss potential class effects. The selected criteria should be clinically relevant, i.e. useful from the point of view of the final beneficiary party, the patient. As a result, all intermediate criteria are excluded from the scope of the discussion, regardless of the perception of their status as surrogate criteria [2]. Economic aspects were excluded from the round table discussion. It seems ambitious, or indeed unrealistic, to aim to capture and summarise exhaustive information on all of the benefits and disadvantages associated with the use of a drug in a single expression, albeit a mathematical formula. Does this ambition then immediately discredit the effort made in terms of definition? The decisions, at population level for regulators and individually for prescribing, are based on an estimate of net benefit which is currently essentially implicit in nature. The purpose of the round table was thus envisaged as fulfilling a supporting role, as well as an invitation to the different decision-makers to clarify the basis for their decisions. Clarification of the basis for medical decision-making is part of the dynamic of evidence-based medicine. Various resources were developed in order to offer decision-makers summaries of the information required for their decisionmaking processes. Among these resources, the British Medical Journal developed the clinical evidence instrument with the aim of identifying treatments for which the benefits clearly exceed medical risks, hence for which net benefit has been established. During the recent analysis on 3000 therapeutic situations [3], the clinical evidence team concluded that only 11% of these situations were beneficial, and 24% probably beneficial. Fifty percent were based on an unknown net benefit. The in-depth presentation of the different magnitudes expressing the benefits and risks of drugs, valued according to the pathological situation, represents in itself a multidimensional expression of net benefit. Should we go further in our approach to summarising and clarifying decisionmaking? This has been attempted numerous times, and the European working group ‘‘PROTECT’’ (pharmacoepidemiological research on outcomes of therapeutics by a European consortium) [4] has proposed a summary of the different approaches envisaged in the literature, which will serve as a technological framework for the round table discussions.

B/R BRB CB CDR CEPS CMR EMA MA MONICA NB NCB NNH NNT Protect Qaly Q-Twist NCCTG Rt+ Rt− SCORE TC TWiST SMR

acute medically III VTE prevention with extended duration betrixaban study benefit-risk ratio benefit-risk balance criteria for benefit criteria for drug risk French economic committee for healthcare products criterion for drug risk European medicines agency marketing authorization multinational monitoring of trends and determinants in cardiovascular disease net benefit net clinical benefit number needed to harm number needed to treat pharmacoepidemiological research on outcomes of therapeutics by a European consortium quality-adjusted life year quality-adjusted survival relative to time without symptoms of disease or toxicity North central cancer treatment group risk difference with treatment risk difference without treatment systematic coronary risk evaluation transparency committee time without symptoms of disease or toxicity of treatment service médical rendu

Introduction Net clinical benefit (NCB or NB) is a concise and quantitative estimation of the benefits and risks of drugs on an absolute scale. It can only be envisaged if five conditions are specified [1]: • the type of drug, including dosage and route of administration; • the situation, existing disease or disorder likely to develop; • the criteria used to estimate the benefits and risks; • the time horizon chosen for these estimates;

How to measure the net benefit of treatment?

53

Figure 1. Stakeholders concerned by the estimation of net benefit and their decision-making processes. B/R: benefit/risk ratio; EMA: European medicines agency; Go: decision to continue drug development; HAS: Haute Autorité de santé (French national authority for health); NB: net benefit; MA: marketing authorisation; rec.: recommendation; TC: transparency committee.

The presentation examines, in succession, the types of situations in which the net benefit should be clarified, the types of results arising from expression of this benefit, the methods for calculating this benefit, and, lastly, a few model situations. The discussions will lead to recommendations concerning the estimation of the net benefit and identify avenues of investigation and improvement to ensure that this estimation meets the needs of the different stakeholders.

Who does the estimation of the net benefit concern? The round table participants listed the main healthcare stakeholders to whom the estimation of the net benefit is relevant, together with the type of decision-making process involved (Fig. 1). Manufacturers question the plausibility of a favourable NB at each stage in the development of the drug. If this is negative, development is suspended. The agencies issue marketing authorisations when they believe the benefit exceeds the risk. They conventionally use the benefit/risk balance (BRB) rather than NB, but the same principle applies. The transparency committee of the Haute Autorité de santé (French national authority for health) evaluates the extent to which the estimation of NB can be evidenced by clear actual benefit (SMR), and assesses the increased value of the drug concerned compared to its competitors. The French economic committee for healthcare products (CEPS) of the French ministry of health estimates the acceptable cost of the NB expressed in terms of public health impact. These different stakeholders determine the NB on a population-based scale. The estimation of the NB underpins all decisions made by physicians and their patients, as well as self-medicating patients, in which case this involves an individual estimation. Individuals with a favourable net benefit shall henceforth be referred to as ‘‘beneficiaries’’.

Extrapolation process from the studied population to the dissemination population The development of a curative or preventive treatment concerns different populations [5] (Fig. 2): i. the population of patients or individuals at risk of an event which we are hoping to prevent; ii. the practitioner’s target population, which excludes individuals not identified as patients or at risk (for example, individuals unaware they suffer from hypertension); iii. the population eligible for a therapeutic trial, which excludes individuals who do not meet the inclusion criteria; beyond study conduct, eligibility criteria are used for extrapolation of the results; iv. the studied population, which excludes individuals refusing to take part in the trial, but includes erroneously included subjects; v. the treatment target population made up of individuals with a favourable net benefit; vi. the dissemination population, exposed to treatment in the post-marketing context. The treatment target population is a theoretical population defined by extrapolation from clinical trial participants. Other rules also exist, based on other premises and other types of reasoning, for instance epidemiological or pathophysiological approaches. These rules offer less protection against different biases, and thus provide a lower level of evidence. Different results are obtained depending on whether the process used to define the target population is restricted to data with a superior level or evidence or not. Hence, the 2013 American recommendations [6] on the control of cholesterol-related complications clearly limit their reference system to clinical trials and the associated meta-analyses, whereas the 2016 European version [7] also comprise epidemiological and pathophysiological data.

54

Figure 2.

F. Gueyffier et al.

Six populations in drug development.

This extrapolation, to define the treatment target population, is based on two major types of rules: • the individuals in this population are similar to those in the studied population and fulfil the eligibility criteria; • their net benefit is expected to be favourable. Regulation of therapeutic products and evidence-based medicine aim to bring the latter two populations (v) and (vi) together, so that all exposed patients are beneficiaries. This extrapolation is one of the most complicated exercises in the medical field. Even though, on average, a net benefit has been demonstrated on the scale of the studied population, being eligible does not necessarily mean being a beneficiary. An example of this would be if the expected benefit is too small to counterbalance the undesirable effects of the treatment. Two major conceptual fundamental points should be taken into account in the estimation of net benefit: the benefit and risk associated with the drug are quantities which vary according to individual patient characteristics. The estimation of these quantities can be modified by different factors: drug interactions with concomitant treatments not used in the studied population, interactions with individual patient characteristics or contexts which have never been studied (Table 1) [8—10].

Table 1 benefit.

How to estimate the net benefit? Offering a good estimate of net benefit aims to help develop choices which are rational, clear and transparent. In addition to the five items required listed in the introduction (active substance, situation, context, criteria, time horizon), this estimation requires the following points to be clarified: the scale to be used, additive or multiplicative, the computation method, and weighting. It should be pointed out from the outset that numerous methods are proposed with a view to estimation of benefit, that computation of this should be placed in context, and that weighting is distinguished from the previous steps due to its subjective nature.

Multiplicative or additive scale? The multiplicative scale has long been used to compare the benefits and risks of a drug in the form of the benefit/risk ratio. This concept is notably applied at the time of the marketing authorisation decision, and is illustrated by the conclusion ‘‘the benefit outweighs the risk in view of the severity of the disease’’ formulated by members of the Committee at the end of the discussions on the dossier. In this context, the expression remains discursive and is not clarified further. It is generally understood that as the

The main strata to be applied when weighting the elements to be taken into account in the estimation of net

Stratification essential to estimation of net benefit

Benefit

Risk

Mortality, total or specific Non-fatal serious events, such as hospitalisation Symptoms

Reduction in cancer-related mortality Hospitalisation for heart failure

Increase in total mortality Anaphylactic shock

Dyspnoea

Cough

How to measure the net benefit of treatment? Table 2

55

Frame: extrapolation process.

Prescribing is justified by a two-step prediction: firstly, of the spontaneous outcome, and, secondly, of the impact of prescribing on this outcome Prediction of the effect of treatment may be based on different reference systems: clinical trials and clinical trial meta-analyses, epidemiological studies, or pathophysiological models Numerous examples illustrate the dangers of epidemiological and pathophysiological extrapolation Epidemiological findings suggest a benefit associated with high beta-carotene intake; however, beta-carotene supplementation increases cardiovascular mortality [8] In an epidemiological context, the cardiovascular risk is at its lowest when glycated haemoglobin is at minimal levels; however, attempting to achieve these levels using drugs has been associated with high mortality rates among patients with advanced diabetes [9] Antiarrhythmics suppress a mechanism for sudden death, but nonetheless increase sudden death [10] If clinical trials had not revealed the divergences of these epidemiological and pathophysiological models, they would still be used and have been responsible for millions of deaths Conclusion: a valid extrapolation is based on clinical trials and their meta-analyses which results are expressed on clinically relevant outcomes

components of the fraction differ in nature, they cannot be divided. Attempts for clarification mainly seem to focus on an additive scale, where the risks of the drug treatment are subtracted from the benefits: NB = DB − DR, where DB indicates the drug benefits and DR the drug risks. More complex expressions have been proposed based on efficacy and risk indexes, such as the NNT, or number needed to treat = number of patients to be treated in order to prevent an event, and its counterpart, the NNH, number needed to harm = number of patients to be treated in order to cause a drug-related event. The NNT is the inverse of absolute benefit, which is the risk difference with (Rt+) and without (Rt−) treatment: NNT = 1/(Rt+ − Rt−). The net benefit may be considered favourable when the NNT is less than the NNH. However, this process should only be possible after taking into account the differences in the weight of the events to be prevented in the estimation of benefit and caused in the estimation of drug risks. This weighting may bring into play the concept of utility weighted by the RUV or relative utility value: weight of events taken into account in the drug risks and weight of events taken into account in the benefits.

A dedicated European working group The Protect group offers an analysis of the different techniques for computation of net benefit, together with a general reasoned approach to estimation [7]. The most satisfactory techniques are used by NICE via the quality-adjusted life year (QALY) approach, and by Canadian regulators via the multiple-criteria decision analysis. The round table participants consider that: • none of the proposed methods can be universally applied; • estimation of net benefit automatically encounters a problem which cannot be solved without an arbitrary decision to weight the events constituting the drug benefits and risks; • it is imperative for the estimation of net benefit to make a systematic distinction between the major categories of criteria with strongly contrasting weights. As an initial approach, the participants propose defining three major categories, to be understood as strata to

be taken into account for weighting and overall estimation (Table 2) [8—10]. We will illustrate the impact of this stratification on results from the acute medically III VTE prevention with extended duration betrixaban study (APEX study) [11]. This randomised clinical trial compared betrixaban with placebo in the prevention of thrombosis among patients hospitalised in a medical context. In Table 2 of the original publication, the authors propose the breakdown into the two comparative groups of patients presenting one of the two main efficacy and safety endpoints, either symptomatic or non-symptomatic thromboembolic events and major bleeding, as an estimate of net benefit. A more clinically relevant criterion with the same level of seriousness could have included symptomatic venous thromboembolic events and major and/or clinically significant bleeding. The net benefit reconstituted from the number of patients presenting these events suggests an imbalance negatively impacting betrixaban: 151 versus 113. It should be noted that this estimation based on the published findings puts forwards the hypothesis that patients presenting one type of event do not present the others. A more accurate estimate should be obtained from the raw data, but the related conclusion would certainly go in the same direction.

A simple computation method for factual elements The effect of preventive treatment, whether beneficial or harmful, is a modification of the probability of the occurrence of events, reduced for a benefit, and increased for an undesirable effect arising from the drug. Whether the objective of treatment is preventive or curative for a symptomatic condition or source of incapacity, it may always be expressed as a success (absence of the event to be prevented, recovery or significant improvement in symptoms or incapacity), or failure (the opposite). Replacing the probability of an event by the probability of failure will ensure the translation from prevention to curation. This effect is deduced from the two probabilities or risks of events, with (Rt+) and without treatment (Rt−). The ratio

56

Figure 3.

F. Gueyffier et al.

Steps for computation of net benefit as per Pocock.

of these two probabilities, or relative risk, is the key parameter for this computation. The natural steps for computation of NB are as follows: • choice of criteria for benefit (CB) and drug risk (CDR); • computation of risks (probabilities) of occurrence of a type CB and type CDR event, Rt+(CB) and Rt−(CDR); • this computation of probability is based on the scores for the risk of events; • computation of relative risk for CB and CDR type events, resulting from a clinical trial or meta-analysis: RR(CB) = Rt+(CB)/Rt−(CB) and RR(CDR) = Rt+(CDR)/Rt−(CDR); • computation of benefit by B = Rt−(CB) × (1-RR[CB]), and drug risk by DR = Rt−(CDR) × (1-RR[CDR]); • computation of NB by B-DR. This computation is a generalization of an approach proposed back in 1995 by Glasziou [12], who put forward the hypothesis for a constant drug risk. In our proposed computation, the drug risk varies according to the same principles as benefit, as currently proposed with the HAS-BLED score [13]. NB may thus be calculated for each individual when the valuation of the individual patient characteristics used to calculate the probability of events is available. S. Pocock suggests incorporating this computation of individual NB and indicating the distribution thereof in clinical trial reports (Fig. 3) [14].

A computation method taking the time aspect into account In oncology, an attempt to use a unique concise criterion expressing the net benefit emerged in the 1980s, with time without symptoms or disease or toxicity of treatment (TWiST) and Q-TWiST, or quality-adjusted survival relative to time without symptoms of disease or toxicity. The idea is that the net benefit that a patient derives from antitumour chemotherapy should take into account the time during which s/he lives without symptoms or recurrence. A good example of this approach is the study published by R. Gelber [15]. This North central cancer treatment group (NCCTG) trial compared radiotherapy alone with a combination of radiotherapy and chemotherapy in patients suffering from rectal cancer. It showed that chemotherapy with 5-fluorouracile (5FU) and semustine significantly reduced the risk of recurrence by 34% and increased overall survival by 29%, with a 5-year survival rate of 65% vs. 49%. The evaluation of the net benefit of chemotherapy in this situation is complex due to the fact that adding chemotherapy is associated with an increase in short- and long-term toxicity. Without going

into the details of the computations, the elements selected in the Q-TWiST model are undesirable effects during treatment or after treatment and recurrence, with actual or estimated toxicity durations. Taking into account quality of life (Q-TWiST) involves the use of utility coefficients for undesirable effects and recurrence. The analytical results for this trial demonstrate the persistence of a ‘‘net’’ benefit of 6.1 months in terms of TWiST for patients receiving chemotherapy (95% CI 0.2—12 months), driven by the prolongation of overall survival, even though patients on chemotherapy experience toxicity for longer periods and die sooner in the event of recurrence. Despite the toxicity periods, the Q-TWiST analysis confirms the superiority of the combined approach. However, it should be acknowledged that this type of integrative model was not developed as anticipated, undoubtedly owing to its complexity and the hindsight required after implementation of the trial in order to obtain all of the elements necessary for the analyses, and the adoption of QALYs, more straightforward and comprehensive. Another difficulty is to apply it at the individual level.

Imperative contextualisation The estimate of net benefit is not an intrinsic and stable characteristic of the active substance, whether expressed on an additive or multiplicative scale. This estimate varies according to patient characteristics and should therefore systematically be placed in context. For example, oral anticoagulants in non-valvular atrial fibrillation prevent approximately two out of three strokes, and cause serious bleeding in 100 patients each year. If we considered that serious bleeding has the same weight as a stroke, these will only be indicated when the spontaneous risk of stroke is considerably greater than 1.5% per year. For patients with a risk of 1.5% per year, treatment causes as many serious bleeding events as strokes prevented, hence, the net benefit is zero. The CHADS2 Vasc score [16] predicts the risk of stroke. The HAS BLED score [13] takes into account the variations in the risk of bleeding according to patient characteristics.

Constant re-evaluation Likewise, net benefit is not an entity which remains fixed and stable over time. On a population scale, it complements the improvement in knowledge, both collective and individual, and should be constantly re-evaluated. Antiinflammatory biotherapy drugs in rheumatoid arthritis are associated with a benefit in terms of symptoms and disease progression, but these also increase the long-term risk of

How to measure the net benefit of treatment? severe infections [17] and skin cancer [18], thereby having an equally negative impact on net benefit. Knowledge of these drug-related complications enables preventive measures to be set in place which, in turn, modify the impact on net benefit when these measures are effective. On an individual scale, the initial valuation of predicted net benefit is complemented by the occurrence (or absence) of undesirable effects, their consequences as to whether treatment is continued or not, and assessment of these effects by the exposed patient. If drug-related undesirable effects lead to treatment discontinuation, the question concerning net benefit of treatment is no longer raised for the individuals concerned. If this involves an event with irreversible sequelae, the assessment of exposure to treatment could be negative. In contrast, for individuals not presenting an undesirable effect, net benefit will be strengthened relative to its estimation in principle. Lastly, providing training for patients on the proper management of drugs is likely to significantly reduce undesirable effects related to inadequate patient compliance or understanding of the issues involved.

Weighting, the ultimate and essential step in estimation of NB Both simple and complex methods have been proposed with regard to weighting. These approaches are useful when working on collective net benefit, such as in an administrative market access decision for a new drug. For example, Sutton et al. [19] proposed to weight events according to the number of life-years that patients were prepared to lose in order to regain their health. Rather than choosing a unique mean value that would summarise all possible opinions, the authors suggest taking into account weight distribution, as part of a Bayesian approach. Weighting of elements constituting drug benefits and risks is always subjective, even when a-priori defined. In the healthcare context, the best judge for this weighting is the patient him/herself. It is once again essential to make this weighting clear to the patient if we wish to work on a numerical estimation. When initiating chemotherapy liable to improve survival, most patients are prepared to take any measures to live longer. Experience of a few hard treatment courses is liable to deeply affect their weighting. Likewise, a context of suffering, whether psychological or physical, will influence patient weighting, differently to a preventive situation where the individual is in good health. Practitioners still receive insufficient training on how to take into account the patient’s point of view in this weighting, and too often tend to impose their own weighting, not realising that it is not universal. On the other hand, their experience in changes in subjective patient weighting may help them clarify patients’ initial decisions and effectively support these changes.

Computation limits Reliability of relative risk The comparative clinical trial methodology reduces biases (systematic errors) for estimation of treatment effects, and the statistical alpha or type 1 risk, i.e. the risk of an

57 error when concluding as to the existence of an effect. It should be noted that controlling the selection bias justifies an intention-to-treat analysis so as to maintain random patient distribution between the compared groups as far as possible during the analysis. However, this intention-totreat analysis reduces the therapeutic contrast between the groups, with patients no longer treated being considered as treated or patients having received the active treatment being considered as treated with placebo. The effect thus estimated, corresponding to the medical decision simulated by the randomisation process (treat, yes or no), includes the uncertainties of treatment exposure once the decision has been made: some patients will stop treatment for good reasons such as the occurrence of undesirable effects, whereas others will take it inappropriately due to inadequate compliance. These different possibilities for deviations from optimum exposure lead to an estimation, which is probably under-estimated on average, of the beneficial efficacy of treatment among perfectly compliant patients. The best level of evidence is reached for the primary endpoint of a clinical trial with a significant result, confirmed by another trial, with a meta-analysis of all studies having raised the same question. The meta-analysis makes it possible to improve the precision of the estimate; however, it summarises studies which do not have the same level of evidence for this estimation. The logical approach is thus to determine whether a study at least offers a reliable response to the question raised, then to improve the estimation of the effect via the summary resulting from the meta-analysis.

Comprehensive and heterogeneous nature of benefits and risks Computation of net benefit requires all beneficial and harmful aspects of the treatment to be taken into account. For example, for aspirin in a primary prevention context, the most frequently encountered reasoning for benefit is to focus on the different events: myocardial infarction, stroke, and other types of events leading to cardiovascular death, stratified according to their level of clinical relevance and seriousness. According to this reasoning, the most satisfactory approach would be to obtain an estimate of the effect on each type of event as well as overall, whereas often estimates of the effect are available only for a few sub-groups of this composite endpoint (cf. APEX study above as an example). It should be noted that this reasoning overlooks less severe events, such as the onset of angina pectoris or the need for angioplasty, or events for which the incidence is too low to contribute to the estimation of benefit, such as end-stage renal disease or onset of peripheral arterial disease. The same observations also apply to bleeding events caused by aspirin.

Precision of relative risk Take, for example, aspirin in a primary prevention context for myocardial infarction, on studied populations consisting of males and females with limited or no selection. The power of clinical trials, increased by the meta-analysis, yields an estimate with a certain degree of precision on average: the relative benefit of aspirin in terms of the risk of myocardial infarction is estimated at 18% on average, with a 95%

58 chance of this risk reduction being between 10 and 25% [20]. This degree of precision is of interest, allowing to assert that aspirin will prevent at least 10% of infarctions on average, and possibly 25%, in the absence of predictive markers modifying this overall estimation.

Existence of predictive markers It is rarely possible to explore the modifying factors for the intensity of effects in a satisfactory manner owing to insufficient power. It is normal to check that there are no obvious modifying factors, and to conclude that, in the absence of modifying factors, it is reasonable to consider that the estimate of the average effect applies to all exposed patients. The steps for calculating NB are therefore those indicated above. If a modifying factor is identified, this modifies the computation by applying a relative risk specific to the group to which the individual belongs. The modifying factors for the effect are known as predictive markers. This is the case for sex, a predictive marker for the effect of aspirin in a primary prevention context in terms of the risk of myocardial infarction [20,21]. The relative risk in men is 0.77 (95% CI: 0.67—0.89) and in women 0.95 (95% CI: 0.77—1.17). A favourable effect on the risk of infarction in men can therefore be predicted for men, with no effect among women. This opposite result is observed for stroke, with a significant effect among women and no effect among men. The relative benefit of aspirin is dependent on sex and the type of event to be prevented. The absolute benefit, from which net benefit will be derived, depends on the risk of the events concerned as well as sex. To illustrate the impact of the basic risk, it has become clear that aspirin is of no interest in Japanese patients in terms of primary prevention [22], which is explained by the low risk of myocardial infarction among Japanese subjects.

Difficult intellectual management of risk Computation of benefit involves multiplying the risk of events without treatment (Rt−) by the relative benefit related to treatment. Let us imagine that an individual is classified as ‘‘high risk’’ due to being in the highest risk categories. This high-risk classification is likely to cause the occurrence of the event to be considered a certainty, even though its probability is substantially less than 50%! Hence if, due to the ‘‘high risk’’ classification, we consider that this individual will experience an event, we often have more than a one in two chance of being wrong. This conceptual shift is very commonplace. When there is a ‘‘strong likelihood’’ of benefit for a patient, treatment is usually considered imperative, even though this often corresponds to a benefit of less than 10%.

Lack of precision for the estimation of the risk of events without treatment An increasing number of risk scores are being published; however, these do not yet cover every situation in which computation of net benefit would be of use, and the performance of these scores often has still to be clarified. The ability of the CHADS2-VASC score to correctly predict the level of risk was compared for different cohorts, revealing considerable variability [23]: the annual rate of stroke

F. Gueyffier et al. varied by a factor of more than 10, from 0.48% to 7.84% for a CHADS-VASC category 2 score, reputed to define eligibility for anticoagulant treatment. It should also be noted that the absolute benefit varies in proportion to the spontaneous risk. Failing to take into account important characteristics limits the interest of predicting the untreated risk, hence, the benefit of treatment: it has long been established that the level of resources or socioprofessional class are powerful and independent markers for risk. However, these types of characteristic are usually ignored by biomedical risk scores. The use of risk scores should be based on locally developed and validated scores, by incorporating all characteristics associated with a higher risk, including socioprofessional indicators. In keeping with this geographical adjustment, the European cardiovascular death score ‘‘Score’’ [24] comprises two application zones: Northern and Southern Europe. This adjustment could be more narrow or precise: a North-South gradient exists in France, as shown by multinational monitoring of trends and determinants in cardiovascular disease (Monica) [25]. Taking this gradient into account would improve the precision and calibration of the prediction scores.

Significance of uncertainty—drug response called into question? Analysis of the concept of net benefit requires awareness of the limitations of the bases for medical decisions. The proposed approach (estimation of the spontaneous risk of incidents or events, application of the relative benefit so as to deduce the absolute benefit) has been available for several decades, particularly in the recommendations for cardiovascular prevention, described as ‘‘overall risk’’ or ‘‘absolute risk’’. This approach is still only applied to a limited extent or in a non-quantitative manner (a CHADS2VASC score above 2 corresponds to a therapeutic indication, ordinarily interpreted in medical practice as a definite unquestionable decision!). Presenting detailed and quantified elements of this information offers those involved, both physicians and patients, considerable freedom of choice. This freedom is undoubtedly awkward regarding the need to take responsibility for the decision. Providing training for stakeholders on these uncertainties seems necessary in order to progress towards genuinely informed and shared decision-making, as illustrated with the case of statins in primary prevention. In numerous situations, the modest benefits associated with prescribing, and the associated uncertainties on an individual level, could challenge the ‘‘drug prescription reflex’’, generated by a wrong certainty of the benefit associated with prescribing decisions.

Patient participation In the context of the ‘‘health democracy’’ movement at work in all Western societies for the past 3 decades, patients want to be more involved in their medical decisions [26] and increasingly prefer a ‘‘discussion’’ model (with a shared medical decision-making process) over a ‘‘paternalistic’’ model in doctor/patient relations [27]. The French Public Health Code stipulates that ‘‘all individuals make decisions concerning their health with the healthcare professional,

How to measure the net benefit of treatment?

59

Table 3 Balance or imbalance between drug benefits and risks, according to their respective level of evidence and seriousness. For a drug in a given indication

Major benefit expected (seriousness of the disease, demonstration of efficacy in terms of morbidity/mortality)

Predominant major risk (seriousness or prevalence)

Herceptin in HER2+ breast cancer Insulin in T1 diabetes mellitus Oral anticoagulants in stroke prevention in the event of AF Natalizumab and PML Morphine and bone metastases Prednisone in giant cell arteritis ® Glivec and chronic myeloid leukaemia

Acceptable minor risk(s) (seriousness or prevalence) (patient)

®

Minor benefit(s) (symptomatic treatment or benefit in terms of intermediate endpoints) Olmesartan and hypertension Sulfonylureas in type 2 diabetes mellitus Neuroleptics and sleep disorders Alpha-blocking agents and benign prostatic hyperplasia Bosentan and pulmonary hypertension

AF: atrial fibrillation; PML: progressive multifocal leukoencephalopathy; S: stroke.

and based on the information and recommendations provided by the healthcare professional’’ [28]. This principle according to which the patient’s point of view should be considered is consensual, and the first publications on this subject date from more than 30 years ago [29]. In his presentation of the model, known as evidence-based medicine, Haynes describes the medical decision-making process based on scientific data, data based on the physician’s own experience, but also patient preferences [30]. Other authors refer to the principle of autonomy [31]. Shared decision-making can be defined as a process in which the patient and physician jointly share the medical decision-making process [32]. The Haute Autorité de santé proposes to formally define a 3-step process [33]: • the physician and patient share medical information, particularly aspects relating to scientific evidence; • the patient receives the necessary support in order to express his/her preferences and consider the possible options; • the choice is made after discussion between the patient and the physician. However, in the large majority of cases, the patients overestimate the benefits of biomedical interventions and medicinal treatments, and underestimate their risks [34]. Likewise, the physicians are not necessarily comfortable with the quantitative aspects of benefits and risks [35]. Decision-making tools should therefore be developed to help patients make choices in keeping with their expectations and values.

Type of situations involving the estimation of net benefit The estimation of net benefit will be used with a varying degree of ease, according to the level of evidence of underlying factual data. According to some, demonstration of efficacy in terms of intermediate endpoints is equivalent to the absence of evidence. Table 3 illustrates the situations in which drug benefits and risks balance or, in contrast, outweigh each other.

Conclusion The participants in the round table on ‘‘the estimation of the net benefit of a treatment’’ noted that the goal of a unique and unchanging net benefit for a treatment in a given situation was an illusion. Based on this observation, questioning is justified in the necessary analysis of individual or collective treatment decisions. This analysis refers to the changes in the decision-making process, from the conventional paternalistic approach to shared decision-making, in which the determining factors are very close to those of the evidence-based medicine approach. Multiple methods for summarising information cannot become established everywhere or in every situation since analysis of net benefit is incredibly complex, involving several stakeholders who may not share the same points of view or interests in principle. The principle of respect for the patient’s autonomy ultimately appears to be the essential dynamic in this approach. Specific skills should be acquired by the different stakeholders, physicians, healthcare professionals, regulators, and the patients themselves in order to achieve genuinely shared decision-making based on a contextualised and sufficiently clear assessment of net benefit, with full awareness of its changing nature and, therefore, the need for regular reevaluation.

Key messages • Net benefit is the implicit basis for decision-making on both an individual and public health level (marketing authorisation [MA], medical service rendered [service medical rendu (SMR)], improvement in medical benefit [amelioration du service medical rendu (ASMR)]). • The estimation of net benefit, as part of individual and collective decision-making, should present information from an absolute and relative perspective whenever possible. • An estimate of net benefit cannot be universal, but must be contextualised, adapted to the characteristics of the patients concerned, and incorporate a time dimension.

60 • It is up to the patient to weight the elements of the individual decision. This can vary over time, for instance, after the patient has experienced serious undesirable effects. • The information required for estimation of net benefit is accumulated over time for long-term treatments — this estimation should be constantly updated. • The use of surrogate criteria should be avoided when estimating net benefit. • Estimation of net benefit should take place prior to any economic considerations.

Proposals/recommendations • Practitioners should receive training on how to manage uncertainties and on the technical bases for estimating net benefit. This training should be incorporated into initial and continuing training for all healthcare professionals. • The population needs to be introduced to managing uncertainties and information on net benefit in order for patients to fully participate in treatment decisions. This should begin at school, as described in ‘‘La main à la pâte’’ regarding general scientific training [36]. • Public health decision-makers should also receive training on the technical instruments for estimating net benefit, the underlying concepts of this computation, and its limitations. • Investment is required in research on the transfer of information and preferences from patients to the physicians accompanying them in the decision-making process, and from physicians who transfer this information to patients. • Using net benefit as a guide to individual decision-making requires access to a comprehensive system which provides the necessary information in order to calculate net benefit. • In certain cases, particularly orphan situations, net benefit may be estimated by incorporating surrogate criteria. In this case, two-step extrapolation takes place, and the associated risks should be limited in the risk management plan, notably by defining how to reduce the degree of uncertainty. Information destined for patients should include the limitations of this estimate of net benefit.

Disclosure of interest The authors declare that they have no competing interest.

References [1] Gueyffier F, Boissel JP, Cucherat M. Integratable message: central problem of communication for therapeutic information. Therapie 1996;51(3):240—5. [2] Gueyffier F. What is the place for intermediate outcomes once cardiovascular prevention is achieved? Therapie 2003;58(1):31—6. [3] BMJ clinical evidence. What conclusions has clinical evidence drawn about what works, what doesn’t based on randomised controlled trial evidence? http://clinicalevidence.bmj.com/ x/set/static/cms/efficacy-categorisations.html [accessed 9 december 2016].

F. Gueyffier et al. [4] PROTECT. Welcome to the PROTECT benefit-risk website. http://protectbenefitrisk.eu/ [accessed 9 december 2016]. [5] Collet JP, Boissel JP. Sick population - trated population: the need for a better definition. Eur J Clin Pharmacol 1991;41:267—71. [6] Stone NJ, Robinson JG, Lichtenstein AH, Bairey Merz CN, Blum CB, Eckel RH, et al. 2013 ACC/AHA guideline on the treatment of blood cholesterol to reduce atherosclerotic cardiovascular risk in adults: a report of the American college of cardiology/American heart association task force on practice guidelines. J Am Coll Cardiol 2014;63(25 Pt B):2889—934. [7] Catapano AL, Graham I, De Backer G, Wiklund O, Chapman MJ, Drexel H, et al. 2016 ESC/EAS guidelines for the management of dyslipidaemias: the task force for the management of dyslipidaemias of the European society of cardiology (ESC) and European atherosclerosis society (EAS) developed with the special contribution of the European assocciation for cardiovascular prevention & rehabilitation (EACPR). Atherosclerosis 2016;253:281—344. [8] Egger M, Schneider M, Davey Smith G. Spurious precision? Metaanalysis of observational studies. BMJ 1998;316(7125):140—4. [9] Gerstein HC, Miller ME, Genuth S, Ismail-Beigi F, Buse JB, Goff Jr DC, et al. Long-term effects of intensive glucose lowering on cardiovascular outcomes. N Engl J Med 2011;364(9):818—28. [10] The cardiac arrhythmia suppression trial (CAST) investigators. Preliminary report: effect of encainide and flecainide on mortality in a randomized trial of arrhythmia suppression after myocardial infarction. N Engl J Med 1989;321(6):406—12. [11] Cohen AT, Harrington RA, Goldhaber SZ, Hull RD, Wiens BL, Gold A, et al. Extended thromboprophylaxis with betrixaban in acutely III medical patients. N Engl J Med 2016;375(6):534—44. [12] Glasziou PP, Irwig LM. An evidence-based approach to individualising treatment. BMJ 1995;311(7016):1356—9. [13] Lip GY, Frison L, Halperin JL, Lane DA. Comparative validation of a novel risk score for predicting bleeding risk in anticoagulated patients with atrial fibrillation: the HAS-BLED (hypertension, abnormal renal/liver function, stroke, bleeding history or predisposition, labile INR, elderly, drugs/alcohol concomitantly) score. J Am Coll Cardiol 2011;57(2):173—80. [14] Pocock SJ, Stone GW, Mehran R, Clayton TC. Individualizing treatment choices using quantitative methods. Am Heart J 2014;168(5):607—10. [15] Gelber RD, Goldhirsch A, Cole BF, Wieand HS, Schroeder G, Krook JE. A quality-adjusted time without symptoms or toxicity (Q-TWiST) analysis of adjuvant radiation therapy and chemotherapy for resectable rectal cancer. J Natl Cancer Inst 1996;88(15):1039—45. [16] Boriani G, Botto GL, Padeletti L, Santini M, Capucci A, Gulizia M, et al. Improving stroke risk stratification using the CHADS2 and CHA2DS2-VASc risk scores in patients with paroxysmal atrial fibrillation by continuous arrhythmia burden monitoring. Stroke 2011;42(6):1768—70. [17] Lahiri M, Dixon WG. Risk of infection with biologic antirheumatic therapies in patients with rheumatoid arthritis. Best Pract Res Clin Rheumatol 2015;29(2):290—305. [18] Raaschou P, Simard JF, Asker Hagelberg C, Askling J, ARTIS study group. Rheumatoid arthritis, anti-tumour necrosis factor treatment, and risk of squamous cell and basal cell skin cancer: cohort study based on nationwide prospectively recorded data from Sweden. BMJ 2016;352:i262. [19] Sutton AJ, Cooper NJ, Abrams KR, Lambert PC, Jones DR. A Bayesian approach to evaluating net clinical benefit allowed for parameter uncertainty. J Clin Epidemiol 2005;58(1):26—40. [20] Baigent C, Blackwell L, Collins R, Emberson J, Godwin J, Antithrombotic Trialists’ (ATT) collaboration, et al. Aspirin in the primary and secondary prevention of vascular disease: collaborative meta-analysis of individual participant data from randomised trials. Lancet 2009;373(9678):1849—60.

How to measure the net benefit of treatment? [21] Ridker PM, Cook NR, Lee IM, Gordon D, Gaziano JM, Manson JE, et al. A randomized trial of low-dose aspirin in the primary prevention of cardiovascular disease in women. N Engl J Med 2005;352(13):1293—304. [22] Ikeda Y, Shimada K, Teramoto T, Uchiyama S, Yamazaki T, Oikawa S, et al. Low-dose aspirin for primary prevention of cardiovascular events in Japanese patients 60 years or older with atherosclerotic risk factors: a randomized clinical trial. JAMA 2014;312(23):2510—20. [23] Quinn GR, Severdija ON, Chang Y, Singer DE. Wide variation in reported rates of stroke across cohorts of patients with atrial fibrillation. Circulation 2016, pii: circulationaha.116. 024057. [24] Conroy RM, Pyörälä K, Fitzgerald AP, Sans S, Menotti A, De Backer G, et al. Estimation of 10-year risk of fatal cardiovascular disease in Europe: the SCORE project. Eur Heart J 2003;24(11):987—1003. [25] Richard JL. The Monica project. A WHO research project in cardiovascular diseases. Rev Epidemiol Sante Publique 1988;36(4—5):325—34. [26] Saout C, Raymond G, Caillé Y. Enjeux de l’« observance » : présentation des recommandations de notre conférence citoyenne des 1er et 2 juin. 2015. http://www.leciss.org/ espace-presse/actualit%C3%A9s/enjeux-de-l-observancepresentation-recommandations [accessed 9 december 2016]. [27] Härter M. Do patients want shared decision making and how is this measured. In: Better doctors, better patients, better decisions envisioning health care 2020. Cambridge: MIT Press Scholarship Online M.G.J. Gigerenzer G, Editor; 2011.

61 [28] Legifrance. Article L1111-4 du Code de la Santé publique. 2016 https://www.legifrance.gouv.fr/affichCodeArticle.do?idArticle =LEGIARTI000006685767&cidTexte=LEGITEXT000006072665 [accessed 9 december 2016]. [29] Strull WM, Lo B, Charles G. Do patients want to participate in medical decision making? JAMA 1984;252(21):2990—4. [30] Haynes RB, Devereaux PJ, Guyatt GH. Guyatt, physicians’ and patients’ choices in evidence based practice. BMJ 2002;324(7350):1350. [31] Beauchamp TL, Childress JF. Respect for autonomy. In: Principles of biomedical ethics. USA: Oxford University Press; 2001. p. 57—112. [32] Charles C, Gafni A, Whelan T. Shared decision-making in the medical encounter: what does it mean? (or it takes at least two to tango). Soc Sci Med 1997;44(5):681—92. [33] Concept, aides destinées aux patients et impact de la « décision médicale partagée ». Haute Autorité de santé. Patients et professionnels : décider ensemble. 2013. http://www.hassante.fr/portail/upload/docs/application/pdf/201310/12iex04 decision medicale partagee mel vd.pdf [accessed 9 december 2016 (76 pp.)]. [34] Hoffmann TC, Del Mar C. Patients’ expectations of the benefits and harms of treatments, screening, and tests: a systematic review. JAMA Intern Med 2015;175(2):274—86. [35] Wegwarth O, Gigerenzer G. Statistical illiteracy in doctors. In: Better doctors, better patients, better decisions envisioning health care 2020. Cambridge: MIT Press Scholarship Online M.G.J. Gigerenzer G, Editor; 2011. [36] Fondation. La main à la pâte. http://www.fondationlamap.org/ [accessed 9 december 2016].