- Email: [email protected]
Evidence generation in medicine
Chapter 115
Evidence generation in medicine Leandro Pecchia, Davide Piaggio School of Engineering, University of Warwick, Coventry, United Kingdom
Introduction Epidemiology, coming from the Greek words ἐπί (upon) δῆμος (the people) λόγος (discourse), literally means “the study on what happens on the people.” Specifically, it is the scientific study of the diffusion and control of diseases. The aims of epidemiological studies vary from investigating the etiology of diseases (i.e., their causes), to identifying risk factors or protective effects and to evaluating health needs or treatment and prevention strategies.
Types of epidemiological studies There are two main types of epidemiological studies: experimental and observational. The former includes randomized control trial (RCT) and nonrandomized control trials (NRCTs), which require the researcher to take part proactively to the study. They are considered the gold standards to generate evidence in medicine in public health, at least as regards drugs since it may be different for medical devices. The so-called “parachute paradox” is the best proof of the inappropriateness of experimental trials for medical devices in some cases. Imagine that a parachute is a new medical device, how could experimental trials be designed? It would be easy to find volunteers for proving the effects of parachutes on a free fall from a helicopter, but could it be easy (or would it be even moral) to find volunteers to be part of the control group (i.e., jump out of the helicopter without any parachute)? Thus, if the parachute was a medical device and the experimental studies were the only solution, it would never hit the market. Indeed, there are many cases like this in medicine, which are incredibly complicated and need to be assessed in a way different from the conventional one. Hence, the observational studies still hold great importance. They are studies in which hypothesis are formulated and consolidated, without exposing people to any risk nor doing anything proactively. They can be divided into:
818
●
●
Analytical studies, in which a control group is used to see if a device/drug is promoting a significant change (e.g., cohort studies, case-control studies and cross-sectional studies); Descriptive studies, in which no control group is used. For example, they allow the study of the evolution in time of a factor [e.g., case-series, ecological, cross-sectional studies (surveys)].
There is a clear hierarchy among all these studies. They start from case series (used for generating hypothesis) and end with randomized controlled trials (used for establishing causality).
Analytical studies The direction in time characterizes these studies. According to which direction you move in time, you can have: ●
●
●
Cohort studies that start from the exposure and finish with the outcome. In short, a hypothesis was made in the past when some measurements were taken, and after some years other measurements are taken to see the outcome (i.e., confirm or discard the hypothesis); Case-control studies that are from the outcome to the exposure. In this kind of studies, there are repeated measures, without a starting hypothesis. This case is much less valuable because there is no idea regarding what to look for in the data; Cross-sectional study that is, when exposure and outcome are at the same time. In this case, there are no repeated measures.
Different kinds for different purposes Case series Usually a consecutive set of cases of a disease collected from a clinical setting. Cases of diseases are described, no Clinical Engineering Handbook. https://doi.org/10.1016/B978-0-12-813467-2.00116-4 Copyright © 2020 Elsevier Inc. All rights reserved.
Evidence generation in medicine Chapter | 115 819
comparison is made. This is only for description purposes, thus no causation can be implied.
They are relatively easy and inexpensive to perform, and they are a kind of study in which information is collected from each subject in the study population at one point in time (e.g., a survey). This study can either be a descriptive survey if the goal is to determine the prevalence of something or an analytical study, if the goal is to investigate possible associations between exposures and specific outcomes. The target of this study is the population, from which a sample is drawn (ideally randomly, because the generalizability of the assumptions made on the sample should be assured). Then two groups, the group of interest (e.g., smokers) and the comparison group of interest (e.g., nonsmokers), are compared, and the prevalence of a specific disease is analyzed for each of them.
have been exposed, “c” is the number of cases who have been unexposed, and “d” is the number of controls who have been unexposed. The OR is part of the descriptive statistics and, as such, it cannot be used to make inferences. As regards its interpretation, if the OR is higher than 1, there is a reason to think that this factor is more recurrent among patients suffering from a certain disease. This type of study is more advantageous when compared to conventional cohort studies because the latter takes a long time, is expensive, and is not suitable for studying rare events, since they are impossibly large. Case-control study. Conventional vs nested. The conventional case-control study requires a retrospective collection of data; while the nested one is nested within a cohort study and requires a collection of data before the disease has developed (from preexisting records or biological samples). An example of the nested one could be the following: a cohort of individuals is followed over time, and biological data are collected at the baseline. Subsequently, these subjects are observed over time to see who develops a disease. At this point, two subgroups are created: one includes the individuals that developed the disease and one includes a subgroup of the ones who did not develop it (the ideal ratio should be 1:3). Afterwards, the cycle starts again, and specific biomarkers are analyzed from the samples that were collected, but only for the individuals that have been finally selected. In this way, the temporal relationship is assured, and the procedure is cost-effective, because only the biomarkers from certain individuals are measured (the statistical power will be the same, if the ideal ratio is followed). Among the other advantages, when compared to conventional case-control studies: incidence rates can be calculated, the population for the sampling of the controls is already defined and recall bias is eliminated because data is obtained before the disease has developed.
Case-control study
Cohort studies
Ecological studies They are essential in the field of chronic and cardiovascular diseases. For example, they can be used to study the role of diets in the prevention of cardiovascular diseases. They are based on a comparison between two populations, and they rely on information derived from national statistics. Since it works on whole population data, it cannot necessarily apply to singular individuals. The latter is a potential weakness of this kind of study, known as “ecological fallacy.” As an example of this, Durkheim in 1897 inferred that suicide rates were higher in European countries that were more heavily Protestants. Nonetheless, it was (and is) wrong to conclude that Protestants must be more likely to commit suicide, only because countries with Protestants showed to have higher suicide rates. Overall, this kind of study is useful for raising hypotheses.
Cross-sectional studies
It is very popular in hospital settings. It involves comparing subjects with a condition (i.e., the cases) to subjects without the conditions (i.e., the controls). The level of exposure to a factor is determined for both groups and compared. In case the prevalence of exposure was higher in the cases rather than in the controls, then the exposure might be a risk factor. In this kind of study, the starting point is the definition of two samples, and then the exposure to certain factors should be looked for retrospectively. How is this type of information reported? With the odds ratio (OR), which can be defined as OR =
ad bc
where “a” is the number of cases (diseased) who have been exposed, “b” is the number of controls (nondiseased) who
They are the gold standard in observational epidemiology as they are the most robust kind of study. A cohort is a group of people who have something in common (e.g., take the same drug). A cohort study is one in which a group of people are followed up over time. This kind of studies is used for disease etiology or disease prognosis. In particular, people without disease are selected and followed up over time. One group is exposed and the other unexposed to a potential cause of disease. Eventually, the incidences of the disease are compared between the two groups. The two indices used in this case are incidence risk ratio (IRR) and relative risk (RR), defined as follows: IRR =
Incidence in exposed Incidence in unexposed
820 SECTION | 12 Health technology assessment
RR =
Risk in exposed Risk in unexposed
In particular, if A is the number of exposed people who develop the disease, B is the number of exposed people who do not develop a disease, C is the number of unexposed people who develop the disease, and D is the number of people who do not develop the disease, then IRR can be defined as IRR =
A × (C + D ) A C ÷ = A + B C + D C × ( A + B)
It is then clear that the IRR requires an incidence rate.
Experimental studies. Randomized controlled trials (RCTs) A trial is an experiment or intervention and not passive observation. This type of study is controlled because it tries to resemble laboratory experiments in basic science. Eventually, it is randomized, as it has a random allocation of exposure. RCT should only be evaluating the effect of an intervention scientifically and safely. The investigators control the intervention or treatment and the unmeasured confounding. This kind of study demonstrates causality and starts from a population. The latter is randomized into two different groups. One group will be testing new treatment and the other a control treatment (sometimes a placebo), and the outcome will be analyzed. This study is very similar to a prospective study except for the randomization process. The latter allows the two groups to be identical except for a different kind of intervention. In this way, the possible confounding effects, which are the limitations of the observational studies, are being taken into account. Nonrandom trials. This kind of study takes place in hospitals, for example, when patients’ therapy (new or standard treatment) is selected by the operator. This is a weaker design, as it introduces confounding effects and selection bias. One problem of randomized trials is still the chance of selection bias, which can happen at two different levels (selecting a sample which is generalizable to a population; taking into considerations the characteristics of nonparticipants that is, those who are nonselected). Overall, randomization is crucial because it gives equal chances of receiving each treatment, groups are likely to have similar characteristics by chance, it reduces selection bias if patients enter trial before randomization and it increases credibility. There are
different ways of obtaining randomization such as tossing an unbiased coin, using random number tables, computergenerated random numbers, block randomization, and factorial design (i.e., multistage randomization). Another central issue of this kind of studies is the placebo effect that is, defined by Pocock as “even if the therapy is irrelevant to the patient’s condition, the patient’s attitude to his or her illness, and indeed the illness itself, may be improved by a feeling that something is being done about it.” Hence, differences between new-treatment and no-treatment groups could be due to either true treatment effect or the placebo effect. One way of preventing this could be concealing the allocation by not disclosing to patients and those involved in recruiting trial participants the allocation sequence before random allocation occurs. Moreover, to ensure that there is no bias from the investigator or the clinical operator blinding should be performed. The latter means not disclosing to participants and outcome assessors the treatment allocations after random allocation. This could be reached by making treatment appear identical in taste, appearance, texture, dosage regime, etc. In particular, a study can be single blinded, if the patient or the assessor or the clinician does not know the treatment allocation. Double blinded if two or more of patient, clinician and assessor do not know the treatment allocation. Ethical issues and stopping rules. To conclude it is worth to have a quick overview of what are the ethical issues implied in RCTs and possible stopping rules. Ideally, when a clinical trial is started, there should be a clinical equipoise that is, the assumption that at the moment of the design of the trial there is no “better” intervention between the one of the control and of the experimental group. If not, there could be some ethical issues linked to the allocation of the patients to the best or the worst treatment. Also, the informed consent of the patient is an essential characteristic of an ethical study. As regards stopping rules, they can be based on: (1) Harm: ● unblinding of the clinician is required if severe side effects or clinical emergency take place; ● the study can be terminated early if one treatment is clearly harmful; (2) Cost: ● The study can be terminated early if preliminary data prove the clear benefit of one treatment in order to save resources.
Evidence generation in medicine Leandro Pecchia, Davide Piaggio School of Engineering, University of Warwick, Coventry, United Kingdom
Introduction Epidemiology, coming from the Greek words ἐπί (upon) δῆμος (the people) λόγος (discourse), literally means “the study on what happens on the people.” Specifically, it is the scientific study of the diffusion and control of diseases. The aims of epidemiological studies vary from investigating the etiology of diseases (i.e., their causes), to identifying risk factors or protective effects and to evaluating health needs or treatment and prevention strategies.
Types of epidemiological studies There are two main types of epidemiological studies: experimental and observational. The former includes randomized control trial (RCT) and nonrandomized control trials (NRCTs), which require the researcher to take part proactively to the study. They are considered the gold standards to generate evidence in medicine in public health, at least as regards drugs since it may be different for medical devices. The so-called “parachute paradox” is the best proof of the inappropriateness of experimental trials for medical devices in some cases. Imagine that a parachute is a new medical device, how could experimental trials be designed? It would be easy to find volunteers for proving the effects of parachutes on a free fall from a helicopter, but could it be easy (or would it be even moral) to find volunteers to be part of the control group (i.e., jump out of the helicopter without any parachute)? Thus, if the parachute was a medical device and the experimental studies were the only solution, it would never hit the market. Indeed, there are many cases like this in medicine, which are incredibly complicated and need to be assessed in a way different from the conventional one. Hence, the observational studies still hold great importance. They are studies in which hypothesis are formulated and consolidated, without exposing people to any risk nor doing anything proactively. They can be divided into:
818
●
●
Analytical studies, in which a control group is used to see if a device/drug is promoting a significant change (e.g., cohort studies, case-control studies and cross-sectional studies); Descriptive studies, in which no control group is used. For example, they allow the study of the evolution in time of a factor [e.g., case-series, ecological, cross-sectional studies (surveys)].
There is a clear hierarchy among all these studies. They start from case series (used for generating hypothesis) and end with randomized controlled trials (used for establishing causality).
Analytical studies The direction in time characterizes these studies. According to which direction you move in time, you can have: ●
●
●
Cohort studies that start from the exposure and finish with the outcome. In short, a hypothesis was made in the past when some measurements were taken, and after some years other measurements are taken to see the outcome (i.e., confirm or discard the hypothesis); Case-control studies that are from the outcome to the exposure. In this kind of studies, there are repeated measures, without a starting hypothesis. This case is much less valuable because there is no idea regarding what to look for in the data; Cross-sectional study that is, when exposure and outcome are at the same time. In this case, there are no repeated measures.
Different kinds for different purposes Case series Usually a consecutive set of cases of a disease collected from a clinical setting. Cases of diseases are described, no Clinical Engineering Handbook. https://doi.org/10.1016/B978-0-12-813467-2.00116-4 Copyright © 2020 Elsevier Inc. All rights reserved.
Evidence generation in medicine Chapter | 115 819
comparison is made. This is only for description purposes, thus no causation can be implied.
They are relatively easy and inexpensive to perform, and they are a kind of study in which information is collected from each subject in the study population at one point in time (e.g., a survey). This study can either be a descriptive survey if the goal is to determine the prevalence of something or an analytical study, if the goal is to investigate possible associations between exposures and specific outcomes. The target of this study is the population, from which a sample is drawn (ideally randomly, because the generalizability of the assumptions made on the sample should be assured). Then two groups, the group of interest (e.g., smokers) and the comparison group of interest (e.g., nonsmokers), are compared, and the prevalence of a specific disease is analyzed for each of them.
have been exposed, “c” is the number of cases who have been unexposed, and “d” is the number of controls who have been unexposed. The OR is part of the descriptive statistics and, as such, it cannot be used to make inferences. As regards its interpretation, if the OR is higher than 1, there is a reason to think that this factor is more recurrent among patients suffering from a certain disease. This type of study is more advantageous when compared to conventional cohort studies because the latter takes a long time, is expensive, and is not suitable for studying rare events, since they are impossibly large. Case-control study. Conventional vs nested. The conventional case-control study requires a retrospective collection of data; while the nested one is nested within a cohort study and requires a collection of data before the disease has developed (from preexisting records or biological samples). An example of the nested one could be the following: a cohort of individuals is followed over time, and biological data are collected at the baseline. Subsequently, these subjects are observed over time to see who develops a disease. At this point, two subgroups are created: one includes the individuals that developed the disease and one includes a subgroup of the ones who did not develop it (the ideal ratio should be 1:3). Afterwards, the cycle starts again, and specific biomarkers are analyzed from the samples that were collected, but only for the individuals that have been finally selected. In this way, the temporal relationship is assured, and the procedure is cost-effective, because only the biomarkers from certain individuals are measured (the statistical power will be the same, if the ideal ratio is followed). Among the other advantages, when compared to conventional case-control studies: incidence rates can be calculated, the population for the sampling of the controls is already defined and recall bias is eliminated because data is obtained before the disease has developed.
Case-control study
Cohort studies
Ecological studies They are essential in the field of chronic and cardiovascular diseases. For example, they can be used to study the role of diets in the prevention of cardiovascular diseases. They are based on a comparison between two populations, and they rely on information derived from national statistics. Since it works on whole population data, it cannot necessarily apply to singular individuals. The latter is a potential weakness of this kind of study, known as “ecological fallacy.” As an example of this, Durkheim in 1897 inferred that suicide rates were higher in European countries that were more heavily Protestants. Nonetheless, it was (and is) wrong to conclude that Protestants must be more likely to commit suicide, only because countries with Protestants showed to have higher suicide rates. Overall, this kind of study is useful for raising hypotheses.
Cross-sectional studies
It is very popular in hospital settings. It involves comparing subjects with a condition (i.e., the cases) to subjects without the conditions (i.e., the controls). The level of exposure to a factor is determined for both groups and compared. In case the prevalence of exposure was higher in the cases rather than in the controls, then the exposure might be a risk factor. In this kind of study, the starting point is the definition of two samples, and then the exposure to certain factors should be looked for retrospectively. How is this type of information reported? With the odds ratio (OR), which can be defined as OR =
ad bc
where “a” is the number of cases (diseased) who have been exposed, “b” is the number of controls (nondiseased) who
They are the gold standard in observational epidemiology as they are the most robust kind of study. A cohort is a group of people who have something in common (e.g., take the same drug). A cohort study is one in which a group of people are followed up over time. This kind of studies is used for disease etiology or disease prognosis. In particular, people without disease are selected and followed up over time. One group is exposed and the other unexposed to a potential cause of disease. Eventually, the incidences of the disease are compared between the two groups. The two indices used in this case are incidence risk ratio (IRR) and relative risk (RR), defined as follows: IRR =
Incidence in exposed Incidence in unexposed
820 SECTION | 12 Health technology assessment
RR =
Risk in exposed Risk in unexposed
In particular, if A is the number of exposed people who develop the disease, B is the number of exposed people who do not develop a disease, C is the number of unexposed people who develop the disease, and D is the number of people who do not develop the disease, then IRR can be defined as IRR =
A × (C + D ) A C ÷ = A + B C + D C × ( A + B)
It is then clear that the IRR requires an incidence rate.
Experimental studies. Randomized controlled trials (RCTs) A trial is an experiment or intervention and not passive observation. This type of study is controlled because it tries to resemble laboratory experiments in basic science. Eventually, it is randomized, as it has a random allocation of exposure. RCT should only be evaluating the effect of an intervention scientifically and safely. The investigators control the intervention or treatment and the unmeasured confounding. This kind of study demonstrates causality and starts from a population. The latter is randomized into two different groups. One group will be testing new treatment and the other a control treatment (sometimes a placebo), and the outcome will be analyzed. This study is very similar to a prospective study except for the randomization process. The latter allows the two groups to be identical except for a different kind of intervention. In this way, the possible confounding effects, which are the limitations of the observational studies, are being taken into account. Nonrandom trials. This kind of study takes place in hospitals, for example, when patients’ therapy (new or standard treatment) is selected by the operator. This is a weaker design, as it introduces confounding effects and selection bias. One problem of randomized trials is still the chance of selection bias, which can happen at two different levels (selecting a sample which is generalizable to a population; taking into considerations the characteristics of nonparticipants that is, those who are nonselected). Overall, randomization is crucial because it gives equal chances of receiving each treatment, groups are likely to have similar characteristics by chance, it reduces selection bias if patients enter trial before randomization and it increases credibility. There are
different ways of obtaining randomization such as tossing an unbiased coin, using random number tables, computergenerated random numbers, block randomization, and factorial design (i.e., multistage randomization). Another central issue of this kind of studies is the placebo effect that is, defined by Pocock as “even if the therapy is irrelevant to the patient’s condition, the patient’s attitude to his or her illness, and indeed the illness itself, may be improved by a feeling that something is being done about it.” Hence, differences between new-treatment and no-treatment groups could be due to either true treatment effect or the placebo effect. One way of preventing this could be concealing the allocation by not disclosing to patients and those involved in recruiting trial participants the allocation sequence before random allocation occurs. Moreover, to ensure that there is no bias from the investigator or the clinical operator blinding should be performed. The latter means not disclosing to participants and outcome assessors the treatment allocations after random allocation. This could be reached by making treatment appear identical in taste, appearance, texture, dosage regime, etc. In particular, a study can be single blinded, if the patient or the assessor or the clinician does not know the treatment allocation. Double blinded if two or more of patient, clinician and assessor do not know the treatment allocation. Ethical issues and stopping rules. To conclude it is worth to have a quick overview of what are the ethical issues implied in RCTs and possible stopping rules. Ideally, when a clinical trial is started, there should be a clinical equipoise that is, the assumption that at the moment of the design of the trial there is no “better” intervention between the one of the control and of the experimental group. If not, there could be some ethical issues linked to the allocation of the patients to the best or the worst treatment. Also, the informed consent of the patient is an essential characteristic of an ethical study. As regards stopping rules, they can be based on: (1) Harm: ● unblinding of the clinician is required if severe side effects or clinical emergency take place; ● the study can be terminated early if one treatment is clearly harmful; (2) Cost: ● The study can be terminated early if preliminary data prove the clear benefit of one treatment in order to save resources.