- Email: [email protected]
Medical databases in post-marketing drug surveillance
Trends in Pharmacological Sciences October 1986
Medical databases in post-marketing drug. surveillance IN MUCH OF THE developed world, the current drug approval process consists of preclinical animal testing followed by three phases of clinical testing 1. Phase I testing is usually conducted in normal volunteers, and is intended for the study of drug tolerance and pharmacokinetics, and to determine the dose range that should be used in humans. Phase II testing involves administration of the drug to a small number of selected patients, to obtain preliminary impressions about its efficacy for one or more conditions and to choose a final dosage form for broader trials. Phase III testing is the final premarketing test of the drug's safety and efficacy; multiple controlled and uncontrolled clinical trials are performed. These studies are usually continued to include between 500 and 3000 patients, even if efficacy can be proven with much smaller sample sizes, in order to be confident of detecting adverse effects that occur once in 100 exposed patients. As is apparent, considerable information remains unknown after the end of phase III testing, including: less common adverse effects; even if serious; delayed adverse effects; efficacy
and toxicity in types of patients usually excluded from pre-marketing testing, e.g., children and pregnant women; efficacy and toxicity in patients with other illnesses and/or ingesting other drugs; the efficacy and toxicity of the drug relative to other drugs used for the same purpose; efficacy and toxicity when used for indications other than those initially tested; the toxic effects of a massive overdose; physicians' prescribing habits; etc. The attempt to obtain this information after marketing is postmarketing drug surveillance (PMS). TABLE 1. Study designs used in post-marketing drug surveillance 1 2 3 4 5 6
Randomized clinical trials Prospective cohort studies Retrospectivecohort studies Case-control studies Case series Case reports
What study designs are used for post-marketing surveillance? Although randomized clinical trials can be used for PMS, their use for PMS is relatively uncommon, as they are complex and a PMS randomized clinical trial will suffer from most of the problems of a Phase III pre-marketing randomized clinical trial, gaining little. Rather, the techniques used in PMS are mostly the other designs listed in Table I. Each will be reviewed in turn.
1986, Elsevier Science Publishers B,V., A m s t e r d a m
0165 - 6147/86/$02.00
378
TIPS - O c t o b e r 1986
CASE-CONTROL STUDIES
I
DISEASE PRESENT (CASES)
cO uJ
o -v o o
ABSENT (CONTROLS)
PRESENT (EXPOSED)
A
B
ABSENT (NOT EXPOSED)
C
D
Fig. 1. Cohort and case-control studies provide similar information, but approach data collection from opposite directions.
R a n d o m i z e d clinical trials, or experimental studies, are studies which compare exposed subjects with unexposed subjects, or with subjects receiving another drug, in which the investigator controls which study group any given patient is assigned to, usually assigning subjects randomly to the different study groups. This is the most convincing and powerful design, as random assignment makes it most likely that the study groups are comparable. However, it is logistically complex, especially when large, artificial, and its application to special groups, like children and pregnant women, raises ethical concerns. Cohort studies are similar to randomized clinical trials, except the investigator does not control which study group a patient is assigned to: he just studies ongoing medical care. Cohort studies compare patients exposed to a drug to a control group of unexposed patients or patients exposed to another drug and look at subsequent clinical events. The rates of these events are then compared between the two groups. They can be performed prospectively, enrolling patients before or as they receive the drug under study and then watching for subsequent events, or retrospectively, studying both exposures and subsequent events which occur in the past. Cohort studies are ideal for answering questions involving a single exposure experienced by a relatively small proportion of a population, which is true of virtually any drug, and multiple possible outcomes. Thus, they are particularly useful for
studies of newly-marketed drugs, looking for drug effects that are either anticipated from premarketing studies or unanticipated. These studies, however, require very large samples of patients in order to detect relatively uncommon drug effects 2. This results in expensive studies which are logistically difficult to perform. Case-control studies obtain the same information as cohort studies, but approach data collection from the opposite direction (see Figure 1). Case-control studies identify a group of patients with a disease of interest and compare them to a control group of patients without the disease. Both groups are examined for antecedent drug exposures and the rates of exposure are compared. This approach is best when studying any possible cause of a single disease. This method has been used extensively in PMS to demonstrate many important associations. A classic example is in-utero exposure to diethylstilbesterol resulting in adenocarcinoma of the vagina 3. These studies can be performed more quickly and less expensively than cohort studies. However, it can be difficult to select the appropriate control group. In addition, since these studies often require questioning patients regarding prior drug exposures, the potential for recall bias exists: patients with an adverse outcome may be likely to remember prior drug exposures differently from those without such an outcome. This is less likely to be a problem when drug use is defined as present only if ingestion is for a long duration 4.
Case series are reports of a series of patients with a particular drug exposure and their subsequent clinical course. They can be useful to quantitate the frequency of medical events after exposure to a drug. However, because they have no control group, these studies cannot determine that the drug is the cause of any of the medical events, unless the medical event is known to be extremely uncommon in the absence of the drug exposure and is found to be much more common with the drug exposure. Finally, case reports are descriptions of individual patients who are exposed to a drug and who develop a suspected adverse drug reaction. They can be useful in alerting the medical community to the existence of an adverse drug reaction, especially if it is acute, of rapid onset, and uncommon in the absence of drug exposure. However, case reports do not provide an estimate of the incidence of the reaction, since the number of exposed patients is unknown. Thus, case reports are mostly useful for generating, rather than testing, hypotheses about drug effects.
How has post-marketing surveillance been conducted? The major differences between PMS studies and other epidemiologic studies lie in how these methods are operated. Many of these studies require unusually large sample sizes. For example, because 500 to 3000 subjects are usually studied prior to marketing, a PMS study conducted as a randomized clinical trial, cohort study, or case series is generally unwarranted unless it can include at least 10000 exposed subjects, and, in the first two, another 10 000 controls. Traditionally, PMS has been performed by physicians voluntarily reporting case reports of suspected adverse drug reactions, either to their national regulatory body, to pharmaceutical companies, or in the medical literature. The World Health Organization aggregates the information from multiple national regulatory bodies. Voluntary methods of reporting adverse reactions are potentially useful for discovering unexpected adverse drug reactions. This voluntary approach incorporates all drugs and prescribers, and it is relatively inexpensive. How-
379
T I P S - O c t o b e r 1986
ever, voluntary reporting requires that those reporting: (1) recognize or suspect an adverse reaction, (2) attribute the 'event' to a drug and (3) then report the incident to an appropriate individual or agency. Event recognition, attribution, and reporting are undoubtedly incomplete, but the degree of incompleteness is unknown. In addition, there is no information on the total population using the drug (the denominator). Therefore, the incidence of adverse reactions cannot be calculated. Thus, although this approach can be useful for generating 'signals' of possible adverse effects, as the United Kingdom's 'yellow card system' has been 5, it is rarely conclusive. Because these deficits can vary considerably, even among drugs of the same therapeutic class, e.g., according to how recently the drug was marketed or how aggressive the manufacturer is in soliciting reports, voluntary systems are of particularly limited use for comparing the relative effects of different drugs. Historically, the first major advance in PMS was the development of intensive short-term hospital-based surveillance. This approach uses nurse monitors to evaluate the effects of drugs received in hospitals, performing short-term cohort studies. For example, the Boston Collaborative Drug Surveillance Program has collected quantitative data on short-term expected and unexpected adverse drug reactions in hospitalized patients 6~. While this approach has made many important contributions, it cannot be used to study medium or longterm drug effects, to study drugs used primarily in outpatients, or to study u n c o m m o n drugs or diseases.
The Boston University Drug Epidemiology Unit has extended this approach of hospital-based studies, conducting case-control surveillance. Nurse monitors collect data on lifelong drug exposures, and case-control studies are conducted looking for associations between the reason for the hospitalization and these antecedent exposures. This has proven to be a very useful resource, capable of studying relatively u n c o m m o n expected adverse reactions and potentially capable of detecting unexpected adverse reactions 9. It
has been and continues to be the source of a considerable amount of important information. However, this approach cannot be used to study non-hospitalized individuals. In addition, like any approach which is not populationbased, it cannot be used to calculate the incidence of adverse drug reactions. The validity of lifetime drug histories determined via an interview has only been tested for selected categories of drugs, and the results may be subject to recall bias and interviewer bias. Finally, because it relies on gathering new data via interviewers, it is expensive, time consuming, and usually cannot quickly answer questions of clinical and/or regulatory importance, unless the illness in question is very common or the subject of a prior investigation. More recently, the Drug Surveillance Research Unit (DSRU) of the University of Southampton, UK, has developed an innovative system which it calls Prescription Event Monitoring 1°'11. After choosing the drugs to be studied, the DSRU obtains copies of all subsequent prescriptions for those drugs in England from the Prescription Pricing Authority, that part of the National Health Service which reimburses pharmacies for prescription drugs dispensed. Questionnaires are then sent to the prescribers, requesting information on all subsequent medical events. Analyses can then be descriptive or analytical, the latter comparing the experience of patients exposed to the study drug to the experience of patients exposed to other similar drugs or to the experience of the same patients after cessation of therapy. This is potentially a very useful and relatively cost-efficient approach. Finally, large scale 'cohort' studies conducted ad h o c by pharmaceutical companies have also been mounted. Typically, a company's sales force is asked to recruit 2000 cooperative physicians, each of w h o m is then asked to report on the experience of five patients who received the drug 12. This approach is extremely expensive, generally well over a million dollars 3, and is subject to biased reporting because of the mechanism of recruitment. Most importantly, though, is that such studies generally lack
control groups, making one dependent on a judgement about whether or not a medical event is likely to be due to a drug, and making the detection of new adverse drug reactions unlikely. Certainly a control group could be added, but this would double the cost. Use of medical databases in postmarketing surveillance As an attempt to address some of these deficiences in a cost-effective way, investigators have begun turning to medical databases to conduct PMS studies. There are a number of computerized collections of medical billing data which have been or could be used for such studies. They have a number of advantages, including size, cost, the absence of recall or interviewer bias, and in some of them, the fact that they are population-based and can be used prospectively. They also have a n u m b e r of disadvantages, including variable data elements; incomplete information on confounding variables; the inclusion of only severe illnesses; and uncertain and variable data validity. As an example, Health Maintenance Organizations can be useful in performing either drug-specific cohort studies or disease-specific case-control studies. However, even the larger Health Maintenance Organizations, e.g., Kaiser Medical Plan, are not as large as may be necessary in some studies, and generally do not have computerized pharmacies. Those that have computerized pharmacies, e.g. the Group Health Cooperative of Puget Sound, used by the Boston Collaborative Drug Surveillance program, are smaller and, so, are only useful when both the drug and the disease are relatively common 13A4. Health Maintenance Organizations generally do not include outpatient diagnoses in their computerized diagnosis database. Also, studies in Health Maintenance Organizations are limited to drugs admitted to the organizations' formularies. As another example, the Saskatchewan Health Plan has billing data on the one million residents of Saskatchewan. This has the major advantages of being a large, population-based system, without the problems of changing eligibility that other insurance plans have.
380 However, its population is not as large as is necessary for some studies. Other limitations include: (1) a population with few blacks; (2) a largely rural population; (3) a somewhat limited formulary (1500 drugs); (4) limited outpatient data; (5) the Health Plan uses data from a billing system in which errors may possibly be increased, since physician r e i m b u r s e m e n t changes dep e n d i n g on the listed diagnosis; and (6) the plan has only recently been used for rigorous drug epidemiology studies. Thus, although it is a potentially very useful resource, more information is needed before its true utility will be known. Similarly, the linked health data available from the one million person population of Rhode Island have potential promise. However, to date, this database includes drug data on only inpatients and a small (100000) subset of outpatients. An alternative source of computerized billing data is Medicaid data. Medicaid data can be particularly useful in achieving very large sample sizes, and outpatient diagnoses are available. It can be used to conduct very large cohort studies or to perform case-control studies, drawing its cases from a large population and drawing a true random sample of a population as controls, avoiding one of the major limitations of casecontrol studies. However, Medicaid databases include data from a skewed population, skewed by socioeconomic status, and do not include inpatient drug exposures. More importantly, it is difficult to differentiate between periods of good health and periods of ineligibility. Most significantly, important questions have been raised about the validity of the diagnosis data in these databases. One such database has recently been reviewed and these issues were
TIPS - October 1986
discussed in detail 15. Data from the Professional Activity Study of the Commission on Professional and Hospital Activities have also been used for PMS studies, especially by the Boston Collaborative Drug Surveillance Program 16. These data have been used particularly for case-finding for case-control studies. In addition to the limitations of casecontrol surveillance (see above), the process of determining prior exposure requires obtaining consent from multiple individual hospitals, then individual physicians, and finally individual patients, which results in logistical difficulties and low participation rates. In conclusion, a n u m b e r of different research resources are now available to perform PMS studies. Each has its advantages and disadvantages and each has been useful in certain situations 17. Included, more recently, are a n u m b e r of medical databases, derived from the billing process. These databases have the major advantage of being able to gather the large n u m b e r s of subjects necessary for PMS studies quickly, efficiently, and inexpensively. Because they are secondary data sources, however, n o n e has all of the data one might want if one were gathering it de novo. Different compromises need to be made when using each. Careful attention to the relative strengths and weaknesses of each will permit an investigator to choose the optimal system to answer his or her PMS question, and the consumer of such research to be aware of its limitations. It is unlikely that any single data source will ever fulfill all of the diverse data needs of this field. Certainly, none n o w does. The development of n e w systems of this type needs to be a high priority for the future.
Acknowledgement
The author would like to thank Dianne Greer and Susan Jordhamo for their technical support in preparing this manuscript and supported in part by grant FD-4000079 from the Food and Drug Administration.
BRIAN L. STROM University of Pennsylvania School of Medicine, Clinical Epidemiology Unit, Room 225L NEB/ $2, Philadelphia, PA 19104-6095, USA.
References
1 Barron,B. A. and Bukantz, S. C. (1967) Arch. Intern. Med. 119, 547-556 2 Vessey,M. P., Doll,R., Peto, R.,Johnson, B. and Wiggins,P. (1976)J. Biosocial. Sci. 8, 373-427 3 Herbst, A. L., Ifelder,H. and Poskanzer, D. C. (1971)N. Engl. J. Med. 284,878-881 4 Shapiro, S. (1984) Acta Med. Scand. suppl. 683, 23-27 5 Inman, W. H. W. and Price Evans,D. A. (1972) Br. Med. J. 13, 746-747 6 Cohen, M. R. (1974) Hosp. Pharm. 9, 437-488 7 lick, H. (1977)in Epidemiological Evaluation of Drugs (Colombo,F., Shapiro, S., SIone, D. and Tognoni, G., eds), pp. 47-53, PSG Publishing Co. 8 Miller,R. R. and Greenblatt,D. I. (eds) (1976) Drug Effects in Hospitalized Patients, John Wiley 9 Slone, D., Shapiro, S. and Miettinen, O. S. (1977)in Epidemiological Evaluation of Drugs (Colombo, F., Shapiro, S., Slone, D. and Tognoni, G., eds), pp. 59-70, PSG Publishing Co. 10 Inman, W. H. (1981) Br. Med. J. 282, 1216-1217 11 Inman, W. H. (1982) Br. Med. J. 285, 809-810 12 Gifford,L. M., Aengle,M. E., Myerson, R. M., Tannenbaum, P. I. (1980)J. Am. Med. Assoc. 243, 1532-1535 13 Walker,A. M. and lick, H. (1979)Am. J. Epidemiol. 110, 41-46 14 Walker, A. M., lick, H., Hunter, J. R., Danford, A. and Rothman, K. J. (1981) J. Am. Med. Assoc. 245, 2315--2317 15 Strom, B. L., Carson, J. L., Morse, M. L. and LeRoy,A. A. (1985)Clin. Pharmacol. Therap. 38, 359-364 16 lick, H. (1979) Am. J. Epidemiol. 109, 625-627 17 Summary of the Final Report of the Joint Commission on Prescription Drug Use, Inc.
January 23, 1980
Medical databases in post-marketing drug. surveillance IN MUCH OF THE developed world, the current drug approval process consists of preclinical animal testing followed by three phases of clinical testing 1. Phase I testing is usually conducted in normal volunteers, and is intended for the study of drug tolerance and pharmacokinetics, and to determine the dose range that should be used in humans. Phase II testing involves administration of the drug to a small number of selected patients, to obtain preliminary impressions about its efficacy for one or more conditions and to choose a final dosage form for broader trials. Phase III testing is the final premarketing test of the drug's safety and efficacy; multiple controlled and uncontrolled clinical trials are performed. These studies are usually continued to include between 500 and 3000 patients, even if efficacy can be proven with much smaller sample sizes, in order to be confident of detecting adverse effects that occur once in 100 exposed patients. As is apparent, considerable information remains unknown after the end of phase III testing, including: less common adverse effects; even if serious; delayed adverse effects; efficacy
and toxicity in types of patients usually excluded from pre-marketing testing, e.g., children and pregnant women; efficacy and toxicity in patients with other illnesses and/or ingesting other drugs; the efficacy and toxicity of the drug relative to other drugs used for the same purpose; efficacy and toxicity when used for indications other than those initially tested; the toxic effects of a massive overdose; physicians' prescribing habits; etc. The attempt to obtain this information after marketing is postmarketing drug surveillance (PMS). TABLE 1. Study designs used in post-marketing drug surveillance 1 2 3 4 5 6
Randomized clinical trials Prospective cohort studies Retrospectivecohort studies Case-control studies Case series Case reports
What study designs are used for post-marketing surveillance? Although randomized clinical trials can be used for PMS, their use for PMS is relatively uncommon, as they are complex and a PMS randomized clinical trial will suffer from most of the problems of a Phase III pre-marketing randomized clinical trial, gaining little. Rather, the techniques used in PMS are mostly the other designs listed in Table I. Each will be reviewed in turn.
1986, Elsevier Science Publishers B,V., A m s t e r d a m
0165 - 6147/86/$02.00
378
TIPS - O c t o b e r 1986
CASE-CONTROL STUDIES
I
DISEASE PRESENT (CASES)
cO uJ
o -v o o
ABSENT (CONTROLS)
PRESENT (EXPOSED)
A
B
ABSENT (NOT EXPOSED)
C
D
Fig. 1. Cohort and case-control studies provide similar information, but approach data collection from opposite directions.
R a n d o m i z e d clinical trials, or experimental studies, are studies which compare exposed subjects with unexposed subjects, or with subjects receiving another drug, in which the investigator controls which study group any given patient is assigned to, usually assigning subjects randomly to the different study groups. This is the most convincing and powerful design, as random assignment makes it most likely that the study groups are comparable. However, it is logistically complex, especially when large, artificial, and its application to special groups, like children and pregnant women, raises ethical concerns. Cohort studies are similar to randomized clinical trials, except the investigator does not control which study group a patient is assigned to: he just studies ongoing medical care. Cohort studies compare patients exposed to a drug to a control group of unexposed patients or patients exposed to another drug and look at subsequent clinical events. The rates of these events are then compared between the two groups. They can be performed prospectively, enrolling patients before or as they receive the drug under study and then watching for subsequent events, or retrospectively, studying both exposures and subsequent events which occur in the past. Cohort studies are ideal for answering questions involving a single exposure experienced by a relatively small proportion of a population, which is true of virtually any drug, and multiple possible outcomes. Thus, they are particularly useful for
studies of newly-marketed drugs, looking for drug effects that are either anticipated from premarketing studies or unanticipated. These studies, however, require very large samples of patients in order to detect relatively uncommon drug effects 2. This results in expensive studies which are logistically difficult to perform. Case-control studies obtain the same information as cohort studies, but approach data collection from the opposite direction (see Figure 1). Case-control studies identify a group of patients with a disease of interest and compare them to a control group of patients without the disease. Both groups are examined for antecedent drug exposures and the rates of exposure are compared. This approach is best when studying any possible cause of a single disease. This method has been used extensively in PMS to demonstrate many important associations. A classic example is in-utero exposure to diethylstilbesterol resulting in adenocarcinoma of the vagina 3. These studies can be performed more quickly and less expensively than cohort studies. However, it can be difficult to select the appropriate control group. In addition, since these studies often require questioning patients regarding prior drug exposures, the potential for recall bias exists: patients with an adverse outcome may be likely to remember prior drug exposures differently from those without such an outcome. This is less likely to be a problem when drug use is defined as present only if ingestion is for a long duration 4.
Case series are reports of a series of patients with a particular drug exposure and their subsequent clinical course. They can be useful to quantitate the frequency of medical events after exposure to a drug. However, because they have no control group, these studies cannot determine that the drug is the cause of any of the medical events, unless the medical event is known to be extremely uncommon in the absence of the drug exposure and is found to be much more common with the drug exposure. Finally, case reports are descriptions of individual patients who are exposed to a drug and who develop a suspected adverse drug reaction. They can be useful in alerting the medical community to the existence of an adverse drug reaction, especially if it is acute, of rapid onset, and uncommon in the absence of drug exposure. However, case reports do not provide an estimate of the incidence of the reaction, since the number of exposed patients is unknown. Thus, case reports are mostly useful for generating, rather than testing, hypotheses about drug effects.
How has post-marketing surveillance been conducted? The major differences between PMS studies and other epidemiologic studies lie in how these methods are operated. Many of these studies require unusually large sample sizes. For example, because 500 to 3000 subjects are usually studied prior to marketing, a PMS study conducted as a randomized clinical trial, cohort study, or case series is generally unwarranted unless it can include at least 10000 exposed subjects, and, in the first two, another 10 000 controls. Traditionally, PMS has been performed by physicians voluntarily reporting case reports of suspected adverse drug reactions, either to their national regulatory body, to pharmaceutical companies, or in the medical literature. The World Health Organization aggregates the information from multiple national regulatory bodies. Voluntary methods of reporting adverse reactions are potentially useful for discovering unexpected adverse drug reactions. This voluntary approach incorporates all drugs and prescribers, and it is relatively inexpensive. How-
379
T I P S - O c t o b e r 1986
ever, voluntary reporting requires that those reporting: (1) recognize or suspect an adverse reaction, (2) attribute the 'event' to a drug and (3) then report the incident to an appropriate individual or agency. Event recognition, attribution, and reporting are undoubtedly incomplete, but the degree of incompleteness is unknown. In addition, there is no information on the total population using the drug (the denominator). Therefore, the incidence of adverse reactions cannot be calculated. Thus, although this approach can be useful for generating 'signals' of possible adverse effects, as the United Kingdom's 'yellow card system' has been 5, it is rarely conclusive. Because these deficits can vary considerably, even among drugs of the same therapeutic class, e.g., according to how recently the drug was marketed or how aggressive the manufacturer is in soliciting reports, voluntary systems are of particularly limited use for comparing the relative effects of different drugs. Historically, the first major advance in PMS was the development of intensive short-term hospital-based surveillance. This approach uses nurse monitors to evaluate the effects of drugs received in hospitals, performing short-term cohort studies. For example, the Boston Collaborative Drug Surveillance Program has collected quantitative data on short-term expected and unexpected adverse drug reactions in hospitalized patients 6~. While this approach has made many important contributions, it cannot be used to study medium or longterm drug effects, to study drugs used primarily in outpatients, or to study u n c o m m o n drugs or diseases.
The Boston University Drug Epidemiology Unit has extended this approach of hospital-based studies, conducting case-control surveillance. Nurse monitors collect data on lifelong drug exposures, and case-control studies are conducted looking for associations between the reason for the hospitalization and these antecedent exposures. This has proven to be a very useful resource, capable of studying relatively u n c o m m o n expected adverse reactions and potentially capable of detecting unexpected adverse reactions 9. It
has been and continues to be the source of a considerable amount of important information. However, this approach cannot be used to study non-hospitalized individuals. In addition, like any approach which is not populationbased, it cannot be used to calculate the incidence of adverse drug reactions. The validity of lifetime drug histories determined via an interview has only been tested for selected categories of drugs, and the results may be subject to recall bias and interviewer bias. Finally, because it relies on gathering new data via interviewers, it is expensive, time consuming, and usually cannot quickly answer questions of clinical and/or regulatory importance, unless the illness in question is very common or the subject of a prior investigation. More recently, the Drug Surveillance Research Unit (DSRU) of the University of Southampton, UK, has developed an innovative system which it calls Prescription Event Monitoring 1°'11. After choosing the drugs to be studied, the DSRU obtains copies of all subsequent prescriptions for those drugs in England from the Prescription Pricing Authority, that part of the National Health Service which reimburses pharmacies for prescription drugs dispensed. Questionnaires are then sent to the prescribers, requesting information on all subsequent medical events. Analyses can then be descriptive or analytical, the latter comparing the experience of patients exposed to the study drug to the experience of patients exposed to other similar drugs or to the experience of the same patients after cessation of therapy. This is potentially a very useful and relatively cost-efficient approach. Finally, large scale 'cohort' studies conducted ad h o c by pharmaceutical companies have also been mounted. Typically, a company's sales force is asked to recruit 2000 cooperative physicians, each of w h o m is then asked to report on the experience of five patients who received the drug 12. This approach is extremely expensive, generally well over a million dollars 3, and is subject to biased reporting because of the mechanism of recruitment. Most importantly, though, is that such studies generally lack
control groups, making one dependent on a judgement about whether or not a medical event is likely to be due to a drug, and making the detection of new adverse drug reactions unlikely. Certainly a control group could be added, but this would double the cost. Use of medical databases in postmarketing surveillance As an attempt to address some of these deficiences in a cost-effective way, investigators have begun turning to medical databases to conduct PMS studies. There are a number of computerized collections of medical billing data which have been or could be used for such studies. They have a number of advantages, including size, cost, the absence of recall or interviewer bias, and in some of them, the fact that they are population-based and can be used prospectively. They also have a n u m b e r of disadvantages, including variable data elements; incomplete information on confounding variables; the inclusion of only severe illnesses; and uncertain and variable data validity. As an example, Health Maintenance Organizations can be useful in performing either drug-specific cohort studies or disease-specific case-control studies. However, even the larger Health Maintenance Organizations, e.g., Kaiser Medical Plan, are not as large as may be necessary in some studies, and generally do not have computerized pharmacies. Those that have computerized pharmacies, e.g. the Group Health Cooperative of Puget Sound, used by the Boston Collaborative Drug Surveillance program, are smaller and, so, are only useful when both the drug and the disease are relatively common 13A4. Health Maintenance Organizations generally do not include outpatient diagnoses in their computerized diagnosis database. Also, studies in Health Maintenance Organizations are limited to drugs admitted to the organizations' formularies. As another example, the Saskatchewan Health Plan has billing data on the one million residents of Saskatchewan. This has the major advantages of being a large, population-based system, without the problems of changing eligibility that other insurance plans have.
380 However, its population is not as large as is necessary for some studies. Other limitations include: (1) a population with few blacks; (2) a largely rural population; (3) a somewhat limited formulary (1500 drugs); (4) limited outpatient data; (5) the Health Plan uses data from a billing system in which errors may possibly be increased, since physician r e i m b u r s e m e n t changes dep e n d i n g on the listed diagnosis; and (6) the plan has only recently been used for rigorous drug epidemiology studies. Thus, although it is a potentially very useful resource, more information is needed before its true utility will be known. Similarly, the linked health data available from the one million person population of Rhode Island have potential promise. However, to date, this database includes drug data on only inpatients and a small (100000) subset of outpatients. An alternative source of computerized billing data is Medicaid data. Medicaid data can be particularly useful in achieving very large sample sizes, and outpatient diagnoses are available. It can be used to conduct very large cohort studies or to perform case-control studies, drawing its cases from a large population and drawing a true random sample of a population as controls, avoiding one of the major limitations of casecontrol studies. However, Medicaid databases include data from a skewed population, skewed by socioeconomic status, and do not include inpatient drug exposures. More importantly, it is difficult to differentiate between periods of good health and periods of ineligibility. Most significantly, important questions have been raised about the validity of the diagnosis data in these databases. One such database has recently been reviewed and these issues were
TIPS - October 1986
discussed in detail 15. Data from the Professional Activity Study of the Commission on Professional and Hospital Activities have also been used for PMS studies, especially by the Boston Collaborative Drug Surveillance Program 16. These data have been used particularly for case-finding for case-control studies. In addition to the limitations of casecontrol surveillance (see above), the process of determining prior exposure requires obtaining consent from multiple individual hospitals, then individual physicians, and finally individual patients, which results in logistical difficulties and low participation rates. In conclusion, a n u m b e r of different research resources are now available to perform PMS studies. Each has its advantages and disadvantages and each has been useful in certain situations 17. Included, more recently, are a n u m b e r of medical databases, derived from the billing process. These databases have the major advantage of being able to gather the large n u m b e r s of subjects necessary for PMS studies quickly, efficiently, and inexpensively. Because they are secondary data sources, however, n o n e has all of the data one might want if one were gathering it de novo. Different compromises need to be made when using each. Careful attention to the relative strengths and weaknesses of each will permit an investigator to choose the optimal system to answer his or her PMS question, and the consumer of such research to be aware of its limitations. It is unlikely that any single data source will ever fulfill all of the diverse data needs of this field. Certainly, none n o w does. The development of n e w systems of this type needs to be a high priority for the future.
Acknowledgement
The author would like to thank Dianne Greer and Susan Jordhamo for their technical support in preparing this manuscript and supported in part by grant FD-4000079 from the Food and Drug Administration.
BRIAN L. STROM University of Pennsylvania School of Medicine, Clinical Epidemiology Unit, Room 225L NEB/ $2, Philadelphia, PA 19104-6095, USA.
References
1 Barron,B. A. and Bukantz, S. C. (1967) Arch. Intern. Med. 119, 547-556 2 Vessey,M. P., Doll,R., Peto, R.,Johnson, B. and Wiggins,P. (1976)J. Biosocial. Sci. 8, 373-427 3 Herbst, A. L., Ifelder,H. and Poskanzer, D. C. (1971)N. Engl. J. Med. 284,878-881 4 Shapiro, S. (1984) Acta Med. Scand. suppl. 683, 23-27 5 Inman, W. H. W. and Price Evans,D. A. (1972) Br. Med. J. 13, 746-747 6 Cohen, M. R. (1974) Hosp. Pharm. 9, 437-488 7 lick, H. (1977)in Epidemiological Evaluation of Drugs (Colombo,F., Shapiro, S., SIone, D. and Tognoni, G., eds), pp. 47-53, PSG Publishing Co. 8 Miller,R. R. and Greenblatt,D. I. (eds) (1976) Drug Effects in Hospitalized Patients, John Wiley 9 Slone, D., Shapiro, S. and Miettinen, O. S. (1977)in Epidemiological Evaluation of Drugs (Colombo, F., Shapiro, S., Slone, D. and Tognoni, G., eds), pp. 59-70, PSG Publishing Co. 10 Inman, W. H. (1981) Br. Med. J. 282, 1216-1217 11 Inman, W. H. (1982) Br. Med. J. 285, 809-810 12 Gifford,L. M., Aengle,M. E., Myerson, R. M., Tannenbaum, P. I. (1980)J. Am. Med. Assoc. 243, 1532-1535 13 Walker,A. M. and lick, H. (1979)Am. J. Epidemiol. 110, 41-46 14 Walker, A. M., lick, H., Hunter, J. R., Danford, A. and Rothman, K. J. (1981) J. Am. Med. Assoc. 245, 2315--2317 15 Strom, B. L., Carson, J. L., Morse, M. L. and LeRoy,A. A. (1985)Clin. Pharmacol. Therap. 38, 359-364 16 lick, H. (1979) Am. J. Epidemiol. 109, 625-627 17 Summary of the Final Report of the Joint Commission on Prescription Drug Use, Inc.
January 23, 1980