- Email: [email protected]
Global index of safety (GIS)
Journal of Clinical Epidemiology 54 (2001) 1120–1125
Global index of safety (GIS): A new instrument to assess drug safety José A. Sacristána,*, Juan C. Gómeza, Xavier Badíab, Paul Kindc a
Clinical Research Department, Lilly S.A., Madrid, Spain b Catalan Institute of Public Health, Barcelona, Spain c Center for Health Economics, University of York, York, UK Received 16 October 2000; received in revised form 30 April 2001; accepted 2 May 2001
Abstract There is an asymmetry between the extraordinary development of measures and tools aimed at studying the beneficial effects of the drugs and the more limited methods to assess their safety profile. The goal of our study was to develop a global measuring tool to assess drugs’ safety. We conducted a survey of Spanish psychiatrists in mental health centers and outpatient treatment units to assess the severity scores that they would assign to a list of the most common adverse events (AEs) that usually occur with antipsychotic treatment. The severity scores were then applied to the list of AEs that really occurred along a naturalistic pharmacoepidemiological study on the use of different antipsychotics in the treatment of schizophrenia. The Global Index of Safety (GIS) of the experimental group treated with olanzapine (OLZ) was compared with the GIS of the control group and with the GIS of specific antipsychotics for which the number of treated patients was greater than 100. A total of 194 psychiatrists rated the severity of each AE on a scale of 1 (insignificant) to 5 (extremely severe). The individual severity was applied to the 2949 schizophrenic patients included in a pharmacoepidemiological study. A GIS was calculated for every group of patients receiving the same treatments. The GIS of the control group was higher (4.3) than that calculated from the experimental group (2.5) (P 0.001). The GIS of the risperidone (3.6) and haloperidol (6.0) subgroups were higher than that calculated from the OLZ group (P 0.001). The development of a GIS may facilitate the comparison of the safety of several drugs and may constitute a very valuable aid for those involved in selecting drugs. © 2001 Elsevier Science Inc. All rights reserved. Keywords: Safety; Adverse events; Antipsychotics; Schizophrenia; Olanzapine; Risperidone; Haloperidol
1. Introduction The drug selection process is based on the comparison of the risk–benefit profile of the different options available. Initially, efficacy and risk effects are assessed using randomized clinical trials (RCTs) in which the new drugs are compared with placebo or standard treatments. The limited number of patients that can be included in RCTs, their experimental nature, and the relatively short follow-up period, make it desirable to complete the information obtained in RCTs with post-marketing data when drugs are used in actual practice. Generally, the methods of studying adverse events are less extensive than the methods used to assess the beneficial effects of a drug. Although there are frequent controversies on the pros and cons of using surrogate endpoints, or on the appropriate temporal horizon to measure beneficial effects, efficacy assessment presents two important advantages with
* Corresponding author. Clinical Research Department, Lilly S.A., Avda. de la Industria 30, 28108 Alcobendas, Madrid, Spain. Fax: 34-916635231. E-mail address: [email protected] (J.A. Sacristán)
respect to safety evaluation: the high number and the sophistication of the measuring instruments, and the possibility of directly comparing identical outcomes (for example, reduction of arterial blood pressure or decrease in mortality). Spontaneous reporting systems remain the most common way to detect rare adverse events (AEs) that are temporally associated with drug use. However, these systems are not a valid option when decisionmakers are interested in detecting the true incidence of anticipated and common harmful effects. To obtain the latter information, appropriate analytic studies such as case-control studies, cohort studies, or postmarketing clinical trials need to be conducted [1]. But a common problem of all analytic methods is that results are often limited to a long list of the frequencies of occurrence of all the AEs reported during the study. This line-item process may assist in achieving an approximate understanding of the safety profile of the drugs studied and may sometimes be sufficient for deciding on the option with the most favorable profile. This is true in cases in which a particular drug is associated with an AE that is much more serious than those of other drugs, or if the frequency of one or more AEs is clearly higher in one of the options. However, other times the choice is not that obvious and the comparison of the
0895-4356/01/$ – see front matter © 2001 Elsevier Science Inc. All rights reserved. PII: S0895-4356(01)00 3 8 4 - 5
J.A. Sacristán et al. / Journal of Clinical Epidemiology 54 (2001) 1120–1125
1121
item-by-item AE does not allow for any conclusions to be drawn. The development of a global safety measuring tool, which would represent a single weighted score for an overall AE profile, may facilitate the safety comparison of several drugs and constitute a valuable aid for those involved in selecting drugs. The objective of this study was to develop a GIS in parallel to a recent naturalistic prospective pharmacoepidemiological study with antipsychotics.
signed, the severity of each AE was calculated (mean, standard deviation [SD], median, and 95% confidence intervals [95% CI] for each AE). On closure of the database and before any statistical analysis, the list was completed by including AEs detected during the pharmacoepidemiological study that were not on the original list. For these AEs, severity scores were assigned using the value of the AE most similar to itself by means of a consensus meeting of the 5 investigators managing this component of the study.
2. Materials and methods
2.3. Application of the AE severity scores to the EFESO and calculation of the GIS for each treatment group
The GIS was developed in the following steps: (1) Selection of AE; (2) Drafting a questionnaire and physician rating of the severity of each AE; (3) Application of AE severity scores to the pharmacoepidemiological study (EFESO) and calculation of the GIS for each treatment group; and (4) Comparison of the GIS of the different groups. 2.1. Selection of the adverse events A list with the most common and severe AE that may occur in patients treated with antipsychotics was constructed. AE selection took place by reviewing the AE reported in the recent literature [2–5]. The initial list was completed by searching the Eli Lilly and Company safety database for the most common AE (frequency 1%) reported for OLZ and comparable drugs, including those that occurred after commercialization. Other known AEs of neuroleptics were also included in our criteria. The AEs with similar terminology were grouped according to the COSTART terminology [6]. 2.2. Drafting of the questionnaire and obtaining the severity scores for each AE During the initial meetings for the start-up of a pharmacoepidemiological study with antipsychotics, the purpose of developing a new safety evaluation tool and its subsequent application to the safety results of the study was explained to the participating psychiatrists. In the GIS questionnaire instructions, the psychiatrists were asked to assign, according to their experience, scores to each of the AEs on the list, rating the severity that each of the AEs would have in schizophrenic patients treated with antipsychotics. The scores ranged from 1–5 on the following assessment scale: (1) Insignificant, (2) Mild, (3) Moderate, (4) Severe, and (5) Extremely severe. This system for scoring the physicianperceived severity of the AE has been reported previously [7]. All questionnaires were administered by either JAS or JCG and all participants received the same instructions. The questionnaire included instructions for its completion. Given that a particular AE could have different degrees of severity in different patients, it was suggested that the score should reflect a “standard” severity—in other words, the severity they had most often observed in the schizophrenic patients they had treated. All psychiatrists participating at the start-up meetings for the study completed the questionnaire. Using the scores as-
EFESO was an observational phase-IV prospective pharmacoepidemiological study with an open comparison in parallel groups to assess the safety of OLZ in comparison with other antipsychotic drugs in the treatment of outpatients with schizophrenia. Data were collected by a total of 293 psychiatrists, mostly in mental health centers or outpatient treatment units. The main results of the study have recently been published elsewhere [8]. To assess the main goal, all AEs spontaneously reported by patients or identified in the course of the doctor–patient interview were recorded. An adverse event was defined as any undesirable experience occurring after administration of the product. An adverse event did not necessarily have a causal relationship with the treatment. Extrapyramidal symptoms (EPS) were collected by means of a short questionnaire, based on the extrapyramidal symptoms section of the UKU scale: dystonia, rigidity, hypokinesia, tremor, dyskinesia, and akathisia [9]. All AEs were coded using the COSTART glossary. Applying the AE previously obtained severity scores to the EFESO AE results, the GIS was calculated for each antipsychotic. The first step was to calculate the safety index for each patient by summing the severity scores for all of the respective AEs in each patient. For example, the safety index for a drug in a patient without any AEs would be 0, whereas it would be 7.5 in a patient with 2 AEs with severity ratings of 3.5 and 4. The second step was to calculate the GIS for each drug; this was determined by averaging the indexes calculated in all patients in the respective drug treatment groups. 2.4. Comparison of the GIS of the different groups The GIS of the experimental group treated with olanzapine (OLZ) was compared with that of the control group (all other antipsychotics). Moreover, the GIS for OLZ was compared with that of specific antipsychotics for which the number of treated patients was greater than 100. Two GIS values were determined. The first included all patients of the study, regardless of AE experience, and the second calculation only included data from patients who experienced at least 1 AE. 2.5. Statistical analysis The EFESO study was designed to confirm the differences in the incidence of extrapyramidal symptoms between
1122
J.A. Sacristán et al. / Journal of Clinical Epidemiology 54 (2001) 1120–1125
the OLZ group and the control group, with a 90% power and a two-tailed risk of 0.05, to which end a sample size of approximately 1000 patients was estimated for each group. In addition, the OLZ group contained twice the number of patients as the control group to detect, with an 80% power and an risk of .05, any adverse effects occurring with a frequency of 1:1000 in the OLZ group. In short, to meet our objectives the study was designed to gather information on approximately 2000 patients treated with OLZ and 1000 patients treated with other antipsychotic drugs [8]. All patients who received OLZ were included in the OLZ group, and patients who did not receive OLZ were included in the control group. Nevertheless, to test for differences between different drugs in the control group, it was decided a priori to conduct a secondary analysis comparing the incidence of AE in the OLZ group versus treatment-specific subgroups with n 100. The data were keyed into two databases simultaneously by different individuals and later compared to eliminate errors. The system used for the verification, validation, and analysis of the data was SAS® version 6.12 for Windows. All patients with available information were included in the analysis. The incidence of each AE in each group was calculated for the number of patients presenting the event at any time during the study for the total number of patients in the group. The incidence of AE was described by means of frequency and percentage and compared with the 2 test or Fisher’s Exact test. The GISs were described using means, medians, SDs, and ranges, and compared using single factor analysis of variance (ANOVA). A two-tailed significance level of 0.05 was applied for all tests. 3. Results Due to space restrictions, the list with the average severity assigned for each AE by the participating psychiatrists is not included in this article (interested readers may request it from the corresponding author). The highest perceived severity scores corresponded to malignant neuroleptic syndrome (mean SD, 4.81 0.76), agranulocytosis (4.64 0.88), and suicide attempt (4.45 0.88). The lowest corresponded to an increase in coughing (1.93 0.69), chills (1.91 0.74), and thirst (1.88 0.71). In the EFESO study, 2128 (72,2%) patients were treated with the experimental drug (OLZ) and 821 (28,8%) patients were in the control group, treated as follows: risperidone (RIS) 417 (14.1%), haloperidol (HAL) 112 (3.8%), sertindole 84 (2.8%) zuclopenthixol 74 (2.5%), fluphenazine 33 (1.1%), trifluoperazine 31 (1.1%), and others 70 (2.3%). Despite the observational nature of the study there were no statistically significant differences between the two groups in age, gender, time from onset, schizophrenia subtype, or baseline severity [8]. The GIS of the OLZ group was compared with the GIS of the control group. Additionally, the GIS of the OLZ group was compared with the GIS of pa-
Table 1 Global Index of Safety (GIS) of olanzapine (OLZ) vs. control group, and OLZ vs. risperidone (RIS) and haloperidol (HAL). All-patients analysis
Mean Median SD Range
OLZa N 1,896
Controlb N 748
OLZc N 1,896
RIS N 383
HAL N 104
2.5 0 3.5 0–24.7
4.3 3 4.6 0–21.3
2.5 0 3.5 0–24.7
3.6 2.7 4.4 0–19.4
6.0 5.8 4.9 0–21.3
p 0.001 versus control (ANOVA). This group includes all patients in the control, including the RIS and HAL subgroups. c p 0.001 versus RIS, and HAL (ANOVA). a
b
tients treated with RIS and HAL (antipsychotics for which the number of treated patients was greater than 100). A higher percentage of the patients in the control group (64%) presented at least one adverse event at some time in the study, a statistically significant difference (P 0.001) when compared with the OLZ group (48%). This percentage was also lower (P 0.001) in the OLZ group compared with the HAL (80%) and RIS (57%) subgroups. In the same way, a significantly lower (P 0.001) percentage of patients developed EPS in the OLZ group (37%) than in the RIS (50%) and HAL (76%) subgroups. These results are consistent with the profile shown in the registration clinical trials [2–5] and are included in the product’s package insert and in the only randomized, double-blind clinical trial that has compared OLZ and RIS [10]. Table 1 compares the GIS for OLZ with the antipsychotic control group as well as with RIS and HAL specifically, when all patients were included in the analysis. The GIS calculated from the control group (4.3) was 72% higher (worse score) than that calculated from the OLZ group (2.5) (P 0.001). The GIS of the RIS (3.6) and HAL (6.0) subgroups were 44% and 140% higher than that calculated from the OLZ group (P 0.001). Table 2 shows the results of this same comparison when only including those patients with at least one AE. In this case, the GIS of OLZ was also significantly lower than that of the antipsychotics used in the control group or with HAL and RIS specifically. In pa-
Table 2 Global Index of Safety (GIS) of olanzapine (OLZ) vs. control group, and OLZ vs. risperidone (RIS) and haloperidol (HAL). Patients with at least one adverse event
Mean Median SD Range
OLZa N 903
Controlb N 476
OLZc N 903
RIS N 218
HAL N 83
5.2 3.2 3.4 2–24.7
6.7 5.8 4.0 2–21.3
5.2 3.2 3.4 2–24.7
6.3 5.8 4.1 2–19.4
7.6 6.2 4.3 2.4–21.3
p 0.001 versus Control (ANOVA). This group includes all patients in the control group, including the RIS and HAL subgroups. c p 0.001 versus RIS, and HAL (ANOVA). a
b
J.A. Sacristán et al. / Journal of Clinical Epidemiology 54 (2001) 1120–1125 Table 3 Incidence of adverse events reported with a frequency higher than 1%
Akathisia Dyskinesia Dystonia Extrapyramidal syndrome Hypertonia Hypokinesia Somnolence Tremor Weight gain
OLZ N 2,128
Control group N 821
N
N
%
%
59 31 24
2.8 1.5 1.1
78 12 24
9.5 1.5 2.9
6 73 104 96 140 146
0.3 3.4 4.9 4.5 6.6 6.9
9 97 105 17 121 14
1.1 11.8 12.8 2.18 14.7 1.7
2 60.539, p 0.001 2 11.927, p 0.001 Fisher, 7.762, p 0.009 2 76.665, p 0.001 2 56.179, p 0.001 2 9.577, p 0.002 2 48.889, p 0.001 2 30.690, p 0.001
Adverse events have been classified and coded according to the COSTART dictionary
tients who experienced at least one AE, HAL (7.6) and RIS (6.3) had a higher a GIS than patients treated with OLZ (5.2). Table 3 lists the AE reported in more than 1% of the OLZ group or in the control group, with an indication of events in which the incidence was statistically significant between the groups. The control group was also associated with a greater incidence of sexual disorders in males, whereas OLZ was more frequently associated with somnolence and weight gain. There were no significant differences in the incidence of anticholinergic symptoms (dry mouth,
1123
constipation, diplopia, urinary retention, difficulties in concentration, and confusion). There were no cases of agranulocytosis in the study. A higher percentage of patients in the control group (57,9%) received some kind of concomitant treatment, in comparison with patients in the OLZ group (36.3%; P 0.001). Specifically, a lower percentage of the patients in the OLZ group received anticholinergic medication compared with the control group or the HAL and RIS subgroups (at 6 months, OLZ 10.2% versus control group 26.8%, P 0.001; versus RIS 19.9%, P 0.001; and versus HAL 44%, P 0.001). Table 4 shows the AEs, classified and coded according the COSTART dictionary, that were reported with a statistically significant difference frequency in the OLZ group compared with the RIS and HAL subgroups. 4. Discussion The calculation of a GIS provides different information than the conventional method of reporting the safety profile of drugs in clinical trials and epidemiological studies. It allows obtaining information on the relative severity assigned by the researchers to different AEs. For example, in this study, the researchers considered that weight gain and somnolence have a lower severity (2.6 and 2.42, respectively) than the EPS most often reported with the other groups: dystonia (3.21), hypokinesia (3.01), akathisia (3.05), tremor
Table 4 Incidence of those adverse events that were reported with a significantly different frequency in the olanzapine group compared with the risperidone and haloperidol subgroups, classified and coded according to the COSTART dictionary
Event
Olanzapine N 2,128 n (%)
Risperidone N 417 n (%)
Haloperidol N 112 n (%)
Akathisia
59 (2.8)
30 (7.2)
19 (17)
Dystonia
24 (1.1)
9 (2.2)
4 (3.6)
6 (0.3)
3 (0.7)
4 (3.6)
Hypertonia
73 (3.4)
35 (8.4)
29 (25.9)
Hypokinesia
104 (4.9)
36 (8.6)
30 (26.8)
Hypotension
4 (0.2)
1 (0.2)
2 (1.8)
Somnolence
96 (4.5)
7 (1.7)
5 (4.5)
Tremor
140 (6.6)
47 (11.3)
29 (25.9)
Weight gain
146 (6.9)
8 (1.9)
1 (0.9)
Males Abnormal Ejaculation
N 1,349 0
N 274 2 (0.7)
N 68 0
Extrapyramidal syndrome
Impotence Females Amenorrhea
0 N 769 5 (0.7)
2 (0.73) N 142 5 (3.5)
0 N 44 0
OLZ versus RIS 2/p
OLZ versus HAL 2/p
2 20.200 p 0.001
2 63.761 p 0.001 2 5.147 p 0.048 2 25.905 p 0.001 2 123.521 p 0.001 2 90.720 p 0.001 2 10.168 p 0.033
2 21.134 p 0 .001 2 9.412 p 0.002 2 7.204 p 0.007 2 11.275 p 0.001 2 14.982 p 0.001 2 9.859 p 0.028 2 9.859 p 0.028 2 9.100 p 0.011
2 56.900 p 0.001 2 6.180 p 0.013
1124
J.A. Sacristán et al. / Journal of Clinical Epidemiology 54 (2001) 1120–1125
(2.74), or hypertonia (3.16). Also, the use of a global index allows a safety comparison to be made between the different groups, using the usual statistical techniques (Table 1 and Table 2). For example, it would be advisable in systematic application to pharmacological groups such as NSAIDs in which the safety profile usually determines the optimal option. Finally, the possible use of a GIS to compare drugs from different therapeutic groups could prove to be attractive for decisionmakers. A classical analysis of the safety profile is in many cases sufficient to determine which of the options presents the most favorable profile. In these cases, the use of a GIS should be considered a complementary tool that may serve to increase available information with the advantages noted earlier. For instance, the comparisons made in Table 3 and Table 4 may suggest that, as a whole, OLZ presents a more favorable profile than the other treatment groups: statistically significant differences were detected in the incidence of EPS in favor of OLZ compared with the other treatment groups; sexual disorders in males were more frequent in the control group, whereas OLZ was more frequently associated with somnolence and weight gain. However, the relative importance of each AE is not always clear; in the absence of a global safety measure, the selection of the safest option may be a controversial decision. Obviously, the use of a GIS derived from scores previously assigned by physicians with experience in the treatment of a specific disease may be a valuable tool to help decisionmakers select the most favorable options. In EFESO, a large proportion of patients from both the OLZ and control groups received concomitant treatment with antipsychotics, benzodiazepines, anticholinergics, or other drugs. Therefore, it is difficult to attribute the safety results unequivocally to OLZ or to other specific drugs. AEs were assigned to groups reflecting the initially prescribed drug in the study. This type of analysis may be appropriate to reflect routine clinical practice where antipsychotics are frequently used in combination. The high degree of variability observed in the GIS values (the SD is greater than the mean) could lead to a recommendation to use the median values. In the case of the EFESO study, the differences in safety between the different drugs would be even more marked: the median of OLZ would be 0 (equivalent to an absence of AE), 3 (equivalent to a moderate AE) for control antipsychotics, and 2.7 and 5.8 for RIS and HAL, respectively. Despite the advantages mentioned above, the proposed method presents several limitations. First, the scores assigned a priori by a group of researchers (in this case psychiatrists who work in the outpatient setting), may not reflect scores that would be assigned by other researchers. For example, psychiatrists working in a hospital setting may assign different severity scores, based on the different conditions of the patients they usually treat. In such cases it would be useful for each collective with different characteristics to establish its own evaluation tool. In this example, it
is likely that hospitalized patients suffer a greater degree of disease severity and may therefore require higher antipsychotic doses. Another drawback is that the same AE may have different severity. For example, the EFESO researchers gave vomiting a severity score of 2.65, but in a specific patient the true severity of the vomiting may vary between a broad range of mild to very severe, and these differences could be related to the drugs administered. In this study these possible differences were not taken into consideration and each AE was given a single severity score. It would be advisable to perfect the method to adjust for the weight of each AE according to its true severity. Furthermore, the interpretation of the GIS is not exempt from difficulties. The GIS of a drug may be high because mild AEs are very common (for example, the high scores shown by some patients in this study may be explained by the simultaneous occurrence of several mild to moderate AEs), or because severe AEs appear less often. Although the clinical significance of both situations would be different, the GIS provides an intuitive idea about the differences in safety between different drugs, keeping in mind the frequency and the severity at the same time. A potential way to develop this assessment tool would be to allow researchers to weight the scores on the scale according to their severity (for example, a very severe AE could weigh 3 times more than a severe one, and 5 times more than a moderate one). Finally, this work presents the inherent limitations of observational studies. Although the OLZ and control groups were not significantly different at baseline in the variables analyzed [8], treatments were selected by the investigators according to purely clinical criteria, so there is a clear potential of bias in treatment assignment that could influence the differences in the incidence of AEs between the groups. There may also have been differences in how patients were assessed or treated throughout the study, given the unblinded nature of the study. It is necessary to take into account that EFESO was only the vehicle to develop the GIS methodology and that the instrument may be perfectly applied to other kind of designs such as randomized clinical trials, database analyses, etc. Several methods for drug decisionmaking, such as the SOJA [11] or the Analytic Hierarchy Process [12] have been developed in the recent years. However, the GIS is the first methodological proposal to assess the safety profile of drugs by using scores previously assigned by clinicians. Although the validation of the instrument needs further investigation, in this study GIS results have been consistent with the conclusions that could be drawn from the conventional analysis of AE: GIS scores were highest in the control group that had a higher incidence of severe neuroleptic adverse events. It would be interesting to apply this instrument to comparisons between groups that do not show such obvious differences as our analysis. It would be desirable to compare the scores assigned by clinicians with those assigned by patients to the same AE, or
J.A. Sacristán et al. / Journal of Clinical Epidemiology 54 (2001) 1120–1125
with the scores assigned by clinicians working in different settings. To that end, our group conducted an interview to obtain the AE scores assigned by researcher participants in a pharmacoepidemiological study of schizophrenic hospitalized patients. The analysis of the ranking of the scores assigned to the same AE by the two groups of psychiatrists (working in the ambulatory and hospital setting) may be a first step in validation of the instrument. We plan to conduct a test-retest of the instrument with a small group of clinicians to assess its reliability. The use of GISs such as the one proposed in this study may serve to achieve a better understanding of the beneficial drug profile and be a great help in making decisions regarding their selection. The GIS is a new instrument that must be perfected and should be considered as complementary tool to the current methods for evaluating drug safety. Further development and validation will be useful in evaluating the advantages and problems of the GIS. Acknowledgments We thank the psychiatrists that participated in the interview, and Raul Vara and Fernando Gonzalo of the Biometrics Department, Phoenix International Spain. This study was supported by Lilly and was conducted under the R&D Spanish Plan for the Pharmaceutical Industry (Farma II) from the Spanish Ministries of Health and Industry.
References [1] Brewer T, Colditz GA. Postmarketing surveillance and adverse drug reactions. Current perspectives and future needs. JAMA 1999;281: 824–9.
1125
[2] Beasley CM, Sanger TM, Satterlee W, Tollefson GD, Tran PV, Hamilton SL. Olanzapine versus placebo results of a double-blind, fixed-dose olanzapine trial. Psychopharmacol 1996;124:159–67. [3] Beasley CM, Tollefson G, Tran P, Satterle W, Sanger T, Hamilton S. Olanzapine versus placebo and haloperidol: Acute phase results of the North American double-blind olanzapine trial. Neuropsychopharmacol 1996;14:111–23. [4] Beasley CM, Hamilton SH, Crawford AM, Dellva MA, Tollefson GD, Tran PV, Blin O, Bevzen JN. Olanzapine versus haloperidol: acute phase results of the international double-blind olanzapine trial. Eur Neuropsychopharmacol 1997;7:125–37. [5] Tollefson GD, Beasley CM Jr, Tran PV, Street JS, Tamura RN, Krueger JA, Graffeo KA, Thieme ME. Olanzapine versus haloperidol in the treatment of schizophrenia, schizoaffective and schizophreniform disorders: results of an international collaborative trial. Am J Psychiatry 1997;154:157–65. [6] COSTART. Coding symbol and thesaurus for adverse event terminology. Rockville, MD: US Department of Health and Human Services, 1990. [7] Kind P, Dolan P, Martínez C. Adjusting outcomes for quality: scaling the severity of adverse drug reactions. 13th Annual Meeting, International Society Technology Assessment Health Care; Barcelona; May 1997. [8] Gómez JC, Sacristán JA, Hernandez J, Breier A, Ruiz P, Antón C, Fontova E. The safety of olanzapine compared to other antipsychotic drugs: results of an observational prospective study in patients with schizophrenia (EFESO study). J Clin Psychiatry 2000;61:335–43. [9] Lingjaerde O, Ahlors UG, Bech P, Dencker SJ, Elgen K. The UKU side effect rating scale. a new comprehensive rating scale for psychotropic drugs and a cross-sectional study of side effects in neuroleptictreated patients. Acta Psychiatr Scand Suppl 1987;334:1–100. [10] Tran PV, Hamilton SH, Kuntz AJ, Potvin JH, Andersen SW, Beasley CM, Tollefson GD. Double-blind comparison of olanzapine versus risperidone in the treatment of schizophrenia and other psychotic disorders. J Clin Psychopharmacol 1997;17:407–18. [11] Janknegt R, Steenhoek A. The system of objective judgement analysis (SOJA). A tool in rationing drug selection for formulary inclusion. Drugs 1997;53:550–62. [12] Dolan JG. Medical decision making using the analytic hierarchy process: choice of initial antimicrobial therapy for acute pyelonephritis. Med Decis Making 1989;9:51–6.
Global index of safety (GIS): A new instrument to assess drug safety José A. Sacristána,*, Juan C. Gómeza, Xavier Badíab, Paul Kindc a
Clinical Research Department, Lilly S.A., Madrid, Spain b Catalan Institute of Public Health, Barcelona, Spain c Center for Health Economics, University of York, York, UK Received 16 October 2000; received in revised form 30 April 2001; accepted 2 May 2001
Abstract There is an asymmetry between the extraordinary development of measures and tools aimed at studying the beneficial effects of the drugs and the more limited methods to assess their safety profile. The goal of our study was to develop a global measuring tool to assess drugs’ safety. We conducted a survey of Spanish psychiatrists in mental health centers and outpatient treatment units to assess the severity scores that they would assign to a list of the most common adverse events (AEs) that usually occur with antipsychotic treatment. The severity scores were then applied to the list of AEs that really occurred along a naturalistic pharmacoepidemiological study on the use of different antipsychotics in the treatment of schizophrenia. The Global Index of Safety (GIS) of the experimental group treated with olanzapine (OLZ) was compared with the GIS of the control group and with the GIS of specific antipsychotics for which the number of treated patients was greater than 100. A total of 194 psychiatrists rated the severity of each AE on a scale of 1 (insignificant) to 5 (extremely severe). The individual severity was applied to the 2949 schizophrenic patients included in a pharmacoepidemiological study. A GIS was calculated for every group of patients receiving the same treatments. The GIS of the control group was higher (4.3) than that calculated from the experimental group (2.5) (P 0.001). The GIS of the risperidone (3.6) and haloperidol (6.0) subgroups were higher than that calculated from the OLZ group (P 0.001). The development of a GIS may facilitate the comparison of the safety of several drugs and may constitute a very valuable aid for those involved in selecting drugs. © 2001 Elsevier Science Inc. All rights reserved. Keywords: Safety; Adverse events; Antipsychotics; Schizophrenia; Olanzapine; Risperidone; Haloperidol
1. Introduction The drug selection process is based on the comparison of the risk–benefit profile of the different options available. Initially, efficacy and risk effects are assessed using randomized clinical trials (RCTs) in which the new drugs are compared with placebo or standard treatments. The limited number of patients that can be included in RCTs, their experimental nature, and the relatively short follow-up period, make it desirable to complete the information obtained in RCTs with post-marketing data when drugs are used in actual practice. Generally, the methods of studying adverse events are less extensive than the methods used to assess the beneficial effects of a drug. Although there are frequent controversies on the pros and cons of using surrogate endpoints, or on the appropriate temporal horizon to measure beneficial effects, efficacy assessment presents two important advantages with
* Corresponding author. Clinical Research Department, Lilly S.A., Avda. de la Industria 30, 28108 Alcobendas, Madrid, Spain. Fax: 34-916635231. E-mail address: [email protected] (J.A. Sacristán)
respect to safety evaluation: the high number and the sophistication of the measuring instruments, and the possibility of directly comparing identical outcomes (for example, reduction of arterial blood pressure or decrease in mortality). Spontaneous reporting systems remain the most common way to detect rare adverse events (AEs) that are temporally associated with drug use. However, these systems are not a valid option when decisionmakers are interested in detecting the true incidence of anticipated and common harmful effects. To obtain the latter information, appropriate analytic studies such as case-control studies, cohort studies, or postmarketing clinical trials need to be conducted [1]. But a common problem of all analytic methods is that results are often limited to a long list of the frequencies of occurrence of all the AEs reported during the study. This line-item process may assist in achieving an approximate understanding of the safety profile of the drugs studied and may sometimes be sufficient for deciding on the option with the most favorable profile. This is true in cases in which a particular drug is associated with an AE that is much more serious than those of other drugs, or if the frequency of one or more AEs is clearly higher in one of the options. However, other times the choice is not that obvious and the comparison of the
0895-4356/01/$ – see front matter © 2001 Elsevier Science Inc. All rights reserved. PII: S0895-4356(01)00 3 8 4 - 5
J.A. Sacristán et al. / Journal of Clinical Epidemiology 54 (2001) 1120–1125
1121
item-by-item AE does not allow for any conclusions to be drawn. The development of a global safety measuring tool, which would represent a single weighted score for an overall AE profile, may facilitate the safety comparison of several drugs and constitute a valuable aid for those involved in selecting drugs. The objective of this study was to develop a GIS in parallel to a recent naturalistic prospective pharmacoepidemiological study with antipsychotics.
signed, the severity of each AE was calculated (mean, standard deviation [SD], median, and 95% confidence intervals [95% CI] for each AE). On closure of the database and before any statistical analysis, the list was completed by including AEs detected during the pharmacoepidemiological study that were not on the original list. For these AEs, severity scores were assigned using the value of the AE most similar to itself by means of a consensus meeting of the 5 investigators managing this component of the study.
2. Materials and methods
2.3. Application of the AE severity scores to the EFESO and calculation of the GIS for each treatment group
The GIS was developed in the following steps: (1) Selection of AE; (2) Drafting a questionnaire and physician rating of the severity of each AE; (3) Application of AE severity scores to the pharmacoepidemiological study (EFESO) and calculation of the GIS for each treatment group; and (4) Comparison of the GIS of the different groups. 2.1. Selection of the adverse events A list with the most common and severe AE that may occur in patients treated with antipsychotics was constructed. AE selection took place by reviewing the AE reported in the recent literature [2–5]. The initial list was completed by searching the Eli Lilly and Company safety database for the most common AE (frequency 1%) reported for OLZ and comparable drugs, including those that occurred after commercialization. Other known AEs of neuroleptics were also included in our criteria. The AEs with similar terminology were grouped according to the COSTART terminology [6]. 2.2. Drafting of the questionnaire and obtaining the severity scores for each AE During the initial meetings for the start-up of a pharmacoepidemiological study with antipsychotics, the purpose of developing a new safety evaluation tool and its subsequent application to the safety results of the study was explained to the participating psychiatrists. In the GIS questionnaire instructions, the psychiatrists were asked to assign, according to their experience, scores to each of the AEs on the list, rating the severity that each of the AEs would have in schizophrenic patients treated with antipsychotics. The scores ranged from 1–5 on the following assessment scale: (1) Insignificant, (2) Mild, (3) Moderate, (4) Severe, and (5) Extremely severe. This system for scoring the physicianperceived severity of the AE has been reported previously [7]. All questionnaires were administered by either JAS or JCG and all participants received the same instructions. The questionnaire included instructions for its completion. Given that a particular AE could have different degrees of severity in different patients, it was suggested that the score should reflect a “standard” severity—in other words, the severity they had most often observed in the schizophrenic patients they had treated. All psychiatrists participating at the start-up meetings for the study completed the questionnaire. Using the scores as-
EFESO was an observational phase-IV prospective pharmacoepidemiological study with an open comparison in parallel groups to assess the safety of OLZ in comparison with other antipsychotic drugs in the treatment of outpatients with schizophrenia. Data were collected by a total of 293 psychiatrists, mostly in mental health centers or outpatient treatment units. The main results of the study have recently been published elsewhere [8]. To assess the main goal, all AEs spontaneously reported by patients or identified in the course of the doctor–patient interview were recorded. An adverse event was defined as any undesirable experience occurring after administration of the product. An adverse event did not necessarily have a causal relationship with the treatment. Extrapyramidal symptoms (EPS) were collected by means of a short questionnaire, based on the extrapyramidal symptoms section of the UKU scale: dystonia, rigidity, hypokinesia, tremor, dyskinesia, and akathisia [9]. All AEs were coded using the COSTART glossary. Applying the AE previously obtained severity scores to the EFESO AE results, the GIS was calculated for each antipsychotic. The first step was to calculate the safety index for each patient by summing the severity scores for all of the respective AEs in each patient. For example, the safety index for a drug in a patient without any AEs would be 0, whereas it would be 7.5 in a patient with 2 AEs with severity ratings of 3.5 and 4. The second step was to calculate the GIS for each drug; this was determined by averaging the indexes calculated in all patients in the respective drug treatment groups. 2.4. Comparison of the GIS of the different groups The GIS of the experimental group treated with olanzapine (OLZ) was compared with that of the control group (all other antipsychotics). Moreover, the GIS for OLZ was compared with that of specific antipsychotics for which the number of treated patients was greater than 100. Two GIS values were determined. The first included all patients of the study, regardless of AE experience, and the second calculation only included data from patients who experienced at least 1 AE. 2.5. Statistical analysis The EFESO study was designed to confirm the differences in the incidence of extrapyramidal symptoms between
1122
J.A. Sacristán et al. / Journal of Clinical Epidemiology 54 (2001) 1120–1125
the OLZ group and the control group, with a 90% power and a two-tailed risk of 0.05, to which end a sample size of approximately 1000 patients was estimated for each group. In addition, the OLZ group contained twice the number of patients as the control group to detect, with an 80% power and an risk of .05, any adverse effects occurring with a frequency of 1:1000 in the OLZ group. In short, to meet our objectives the study was designed to gather information on approximately 2000 patients treated with OLZ and 1000 patients treated with other antipsychotic drugs [8]. All patients who received OLZ were included in the OLZ group, and patients who did not receive OLZ were included in the control group. Nevertheless, to test for differences between different drugs in the control group, it was decided a priori to conduct a secondary analysis comparing the incidence of AE in the OLZ group versus treatment-specific subgroups with n 100. The data were keyed into two databases simultaneously by different individuals and later compared to eliminate errors. The system used for the verification, validation, and analysis of the data was SAS® version 6.12 for Windows. All patients with available information were included in the analysis. The incidence of each AE in each group was calculated for the number of patients presenting the event at any time during the study for the total number of patients in the group. The incidence of AE was described by means of frequency and percentage and compared with the 2 test or Fisher’s Exact test. The GISs were described using means, medians, SDs, and ranges, and compared using single factor analysis of variance (ANOVA). A two-tailed significance level of 0.05 was applied for all tests. 3. Results Due to space restrictions, the list with the average severity assigned for each AE by the participating psychiatrists is not included in this article (interested readers may request it from the corresponding author). The highest perceived severity scores corresponded to malignant neuroleptic syndrome (mean SD, 4.81 0.76), agranulocytosis (4.64 0.88), and suicide attempt (4.45 0.88). The lowest corresponded to an increase in coughing (1.93 0.69), chills (1.91 0.74), and thirst (1.88 0.71). In the EFESO study, 2128 (72,2%) patients were treated with the experimental drug (OLZ) and 821 (28,8%) patients were in the control group, treated as follows: risperidone (RIS) 417 (14.1%), haloperidol (HAL) 112 (3.8%), sertindole 84 (2.8%) zuclopenthixol 74 (2.5%), fluphenazine 33 (1.1%), trifluoperazine 31 (1.1%), and others 70 (2.3%). Despite the observational nature of the study there were no statistically significant differences between the two groups in age, gender, time from onset, schizophrenia subtype, or baseline severity [8]. The GIS of the OLZ group was compared with the GIS of the control group. Additionally, the GIS of the OLZ group was compared with the GIS of pa-
Table 1 Global Index of Safety (GIS) of olanzapine (OLZ) vs. control group, and OLZ vs. risperidone (RIS) and haloperidol (HAL). All-patients analysis
Mean Median SD Range
OLZa N 1,896
Controlb N 748
OLZc N 1,896
RIS N 383
HAL N 104
2.5 0 3.5 0–24.7
4.3 3 4.6 0–21.3
2.5 0 3.5 0–24.7
3.6 2.7 4.4 0–19.4
6.0 5.8 4.9 0–21.3
p 0.001 versus control (ANOVA). This group includes all patients in the control, including the RIS and HAL subgroups. c p 0.001 versus RIS, and HAL (ANOVA). a
b
tients treated with RIS and HAL (antipsychotics for which the number of treated patients was greater than 100). A higher percentage of the patients in the control group (64%) presented at least one adverse event at some time in the study, a statistically significant difference (P 0.001) when compared with the OLZ group (48%). This percentage was also lower (P 0.001) in the OLZ group compared with the HAL (80%) and RIS (57%) subgroups. In the same way, a significantly lower (P 0.001) percentage of patients developed EPS in the OLZ group (37%) than in the RIS (50%) and HAL (76%) subgroups. These results are consistent with the profile shown in the registration clinical trials [2–5] and are included in the product’s package insert and in the only randomized, double-blind clinical trial that has compared OLZ and RIS [10]. Table 1 compares the GIS for OLZ with the antipsychotic control group as well as with RIS and HAL specifically, when all patients were included in the analysis. The GIS calculated from the control group (4.3) was 72% higher (worse score) than that calculated from the OLZ group (2.5) (P 0.001). The GIS of the RIS (3.6) and HAL (6.0) subgroups were 44% and 140% higher than that calculated from the OLZ group (P 0.001). Table 2 shows the results of this same comparison when only including those patients with at least one AE. In this case, the GIS of OLZ was also significantly lower than that of the antipsychotics used in the control group or with HAL and RIS specifically. In pa-
Table 2 Global Index of Safety (GIS) of olanzapine (OLZ) vs. control group, and OLZ vs. risperidone (RIS) and haloperidol (HAL). Patients with at least one adverse event
Mean Median SD Range
OLZa N 903
Controlb N 476
OLZc N 903
RIS N 218
HAL N 83
5.2 3.2 3.4 2–24.7
6.7 5.8 4.0 2–21.3
5.2 3.2 3.4 2–24.7
6.3 5.8 4.1 2–19.4
7.6 6.2 4.3 2.4–21.3
p 0.001 versus Control (ANOVA). This group includes all patients in the control group, including the RIS and HAL subgroups. c p 0.001 versus RIS, and HAL (ANOVA). a
b
J.A. Sacristán et al. / Journal of Clinical Epidemiology 54 (2001) 1120–1125 Table 3 Incidence of adverse events reported with a frequency higher than 1%
Akathisia Dyskinesia Dystonia Extrapyramidal syndrome Hypertonia Hypokinesia Somnolence Tremor Weight gain
OLZ N 2,128
Control group N 821
N
N
%
%
59 31 24
2.8 1.5 1.1
78 12 24
9.5 1.5 2.9
6 73 104 96 140 146
0.3 3.4 4.9 4.5 6.6 6.9
9 97 105 17 121 14
1.1 11.8 12.8 2.18 14.7 1.7
2 60.539, p 0.001 2 11.927, p 0.001 Fisher, 7.762, p 0.009 2 76.665, p 0.001 2 56.179, p 0.001 2 9.577, p 0.002 2 48.889, p 0.001 2 30.690, p 0.001
Adverse events have been classified and coded according to the COSTART dictionary
tients who experienced at least one AE, HAL (7.6) and RIS (6.3) had a higher a GIS than patients treated with OLZ (5.2). Table 3 lists the AE reported in more than 1% of the OLZ group or in the control group, with an indication of events in which the incidence was statistically significant between the groups. The control group was also associated with a greater incidence of sexual disorders in males, whereas OLZ was more frequently associated with somnolence and weight gain. There were no significant differences in the incidence of anticholinergic symptoms (dry mouth,
1123
constipation, diplopia, urinary retention, difficulties in concentration, and confusion). There were no cases of agranulocytosis in the study. A higher percentage of patients in the control group (57,9%) received some kind of concomitant treatment, in comparison with patients in the OLZ group (36.3%; P 0.001). Specifically, a lower percentage of the patients in the OLZ group received anticholinergic medication compared with the control group or the HAL and RIS subgroups (at 6 months, OLZ 10.2% versus control group 26.8%, P 0.001; versus RIS 19.9%, P 0.001; and versus HAL 44%, P 0.001). Table 4 shows the AEs, classified and coded according the COSTART dictionary, that were reported with a statistically significant difference frequency in the OLZ group compared with the RIS and HAL subgroups. 4. Discussion The calculation of a GIS provides different information than the conventional method of reporting the safety profile of drugs in clinical trials and epidemiological studies. It allows obtaining information on the relative severity assigned by the researchers to different AEs. For example, in this study, the researchers considered that weight gain and somnolence have a lower severity (2.6 and 2.42, respectively) than the EPS most often reported with the other groups: dystonia (3.21), hypokinesia (3.01), akathisia (3.05), tremor
Table 4 Incidence of those adverse events that were reported with a significantly different frequency in the olanzapine group compared with the risperidone and haloperidol subgroups, classified and coded according to the COSTART dictionary
Event
Olanzapine N 2,128 n (%)
Risperidone N 417 n (%)
Haloperidol N 112 n (%)
Akathisia
59 (2.8)
30 (7.2)
19 (17)
Dystonia
24 (1.1)
9 (2.2)
4 (3.6)
6 (0.3)
3 (0.7)
4 (3.6)
Hypertonia
73 (3.4)
35 (8.4)
29 (25.9)
Hypokinesia
104 (4.9)
36 (8.6)
30 (26.8)
Hypotension
4 (0.2)
1 (0.2)
2 (1.8)
Somnolence
96 (4.5)
7 (1.7)
5 (4.5)
Tremor
140 (6.6)
47 (11.3)
29 (25.9)
Weight gain
146 (6.9)
8 (1.9)
1 (0.9)
Males Abnormal Ejaculation
N 1,349 0
N 274 2 (0.7)
N 68 0
Extrapyramidal syndrome
Impotence Females Amenorrhea
0 N 769 5 (0.7)
2 (0.73) N 142 5 (3.5)
0 N 44 0
OLZ versus RIS 2/p
OLZ versus HAL 2/p
2 20.200 p 0.001
2 63.761 p 0.001 2 5.147 p 0.048 2 25.905 p 0.001 2 123.521 p 0.001 2 90.720 p 0.001 2 10.168 p 0.033
2 21.134 p 0 .001 2 9.412 p 0.002 2 7.204 p 0.007 2 11.275 p 0.001 2 14.982 p 0.001 2 9.859 p 0.028 2 9.859 p 0.028 2 9.100 p 0.011
2 56.900 p 0.001 2 6.180 p 0.013
1124
J.A. Sacristán et al. / Journal of Clinical Epidemiology 54 (2001) 1120–1125
(2.74), or hypertonia (3.16). Also, the use of a global index allows a safety comparison to be made between the different groups, using the usual statistical techniques (Table 1 and Table 2). For example, it would be advisable in systematic application to pharmacological groups such as NSAIDs in which the safety profile usually determines the optimal option. Finally, the possible use of a GIS to compare drugs from different therapeutic groups could prove to be attractive for decisionmakers. A classical analysis of the safety profile is in many cases sufficient to determine which of the options presents the most favorable profile. In these cases, the use of a GIS should be considered a complementary tool that may serve to increase available information with the advantages noted earlier. For instance, the comparisons made in Table 3 and Table 4 may suggest that, as a whole, OLZ presents a more favorable profile than the other treatment groups: statistically significant differences were detected in the incidence of EPS in favor of OLZ compared with the other treatment groups; sexual disorders in males were more frequent in the control group, whereas OLZ was more frequently associated with somnolence and weight gain. However, the relative importance of each AE is not always clear; in the absence of a global safety measure, the selection of the safest option may be a controversial decision. Obviously, the use of a GIS derived from scores previously assigned by physicians with experience in the treatment of a specific disease may be a valuable tool to help decisionmakers select the most favorable options. In EFESO, a large proportion of patients from both the OLZ and control groups received concomitant treatment with antipsychotics, benzodiazepines, anticholinergics, or other drugs. Therefore, it is difficult to attribute the safety results unequivocally to OLZ or to other specific drugs. AEs were assigned to groups reflecting the initially prescribed drug in the study. This type of analysis may be appropriate to reflect routine clinical practice where antipsychotics are frequently used in combination. The high degree of variability observed in the GIS values (the SD is greater than the mean) could lead to a recommendation to use the median values. In the case of the EFESO study, the differences in safety between the different drugs would be even more marked: the median of OLZ would be 0 (equivalent to an absence of AE), 3 (equivalent to a moderate AE) for control antipsychotics, and 2.7 and 5.8 for RIS and HAL, respectively. Despite the advantages mentioned above, the proposed method presents several limitations. First, the scores assigned a priori by a group of researchers (in this case psychiatrists who work in the outpatient setting), may not reflect scores that would be assigned by other researchers. For example, psychiatrists working in a hospital setting may assign different severity scores, based on the different conditions of the patients they usually treat. In such cases it would be useful for each collective with different characteristics to establish its own evaluation tool. In this example, it
is likely that hospitalized patients suffer a greater degree of disease severity and may therefore require higher antipsychotic doses. Another drawback is that the same AE may have different severity. For example, the EFESO researchers gave vomiting a severity score of 2.65, but in a specific patient the true severity of the vomiting may vary between a broad range of mild to very severe, and these differences could be related to the drugs administered. In this study these possible differences were not taken into consideration and each AE was given a single severity score. It would be advisable to perfect the method to adjust for the weight of each AE according to its true severity. Furthermore, the interpretation of the GIS is not exempt from difficulties. The GIS of a drug may be high because mild AEs are very common (for example, the high scores shown by some patients in this study may be explained by the simultaneous occurrence of several mild to moderate AEs), or because severe AEs appear less often. Although the clinical significance of both situations would be different, the GIS provides an intuitive idea about the differences in safety between different drugs, keeping in mind the frequency and the severity at the same time. A potential way to develop this assessment tool would be to allow researchers to weight the scores on the scale according to their severity (for example, a very severe AE could weigh 3 times more than a severe one, and 5 times more than a moderate one). Finally, this work presents the inherent limitations of observational studies. Although the OLZ and control groups were not significantly different at baseline in the variables analyzed [8], treatments were selected by the investigators according to purely clinical criteria, so there is a clear potential of bias in treatment assignment that could influence the differences in the incidence of AEs between the groups. There may also have been differences in how patients were assessed or treated throughout the study, given the unblinded nature of the study. It is necessary to take into account that EFESO was only the vehicle to develop the GIS methodology and that the instrument may be perfectly applied to other kind of designs such as randomized clinical trials, database analyses, etc. Several methods for drug decisionmaking, such as the SOJA [11] or the Analytic Hierarchy Process [12] have been developed in the recent years. However, the GIS is the first methodological proposal to assess the safety profile of drugs by using scores previously assigned by clinicians. Although the validation of the instrument needs further investigation, in this study GIS results have been consistent with the conclusions that could be drawn from the conventional analysis of AE: GIS scores were highest in the control group that had a higher incidence of severe neuroleptic adverse events. It would be interesting to apply this instrument to comparisons between groups that do not show such obvious differences as our analysis. It would be desirable to compare the scores assigned by clinicians with those assigned by patients to the same AE, or
J.A. Sacristán et al. / Journal of Clinical Epidemiology 54 (2001) 1120–1125
with the scores assigned by clinicians working in different settings. To that end, our group conducted an interview to obtain the AE scores assigned by researcher participants in a pharmacoepidemiological study of schizophrenic hospitalized patients. The analysis of the ranking of the scores assigned to the same AE by the two groups of psychiatrists (working in the ambulatory and hospital setting) may be a first step in validation of the instrument. We plan to conduct a test-retest of the instrument with a small group of clinicians to assess its reliability. The use of GISs such as the one proposed in this study may serve to achieve a better understanding of the beneficial drug profile and be a great help in making decisions regarding their selection. The GIS is a new instrument that must be perfected and should be considered as complementary tool to the current methods for evaluating drug safety. Further development and validation will be useful in evaluating the advantages and problems of the GIS. Acknowledgments We thank the psychiatrists that participated in the interview, and Raul Vara and Fernando Gonzalo of the Biometrics Department, Phoenix International Spain. This study was supported by Lilly and was conducted under the R&D Spanish Plan for the Pharmaceutical Industry (Farma II) from the Spanish Ministries of Health and Industry.
References [1] Brewer T, Colditz GA. Postmarketing surveillance and adverse drug reactions. Current perspectives and future needs. JAMA 1999;281: 824–9.
1125
[2] Beasley CM, Sanger TM, Satterlee W, Tollefson GD, Tran PV, Hamilton SL. Olanzapine versus placebo results of a double-blind, fixed-dose olanzapine trial. Psychopharmacol 1996;124:159–67. [3] Beasley CM, Tollefson G, Tran P, Satterle W, Sanger T, Hamilton S. Olanzapine versus placebo and haloperidol: Acute phase results of the North American double-blind olanzapine trial. Neuropsychopharmacol 1996;14:111–23. [4] Beasley CM, Hamilton SH, Crawford AM, Dellva MA, Tollefson GD, Tran PV, Blin O, Bevzen JN. Olanzapine versus haloperidol: acute phase results of the international double-blind olanzapine trial. Eur Neuropsychopharmacol 1997;7:125–37. [5] Tollefson GD, Beasley CM Jr, Tran PV, Street JS, Tamura RN, Krueger JA, Graffeo KA, Thieme ME. Olanzapine versus haloperidol in the treatment of schizophrenia, schizoaffective and schizophreniform disorders: results of an international collaborative trial. Am J Psychiatry 1997;154:157–65. [6] COSTART. Coding symbol and thesaurus for adverse event terminology. Rockville, MD: US Department of Health and Human Services, 1990. [7] Kind P, Dolan P, Martínez C. Adjusting outcomes for quality: scaling the severity of adverse drug reactions. 13th Annual Meeting, International Society Technology Assessment Health Care; Barcelona; May 1997. [8] Gómez JC, Sacristán JA, Hernandez J, Breier A, Ruiz P, Antón C, Fontova E. The safety of olanzapine compared to other antipsychotic drugs: results of an observational prospective study in patients with schizophrenia (EFESO study). J Clin Psychiatry 2000;61:335–43. [9] Lingjaerde O, Ahlors UG, Bech P, Dencker SJ, Elgen K. The UKU side effect rating scale. a new comprehensive rating scale for psychotropic drugs and a cross-sectional study of side effects in neuroleptictreated patients. Acta Psychiatr Scand Suppl 1987;334:1–100. [10] Tran PV, Hamilton SH, Kuntz AJ, Potvin JH, Andersen SW, Beasley CM, Tollefson GD. Double-blind comparison of olanzapine versus risperidone in the treatment of schizophrenia and other psychotic disorders. J Clin Psychopharmacol 1997;17:407–18. [11] Janknegt R, Steenhoek A. The system of objective judgement analysis (SOJA). A tool in rationing drug selection for formulary inclusion. Drugs 1997;53:550–62. [12] Dolan JG. Medical decision making using the analytic hierarchy process: choice of initial antimicrobial therapy for acute pyelonephritis. Med Decis Making 1989;9:51–6.