- Email: [email protected]
Penetrating the blind in a study of an SSRI
Journal of Behavior Therapy and Experimental Psychiatry 33 (2002) 67–71
Penetrating the blind in a study of an SSRI Melissa P. Piaseckia,*, David O. Antonuccioa, Gerri M. Steinagela, Barbara S. Kohlenberga, Karen Kapadarb a
University of Nevada School of Medicine, Department of Psychiatry/354, Reno, NV 89557 0046, USA b Department of Mathematics, University of Colorado-Denver, Denver, CO 80217 3364, USA Received 8 January 2001; received in revised form 5 April 2002; accepted 8 April 2002
Abstract We assessed blind integrity in a double-blinded study comparing paroxetine 20 mg with inert placebo in 20 volunteer subjects who were attempting to stop using methamphetamines. At the end of the study, the blinded clinicians reviewed subject charts and attempted to identify the assigned conditions for the 13 subjects who completed two or more weeks of the study. The three subjects who completed the entire study also attempted to identify their conditions on a questionnaire. We conclude that the blind may unwittingly be broken when the treatments under study are placebo and the selective serotonin reuptake inhibitor (SSRI) paroxetine. The integrity of the blind should be tested in all double-blind SSRI studies. Published by Elsevier Science Ltd.
1. Introduction The double-blind study is the gold standard in controlled medication studies. When subjects have been properly randomized to treatment arms, and both investigators and subjects are blind to the treatment conditions, expectancy effects and potential biases are minimized and treatment effects can be attributed to the agents themselves. Placebo controlled psychotropic medication trials routinely attempt to keep both subjects and clinicians blind to the randomly assigned treatment conditions. However, the effectiveness of this blinding procedure increasingly has been questioned in medication trials, especially when investigators use inert placebos that do not have side effects comparable to the study medication (Antonuccio, Danton, DeNelsky, Greenberg, & Gordon, 1999). Penetration of the *Tel.: +1-775-784-4917; fax: +1-775-784-1428. E-mail address: [email protected] (M.P. Piasecki). 0005-7916/02/$ - see front matter Published by Elsevier Science Ltd. PII: S 0 0 0 5 - 7 9 1 6 ( 0 2 ) 0 0 0 1 3 - 7
68
M.P. Piasecki et al. / J. Behav. Ther. & Exp. Psychiat. 33 (2002) 67–71
blind, which may introduce bias into study findings, is not routinely tested. When tested, it is often found that the clinicians or subjects are able to discriminate placebo from active agent. In a review of the literature, Fisher and Greenberg (1993) found 23 placebo controlled psychotropic drug studies where the double blind was penetrated and three studies where it was assessed but not penetrated. Assessment of the blind usually is accomplished by asking clinicians and subjects to indicate whether or not, in their opinion, the subjects received placebo or active drug. The breach of the double blind is often attributable to the side effects or therapeutic effects of a medication. Some older antidepressant medications, such as TCAs and monoamine oxidase inhibitors, have obvious side effect profiles that make it relatively easy to detect the active condition. For example, in a study of the double blind in a placebo controlled trial of a tricyclic antidepressant (TCA) and an monamine oxidaxe inhibitor (MAOI), patients correctly identified their conditions 79% of the time and the doctors correctly identified the assigned conditions 87% of the time (Rabkin et al., 1986). Similarly, when a low dose of a tricyclic (10 mg of imipramine) was used as an active placebo in comparison to a higher dose of the same drug (150 mg), 79% of the subjects in the low dose condition and 50% in the high dose condition correctly guessed their dose (Galloway, Newmeyer, Knapp, Stalcup, & Smith, 1996). The selective serotonin reuptake inhibitor (SSRI’s) purportedly have fewer and less troublesome side effects. Accordingly, the detection of an SSRI versus an inert placebo may be more difficult. A Medline review did not reveal any published reports specifically relating to unblinding with SSRI’s. In this report, we describe the results of querying both patients and clinicians about the assigned treatment condition in a small study of paroxetine for methamphetamine craving. The complete details of this trial will be reported elsewhere (Piasecki, Steinagel, Thienhaus, & Kohlenberg, in press).
2. Method Twenty methamphetamine users were recruited from the community for volunteer participation in a study using paroxetine versus inert placebo for the treatment of methamphetamine craving. After having received a complete description of the study, each subject signed an informed consent form. Both active medication and placebo were encapsulated in identical opaque gel caps, with no difference in size, shape, color or odor. Before the study began, the investigators attempted to distinguish between active and placebos pills and were unable to discern any differences. Subjects received a coded bottle of seven pills at each visit. At the end of the study, before revealing the subjects’ conditions, the two clinicians (MP and GS) attempted to identify the condition for each subject by reviewing the progress notes in the files of the 13 subjects who had attended at least one session after the intake appointment. The clinician who worked with each subject attempted to identify whether the subject received active treatment or placebo. The clinician’s responses were based on the type and severity of side effects and other effects reported by the subject.
M.P. Piasecki et al. / J. Behav. Ther. & Exp. Psychiat. 33 (2002) 67–71
69
The three subjects who completed the study were asked on a form at the end of the 8 week treatment protocol to guess whether they were on active drug or placebo and the basis for their responses. Unfortunately, we did not obtain guesses from all subjects during the course of the treatment and were limited to the three completing subjects.
3. Results Seven subjects dropped out of the study after the initial intake and three completed the 8 week trial. Three placebo subjects reported side effects including sedation, vomiting, headache, nausea and chemical taste. Two placebo subjects reported no side effects. Seven of eight active drug subjects reported side effects including dry mouth, ‘‘chattery’’ jaw, nausea, fatigue, insomnia, weight gain, decreased energy, blurred vision and sexual dysfunction. According to the Fisher’s exact test of the difference in the proportions of patients experiencing side effects on drug versus placebo, the p value is 0.3147. For the 13 subjects who returned for at least one follow up visit, the clinicians correctly guessed the condition for 12 of 13 (92%). Based on the binomial distribution, this result indicates that the probability of correctly identifying the patients’ condition merely by random guessing is significantly different from 1/2 (po0:01). All three completer subjects indicated that they thought they were in the active medication condition, although one of these subjects was actually assigned to placebo. Reasons reported by subjects for guessing active medication included side effects (e.g., ‘‘dry mouth, jaw chattering, upset stomach’’) and efficacy (e.g., ‘‘less craving’’, ‘‘curbed my appetite for crank’’). All three subjects rated the confidence of their guesses as ‘‘very confident’’. One subject, who was assigned to active treatment, had objective evidence of efficacy in a negative urine test for methamphetamine.
4. Discussion In this study, the clinicians were able to correctly identify the assigned condition of most subjects based on reported side effects. Our results must be considered exploratory because so few subjects were able to complete treatment, potentially skewing the sample in unknown ways. There is also no way to know whether the findings of this study will generalize to patients who do not use amphetamines, as is typical in other SSRI studies. It is clear that efficacy did not contribute to clinicians’ ability to correctly identify the condition because only one subject in the entire study was able to abstain from methamphetamine use by the end of the protocol (assuming those who dropped out before eight weeks returned to methamphetamine use). Since side effects were reported by patients in both conditions, the specific pattern of paroxetine side effects may have signaled the actual condition. The high rate at which clinicians correctly identified the assigned conditions in this trial of an SSRI raises questions about the validity of studies that use inert placebos in comparison to SSRI’s (Antonuccio et al., 1999). For a drug like paroxetine, with
70
M.P. Piasecki et al. / J. Behav. Ther. & Exp. Psychiat. 33 (2002) 67–71
side effects such as dry mouth and sedation, an active placebo such as diphenhydramine may be more effective in preserving the blind. For other SSRI’s, low dose psuedoephredrine or caffeine may more effectively blind subjects and clinical researchers to the stimulating effects of ‘‘activating’’ drugs such as fluoxetine or sertraline. Hughes and Krahn (1985) describe a method for measuring the blindness of a study and whether penetration of the blind affects the study’s validity. Their proposed method involves asking subjects not only to identify their conditions, but also for their rationale for and confidence in their beliefs. If all studies, including those with active placebo conditions, were to assess blindness during and after the study, the effect of accurately identifying the treatment condition could be compared to the actual drug effect, allowing the influence on the study’s validity to be evaluated and reported (Hughes & Krahn, 1985). Our study with paroxetine suggests that such an assessment could be critical in assessing the validity in double blind randomized studies with SSRI medications. Because of low demonstrated efficacy and small sample size in this study, the clinicians’ ability to correctly identify the subjects’ conditions could not possibly impact outcome. However, penetration of the blind and its potential impact on outcome raises questions about the internal validity of any study that demonstrates significant differences between active treatment and placebo conditions. Despite arguments to the contrary (Quitkin, Rabkin, Gerald, Davis, & Klein, 2000), the procedure of asking clinicians and subjects to identify the treatment condition, can serve as a model for assessing blindness in larger scale studies with positive results. This strategy is simple to perform, cost-effective, and may enhance our ability to evaluate the internal validity of any outcome study. Double-blind studies are based on the assumption that an active and control condition are alike in every way except for an active chemical agent. Either the clinicians in the present study were particularly lucky or good at identifying the actual treatment conditions or this assumption is not valid. Therefore, we propose that any psychopharmacologic study claiming double-blind status be required to evaluate and report on the integrity of the double blind in order to satisfy peer review standards.
Acknowledgements The authors acknowledge the support of SmithKline Beecham pharmaceuticals (Tracy Hemmert) in providing paroxetine samples for this study. We also gratefully acknowledge the assistance of Jody Eble, Ruth Atwell, Elizabeth Gifford, Ken Bender, and Joseph Phelan.
References Antonuccio, D. O., Danton, W. G., DeNelsky, G. Y., Greenberg, R. P., & Gordon, J. S. (1999). Raising questions about antidepressants. Psychotherapy and Psychosomatics, 68, 3–14.
M.P. Piasecki et al. / J. Behav. Ther. & Exp. Psychiat. 33 (2002) 67–71
71
Fisher, S., & Greenberg, R. P. (1993). How sound is the double blind design for evaluating psychotropic drugs? Journal of Nervous and Mental Disease, 181, 345–350. Galloway, G. P., Newmeyer, J., Knapp, T., Stalcup, S. A., & Smith, D. (1996). A controlled trial of imipramine for the treatment of methamphetamine dependence. Journal of Subst Abuse Treat, 13, 493–497. Hughes, J. R., & Krahn, D. (1985). Blindness and the validity of the double-blind procedure. Journal of Clinical Psychopharmacology, 5, 138–142. Piasecki, M. P., Steinagel, G. M., Thienhaus, O. J., Kohlenberg, B. S. (in press). An Exploratory study: The use of paroxetine for methamphetamine craving. Journal of Psychoactive Drugs. Quitkin, F. M., Rabkin, J. G., Gerald, J., Davis, J. M., & Klein, D. F. (2000). Validity of clinical trials of antidepressants. American Journal of Psychiatry, 157, 327–337. Rabkin, J. G., Markowitz, J. S., Stewart, J., McGrath, P., Harrison, W., Quitkin, F. M., & Klein, D. F. (1986). How blind is blind? Assessment of patient and doctor medication guesses in a placebo controlled trial of imipramine and phenelzine. Psychiatry Research, 19, 75–86.
Penetrating the blind in a study of an SSRI Melissa P. Piaseckia,*, David O. Antonuccioa, Gerri M. Steinagela, Barbara S. Kohlenberga, Karen Kapadarb a
University of Nevada School of Medicine, Department of Psychiatry/354, Reno, NV 89557 0046, USA b Department of Mathematics, University of Colorado-Denver, Denver, CO 80217 3364, USA Received 8 January 2001; received in revised form 5 April 2002; accepted 8 April 2002
Abstract We assessed blind integrity in a double-blinded study comparing paroxetine 20 mg with inert placebo in 20 volunteer subjects who were attempting to stop using methamphetamines. At the end of the study, the blinded clinicians reviewed subject charts and attempted to identify the assigned conditions for the 13 subjects who completed two or more weeks of the study. The three subjects who completed the entire study also attempted to identify their conditions on a questionnaire. We conclude that the blind may unwittingly be broken when the treatments under study are placebo and the selective serotonin reuptake inhibitor (SSRI) paroxetine. The integrity of the blind should be tested in all double-blind SSRI studies. Published by Elsevier Science Ltd.
1. Introduction The double-blind study is the gold standard in controlled medication studies. When subjects have been properly randomized to treatment arms, and both investigators and subjects are blind to the treatment conditions, expectancy effects and potential biases are minimized and treatment effects can be attributed to the agents themselves. Placebo controlled psychotropic medication trials routinely attempt to keep both subjects and clinicians blind to the randomly assigned treatment conditions. However, the effectiveness of this blinding procedure increasingly has been questioned in medication trials, especially when investigators use inert placebos that do not have side effects comparable to the study medication (Antonuccio, Danton, DeNelsky, Greenberg, & Gordon, 1999). Penetration of the *Tel.: +1-775-784-4917; fax: +1-775-784-1428. E-mail address: [email protected] (M.P. Piasecki). 0005-7916/02/$ - see front matter Published by Elsevier Science Ltd. PII: S 0 0 0 5 - 7 9 1 6 ( 0 2 ) 0 0 0 1 3 - 7
68
M.P. Piasecki et al. / J. Behav. Ther. & Exp. Psychiat. 33 (2002) 67–71
blind, which may introduce bias into study findings, is not routinely tested. When tested, it is often found that the clinicians or subjects are able to discriminate placebo from active agent. In a review of the literature, Fisher and Greenberg (1993) found 23 placebo controlled psychotropic drug studies where the double blind was penetrated and three studies where it was assessed but not penetrated. Assessment of the blind usually is accomplished by asking clinicians and subjects to indicate whether or not, in their opinion, the subjects received placebo or active drug. The breach of the double blind is often attributable to the side effects or therapeutic effects of a medication. Some older antidepressant medications, such as TCAs and monoamine oxidase inhibitors, have obvious side effect profiles that make it relatively easy to detect the active condition. For example, in a study of the double blind in a placebo controlled trial of a tricyclic antidepressant (TCA) and an monamine oxidaxe inhibitor (MAOI), patients correctly identified their conditions 79% of the time and the doctors correctly identified the assigned conditions 87% of the time (Rabkin et al., 1986). Similarly, when a low dose of a tricyclic (10 mg of imipramine) was used as an active placebo in comparison to a higher dose of the same drug (150 mg), 79% of the subjects in the low dose condition and 50% in the high dose condition correctly guessed their dose (Galloway, Newmeyer, Knapp, Stalcup, & Smith, 1996). The selective serotonin reuptake inhibitor (SSRI’s) purportedly have fewer and less troublesome side effects. Accordingly, the detection of an SSRI versus an inert placebo may be more difficult. A Medline review did not reveal any published reports specifically relating to unblinding with SSRI’s. In this report, we describe the results of querying both patients and clinicians about the assigned treatment condition in a small study of paroxetine for methamphetamine craving. The complete details of this trial will be reported elsewhere (Piasecki, Steinagel, Thienhaus, & Kohlenberg, in press).
2. Method Twenty methamphetamine users were recruited from the community for volunteer participation in a study using paroxetine versus inert placebo for the treatment of methamphetamine craving. After having received a complete description of the study, each subject signed an informed consent form. Both active medication and placebo were encapsulated in identical opaque gel caps, with no difference in size, shape, color or odor. Before the study began, the investigators attempted to distinguish between active and placebos pills and were unable to discern any differences. Subjects received a coded bottle of seven pills at each visit. At the end of the study, before revealing the subjects’ conditions, the two clinicians (MP and GS) attempted to identify the condition for each subject by reviewing the progress notes in the files of the 13 subjects who had attended at least one session after the intake appointment. The clinician who worked with each subject attempted to identify whether the subject received active treatment or placebo. The clinician’s responses were based on the type and severity of side effects and other effects reported by the subject.
M.P. Piasecki et al. / J. Behav. Ther. & Exp. Psychiat. 33 (2002) 67–71
69
The three subjects who completed the study were asked on a form at the end of the 8 week treatment protocol to guess whether they were on active drug or placebo and the basis for their responses. Unfortunately, we did not obtain guesses from all subjects during the course of the treatment and were limited to the three completing subjects.
3. Results Seven subjects dropped out of the study after the initial intake and three completed the 8 week trial. Three placebo subjects reported side effects including sedation, vomiting, headache, nausea and chemical taste. Two placebo subjects reported no side effects. Seven of eight active drug subjects reported side effects including dry mouth, ‘‘chattery’’ jaw, nausea, fatigue, insomnia, weight gain, decreased energy, blurred vision and sexual dysfunction. According to the Fisher’s exact test of the difference in the proportions of patients experiencing side effects on drug versus placebo, the p value is 0.3147. For the 13 subjects who returned for at least one follow up visit, the clinicians correctly guessed the condition for 12 of 13 (92%). Based on the binomial distribution, this result indicates that the probability of correctly identifying the patients’ condition merely by random guessing is significantly different from 1/2 (po0:01). All three completer subjects indicated that they thought they were in the active medication condition, although one of these subjects was actually assigned to placebo. Reasons reported by subjects for guessing active medication included side effects (e.g., ‘‘dry mouth, jaw chattering, upset stomach’’) and efficacy (e.g., ‘‘less craving’’, ‘‘curbed my appetite for crank’’). All three subjects rated the confidence of their guesses as ‘‘very confident’’. One subject, who was assigned to active treatment, had objective evidence of efficacy in a negative urine test for methamphetamine.
4. Discussion In this study, the clinicians were able to correctly identify the assigned condition of most subjects based on reported side effects. Our results must be considered exploratory because so few subjects were able to complete treatment, potentially skewing the sample in unknown ways. There is also no way to know whether the findings of this study will generalize to patients who do not use amphetamines, as is typical in other SSRI studies. It is clear that efficacy did not contribute to clinicians’ ability to correctly identify the condition because only one subject in the entire study was able to abstain from methamphetamine use by the end of the protocol (assuming those who dropped out before eight weeks returned to methamphetamine use). Since side effects were reported by patients in both conditions, the specific pattern of paroxetine side effects may have signaled the actual condition. The high rate at which clinicians correctly identified the assigned conditions in this trial of an SSRI raises questions about the validity of studies that use inert placebos in comparison to SSRI’s (Antonuccio et al., 1999). For a drug like paroxetine, with
70
M.P. Piasecki et al. / J. Behav. Ther. & Exp. Psychiat. 33 (2002) 67–71
side effects such as dry mouth and sedation, an active placebo such as diphenhydramine may be more effective in preserving the blind. For other SSRI’s, low dose psuedoephredrine or caffeine may more effectively blind subjects and clinical researchers to the stimulating effects of ‘‘activating’’ drugs such as fluoxetine or sertraline. Hughes and Krahn (1985) describe a method for measuring the blindness of a study and whether penetration of the blind affects the study’s validity. Their proposed method involves asking subjects not only to identify their conditions, but also for their rationale for and confidence in their beliefs. If all studies, including those with active placebo conditions, were to assess blindness during and after the study, the effect of accurately identifying the treatment condition could be compared to the actual drug effect, allowing the influence on the study’s validity to be evaluated and reported (Hughes & Krahn, 1985). Our study with paroxetine suggests that such an assessment could be critical in assessing the validity in double blind randomized studies with SSRI medications. Because of low demonstrated efficacy and small sample size in this study, the clinicians’ ability to correctly identify the subjects’ conditions could not possibly impact outcome. However, penetration of the blind and its potential impact on outcome raises questions about the internal validity of any study that demonstrates significant differences between active treatment and placebo conditions. Despite arguments to the contrary (Quitkin, Rabkin, Gerald, Davis, & Klein, 2000), the procedure of asking clinicians and subjects to identify the treatment condition, can serve as a model for assessing blindness in larger scale studies with positive results. This strategy is simple to perform, cost-effective, and may enhance our ability to evaluate the internal validity of any outcome study. Double-blind studies are based on the assumption that an active and control condition are alike in every way except for an active chemical agent. Either the clinicians in the present study were particularly lucky or good at identifying the actual treatment conditions or this assumption is not valid. Therefore, we propose that any psychopharmacologic study claiming double-blind status be required to evaluate and report on the integrity of the double blind in order to satisfy peer review standards.
Acknowledgements The authors acknowledge the support of SmithKline Beecham pharmaceuticals (Tracy Hemmert) in providing paroxetine samples for this study. We also gratefully acknowledge the assistance of Jody Eble, Ruth Atwell, Elizabeth Gifford, Ken Bender, and Joseph Phelan.
References Antonuccio, D. O., Danton, W. G., DeNelsky, G. Y., Greenberg, R. P., & Gordon, J. S. (1999). Raising questions about antidepressants. Psychotherapy and Psychosomatics, 68, 3–14.
M.P. Piasecki et al. / J. Behav. Ther. & Exp. Psychiat. 33 (2002) 67–71
71
Fisher, S., & Greenberg, R. P. (1993). How sound is the double blind design for evaluating psychotropic drugs? Journal of Nervous and Mental Disease, 181, 345–350. Galloway, G. P., Newmeyer, J., Knapp, T., Stalcup, S. A., & Smith, D. (1996). A controlled trial of imipramine for the treatment of methamphetamine dependence. Journal of Subst Abuse Treat, 13, 493–497. Hughes, J. R., & Krahn, D. (1985). Blindness and the validity of the double-blind procedure. Journal of Clinical Psychopharmacology, 5, 138–142. Piasecki, M. P., Steinagel, G. M., Thienhaus, O. J., Kohlenberg, B. S. (in press). An Exploratory study: The use of paroxetine for methamphetamine craving. Journal of Psychoactive Drugs. Quitkin, F. M., Rabkin, J. G., Gerald, J., Davis, J. M., & Klein, D. F. (2000). Validity of clinical trials of antidepressants. American Journal of Psychiatry, 157, 327–337. Rabkin, J. G., Markowitz, J. S., Stewart, J., McGrath, P., Harrison, W., Quitkin, F. M., & Klein, D. F. (1986). How blind is blind? Assessment of patient and doctor medication guesses in a placebo controlled trial of imipramine and phenelzine. Psychiatry Research, 19, 75–86.