- Email: [email protected]
Methods to reduce the incidence of false negative trial results in substance use treatment research
Accepted Manuscript Title: Methods to Reduce the Incidence of False Negative Trial Results in Substance Use Treatment Research Authors: Rachel L Tomko, Erin A McClure, Lindsay M Squeglia, Hayley Treloar Padovano, Aimee L McRae-Clark, Nathaniel L Baker, Matthew J Carpenter, Kevin M Gray PII: DOI: Reference:
S2352-250X(18)30266-5 https://doi.org/10.1016/j.copsyc.2019.01.009 COPSYC 778
To appear in:
Please cite this article as: Tomko RL, McClure EA, Squeglia LM, Treloar Padovano H, McRae-Clark AL, Baker NL, Carpenter MJ, Gray KM, Methods to Reduce the Incidence of False Negative Trial Results in Substance Use Treatment Research, Current Opinion in Psychology (2019), https://doi.org/10.1016/j.copsyc.2019.01.009 This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Methods to Reduce the Incidence of False Negative Trial Results in Substance Use Treatment Research Running Head: Reducing False Negatives in SUD Treatment Research
SC RI PT
Rachel L. Tomko1,* [email protected], Erin A. McClure1, Lindsay M. Squeglia1, Hayley Treloar Padovano2, Aimee L. McRae-Clark1, Nathaniel L. Baker3, Matthew J. Carpenter1, Kevin M. Gray1 1
Department of Psychiatry and Behavioral Sciences, Medical University of South Carolina1, Department of Psychiatry and Human Behavior, Brown University2 3 Department of Public Health Sciences, Medical University of South Carolina3 2
*
U
Correspondence concerning this article may be addressed to Rachel L. Tomko,
N
Medical University of South Carolina, Department of Psychiatry and Behavioral Sciences,
A
MSC 861, 67 President Street, Charleston, SC 29425-8610
Rachel L. Tomko, Erin A. McClure, Lindsay M. Squeglia, Hayley Treloar
M
Padovano, Aimee L. McRae-Clark, and Nathaniel L. Baker declare that they have no conflicts of interest. Matthew J. Carpenter and Kevin M. Gray have provided consultation to
D
Pfizer, Inc.
TE
This manuscript was supported by National Institutes of Health grants from the National Institute of Drug Abuse (R01DA042114, R01DA038700, U01DA031779,
EP
UG3DA043231, K01DA036739, K24DA038240), Eunice Kennedy Shriver National Institute of Child Health and Human Development (K12HD055885), and National Institute
CC
on Alcohol Abuse and Alcoholism (K23AA025399, K23AA024808).
A
Word Count: 2074 Tables and Figures: 2
ABSTRACT Treatment development and evaluation for substance use disorders is hindered when randomized controlled trials fail to show a treatment effect when one exists. This manuscript
2
provides an overview of addressable methodological factors that may contribute to incorrect trial results. The collection of remote, naturalistic, real-time adherence and substance use data through ambulatory assessment methods in everyday life is presented as a partial
SC RI PT
solution. Other recommendations related to participant recruitment and selection, ensuring adequate consistency/fidelity and dose of treatment, and rigorously assessing clinical
outcomes are discussed. With implementation of eligibility criteria verification, treatment
adherence monitoring, and remote assessment of substance use and biomarkers, ambulatory assessment may help improve clinical trial success rates by improving precision, increasing
U
reproducibility, and reducing the impact of methodological issues that may lead to
N
inaccurate trial results.
A
Keywords: Randomized Controlled Trials, Pharmacotherapy, Psychotherapy, Medication
D
M
Adherence, Remote Technology, Ecological Momentary Assessment, Type II Error
TE
Despite effective evidence-based behavioral and pharmacological treatments for
EP
substance use disorders (SUDs),1,2 novel treatment development has not kept pace with advances in mechanistic understanding of SUDs. 2 Treatment development and
CC
dissemination remains slow1,2 and randomized controlled trials (RCTs) with spurious results can delay treatment development further. When an ineffective treatment is interpreted to be
A
effective (Type I error), it may be administered to an increasing number of patients, potentially exposing them to adverse effects without any benefit. It is also problematic when an effective treatment is incorrectly interpreted to be ineffective (Type II error), which may lead to the abandonment of future trials utilizing similar treatment strategies. Error rates in
3
SUD RCTs are difficult to estimate, but existing evidence suggests they are common. For example, 44% of successful Phase II psychiatric pharmacological interventions (smaller trials examining initial efficacy and safety in the population of interest) failed in larger
SC RI PT
Phase III trials (larger trials designed to examine efficacy).3 This is lower than the 55% Phase III trial success rate across all medical fields.3 Additionally, two meta-analyses
suggested that 12-45% of the variation in odds ratios in alcohol and tobacco use disorder
pharmacological trials were due to study characteristics, such as retention rate, biochemical verification of outcomes, funding source, and publication date.4,5 This suggests that a
U
number of RCTs result in null outcomes through no consequence of the intervention itself,
A
produce Type II errors in treatment evaluation.
N
leading to the notion that non-specific treatment factors, i.e., methodological factors, can
M
Here, we review methodological barriers that negatively impact RCTs. Randomization and statistical procedure selection considerations have been reviewed
D
elsewhere.6 We instead focus on participant recruitment and selection, quality and
TE
consistency of treatment, and measurement of clinical outcomes in efficacy trials
EP
(summarized in Table 1). Opportunities to decrease these errors through ambulatory assessment [“the acquisition of psychological data and/or physiological measures in
CC
everyday life (i.e., natural settings)”24] are emphasized.
A
1) Participant Recruitment and Selection Appropriately identifying inclusion criteria can improve the likelihood of detecting a
treatment effect. SUD diagnosis is a common inclusion criterion. In the future, diagnostic criteria may be refined or inclusion criteria may shift from diagnosis to phenotypes/endophenotypes that better reflect underlying neurobiological mechanisms. The
4
Phase III RCT success rate when patients are selected based on genetic biomarkers is 76% across medical fields, exceeding the average success rate of 55% without biomarker-based selection.3 This suggests that current inclusion criteria practices could be refined to enhance
SC RI PT
RCT outcomes, but more SUD research is necessary before such a shift could occur. Reducing exclusion criteria to the minimum possible for safety is recommended to maintain generalizability.9,10 Overly restrictive medical and psychiatric exclusion criteria
may also inadvertently attenuate treatment effects.9 It is thus important to carefully consider
eligibility criteria and to objectively verify that criteria are met. One survey found that nearly
U
40% of participants who had enrolled in more than one research study within the past year
N
acknowledged fabricating or exaggerating symptoms to meet criteria.25 Participants who
A
falsely report SUD symptoms are problematic because they are often “destined to succeed”
M
in the trial.7 The increase in placebo response in clinical trials may be due to a rise in these individuals who appear as treatment responders regardless of treatment assignment.7,26
D
Selection of individuals who are “destined to succeed” has a substantial negative impact on
TE
power in RCTs. McCann and colleagues7 provide a hypothetical example in which a sample
EP
size calculation suggested 282 patients were required to be adequately powered. If even 10% of the sample is “destined to succeed”, more than double the sample size would be
CC
necessary for adequate power and the estimated effect size would still be attenuated. Trial eligibility is often determined after a single in-person screening visit, which
A
often cannot offer a comprehensive clinical picture. For example, a non-smoker may attempt to enter a trial by smoking a few cigarettes prior to screening. They may then pass a breath carbon monoxide (CO) cut-off indicative of recent combustible tobacco use. However, it would be less likely that a non-smoker would meet eligibility criteria if breath CO is
5
repeatedly monitored in daily life with remote technology.27-29 Additionally, measurement error is reduced when the average of several longitudinal assessments is used as the baseline,30 which ambulatory assessment could make more feasible. Other strategies to
SC RI PT
prevent enrollment of deceptive participants may be considered, including recruiting directly from clinical settings, limiting disclosure of enrollment criteria, reducing financial
incentives, referencing medical records to verify criteria, and utilizing participant registries (Table 1).7,8,11
2. Dose, Consistency, and Fidelity of Treatment Administration
U
Efficacious treatments will appear ineffective in RCTs if patients receive an
N
insufficient amount of the treatment. This may occur if 1) treatment is not administered to
A
fidelity, 2) pharmacotherapy varies in preparation or potency, 3) participants do not adhere
M
to treatment recommendations (i.e., complete homework assignments, take medication), or 4) participants discontinue treatment early. Research on how behavioral treatment fidelity
D
affects outcomes is mixed, but appears to have nuanced effects on outcomes.13-15
TE
Researchers should ensure that treatment is being delivered to fidelity via regular
EP
monitoring to minimize Type II error. Likewise, pharmacotherapy trials should include verification of consistency, quality, and potency of medications.16 Indeed, the National
CC
Institutes of Health now requires new grant applications to acknowledge how the integrity
A
of chemical and biological resources will be determined. Even when researchers maximize the likelihood of high treatment integrity, patient
non-adherence and drop-out is pervasive in SUD RCTs. In pharmacotherapy trials, medication adherence is significantly over-estimated when patient self-report or pill counts are used, typically reported at >80%.7, 31,32 Trials that include biomarkers of adherence find
6
up to 35-55% of participants may be non-adherent despite high rates of self-reported adherence.7,32,33 Non-adherence with standard cognitive-behavioral treatment assignments is similar, with studies estimating 35-60% assignment completion rates among individuals
SC RI PT
with heroin or cocaine use disorders.34-36 Non-adherent participants significantly reduce statistical power in RCTs resulting in inadequacy to detect meaningful differences at the a priori defined sample size.7,8
Numerous solutions to objectively track medication adherence are being utilized.7,37 Testing for the presence of medication byproducts or added adherence markers (e.g.,
U
riboflavin38, acetazolamide39) in the urine is common and encouraged. However, additional
N
strategies must be used to determine precise timing or number of doses.
A
Ambulatory assessment methods provide real-time adherence data for
M
pharmacotherapy and behavioral treatments. Pill bottles or caps that detect and timestamp when the bottle is opened provide precise timing and higher estimates of medication non-
D
adherence relative to self-report and pill count.40 Remote observation of medication taking
TE
has also been implemented in SUD clinical trials via video connection with a staff member
EP
in real-time,41 video recording and uploading of medication taking in real-time to be reviewed by staff,29 or use of video technology that can detect medication ingestion (Ai
CC
Cure Technologies). These approaches may also increase adherence through incorporation of dose reminder messaging and contacting study participants when a dose is missed. More
A
recently, medication capsules equipped with an ingestible sensor have been developed that emit signals once the medication has reached the stomach.42 It is arguably easier to track adherence with assignments in behavioral therapy, particularly when the assignment can be
7
completed online or via a mobile device. Electronic reminders can be used and progress on behavioral assignments can be tracked in real-time.36 With improved adherence data accuracy, analyses may examine adherence as a
SC RI PT
moderator of treatment response. In addition, a run-in period prior to treatment randomization has been suggested.7 All participants receive placebo (or inert homework assignments) during this run-in period and ambulatory techniques can assess adherence.
Investigators may choose to only randomize participants demonstrating a certain level of
adherence to placebo or, in order to maintain generalizability, include only run-in adherent
U
participants in the primary analysis but retain all participants in the trial.7 Lastly, more
N
accurate estimation of adherence can help inform clinical trial sample size estimates to
A
account for expected non-adherence and attenuated effect sizes. The run-in period may also
M
reduce drop-out since those most susceptible to early drop-out may do so during this period. 3. Selection and Measurement of Outcomes
D
RCTs may produce erroneous results simply due to measurement error in the
TE
primary outcome. The typical outcome in SUD RCTs is change in substance use.17
EP
However, both biological confirmation of substance use and self-reported outcomes have limitations. It is thus generally recommended that substance use be assessed via both
CC
biomarkers and self-report.17 Traditionally, researchers test for the presence of the drug (or metabolites) from biological samples provided by participants at once to thrice weekly
A
visits, which is generally the gold standard for assessing abstinence or reduction in use. However, analysis of biomarkers a few times weekly does not provide insight regarding timing or specific quantity of substance use. Biomarkers typically have half-lives that are too short to confirm continuous abstinence (e.g., breath CO in smokers, breath alcohol
8
concentration in drinkers) or too long to capture change in use over shorter intervals (e.g., urinary Δ9-tetrahydrocannabinol in cannabis users43,44). Self-reported use is often assessed retrospectively at study visits, introducing potential recall biases and intentional
SC RI PT
misreporting.44,45 Ambulatory assessment may increase validity and rigor of clinical outcome
assessment.37 Near real-time self-report reduces retrospective recall and can be combined with more frequent remote biomarker detection.21-23 Substance use can also be detected indirectly based on real-time assessment of associated physiology, movement, or
U
behavior.23,27-29,46-59 Current examples of direct and indirect ambulatory assessment of
N
substance use are summarized in Table 2. To our knowledge, some methods have only been
A
applied to specific substances (e.g., cell phone usage data for high-risk alcohol use
M
detection59). However, future research may implement similar methods to detect other substance use. Because the success of a clinical trial hinges upon the accuracy of substance
TE
Summary and Conclusions
D
use assessment, highly sensitive and specific measures are critical.
EP
Methodological decisions can greatly impact the success of a RCT and, consequently, treatment development for SUDs. First, careful consideration of
CC
inclusion/exclusion criteria and the use of a brief, remote run-in period to monitor and verify inclusion criteria may be instrumental in ruling out inappropriate participants. Second,
A
treatment integrity and adherence should be monitored via viable methods and technologies. Real-time assessment may improve adherence in RCTs while also providing a more accurate estimate of non-adherence for analyses. Third, efforts should be made to enhance
9
the validity of substance use measurement. Incorporation of these strategies RCTs focused on establishing efficacy may decrease the rate of Type I and II errors. These recommendations come with additional challenges. First, most require
SC RI PT
additional assessment and costs. Technologies requiring programming may be costly; however, more affordable options are becoming increasingly available.29 Second, increased
assessment burden may impact study retention, perhaps in systematic ways. Future research should test whether increased monitoring leads to selection bias and/or increased non-
retention. Third, it is important to consider the effect that active monitoring may have on
U
outcomes. If increased monitoring of one’s behavior results in positive behavioral change,
N
placebo group response rates may increase. For this reason (and issues of burden), passive
A
monitoring that does not rely on participant engagement or requires minimal change in
M
typical behavior (e.g., transdermal monitoring; accelerometer data) are preferred. Fourth, ethical and logistical issues arise when delaying treatment for additional assessment. The
D
run-in period should be as brief as possible (e.g., one week7) while also allowing for
TE
sufficient assessment. Eligibility determination visits and randomization/treatment initiation
EP
visits often occur on separate occasions. Thus, inclusion of assessment between these two visits likely does not significantly delay treatment. Fifth, increased monitoring of treatment
CC
adherence does not combat all forms of non-adherence. Adverse effects and intentional nonadherence due to lack of perceived benefit will still occur. Sixth, appropriately identifying
A
the study population requires understanding of treatment mechanisms. This increases the importance of examining treatment mechanisms during a RCT. Though not directly related to the success of the RCT, ambulatory assessment can also be instrumental in helping identify treatment mechanisms.60 Finally, the use of run-in periods and more rigorous
10
assessment of inclusion criteria will result in fewer eligible and enrolled participants. While this process should ultimately increase the appropriateness of participants and improve detection of a meaningful effect, this may increase costs and slow recruitment. Fortunately,
SC RI PT
the use of mobile and remote assessments may allow recruitment from a larger geographical region due to a decreased need for in-person visits.
In sum, novel SUD treatments are needed and advances in technology allow
increased rigor in the conduct of RCTs, facilitating treatment development. Researchers may consider using these strategies to sensitively detect treatment effects and aid in the
A
CC
EP
TE
D
M
A
N
U
reduction of error rates.
11
References 1. Kiluk BD, Carroll KM. New developments in behavioral treatments for substance use disorders. Curr Psychiatry Rep. 2013 Nov;15(12):420.
SC RI PT
2. Volkow ND, Skolnick P. New medications for substance use disorders: challenges and opportunities. Neuropsychopharmacology. 2012 Jan;37(1):290-292.
3. **Thomas DW, Burns J, Audette J, Carroll A, Dow-Hygelund C, Hay M. Clinical development success rates 2006-2015. San Diego: Biomedtracker/Washington, DC: BIO/Bend: Amplion.
2016 Jun. Accessed 3/12/18 from http://www.amplion.com/clinical-development-success-rates.
M
A
N
U
This report examined success rates across >10,000 clinical trials and provides descriptive information on trial success rate across medical fields and phases of medication development. Psychiatry medications consistently had lower than average success rates at each phase of the medication development process. Success at Phase I was 6.2% for psychiatry and 9.6% for all indications. Success at Phase II was 24% for psychiatry and 31% for all indications. Success at Phase III was closer to the average with 56% of psychiatry medications and 58% of all medications having positive results. Studies using genetic biomarkers as inclusion criteria increased positive trial outcomes from 55% to 76%.
D
4. **Klemperer EM, Hughes JR, Naud S. Study characteristics influence the efficacy of substance
TE
abuse treatments: A meta-analysis of medications for alcohol use disorder. Drug and alcohol dependence. 2018 Jul 24.
CC
EP
In a meta-analysis of medications for alcohol use disorder, study characteristics accounted for almost half of the variance in treatment efficacy, with little consistency about which characteristics accounted for this effect. Findings highlight the need for comprehensively reporting study characteristics in clinical trials. 5. **Klemperer EM, Hughes JR, Naud S. Study characteristics influence the efficacy of substance
A
abuse treatments: A meta-analysis of medications for smoking cessation. Nicotine & Tobacco Research. In press. In a meta-analysis of pharmacological treatments for tobacco use disorder, study characteristics accounted for 12% (nicotine patch) to 36% (buproprion) of variance in odds ratio. Findings highlight the need for comprehensively reporting study characteristics in clinical trials. The rate of abstinence in control conditions varied depending on study
12
characteristics including funding source and date of publication, with industry funding and earlier dates of publication showing lower control group response rates for certain medications. 6. Witkiewitz K, Finney JW, Harris, AHS, Kivlahan DR, Kranzler HR. Recommendations for the
SC RI PT
design and analysis of treatment trials for alcohol use disorders. Alcohol Clin Exp Res. 2015 Sept; 39(9): 1557–1570.
7. **McCann DJ, Petry NM, Bresell A, Isacsson E, Wilson E, Alexander RC. Medication nonadherence, “professional subjects,” and apparent placebo responders: Overlapping
challenges for medications development. J Clin Psychopharmacol. 2015 Oct;35:566-573.
A
N
U
This manuscript encourages innovative efforts to prevent the enrollment of inappropriate subjects in clinical trials and to increase medication adherence. Authors promote the use of “Run-in with Adherence Monitoring for Prequalification but Undiminished Participation” (RAMPUP) study design for efficacy trials. This method can be used to lessen the impact of professional subjects (i.e., those “destined to succeed”) and increase medication adherence.
M
8. *Shiovitz TM, Bain EE, McCann DJ, Skolnick P, Laughren T, Hanina A, Burch D. Mitigating the effects of nonadherence in clinical trials. The Journal of Clinical Pharmacology. 2016 Sep
D
1;56(9):1151-1164.
EP
TE
The International Society for CNS Clinical Trial Methodology (ISCTM) Working Group on Nonadherence in Clinical Trials puts forth recommendations for maximizing medication adherence in all stages of clinical trials. These include pre-screen strategies to rule out participants, pre-randomization strategies to predict non-adherence, strategies to enhance adherence during the trial, and analytic strategies to account for non-adherence.
CC
9. Humphreys K, Harris AHS, Weingardt KR. Subject eligibility criteria can substantially influence the results of alcohol-treatment outcome research. J Stud Alcohol Drugs. 2008;
A
69:757–764.
10. Dennis BB, Roshanov PS, Naji L, et al. Opioid substitution and antagonist therapy trials exclude the common addiction patient: a systematic review and analysis of eligibility criteria. Trials. 2015;16:475.
13
11. *McCaul ME, Wand GS. Detecting deception in our research participants: Are your participants who you think they are? Alcoholism: Clinical and Experimental Research. 2018 Feb;42(2):230237.
SC RI PT
This critical review discusses the issue of failing to detect and inclusion of deceptive research participants in alcohol and other drug use trials. Recommendations are made (alcohol-specific) for improved detection of deceptive participants that may be implemented during the advertising, pre-screening, and in-person screening visits prior to enrollment in trials (e.g., advertising strategies, participant registries, biomarker collection, access to medical records, repeat testing, and many more). 12. Zweben A, Fucito LM, O'Malley SS. Effective strategies for maintaining research participation in clinical trials. Drug information journal. 2009 Jul;43(4):459-467.
U
13. Webb CA, DeRubeis RJ, Barber JP. Therapist adherence/competence and treatment outcome: A
N
meta-analytic review. Journal of consulting and clinical psychology. 2010 Apr;78(2):200-211.
A
14. Webb CA, DeRubeis RJ, Dimidjian S, Hollon SD, Amsterdam JD, Shelton RC. Predictors of
M
patient cognitive therapy skills and symptom change in two randomized clinical trials: The role
D
of therapist adherence and the therapeutic alliance. Journal of Consulting and Clinical
TE
Psychology. 2012 Jun;80(3):373-381.
15. Campos-Melady M, Smith JE, Meyers RJ, Godley SH, Godley MD. The effect of therapists’
EP
adherence and competence in delivering the adolescent community reinforcement approach on
CC
client outcomes. Psychology of Addictive Behaviors. 2017 Feb;31(1):117. 16. Wolsko PM, Solondz DK, Phillips RS, Schachter SC, Eisenberg DM. Lack of herbal
A
supplement characterization in published randomized controlled trials. The American journal of medicine. 2005 Oct 1;118(10):1087-1093.
17. Donovan DM, Bigelow GE, Brigham GS, Carroll KM, Cohen AJ, Gardin JG, Hamilton JA, Huestis MA, Hughes JR, Lindblad R, Marlatt GA. Primary outcome indices in illicit drug
14
dependence treatment research: Systematic approach to selection and measurement of drug use end‐points in clinical trials. Addiction. 2012 Apr 1;107(4):694-708. 18. Kiluk BD, Carroll KM, Duhig A, et al. Measures of outcome for stimulant trials: ACTTION
SC RI PT
recommendations and research agenda. Drug Alcohol Depend. 2015;158:1-7. 19. Carroll KM, Kiluk BD, Nich C, DeVito EE, Decker S, LaPaglia D, Duffey D, Babuscio TA, Ball SA. Toward empirical identification of a clinically meaningful indicator of treatment
outcome: features of candidate indicators and evaluation of sensitivity to treatment effects and
relationship to one year follow up cocaine use outcomes. Drug and Alcohol Dependence. 2014
U
Apr 1;137:3-19.
N
20. Fitzmaurice GM, Lipsitz SR, Weiss RD. Statistical considerations in the choice of endpoint for
A
drug use disorder trials. Drug and alcohol dependence. 2017 Dec 1;181:219-222.
M
21. Bertz JW, Epstein DH, Preston KL. Combining ecological momentary assessment with objective, ambulatory measures of behavior and physiology in substance-use research.
D
Addictive behaviors. 2018 Aug 1;83:5-17.
TE
22. Adams ZW, McClure EA, Gray KM, Danielson CK, Treiber FA, Ruggiero KJ. Mobile devices
EP
for the remote acquisition of physiological and behavioral biomarkers in psychiatric clinical research. Journal of Psychiatric Research. 2017 Feb 1;85:1-4.
CC
23. *Roberts W, McKee SA. Mobile Alcohol Biosensors and Pharmacotherapy Development Research. Alcohol. 2018 Aug 3.
A
This critical review and commentary describes how wearable sensors can advance the medication development process for alcohol use disorders. Specifically, the potential utility of wearable technology in all phases of clinical trials are described.
15
24. Fahrenberg J. Ambulatory assessment: Issues and perspectives. In: Fahrenberg J, Myrtek M, editors. Ambulatory assessment: Computer assisted psychological and psychophysiological methods in monitoring and field studies. Seattle, WA: Hogrefe & Huber; 1996. p. 3-20.
SC RI PT
25. Devine EG, Waters ME, Putnam M, et al. Concealment and fabrication by experienced research subjects. Clin Trials 2013;10:935-948.
26. Shiovitz TM, Zarrow ME, Shiovitz AM, Bystritsky AM. Failure rate and "professional
subjects" in clinical trials of major depressive disorder. The Journal of clinical psychiatry. 2011 Sep;72(9):1284-1285.
U
27. Dallery J, Raiff BR, Kim SJ, Marsch LA, Stitzer M, Grabinski MJ. Nationwide access to an
N
internet-based contingency management intervention to promote smoking cessation: a
A
randomized controlled trial. Addiction. 2017;112(5):875-883.
M
28. McClure EA, Tomko RL, Carpenter MJ, Treiber FA, Gray KM. Acceptability and compliance with a remote monitoring system to track smoking and abstinence among young smokers. The
D
American journal of drug and alcohol abuse. 2018; 44:561-570.
TE
29. Tomko RL, Gray KM, Oppenheimer SR, Wahlquist AE, McClure EA. Using REDCap for
EP
ambulatory assessment: Implementation in a clinical trial for smoking cessation to augment inperson data collection. American Journal of Drug and Alcohol Abuse. [Epub ahead of print]
CC
30. Friedman LM, Furberg C, DeMets DL, Reboussin D, Granger CB. Fundamentals of clinical trials. Springer-Verlag; 2015 Aug 27.
A
31. *McRae-Clark AL, Baker NL, Sonne SC, DeVane CL, Wagner A, Norton J. Concordance of direct and indirect measures of medication adherence in a treatment trial for cannabis dependence. J Subst Abuse Treat. 2015 Oct;57:70-74. Multiple measures of medication adherence were compared in a 12 week randomized clinical trial examining buspirone versus placebo for cannabis dependence. In addition to staff pill
16
counts, participants (N=109) kept daily diaries of doses taken and provided weekly urine samples which were analyzed for riboflavin (added to both active capsules and placebo) and 6OH-buspirone levels (for those receiving buspirone). Pill counts and daily diaries resulted in a steady >90% adherence across the 12 week study. However, riboflavin and 6-OH-buspirone suggested that adherence declined over time and produced lower estimates ranging from ~75% at week 6 to ~55-60% at week 12.
SC RI PT
32. Peng AR, Morales M, Wileyto EP, Hawk LW, Cinciripini P, George TP, Benowitz NL, Nollen
NL, Lerman C, Tyndale RF, Schnoll R. Measures and predictors of varenicline adherence in the treatment of nicotine dependence. Addictive behaviors. 2017 Dec 1;75:122-129.
33. Anderson AL, Li SH, Markova D, Holmes TH, Chiang N, Kahn R, Campbell J, Dickerson DL, Galloway GP, Haning W, Roache JD. Bupropion for the treatment of methamphetamine
U
dependence in non-daily users: a randomized, double-blind, placebo-controlled trial. Drug and
N
Alcohol Dependence. 2015 May 1;150:170-174.
A
34. Gonzalez VM, Schmitz JM, DeLaune KA. The role of homework in cognitive-behavioral
M
therapy for cocaine dependence. Journal of Consulting and Clinical Psychology. 2006
D
Jun;74(3):633-637.
TE
35. *Decker SE, Kiluk BD, Frankforter T, Babuscio T, Nich C, Carroll KM. Just showing up is not enough: Homework adherence and outcome in cognitive–behavioral therapy for cocaine
EP
dependence. Journal of consulting and clinical psychology. 2016 Oct;84(10):907-912.
A
CC
Data from four clinical trials of Cognitive-Behavioral Therapy for cocaine dependence suggest that rates of homework completion are around 41% among individuals attending at least two sessions. Homework completion was not associated with participant demographics, baseline cocaine use, or number of sessions attended. However, completion of homework was associated with better response to treatment (i.e., negative toxicology screens). 36. Willner-Reid, J., Whitaker, D., Epstein, D. H., Phillips, K. A., Pulaski, A. R., Preston, K. L., & Willner, P. (2015). Cognitive-behavioural therapy for heroin and cocaine use: Ecological momentary assessment of homework simplification and compliance. Psychology and psychotherapy, 89(3), 276-93.
17
37. *Gurvich EM, Kenna GA, Leggio L. Use of novel technology-based techniques to improve alcohol-related outcomes in clinical trials. Alcohol and Alcoholism. 2013;48(6):712-719.
SC RI PT
Technology-based methods of assessing and improving medication adherence in pharmacotherapy trials for alcohol misuse are reviewed, including the medication-event monitoring system, transdermal sensors, ecological momentary assessment, and interactive voice response. Evidence of superiority of these novel methods to traditional patient-adherence monitoring, e.g., pill counting and timeline followback, for improving the quality and accuracy of alcohol pharmacotherapy trial results is provided. 38. Ramanujam VS, Anderson KE, Grady JJ, Nayeem F, Lu LJ. Riboflavin as an oral tracer for monitoring compliance in clinical research. The open biomarkers journal. 2011; (4):1.
39. Hampson AJ, Babalonis S, Lofwall MR, Nuzzo PA, Krieter P, Walsh SL. A pharmacokinetic
N
clinical psychopharmacology. 2016 Aug;36(4):324-332.
U
study examining acetazolamide as a novel adherence marker for clinical trials. Journal of
A
40. **El Alili M, Vrijens B, Demonceau J, Evers SM, Hiligsmann M. A scoping review of studies
M
comparing the medication event monitoring system (MEMS) with alternative methods for
D
measuring medication adherence. British journal of clinical pharmacology. 2016 Jul
TE
1;82(1):268-279.
CC
EP
In this review of 117 relevant articles, authors compared medication adherence via the Medication Event Monitoring System (MEMS) to several other adherence measures. MEMS are pill bottle caps that detect openings and save a timestamp for download at a later time, while also displaying the number of openings per 24-hour period on the cap. This review found that self-report and pill counts provided higher estimates of medication adherence compared to MEMS, though stronger and more comparable methodologies are needed to rigorously compare different measures of medication adherence.
A
41. *DeWorsop D, Creatura G, Bluez G, Thurnauer H, Forselius-Bielen K, Ranganathan M, Deaso E, Bhat JA, D’Souza DC. Feasibility and success of cell-phone assisted remote observation of medication adherence (CAROMA) in clinical trials. Drug and Alcohol Dependence. 2016 Jun 1;163:24-30.
18
SC RI PT
A subsample of participants (n=20) participating in a larger randomized controlled trial for Cannabis Use Disorder piloted Cellphone Assisted Remote Observation of Medication Adherence (CAROMA). Participants were randomized to receive a Fatty Acid Amide Hydrolase inhibitor (FAAH-I) or placebo for 4 weeks. At a predetermined time each weekday morning, research staff called the participant via Skype ® on a mobile phone. The participant showed the pill, took the pill, and then opened their mouth to show the pill was taken. CAROMA resulted in 96% of expected doses taken and cost ~$100/per participant per week. 42. Chai PR, Carreiro S, Innes BJ, Chapman B, Schreiber KL, Edwards RR, Carrico AW, Boyer EW. Oxycodone ingestion patterns in acute fracture pain with digital pills. Anesthesia & Analgesia. 2017 Dec 1;125(6):2105-2112.
43. Lowe RH, Abraham TT, Darwin WD, Herning R, Cadet JL, Huestis MA. Extended urinary Δ9-
U
tetrahydrocannabinol excretion in chronic cannabis users precludes use as a biomarker of new
N
drug exposure. Drug & Alcohol Dependence. 2009 Nov 1;105(1):24-32.
A
44. Baker NL, Gray KM, Sherman BJ, Morella K, Sahlem GL, Wagner AM, McRae-Clark AL.
M
Biological correlates of self-reported new and continued abstinence in cannabis cessation
D
treatment clinical trials. Drug and alcohol dependence. 2018 Jun 1;187:270-277.
TE
45. Monk, R. L., Heim, D., Qureshi, A., & Price, A. (2015). “I have no clue what I drunk last night” using smartphone technology to compare in-vivo and retrospective self-reports of alcohol
EP
consumption. PLoS One, 10(5), e0126209.
CC
46. Koffarnus MN, Bickel WK, Kablinger AS. Remote Alcohol Monitoring to Facilitate Incentive‐ Based Treatment for Alcohol Use Disorder: A Randomized Trial. Alcoholism: Clinical and
A
Experimental Research. [Epub ahead of print]. 47. Leffingwell TR, Cooney NJ, Murphy JG, Luczak S, Rosen G, Dougherty DM, Barnett NP. Continuous objective monitoring of alcohol use: twenty‐first century measurement using transdermal sensors. Alcoholism: Clinical and Experimental Research. 2013 Jan 1;37(1):16-22.
19
48. *Linas BS, Genz A, Westergaard RP, Chang LW, Bollinger RC, Latkin C, Kirk GD. Ecological momentary assessment of illicit drug use compared to biological and self-reported methods. JMIR mHealth and uHealth. 2016 Jan;4(1).
SC RI PT
Adults who used cocaine and/or heroin recorded their substance use in real-time for 30 days on an electronic device. Participants also wore PharmChek Drugs of Abuse patches that collected sweat and could be analyzed for cocaine and heroin and/or their metabolites. Patches were collected once weekly for analysis. Self-report and biological confirmation of cocaine and heroin use via the sweat patch were concordant about 70% of the time.
49. Kim J, Jeerapan I, Imani S, Cho TN, Bandodkar A, Cinti S, Mercier PP, Wang J. Noninvasive alcohol monitoring using a wearable tattoo-based iontophoretic-biosensing system. Acs
U
Sensors. 2016 Jul 22;1(8):1011-1009.
N
50. Skinner AL, Stone CJ, Doughty H, Munafὸ MR. StopWatch: The preliminary evaluation of a
M
Research. 2019 Jan;21(2):257-261.
A
smartwatch-based system for passive detection of cigarette smoking. Nicotine & Tobacco
D
51. Gharani P, Suffoletto B, Chung T, Karimi HA. An artificial neural network for movement
TE
pattern analysis to estimate blood alcohol content level. Sensors. 2017 Dec 13;17(12):2897. 52. Hossain SM, Ali AA, Rahman M, et al. Identifying Drug (Cocaine) Intake Events from Acute
EP
Physiological Response in the Presence of Free-living Physical Activity. IPSN. 2014; 71-82.
CC
53. Carreiro S, Fang H, Zhang J, Wittbold K, Weng S, Mullins R, Smelson D, Boyer EW. iMStrong: deployment of a biosensor system to detect cocaine use. Journal of medical systems.
A
2015 Dec 1;39(12):186. 54. Carreiro S, Smelson D, Ranney M, Horvath KJ, Picard RW, Boudreaux ED, Hayes R, Boyer EW. Real-time mobile detection of drug use with wearable biosensors: a pilot study. Journal of Medical Toxicology. 2015 Mar 1;11(1):73-79. 55. Hatsukami D, Morgan SF, Pickens RW, Hughes JR. Smoking topography in a nonlaboratory
20
environment. International Journal of the Addictions Aug;1987 22:719–725. 56. Winhusen T, Theobald J, Lewis D. Design considerations for a pilot trial using a novel approach for evaluating smoking-cessation medication in methadone-maintained smokers.
SC RI PT
Contemporary Clinical Trials. 2016 Mar;47:334-339. 57. Pearson JL, Elmasry H, Das B, Smiley SL, Rubin LF, DeAtley T, Harvey E, Zhou Y, Niaura R, Abrams DB. Comparison of ecological momentary assessment versus direct measurement of ecigarette use with a bluetooth-enabled e-cigarette: a pilot study. JMIR research protocols. 2017 May;6(5).
U
58. Blank M, Hoek J, George M, Gendall P, Conner TS, Thrul J, Ling PM, Langlotz T. An
N
exploration of smoking-to-vaping transition attempts using a “smart” Electronic Nicotine
A
Delivery System. Nicotine & Tobacco Research. [in press]
M
59. *Bae S, Chung T, Ferreira D, Dey AK, Suffoletto B. Mobile phone sensors and supervised machine learning to identify alcohol use events in young adults: Implications for just-in-time
D
adaptive interventions. Addictive Behaviors. 2018 Aug; 83:42-47.
CC
EP
TE
This study provides an example of how mobile phone sensor data can be used to infer high-risk substance use behavior in real-time. Non-treatment seeking heavy drinkers recorded their alcohol use daily for 28 days while mobile phone sensor data, such as screen duration, call duration, phone screen unlocks, movement, and time of day, was recorded on their mobile devices. Using mobile phone sensor data, supervised machine learning was able to correctly classify time windows in which high-risk drinking occurred 90.9% of the time.
60. *Treloar Padovano H, Miranda R. Using ecological momentary assessment to identify
A
mechanisms of change: An application from a pharmacotherapy trial with adolescent cannabis users. Journal of studies on alcohol and drugs. 2018 Mar 18;79(2):190-198. This study provides an example for applying ecological momentary assessment to capture the temporal order of putative mechanisms of behavior change in pharmacotherapy trials. In a small trial of 40 young cannabis users (ages 15-24 years), topiramate exerted beneficial effects on reduced grams of cannabis used through reducing subjective high from smoking.
D
TE
EP
CC
A
SC RI PT
U
N
A
M
21
22
Table 1. Recommendations for Increasing Precision and Reducing Type I and II Error in SUD Clinical Trials
A
CC
EP
TE
D
M
A
N
U
SC RI PT
General Recommendations Participant Recruitment and Selection 1) Recruit directly from clinical settings, when possible7 2) Consider reducing financial incentives8 3) When possible, avoid excluding participants for non-safety related reasons9,10 4) Minimize disclosure of inclusion and exclusion criteria8,11 5) Objectively confirm inclusion and exclusion criteria11 6) Check medical records to confirm inclusion and exclusion criteria8,11 7) Use participant registries and require identification to avoid dual study participation and repeat participants7,8,11 8) Consider longitudinal assessment of symptoms prior to randomization11 Amount and Integrity of Treatment 1) Consider a run-in period to measure adherence prior to randomization and decide a priori how non-adherence will be addressed (e.g., RAMPUP design)7,8 2) Monitor treatment adherence7 3) Encourage participant adherence to intervention and protocol via adherence incentives, and/or reminders7,11 4) Maintain participants in trial through regular engagement and problem-solving barriers to study participation12 5) Monitor therapist fidelity to treatment protocol13-15 6) Test for quality and consistency of medication capsules, etc.16 Selection and Measurement of Outcomes 1) Select multiple, highly sensitive, specific, valid, and reliable measures (preferably combining self-report with an objective measure) 6,17,18 2) Select measures that are able to capture change over the course of the trial19 3) Consider use of quantitative, continuous measures (even if only for secondary outcomes) to increase statistical power19,20 4) Directly assess substance use in real-time with objective assessments, when possible, to confirm self-report21-23 5) When available, use passive assessments to minimize burden/reactivity23
23
Substance Passive Any No
Transdermal sensor23,47 Wearable patch/tattoo48,49 Hand/arm movement 50
Alcohol, Nicotine Alcohol
No
Direct Objective Outcome Yes No Quantity, Frequency, Abstinence Yes Yes Abstinence
Yes
Yes
Yes
Cocaine, Opiates, Alcohol Cigarette Smoking
Yes
Yes
Yes
Yes
No
Yes
No
Yes
No
Yes
Yes
Yes
No
Yes
Yes
Yes
SC RI PT
Remote Measure Self-report (Ecological Momentary Assessment/Interactive Voice Recording)21,37 Breath samples23,27-29, 46
U
Table 2. Methods for Measuring Substance Use in Real-World Settings
Gait analysis51
Alcohol
Psychophysiology52-54
Cocaine, Yes Opiates Cigarette No Smoking Cigarette Moderate Smoking Nicotine, Moderate Other constituents used in vaporizers Alcohol Yes
M
Portable puff topography devices55
A
N
Yes
Electronic, smart lighter56
TE
D
Smart e-cigarette or vaporizer57,58
Frequency, Abstinence Abstinence
Quantity, Frequency, Abstinence High-risk quantities Abstinence Quantity, Frequency Quantity, Frequency Quantity, Frequency, Abstinence
A
CC
EP
Mobile phone usage data (via supervised No Yes High-risk 59 machine learning) quantities Note. Passive measures do not require any input or activity from the individual beyond initial set-up of a device. Measures are considered moderately passive if they require user-input and adherence, but without any additional time burden. Direct measures assess substance use directly, while indirect measures assess a behavior or process related to substance use. Objective measures are not subject to reporting biases.
S2352-250X(18)30266-5 https://doi.org/10.1016/j.copsyc.2019.01.009 COPSYC 778
To appear in:
Please cite this article as: Tomko RL, McClure EA, Squeglia LM, Treloar Padovano H, McRae-Clark AL, Baker NL, Carpenter MJ, Gray KM, Methods to Reduce the Incidence of False Negative Trial Results in Substance Use Treatment Research, Current Opinion in Psychology (2019), https://doi.org/10.1016/j.copsyc.2019.01.009 This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Methods to Reduce the Incidence of False Negative Trial Results in Substance Use Treatment Research Running Head: Reducing False Negatives in SUD Treatment Research
SC RI PT
Rachel L. Tomko1,* [email protected], Erin A. McClure1, Lindsay M. Squeglia1, Hayley Treloar Padovano2, Aimee L. McRae-Clark1, Nathaniel L. Baker3, Matthew J. Carpenter1, Kevin M. Gray1 1
Department of Psychiatry and Behavioral Sciences, Medical University of South Carolina1, Department of Psychiatry and Human Behavior, Brown University2 3 Department of Public Health Sciences, Medical University of South Carolina3 2
*
U
Correspondence concerning this article may be addressed to Rachel L. Tomko,
N
Medical University of South Carolina, Department of Psychiatry and Behavioral Sciences,
A
MSC 861, 67 President Street, Charleston, SC 29425-8610
Rachel L. Tomko, Erin A. McClure, Lindsay M. Squeglia, Hayley Treloar
M
Padovano, Aimee L. McRae-Clark, and Nathaniel L. Baker declare that they have no conflicts of interest. Matthew J. Carpenter and Kevin M. Gray have provided consultation to
D
Pfizer, Inc.
TE
This manuscript was supported by National Institutes of Health grants from the National Institute of Drug Abuse (R01DA042114, R01DA038700, U01DA031779,
EP
UG3DA043231, K01DA036739, K24DA038240), Eunice Kennedy Shriver National Institute of Child Health and Human Development (K12HD055885), and National Institute
CC
on Alcohol Abuse and Alcoholism (K23AA025399, K23AA024808).
A
Word Count: 2074 Tables and Figures: 2
ABSTRACT Treatment development and evaluation for substance use disorders is hindered when randomized controlled trials fail to show a treatment effect when one exists. This manuscript
2
provides an overview of addressable methodological factors that may contribute to incorrect trial results. The collection of remote, naturalistic, real-time adherence and substance use data through ambulatory assessment methods in everyday life is presented as a partial
SC RI PT
solution. Other recommendations related to participant recruitment and selection, ensuring adequate consistency/fidelity and dose of treatment, and rigorously assessing clinical
outcomes are discussed. With implementation of eligibility criteria verification, treatment
adherence monitoring, and remote assessment of substance use and biomarkers, ambulatory assessment may help improve clinical trial success rates by improving precision, increasing
U
reproducibility, and reducing the impact of methodological issues that may lead to
N
inaccurate trial results.
A
Keywords: Randomized Controlled Trials, Pharmacotherapy, Psychotherapy, Medication
D
M
Adherence, Remote Technology, Ecological Momentary Assessment, Type II Error
TE
Despite effective evidence-based behavioral and pharmacological treatments for
EP
substance use disorders (SUDs),1,2 novel treatment development has not kept pace with advances in mechanistic understanding of SUDs. 2 Treatment development and
CC
dissemination remains slow1,2 and randomized controlled trials (RCTs) with spurious results can delay treatment development further. When an ineffective treatment is interpreted to be
A
effective (Type I error), it may be administered to an increasing number of patients, potentially exposing them to adverse effects without any benefit. It is also problematic when an effective treatment is incorrectly interpreted to be ineffective (Type II error), which may lead to the abandonment of future trials utilizing similar treatment strategies. Error rates in
3
SUD RCTs are difficult to estimate, but existing evidence suggests they are common. For example, 44% of successful Phase II psychiatric pharmacological interventions (smaller trials examining initial efficacy and safety in the population of interest) failed in larger
SC RI PT
Phase III trials (larger trials designed to examine efficacy).3 This is lower than the 55% Phase III trial success rate across all medical fields.3 Additionally, two meta-analyses
suggested that 12-45% of the variation in odds ratios in alcohol and tobacco use disorder
pharmacological trials were due to study characteristics, such as retention rate, biochemical verification of outcomes, funding source, and publication date.4,5 This suggests that a
U
number of RCTs result in null outcomes through no consequence of the intervention itself,
A
produce Type II errors in treatment evaluation.
N
leading to the notion that non-specific treatment factors, i.e., methodological factors, can
M
Here, we review methodological barriers that negatively impact RCTs. Randomization and statistical procedure selection considerations have been reviewed
D
elsewhere.6 We instead focus on participant recruitment and selection, quality and
TE
consistency of treatment, and measurement of clinical outcomes in efficacy trials
EP
(summarized in Table 1). Opportunities to decrease these errors through ambulatory assessment [“the acquisition of psychological data and/or physiological measures in
CC
everyday life (i.e., natural settings)”24] are emphasized.
A
1) Participant Recruitment and Selection Appropriately identifying inclusion criteria can improve the likelihood of detecting a
treatment effect. SUD diagnosis is a common inclusion criterion. In the future, diagnostic criteria may be refined or inclusion criteria may shift from diagnosis to phenotypes/endophenotypes that better reflect underlying neurobiological mechanisms. The
4
Phase III RCT success rate when patients are selected based on genetic biomarkers is 76% across medical fields, exceeding the average success rate of 55% without biomarker-based selection.3 This suggests that current inclusion criteria practices could be refined to enhance
SC RI PT
RCT outcomes, but more SUD research is necessary before such a shift could occur. Reducing exclusion criteria to the minimum possible for safety is recommended to maintain generalizability.9,10 Overly restrictive medical and psychiatric exclusion criteria
may also inadvertently attenuate treatment effects.9 It is thus important to carefully consider
eligibility criteria and to objectively verify that criteria are met. One survey found that nearly
U
40% of participants who had enrolled in more than one research study within the past year
N
acknowledged fabricating or exaggerating symptoms to meet criteria.25 Participants who
A
falsely report SUD symptoms are problematic because they are often “destined to succeed”
M
in the trial.7 The increase in placebo response in clinical trials may be due to a rise in these individuals who appear as treatment responders regardless of treatment assignment.7,26
D
Selection of individuals who are “destined to succeed” has a substantial negative impact on
TE
power in RCTs. McCann and colleagues7 provide a hypothetical example in which a sample
EP
size calculation suggested 282 patients were required to be adequately powered. If even 10% of the sample is “destined to succeed”, more than double the sample size would be
CC
necessary for adequate power and the estimated effect size would still be attenuated. Trial eligibility is often determined after a single in-person screening visit, which
A
often cannot offer a comprehensive clinical picture. For example, a non-smoker may attempt to enter a trial by smoking a few cigarettes prior to screening. They may then pass a breath carbon monoxide (CO) cut-off indicative of recent combustible tobacco use. However, it would be less likely that a non-smoker would meet eligibility criteria if breath CO is
5
repeatedly monitored in daily life with remote technology.27-29 Additionally, measurement error is reduced when the average of several longitudinal assessments is used as the baseline,30 which ambulatory assessment could make more feasible. Other strategies to
SC RI PT
prevent enrollment of deceptive participants may be considered, including recruiting directly from clinical settings, limiting disclosure of enrollment criteria, reducing financial
incentives, referencing medical records to verify criteria, and utilizing participant registries (Table 1).7,8,11
2. Dose, Consistency, and Fidelity of Treatment Administration
U
Efficacious treatments will appear ineffective in RCTs if patients receive an
N
insufficient amount of the treatment. This may occur if 1) treatment is not administered to
A
fidelity, 2) pharmacotherapy varies in preparation or potency, 3) participants do not adhere
M
to treatment recommendations (i.e., complete homework assignments, take medication), or 4) participants discontinue treatment early. Research on how behavioral treatment fidelity
D
affects outcomes is mixed, but appears to have nuanced effects on outcomes.13-15
TE
Researchers should ensure that treatment is being delivered to fidelity via regular
EP
monitoring to minimize Type II error. Likewise, pharmacotherapy trials should include verification of consistency, quality, and potency of medications.16 Indeed, the National
CC
Institutes of Health now requires new grant applications to acknowledge how the integrity
A
of chemical and biological resources will be determined. Even when researchers maximize the likelihood of high treatment integrity, patient
non-adherence and drop-out is pervasive in SUD RCTs. In pharmacotherapy trials, medication adherence is significantly over-estimated when patient self-report or pill counts are used, typically reported at >80%.7, 31,32 Trials that include biomarkers of adherence find
6
up to 35-55% of participants may be non-adherent despite high rates of self-reported adherence.7,32,33 Non-adherence with standard cognitive-behavioral treatment assignments is similar, with studies estimating 35-60% assignment completion rates among individuals
SC RI PT
with heroin or cocaine use disorders.34-36 Non-adherent participants significantly reduce statistical power in RCTs resulting in inadequacy to detect meaningful differences at the a priori defined sample size.7,8
Numerous solutions to objectively track medication adherence are being utilized.7,37 Testing for the presence of medication byproducts or added adherence markers (e.g.,
U
riboflavin38, acetazolamide39) in the urine is common and encouraged. However, additional
N
strategies must be used to determine precise timing or number of doses.
A
Ambulatory assessment methods provide real-time adherence data for
M
pharmacotherapy and behavioral treatments. Pill bottles or caps that detect and timestamp when the bottle is opened provide precise timing and higher estimates of medication non-
D
adherence relative to self-report and pill count.40 Remote observation of medication taking
TE
has also been implemented in SUD clinical trials via video connection with a staff member
EP
in real-time,41 video recording and uploading of medication taking in real-time to be reviewed by staff,29 or use of video technology that can detect medication ingestion (Ai
CC
Cure Technologies). These approaches may also increase adherence through incorporation of dose reminder messaging and contacting study participants when a dose is missed. More
A
recently, medication capsules equipped with an ingestible sensor have been developed that emit signals once the medication has reached the stomach.42 It is arguably easier to track adherence with assignments in behavioral therapy, particularly when the assignment can be
7
completed online or via a mobile device. Electronic reminders can be used and progress on behavioral assignments can be tracked in real-time.36 With improved adherence data accuracy, analyses may examine adherence as a
SC RI PT
moderator of treatment response. In addition, a run-in period prior to treatment randomization has been suggested.7 All participants receive placebo (or inert homework assignments) during this run-in period and ambulatory techniques can assess adherence.
Investigators may choose to only randomize participants demonstrating a certain level of
adherence to placebo or, in order to maintain generalizability, include only run-in adherent
U
participants in the primary analysis but retain all participants in the trial.7 Lastly, more
N
accurate estimation of adherence can help inform clinical trial sample size estimates to
A
account for expected non-adherence and attenuated effect sizes. The run-in period may also
M
reduce drop-out since those most susceptible to early drop-out may do so during this period. 3. Selection and Measurement of Outcomes
D
RCTs may produce erroneous results simply due to measurement error in the
TE
primary outcome. The typical outcome in SUD RCTs is change in substance use.17
EP
However, both biological confirmation of substance use and self-reported outcomes have limitations. It is thus generally recommended that substance use be assessed via both
CC
biomarkers and self-report.17 Traditionally, researchers test for the presence of the drug (or metabolites) from biological samples provided by participants at once to thrice weekly
A
visits, which is generally the gold standard for assessing abstinence or reduction in use. However, analysis of biomarkers a few times weekly does not provide insight regarding timing or specific quantity of substance use. Biomarkers typically have half-lives that are too short to confirm continuous abstinence (e.g., breath CO in smokers, breath alcohol
8
concentration in drinkers) or too long to capture change in use over shorter intervals (e.g., urinary Δ9-tetrahydrocannabinol in cannabis users43,44). Self-reported use is often assessed retrospectively at study visits, introducing potential recall biases and intentional
SC RI PT
misreporting.44,45 Ambulatory assessment may increase validity and rigor of clinical outcome
assessment.37 Near real-time self-report reduces retrospective recall and can be combined with more frequent remote biomarker detection.21-23 Substance use can also be detected indirectly based on real-time assessment of associated physiology, movement, or
U
behavior.23,27-29,46-59 Current examples of direct and indirect ambulatory assessment of
N
substance use are summarized in Table 2. To our knowledge, some methods have only been
A
applied to specific substances (e.g., cell phone usage data for high-risk alcohol use
M
detection59). However, future research may implement similar methods to detect other substance use. Because the success of a clinical trial hinges upon the accuracy of substance
TE
Summary and Conclusions
D
use assessment, highly sensitive and specific measures are critical.
EP
Methodological decisions can greatly impact the success of a RCT and, consequently, treatment development for SUDs. First, careful consideration of
CC
inclusion/exclusion criteria and the use of a brief, remote run-in period to monitor and verify inclusion criteria may be instrumental in ruling out inappropriate participants. Second,
A
treatment integrity and adherence should be monitored via viable methods and technologies. Real-time assessment may improve adherence in RCTs while also providing a more accurate estimate of non-adherence for analyses. Third, efforts should be made to enhance
9
the validity of substance use measurement. Incorporation of these strategies RCTs focused on establishing efficacy may decrease the rate of Type I and II errors. These recommendations come with additional challenges. First, most require
SC RI PT
additional assessment and costs. Technologies requiring programming may be costly; however, more affordable options are becoming increasingly available.29 Second, increased
assessment burden may impact study retention, perhaps in systematic ways. Future research should test whether increased monitoring leads to selection bias and/or increased non-
retention. Third, it is important to consider the effect that active monitoring may have on
U
outcomes. If increased monitoring of one’s behavior results in positive behavioral change,
N
placebo group response rates may increase. For this reason (and issues of burden), passive
A
monitoring that does not rely on participant engagement or requires minimal change in
M
typical behavior (e.g., transdermal monitoring; accelerometer data) are preferred. Fourth, ethical and logistical issues arise when delaying treatment for additional assessment. The
D
run-in period should be as brief as possible (e.g., one week7) while also allowing for
TE
sufficient assessment. Eligibility determination visits and randomization/treatment initiation
EP
visits often occur on separate occasions. Thus, inclusion of assessment between these two visits likely does not significantly delay treatment. Fifth, increased monitoring of treatment
CC
adherence does not combat all forms of non-adherence. Adverse effects and intentional nonadherence due to lack of perceived benefit will still occur. Sixth, appropriately identifying
A
the study population requires understanding of treatment mechanisms. This increases the importance of examining treatment mechanisms during a RCT. Though not directly related to the success of the RCT, ambulatory assessment can also be instrumental in helping identify treatment mechanisms.60 Finally, the use of run-in periods and more rigorous
10
assessment of inclusion criteria will result in fewer eligible and enrolled participants. While this process should ultimately increase the appropriateness of participants and improve detection of a meaningful effect, this may increase costs and slow recruitment. Fortunately,
SC RI PT
the use of mobile and remote assessments may allow recruitment from a larger geographical region due to a decreased need for in-person visits.
In sum, novel SUD treatments are needed and advances in technology allow
increased rigor in the conduct of RCTs, facilitating treatment development. Researchers may consider using these strategies to sensitively detect treatment effects and aid in the
A
CC
EP
TE
D
M
A
N
U
reduction of error rates.
11
References 1. Kiluk BD, Carroll KM. New developments in behavioral treatments for substance use disorders. Curr Psychiatry Rep. 2013 Nov;15(12):420.
SC RI PT
2. Volkow ND, Skolnick P. New medications for substance use disorders: challenges and opportunities. Neuropsychopharmacology. 2012 Jan;37(1):290-292.
3. **Thomas DW, Burns J, Audette J, Carroll A, Dow-Hygelund C, Hay M. Clinical development success rates 2006-2015. San Diego: Biomedtracker/Washington, DC: BIO/Bend: Amplion.
2016 Jun. Accessed 3/12/18 from http://www.amplion.com/clinical-development-success-rates.
M
A
N
U
This report examined success rates across >10,000 clinical trials and provides descriptive information on trial success rate across medical fields and phases of medication development. Psychiatry medications consistently had lower than average success rates at each phase of the medication development process. Success at Phase I was 6.2% for psychiatry and 9.6% for all indications. Success at Phase II was 24% for psychiatry and 31% for all indications. Success at Phase III was closer to the average with 56% of psychiatry medications and 58% of all medications having positive results. Studies using genetic biomarkers as inclusion criteria increased positive trial outcomes from 55% to 76%.
D
4. **Klemperer EM, Hughes JR, Naud S. Study characteristics influence the efficacy of substance
TE
abuse treatments: A meta-analysis of medications for alcohol use disorder. Drug and alcohol dependence. 2018 Jul 24.
CC
EP
In a meta-analysis of medications for alcohol use disorder, study characteristics accounted for almost half of the variance in treatment efficacy, with little consistency about which characteristics accounted for this effect. Findings highlight the need for comprehensively reporting study characteristics in clinical trials. 5. **Klemperer EM, Hughes JR, Naud S. Study characteristics influence the efficacy of substance
A
abuse treatments: A meta-analysis of medications for smoking cessation. Nicotine & Tobacco Research. In press. In a meta-analysis of pharmacological treatments for tobacco use disorder, study characteristics accounted for 12% (nicotine patch) to 36% (buproprion) of variance in odds ratio. Findings highlight the need for comprehensively reporting study characteristics in clinical trials. The rate of abstinence in control conditions varied depending on study
12
characteristics including funding source and date of publication, with industry funding and earlier dates of publication showing lower control group response rates for certain medications. 6. Witkiewitz K, Finney JW, Harris, AHS, Kivlahan DR, Kranzler HR. Recommendations for the
SC RI PT
design and analysis of treatment trials for alcohol use disorders. Alcohol Clin Exp Res. 2015 Sept; 39(9): 1557–1570.
7. **McCann DJ, Petry NM, Bresell A, Isacsson E, Wilson E, Alexander RC. Medication nonadherence, “professional subjects,” and apparent placebo responders: Overlapping
challenges for medications development. J Clin Psychopharmacol. 2015 Oct;35:566-573.
A
N
U
This manuscript encourages innovative efforts to prevent the enrollment of inappropriate subjects in clinical trials and to increase medication adherence. Authors promote the use of “Run-in with Adherence Monitoring for Prequalification but Undiminished Participation” (RAMPUP) study design for efficacy trials. This method can be used to lessen the impact of professional subjects (i.e., those “destined to succeed”) and increase medication adherence.
M
8. *Shiovitz TM, Bain EE, McCann DJ, Skolnick P, Laughren T, Hanina A, Burch D. Mitigating the effects of nonadherence in clinical trials. The Journal of Clinical Pharmacology. 2016 Sep
D
1;56(9):1151-1164.
EP
TE
The International Society for CNS Clinical Trial Methodology (ISCTM) Working Group on Nonadherence in Clinical Trials puts forth recommendations for maximizing medication adherence in all stages of clinical trials. These include pre-screen strategies to rule out participants, pre-randomization strategies to predict non-adherence, strategies to enhance adherence during the trial, and analytic strategies to account for non-adherence.
CC
9. Humphreys K, Harris AHS, Weingardt KR. Subject eligibility criteria can substantially influence the results of alcohol-treatment outcome research. J Stud Alcohol Drugs. 2008;
A
69:757–764.
10. Dennis BB, Roshanov PS, Naji L, et al. Opioid substitution and antagonist therapy trials exclude the common addiction patient: a systematic review and analysis of eligibility criteria. Trials. 2015;16:475.
13
11. *McCaul ME, Wand GS. Detecting deception in our research participants: Are your participants who you think they are? Alcoholism: Clinical and Experimental Research. 2018 Feb;42(2):230237.
SC RI PT
This critical review discusses the issue of failing to detect and inclusion of deceptive research participants in alcohol and other drug use trials. Recommendations are made (alcohol-specific) for improved detection of deceptive participants that may be implemented during the advertising, pre-screening, and in-person screening visits prior to enrollment in trials (e.g., advertising strategies, participant registries, biomarker collection, access to medical records, repeat testing, and many more). 12. Zweben A, Fucito LM, O'Malley SS. Effective strategies for maintaining research participation in clinical trials. Drug information journal. 2009 Jul;43(4):459-467.
U
13. Webb CA, DeRubeis RJ, Barber JP. Therapist adherence/competence and treatment outcome: A
N
meta-analytic review. Journal of consulting and clinical psychology. 2010 Apr;78(2):200-211.
A
14. Webb CA, DeRubeis RJ, Dimidjian S, Hollon SD, Amsterdam JD, Shelton RC. Predictors of
M
patient cognitive therapy skills and symptom change in two randomized clinical trials: The role
D
of therapist adherence and the therapeutic alliance. Journal of Consulting and Clinical
TE
Psychology. 2012 Jun;80(3):373-381.
15. Campos-Melady M, Smith JE, Meyers RJ, Godley SH, Godley MD. The effect of therapists’
EP
adherence and competence in delivering the adolescent community reinforcement approach on
CC
client outcomes. Psychology of Addictive Behaviors. 2017 Feb;31(1):117. 16. Wolsko PM, Solondz DK, Phillips RS, Schachter SC, Eisenberg DM. Lack of herbal
A
supplement characterization in published randomized controlled trials. The American journal of medicine. 2005 Oct 1;118(10):1087-1093.
17. Donovan DM, Bigelow GE, Brigham GS, Carroll KM, Cohen AJ, Gardin JG, Hamilton JA, Huestis MA, Hughes JR, Lindblad R, Marlatt GA. Primary outcome indices in illicit drug
14
dependence treatment research: Systematic approach to selection and measurement of drug use end‐points in clinical trials. Addiction. 2012 Apr 1;107(4):694-708. 18. Kiluk BD, Carroll KM, Duhig A, et al. Measures of outcome for stimulant trials: ACTTION
SC RI PT
recommendations and research agenda. Drug Alcohol Depend. 2015;158:1-7. 19. Carroll KM, Kiluk BD, Nich C, DeVito EE, Decker S, LaPaglia D, Duffey D, Babuscio TA, Ball SA. Toward empirical identification of a clinically meaningful indicator of treatment
outcome: features of candidate indicators and evaluation of sensitivity to treatment effects and
relationship to one year follow up cocaine use outcomes. Drug and Alcohol Dependence. 2014
U
Apr 1;137:3-19.
N
20. Fitzmaurice GM, Lipsitz SR, Weiss RD. Statistical considerations in the choice of endpoint for
A
drug use disorder trials. Drug and alcohol dependence. 2017 Dec 1;181:219-222.
M
21. Bertz JW, Epstein DH, Preston KL. Combining ecological momentary assessment with objective, ambulatory measures of behavior and physiology in substance-use research.
D
Addictive behaviors. 2018 Aug 1;83:5-17.
TE
22. Adams ZW, McClure EA, Gray KM, Danielson CK, Treiber FA, Ruggiero KJ. Mobile devices
EP
for the remote acquisition of physiological and behavioral biomarkers in psychiatric clinical research. Journal of Psychiatric Research. 2017 Feb 1;85:1-4.
CC
23. *Roberts W, McKee SA. Mobile Alcohol Biosensors and Pharmacotherapy Development Research. Alcohol. 2018 Aug 3.
A
This critical review and commentary describes how wearable sensors can advance the medication development process for alcohol use disorders. Specifically, the potential utility of wearable technology in all phases of clinical trials are described.
15
24. Fahrenberg J. Ambulatory assessment: Issues and perspectives. In: Fahrenberg J, Myrtek M, editors. Ambulatory assessment: Computer assisted psychological and psychophysiological methods in monitoring and field studies. Seattle, WA: Hogrefe & Huber; 1996. p. 3-20.
SC RI PT
25. Devine EG, Waters ME, Putnam M, et al. Concealment and fabrication by experienced research subjects. Clin Trials 2013;10:935-948.
26. Shiovitz TM, Zarrow ME, Shiovitz AM, Bystritsky AM. Failure rate and "professional
subjects" in clinical trials of major depressive disorder. The Journal of clinical psychiatry. 2011 Sep;72(9):1284-1285.
U
27. Dallery J, Raiff BR, Kim SJ, Marsch LA, Stitzer M, Grabinski MJ. Nationwide access to an
N
internet-based contingency management intervention to promote smoking cessation: a
A
randomized controlled trial. Addiction. 2017;112(5):875-883.
M
28. McClure EA, Tomko RL, Carpenter MJ, Treiber FA, Gray KM. Acceptability and compliance with a remote monitoring system to track smoking and abstinence among young smokers. The
D
American journal of drug and alcohol abuse. 2018; 44:561-570.
TE
29. Tomko RL, Gray KM, Oppenheimer SR, Wahlquist AE, McClure EA. Using REDCap for
EP
ambulatory assessment: Implementation in a clinical trial for smoking cessation to augment inperson data collection. American Journal of Drug and Alcohol Abuse. [Epub ahead of print]
CC
30. Friedman LM, Furberg C, DeMets DL, Reboussin D, Granger CB. Fundamentals of clinical trials. Springer-Verlag; 2015 Aug 27.
A
31. *McRae-Clark AL, Baker NL, Sonne SC, DeVane CL, Wagner A, Norton J. Concordance of direct and indirect measures of medication adherence in a treatment trial for cannabis dependence. J Subst Abuse Treat. 2015 Oct;57:70-74. Multiple measures of medication adherence were compared in a 12 week randomized clinical trial examining buspirone versus placebo for cannabis dependence. In addition to staff pill
16
counts, participants (N=109) kept daily diaries of doses taken and provided weekly urine samples which were analyzed for riboflavin (added to both active capsules and placebo) and 6OH-buspirone levels (for those receiving buspirone). Pill counts and daily diaries resulted in a steady >90% adherence across the 12 week study. However, riboflavin and 6-OH-buspirone suggested that adherence declined over time and produced lower estimates ranging from ~75% at week 6 to ~55-60% at week 12.
SC RI PT
32. Peng AR, Morales M, Wileyto EP, Hawk LW, Cinciripini P, George TP, Benowitz NL, Nollen
NL, Lerman C, Tyndale RF, Schnoll R. Measures and predictors of varenicline adherence in the treatment of nicotine dependence. Addictive behaviors. 2017 Dec 1;75:122-129.
33. Anderson AL, Li SH, Markova D, Holmes TH, Chiang N, Kahn R, Campbell J, Dickerson DL, Galloway GP, Haning W, Roache JD. Bupropion for the treatment of methamphetamine
U
dependence in non-daily users: a randomized, double-blind, placebo-controlled trial. Drug and
N
Alcohol Dependence. 2015 May 1;150:170-174.
A
34. Gonzalez VM, Schmitz JM, DeLaune KA. The role of homework in cognitive-behavioral
M
therapy for cocaine dependence. Journal of Consulting and Clinical Psychology. 2006
D
Jun;74(3):633-637.
TE
35. *Decker SE, Kiluk BD, Frankforter T, Babuscio T, Nich C, Carroll KM. Just showing up is not enough: Homework adherence and outcome in cognitive–behavioral therapy for cocaine
EP
dependence. Journal of consulting and clinical psychology. 2016 Oct;84(10):907-912.
A
CC
Data from four clinical trials of Cognitive-Behavioral Therapy for cocaine dependence suggest that rates of homework completion are around 41% among individuals attending at least two sessions. Homework completion was not associated with participant demographics, baseline cocaine use, or number of sessions attended. However, completion of homework was associated with better response to treatment (i.e., negative toxicology screens). 36. Willner-Reid, J., Whitaker, D., Epstein, D. H., Phillips, K. A., Pulaski, A. R., Preston, K. L., & Willner, P. (2015). Cognitive-behavioural therapy for heroin and cocaine use: Ecological momentary assessment of homework simplification and compliance. Psychology and psychotherapy, 89(3), 276-93.
17
37. *Gurvich EM, Kenna GA, Leggio L. Use of novel technology-based techniques to improve alcohol-related outcomes in clinical trials. Alcohol and Alcoholism. 2013;48(6):712-719.
SC RI PT
Technology-based methods of assessing and improving medication adherence in pharmacotherapy trials for alcohol misuse are reviewed, including the medication-event monitoring system, transdermal sensors, ecological momentary assessment, and interactive voice response. Evidence of superiority of these novel methods to traditional patient-adherence monitoring, e.g., pill counting and timeline followback, for improving the quality and accuracy of alcohol pharmacotherapy trial results is provided. 38. Ramanujam VS, Anderson KE, Grady JJ, Nayeem F, Lu LJ. Riboflavin as an oral tracer for monitoring compliance in clinical research. The open biomarkers journal. 2011; (4):1.
39. Hampson AJ, Babalonis S, Lofwall MR, Nuzzo PA, Krieter P, Walsh SL. A pharmacokinetic
N
clinical psychopharmacology. 2016 Aug;36(4):324-332.
U
study examining acetazolamide as a novel adherence marker for clinical trials. Journal of
A
40. **El Alili M, Vrijens B, Demonceau J, Evers SM, Hiligsmann M. A scoping review of studies
M
comparing the medication event monitoring system (MEMS) with alternative methods for
D
measuring medication adherence. British journal of clinical pharmacology. 2016 Jul
TE
1;82(1):268-279.
CC
EP
In this review of 117 relevant articles, authors compared medication adherence via the Medication Event Monitoring System (MEMS) to several other adherence measures. MEMS are pill bottle caps that detect openings and save a timestamp for download at a later time, while also displaying the number of openings per 24-hour period on the cap. This review found that self-report and pill counts provided higher estimates of medication adherence compared to MEMS, though stronger and more comparable methodologies are needed to rigorously compare different measures of medication adherence.
A
41. *DeWorsop D, Creatura G, Bluez G, Thurnauer H, Forselius-Bielen K, Ranganathan M, Deaso E, Bhat JA, D’Souza DC. Feasibility and success of cell-phone assisted remote observation of medication adherence (CAROMA) in clinical trials. Drug and Alcohol Dependence. 2016 Jun 1;163:24-30.
18
SC RI PT
A subsample of participants (n=20) participating in a larger randomized controlled trial for Cannabis Use Disorder piloted Cellphone Assisted Remote Observation of Medication Adherence (CAROMA). Participants were randomized to receive a Fatty Acid Amide Hydrolase inhibitor (FAAH-I) or placebo for 4 weeks. At a predetermined time each weekday morning, research staff called the participant via Skype ® on a mobile phone. The participant showed the pill, took the pill, and then opened their mouth to show the pill was taken. CAROMA resulted in 96% of expected doses taken and cost ~$100/per participant per week. 42. Chai PR, Carreiro S, Innes BJ, Chapman B, Schreiber KL, Edwards RR, Carrico AW, Boyer EW. Oxycodone ingestion patterns in acute fracture pain with digital pills. Anesthesia & Analgesia. 2017 Dec 1;125(6):2105-2112.
43. Lowe RH, Abraham TT, Darwin WD, Herning R, Cadet JL, Huestis MA. Extended urinary Δ9-
U
tetrahydrocannabinol excretion in chronic cannabis users precludes use as a biomarker of new
N
drug exposure. Drug & Alcohol Dependence. 2009 Nov 1;105(1):24-32.
A
44. Baker NL, Gray KM, Sherman BJ, Morella K, Sahlem GL, Wagner AM, McRae-Clark AL.
M
Biological correlates of self-reported new and continued abstinence in cannabis cessation
D
treatment clinical trials. Drug and alcohol dependence. 2018 Jun 1;187:270-277.
TE
45. Monk, R. L., Heim, D., Qureshi, A., & Price, A. (2015). “I have no clue what I drunk last night” using smartphone technology to compare in-vivo and retrospective self-reports of alcohol
EP
consumption. PLoS One, 10(5), e0126209.
CC
46. Koffarnus MN, Bickel WK, Kablinger AS. Remote Alcohol Monitoring to Facilitate Incentive‐ Based Treatment for Alcohol Use Disorder: A Randomized Trial. Alcoholism: Clinical and
A
Experimental Research. [Epub ahead of print]. 47. Leffingwell TR, Cooney NJ, Murphy JG, Luczak S, Rosen G, Dougherty DM, Barnett NP. Continuous objective monitoring of alcohol use: twenty‐first century measurement using transdermal sensors. Alcoholism: Clinical and Experimental Research. 2013 Jan 1;37(1):16-22.
19
48. *Linas BS, Genz A, Westergaard RP, Chang LW, Bollinger RC, Latkin C, Kirk GD. Ecological momentary assessment of illicit drug use compared to biological and self-reported methods. JMIR mHealth and uHealth. 2016 Jan;4(1).
SC RI PT
Adults who used cocaine and/or heroin recorded their substance use in real-time for 30 days on an electronic device. Participants also wore PharmChek Drugs of Abuse patches that collected sweat and could be analyzed for cocaine and heroin and/or their metabolites. Patches were collected once weekly for analysis. Self-report and biological confirmation of cocaine and heroin use via the sweat patch were concordant about 70% of the time.
49. Kim J, Jeerapan I, Imani S, Cho TN, Bandodkar A, Cinti S, Mercier PP, Wang J. Noninvasive alcohol monitoring using a wearable tattoo-based iontophoretic-biosensing system. Acs
U
Sensors. 2016 Jul 22;1(8):1011-1009.
N
50. Skinner AL, Stone CJ, Doughty H, Munafὸ MR. StopWatch: The preliminary evaluation of a
M
Research. 2019 Jan;21(2):257-261.
A
smartwatch-based system for passive detection of cigarette smoking. Nicotine & Tobacco
D
51. Gharani P, Suffoletto B, Chung T, Karimi HA. An artificial neural network for movement
TE
pattern analysis to estimate blood alcohol content level. Sensors. 2017 Dec 13;17(12):2897. 52. Hossain SM, Ali AA, Rahman M, et al. Identifying Drug (Cocaine) Intake Events from Acute
EP
Physiological Response in the Presence of Free-living Physical Activity. IPSN. 2014; 71-82.
CC
53. Carreiro S, Fang H, Zhang J, Wittbold K, Weng S, Mullins R, Smelson D, Boyer EW. iMStrong: deployment of a biosensor system to detect cocaine use. Journal of medical systems.
A
2015 Dec 1;39(12):186. 54. Carreiro S, Smelson D, Ranney M, Horvath KJ, Picard RW, Boudreaux ED, Hayes R, Boyer EW. Real-time mobile detection of drug use with wearable biosensors: a pilot study. Journal of Medical Toxicology. 2015 Mar 1;11(1):73-79. 55. Hatsukami D, Morgan SF, Pickens RW, Hughes JR. Smoking topography in a nonlaboratory
20
environment. International Journal of the Addictions Aug;1987 22:719–725. 56. Winhusen T, Theobald J, Lewis D. Design considerations for a pilot trial using a novel approach for evaluating smoking-cessation medication in methadone-maintained smokers.
SC RI PT
Contemporary Clinical Trials. 2016 Mar;47:334-339. 57. Pearson JL, Elmasry H, Das B, Smiley SL, Rubin LF, DeAtley T, Harvey E, Zhou Y, Niaura R, Abrams DB. Comparison of ecological momentary assessment versus direct measurement of ecigarette use with a bluetooth-enabled e-cigarette: a pilot study. JMIR research protocols. 2017 May;6(5).
U
58. Blank M, Hoek J, George M, Gendall P, Conner TS, Thrul J, Ling PM, Langlotz T. An
N
exploration of smoking-to-vaping transition attempts using a “smart” Electronic Nicotine
A
Delivery System. Nicotine & Tobacco Research. [in press]
M
59. *Bae S, Chung T, Ferreira D, Dey AK, Suffoletto B. Mobile phone sensors and supervised machine learning to identify alcohol use events in young adults: Implications for just-in-time
D
adaptive interventions. Addictive Behaviors. 2018 Aug; 83:42-47.
CC
EP
TE
This study provides an example of how mobile phone sensor data can be used to infer high-risk substance use behavior in real-time. Non-treatment seeking heavy drinkers recorded their alcohol use daily for 28 days while mobile phone sensor data, such as screen duration, call duration, phone screen unlocks, movement, and time of day, was recorded on their mobile devices. Using mobile phone sensor data, supervised machine learning was able to correctly classify time windows in which high-risk drinking occurred 90.9% of the time.
60. *Treloar Padovano H, Miranda R. Using ecological momentary assessment to identify
A
mechanisms of change: An application from a pharmacotherapy trial with adolescent cannabis users. Journal of studies on alcohol and drugs. 2018 Mar 18;79(2):190-198. This study provides an example for applying ecological momentary assessment to capture the temporal order of putative mechanisms of behavior change in pharmacotherapy trials. In a small trial of 40 young cannabis users (ages 15-24 years), topiramate exerted beneficial effects on reduced grams of cannabis used through reducing subjective high from smoking.
D
TE
EP
CC
A
SC RI PT
U
N
A
M
21
22
Table 1. Recommendations for Increasing Precision and Reducing Type I and II Error in SUD Clinical Trials
A
CC
EP
TE
D
M
A
N
U
SC RI PT
General Recommendations Participant Recruitment and Selection 1) Recruit directly from clinical settings, when possible7 2) Consider reducing financial incentives8 3) When possible, avoid excluding participants for non-safety related reasons9,10 4) Minimize disclosure of inclusion and exclusion criteria8,11 5) Objectively confirm inclusion and exclusion criteria11 6) Check medical records to confirm inclusion and exclusion criteria8,11 7) Use participant registries and require identification to avoid dual study participation and repeat participants7,8,11 8) Consider longitudinal assessment of symptoms prior to randomization11 Amount and Integrity of Treatment 1) Consider a run-in period to measure adherence prior to randomization and decide a priori how non-adherence will be addressed (e.g., RAMPUP design)7,8 2) Monitor treatment adherence7 3) Encourage participant adherence to intervention and protocol via adherence incentives, and/or reminders7,11 4) Maintain participants in trial through regular engagement and problem-solving barriers to study participation12 5) Monitor therapist fidelity to treatment protocol13-15 6) Test for quality and consistency of medication capsules, etc.16 Selection and Measurement of Outcomes 1) Select multiple, highly sensitive, specific, valid, and reliable measures (preferably combining self-report with an objective measure) 6,17,18 2) Select measures that are able to capture change over the course of the trial19 3) Consider use of quantitative, continuous measures (even if only for secondary outcomes) to increase statistical power19,20 4) Directly assess substance use in real-time with objective assessments, when possible, to confirm self-report21-23 5) When available, use passive assessments to minimize burden/reactivity23
23
Substance Passive Any No
Transdermal sensor23,47 Wearable patch/tattoo48,49 Hand/arm movement 50
Alcohol, Nicotine Alcohol
No
Direct Objective Outcome Yes No Quantity, Frequency, Abstinence Yes Yes Abstinence
Yes
Yes
Yes
Cocaine, Opiates, Alcohol Cigarette Smoking
Yes
Yes
Yes
Yes
No
Yes
No
Yes
No
Yes
Yes
Yes
No
Yes
Yes
Yes
SC RI PT
Remote Measure Self-report (Ecological Momentary Assessment/Interactive Voice Recording)21,37 Breath samples23,27-29, 46
U
Table 2. Methods for Measuring Substance Use in Real-World Settings
Gait analysis51
Alcohol
Psychophysiology52-54
Cocaine, Yes Opiates Cigarette No Smoking Cigarette Moderate Smoking Nicotine, Moderate Other constituents used in vaporizers Alcohol Yes
M
Portable puff topography devices55
A
N
Yes
Electronic, smart lighter56
TE
D
Smart e-cigarette or vaporizer57,58
Frequency, Abstinence Abstinence
Quantity, Frequency, Abstinence High-risk quantities Abstinence Quantity, Frequency Quantity, Frequency Quantity, Frequency, Abstinence
A
CC
EP
Mobile phone usage data (via supervised No Yes High-risk 59 machine learning) quantities Note. Passive measures do not require any input or activity from the individual beyond initial set-up of a device. Measures are considered moderately passive if they require user-input and adherence, but without any additional time burden. Direct measures assess substance use directly, while indirect measures assess a behavior or process related to substance use. Objective measures are not subject to reporting biases.